Ralph Loops

Topic

In agent architecture, I have heard the term Ralph Loop thrown around quite a bit. I want to learn a little bit more on what it is an how I can apply them in my agent workflow.

Conjecture

I mean the fact that it is called a loop means that essentially we are using the output from an LLM as the input to another LLM. I want to use the word recursive but I am not quite sure if that is the right term.

Research

Ok so this is Ralph. He is the namesake of the Ralph loop. A Ralph loop is just a shell script that calls the LLM over and over again until all the criteria of a PRD are met or all of the PRDs that you provide it are accomplished.

Here is a repo that that implements this technique:

https://github.com/snarktank/ralph?tab=readme-ov-file

And then this is the flow chart for how the ralph loop works. It essentially takes all available PRDs and then commits the changes updates the PRD, logs the process and then looks for another PRD.

The power in this is that if you spend more time planning out your PRDs you can actually have the agent do a lot more work autonomously. This has its pros and cons. It is worth experimenting with, but my concern is that the agent goes off the rails on one of the tasks and, because it has produced so much code by the time you look, the error isn’t immediately apparent.

Another issue is that you get to the end of all the stories and then you have a bunch of stuff that is ok but you kind of have to go back to the first story and fix things that then change things in the dependent story.

But this could be an interesting way to generate a lot of code quickly and effectively. Its worth exploring more.

Final Summary

The Ralph loop seems to be just a way of queuing a bunch of tasks for a coding agent so that you can have it work autonomously for a much longer period of time without supervision. I do have concerns about letting an agent run wild on a problem for that long, but it definitely could have value. Both of my objections point the same direction though: the loop is only as good as the verification inside it. If each pass has to leave the build green and the tests passing before it moves on, a bad story gets caught on the story that produced it rather than five stories later. The planning isn’t the expensive part — the checkpoint is.