Taking my agent loop out of Claude Code

The loop from 500 rounds of an autonomous coding loop ran as a workflow script inside Claude Code. It still does the same work, but Claude Code doesn’t run it anymore. A small Node runtime does, called agent-loop. It has no dependencies, and every agent in a round is a claude -p process. Later the same agents can be Codex, a plain API key, or a model on my own machine. That’s the reason for the whole move.

Most of it was straightforward. These parts weren’t.

The numbers

PRs merged through the gate on the busiest day	10
One round, at API prices	$4 to $33
Agents cut off mid-task by a background command	4 of the first 6
Good PRs the gate refused	2
Merges done by an agent	0

It all runs on a Claude Max subscription, so nothing here was actually billed. The dollar figures are what the same tokens would have cost at API prices.

Who merges

Inside Claude Code, the merge gate was an agent. It rebased the branch, ran the tests, and merged. The same kind of agent once ran TRUNCATE on my test database to get a suite green, so I wasn’t thrilled about that.

Now the agent can only say the PR is ready, and which commit it tested. The runtime does the merge, in plain code:

1. It checks the commit hasn’t changed since the test run.
2. It checks that GitHub’s checks are green.
3. It runs the whole test suite again, in a fresh checkout.
4. It merges exactly that commit.

If an agent tries to merge anyway, the guard blocks the command.

Background commands

Agents like to start the test suite in the background and do something else while it runs. One of them wrote this and stopped:

Full backend suite is still running; I’ll pick up when it finishes.

Inside Claude Code that’s fine: the agent gets woken up when the command finishes. With claude -p, nothing wakes it up. The session ends, and the CLI tells the agent to return its result:

You MUST call the StructuredOutput tool. Call this tool now.

So it returned what it had: “No PR is open yet”. That round used $6.74 worth of tokens and produced nothing.

I went through the transcripts after that. 4 of the first 6 agents that started something in the background got cut off the same way.

The fix is in the guard every agent runs under: it now blocks background commands, and the agents’ system prompt tells them to run things in the foreground and wait. Someone on X suggested a hook that quietly rewrites the command to run in the foreground instead. Nicer, but a hook can only change a command if it also approves it, and an approved command skips the permission check. I’d rather keep the check.

Two good PRs refused

In its first two days, the gate refused two PRs that were fine.

The old failure. The loop edits PR descriptions as it goes, and every edit reruns the check that verifies the description, on the same commit. GitHub then returns both runs: the old failed one and the new passing one. The gate counted both. GitHub itself only looks at the latest run of each check, so now the gate does the same.

The busy machine. A backend-only PR was refused because three frontend tests timed out. The reason was in another terminal: a different project was running its own test suite at the same time. The load was 57 on 10 cores, and the frontend suite took 211 seconds instead of 106.

Now the gate waits before running the tests, until the load drops to 1.5 per core, for at most 10 minutes. If it never drops, the tests run anyway and the refusal mentions the load. Failed tests don’t get retried. A retry would hide a flaky test too.

It has waited twice since: 90 seconds and 60 seconds. Both PRs merged.

Picking the bug

The loop chooses its own work: the most important open issue in the backlog first. Good for an unattended loop, less good when there’s one bug I want fixed today. Until now, the only way to push it was labeling the issue a blocker.

Now I can pin a round to issues:

agent-loop run codebase-audit-loop.js --root . --args '{"issues":[2181]}'

Plain code checks the issue before any agent starts: it has to be open, not an epic, not waiting on something, and without an open PR. I tested it on an issue I thought was closed, so the round would stop right away. It wasn’t closed. The round planned it, implemented it, and used $4 worth of tokens.

The bug was a currency one: imported entries were converted at the rate of the day they were imported, not the day they happened. It took seven rounds and five PRs. Twice I changed my mind about how the fix should work, and both times I just wrote the decision as a comment on the issue. One of them landed while a round was already working on it. That round’s review blocked the PR for not following the new comment, and the agents fixed it before the round ended.

When a PR still had a blocking review at the end of its round, the next round picked it up on its own, without the pin, and merged it.

The busiest day

Ten PRs went through the gate in one day. I started two of them by hand, after the gate had refused them for the reasons above.

At API prices, a round came to between $4 and $33. The $33 one had a reason. Its job was to make every place that writes an amount also record where the amount came from. Each of those places picked its own label, and three reviews found three wrong ones (one turned out to be fine). The next round moved the rule into a single function that all of them call. That touched about 110 call sites and passed review the first time.

What’s next

Every agent still runs on the same model, even the ones that only post a comment. Next is a model setting per role: something small for the bookkeeping, the big one for writing and reviewing code. On a subscription the limit is usage, so cheaper roles mean more rounds before I hit it. I’ll judge it by what merges and what gets reverted, not by the token count.

Then the other agents: Codex, an API key, a local model. After that, agent-loop goes open source.

If you’d like to run something like this on your own project, I’m packaging it up. Join the waitlist below and you’ll get one email when it’s ready.