What my agent loop wrote about itself

The last post ended with my autonomous coding loop running on Blackstart, my Unity game, five rounds at a time. It has been running since. It also writes something I had mostly been ignoring: notes about itself.

At the end of every round, a retrospective agent looks back at what got in the way. Not bugs in the game, but problems with the loop’s own process. A checker that skipped a file, a step that asked me something it could have worked out, a rule that fired on the wrong thing. Each one becomes a note, and the notes piled up. This week I built a way to decide them, and went through every one.

The numbers

Rounds on Blackstart since the last post	19
PRs merged by those rounds	20
Notes the loop has written about itself	56
Notes I decided	55
…that needed agent-loop itself changed	35
Size of the implement agent’s prompt	140 KB
Private issue numbers in the loop’s prompts	193

It all runs on a Claude Max subscription. At API prices those 19 rounds would have been $175, about $10 a round.

Three ways to answer a note

The notes now show up in a cockpit, a small local page that also shows what the loop is doing and what’s waiting for me. Each note gets one of three answers:

• Accept it as a rule for this project. The rule goes into tuning.json in the project, and every round from then on gets it in the prompts of the steps it names.
• Dismiss it, with a reason, so the loop stops raising it.
• Core: the fix belongs in agent-loop itself, not in this project.

The cockpit collects my answers and opens one PR with them, which I merge like any other change.

Of the 55, 35 were core. Most of what the loop complained about wasn’t the game. It was the loop.

A review that wasn’t there

One round merged a PR with no review comment on it. The review had run. The agent asked to post the verdict had tried, a hook refused the post because the body quoted a file path, and the agent reported back “NOT posted”. The script counted any reply as published, so the merge went ahead.

The fix was to stop asking an agent to do it. The runtime posts the review now, so a failed post is an error the script sees, not a sentence in a reply.

Each fix filed the next one

The loop has a habit I like in principle. When two parts of the code decide the same thing separately, it moves the decision into one owner and files an issue to route everything else through it. “Route every reader of X through Y.”

In practice, each of those PRs left a small nit: the old way in is still public. The review filed the nit as a bug, the next round picked the bug, and that PR left a nit of its own. Six refactors in two days, each one taking a whole round, while eleven feature issues I had filed sat untouched. The loop noticed before I did, in one of its notes:

TREND (each owner fix files the next owner fix)

So I accepted a rule: pick one of my feature issues over a loop-filed refactor, never two refactors in a row.

A rule in the wrong place

The next two rounds picked two refactors in a row.

The rule applied to the Plan step, because that’s where I thought issues were chosen. They aren’t. A step called the backlog scan picks the issue, and Plan only plans the one it was handed. The rule was in every Plan prompt and never reached the agent that made the choice. On top of that, the loop’s ranking puts bugs ahead of features, and the refactors were filed as bugs.

The loop reported this one too:

RULE NOT ENFORCED (planner selection)

I could have added the rule to the backlog scan’s prompt too. Instead the ranking moved into code. Before the scan starts, the loop works out which open issues are its own refactors, and whether the last merged PR already closed one. The scan gets one line saying so. Whether a particular refactor fixes something wrong today is still its call.

One more thing had to be fixed first. My ruling on an open question started with “Decision:”, and the loop only recognised “Decision (maintainer):”, so it would have asked me the same question again. With both fixes in, the next round picked that issue and fixed it.

Prompts only grow

Every lesson the loop learned had gone in the same way: as more text in a prompt. The script it runs is one file, and that file sat at 509,748 bytes against a cap of 510,000. The cap was there to keep under a limit of the tool the loop used to run in. The loop doesn’t run there anymore, so the cap protected nothing.

The real cost was elsewhere. Every sentence in a prompt is read by that step on every round, on every project, and weighed against every other rule in it. I started recording how big each agent’s prompt actually is. The implement agent reads 140 KB before it starts, about 35,000 tokens.

So I changed how lessons go in. If the loop can decide something mechanically, it goes in code, and the prompt gets the result. Prose is for judgment.

The best example was a set of states for a question the loop parks for me: still waiting, posted, answered before it was posted, and so on. The prose describing them was written out three times, in three different steps, about 5 KB in total, and each agent worked it out again from the same GitHub reads. Now one function does it, and each step gets one line.

The cap became a ratchet. The script may never grow past its budget unless the change says what the growth buys and why it couldn’t be code, and when it shrinks the budget goes down with it.

It has a flaw I found on the first try. Moving a rule into code made the file 1.8 KB bigger, because code is bytes too, while what agents read barely changed. The next version should hold each step’s prompt to its measured size instead of holding the file. The data for that is now being recorded.

Fixed means it stopped saying it

The 35 core notes are agent-loop’s to fix, so each one now links to the PR that fixed it, and I close it once that fix is in the version I run.

Closing turned out to matter for a less obvious reason. The loop is told which notes were already decided, so it doesn’t raise the same thing twice. A core note was listed there as “already recorded” forever. If a fix didn’t hold, the loop would never say so again. A closed note now drops off that list, so a problem that comes back gets reported again.

Tallyroot everywhere, again

I want to open-source agent-loop, so I had every file checked for anything that belongs to one project. The last post found a hundred places that said main. This found more:

• 158 mentions of my projects by name.
• About 640 private issue numbers, 193 of them inside prompts, as in “(issue 2338)”. Nobody outside my repos can look those up, and agents gain nothing from them either.
• Tallyroot’s real configuration kept as a test fixture, with its product description and local database credentials.

Some of it wasn’t just wording. Three things were wrong on the game right now:

• A finance audit. When its backlog is empty, the loop audits the code along a few dimensions. One of them was hard-coded: bond coupons, loan amortisation, property valuation. Every project would get it, including a game about dead planets.
• Questions that never reached me. When the loop needs a decision, it parks the question and a GitHub Action posts it on the issue. Tallyroot has that Action. Blackstart doesn’t, so its questions just sat in a list nobody reads, while the log said “the 6-hour cron still relays it”. There is no cron.
• A ledger that always failed. Every round ended by calling a command only Tallyroot defines, and on Blackstart it recorded “ledger failed” every time.

The rule now is that agent-loop holds nothing project-specific: no names, no issue numbers, no test that checks one project’s facts. Whatever is true of one project goes in that project’s own files. The loop relays its own questions when a project has nothing to do it, the finance audit lives in Tallyroot’s profile, and the tests run against a made-up example project instead of a real one.

What’s next

The cleanup is about halfway. The issue numbers come out of the prompts next, which should make every agent’s prompt a little smaller too. After that, the project rules I accepted need a way to retire once agent-loop does the same thing itself, so they stop being sent along forever.

On Blackstart, the last two rounds picked one of my features and the issue I had ruled on. No refactors.

If you’d like to run something like this on your own project, I’m packaging it up. Join the waitlist below and you’ll get one email when it’s ready.