Coding agents

Who's on call for code nobody wrote?

"The agent wrote it" doesn't settle who owns a change. Whoever approves and merges it owns it in production, and approvals, on-call rotations and incident reviews should reflect that.

Matt Kindy · · 6 min read

Say your team runs an online store. Its checkout service takes payment for every order and has run without trouble for months, until one day it starts throwing intermittent 500s and some customers can’t pay. The engineer on call opens the deploy log and finds three changes that went out that afternoon. An agent wrote all three, and each one was approved by someone who skimmed it. None of the people on the rotation wrote the failing code or read it.

Any team that merges agent-written code will have to decide who owns it. Most teams will first face that question during an incident like this one, when there’s the least time to think it through, so it’s better settled beforehand.

“The agent wrote it” doesn’t answer anything

In an incident review, “the agent wrote it” says where the code came from and leaves open the two questions the review needs answered: why the change shipped, and who decided it was safe.

The agent isn’t on the rotation and won’t be at the review, so nobody can page it or ask it in the retro why it swallowed an exception. The session that wrote the change ended hours earlier. A new session will offer a plausible explanation, but from a model that never saw the original reasoning.

So responsibility for the change goes wherever the team decides it goes. If the team hasn’t decided, nobody is responsible for it until something breaks, and then it falls to whoever is on call.

I’ve argued parts of this before. In an earlier post I said the developer who accepts an agent’s code is at best a reviewer of it, and that holding them fully responsible for code they don’t have the context to own is a setup for failure. Later, in When the model is down, I said owning a change means being able to explain it. Taken together, those lead to my position here: give the people who approve agent code the context and the time to understand it, let them refuse what they can’t explain, and then treat what they approve as theirs. That means changing what an approval means, who goes on call, and what the incident review asks.

The approver is the closest thing to an author

When an agent writes a change, the person who approves it is the last human to decide that it should exist. That makes the approval the most consequential step in shipping the change, but on most teams it’s treated as a formality.

According to GitHub, Approve “signals that the changes are ready to merge.” Google asks much more of its own reviewers: “a reviewer has ownership and responsibility over the code they are reviewing.”

The difference between those two definitions mattered less when a human author also owned the change. When an agent wrote it, the approver may be the only owner the change has. So I think an approval on an agent’s pull request should mean “I can explain what this does in production, and I’m on the hook for it,” and a reviewer who can’t say that shouldn’t approve. (I went through what an approval should take in LGTM is not a review.)

Whoever merges it should be on call for it

Operations has an old rule that the people who build a service should also run it. It made sense before agents, and with agents it’s the only arrangement that keeps an owner attached to the code.

But many teams split those jobs. Product engineers merge changes, and a platform or SRE team holds the pager for the services those changes land in. That split caused problems before agents, and agents make it worse, because the people merging the code may not have read it and the people paged for it almost certainly haven’t. As agents write more of the code, the person who changed the system and the person who has to understand it during an outage drift further apart.

I’d close that distance by putting the people who approve and merge changes into a service on call for it, or at least on its escalation path by name. They’ll get more pages than they’re used to, and I expect that to make them more careful about what they approve.

Six steps from merge to incident review: an agent writes the change; someone approves it, the last human to decide it should exist; it merges and deploys; the pager goes off because checkout starts throwing 500s; the person on call opens the deploy log and finds three changes, all written by an agent; the incident review asks whether the approver could have explained it and whether on call could read it without an agent. A dashed arrow from the approval step to the pager step is labeled same person. An agent writes the change Someone approves it the last human to decide it should exist It merges and deploys The pager goes off checkout starts throwing 500s On call opens the deploy log three changes, all written by an agent Incident review Could the approver have explained it? Could on call read it without an agent? same person
From merge to incident review. Under this arrangement, the person who approves the change is the person who gets paged for it.

Look at the approval in the incident review

When an agent’s change does break, blameless postmortems are the right model for the review. Google’s SRE book describes one as focusing “on identifying the contributing causes of the incident without indicting any individual or team.” It also makes the point that you can’t fix people, but you can fix the systems and processes around them so they make better choices.

Agents add a contributing cause that most postmortem templates don’t ask about, which is that nobody understood the change before it shipped. That’s a failure of process, and the process in question is the approval, so the template needs two more questions:

  • Could the person who approved the change have explained the failing path before it merged?
  • Could the person who got paged read the code they were responsible for, without asking an agent what it does?

If the answer to either is no, telling people to be more careful won’t help. The fix belongs in the process: what an approval requires, who is on call, and whether a change nobody can explain is allowed to merge into a service that pages people.

Being able to explain it is the job

Both questions ask whether someone could explain the code. Owning code in production means being able to say what it does when it runs: what it reads and writes, what it returns when the database is slow, and which errors it handles and which it passes on. That takes having read the code, though not necessarily having written it, and being able to find your way around it quickly during an incident.

Tools make that easier, though they don’t replace an owner. Tests pin down the behavior that matters, and traces show the path a request took. A description generated from the code helps too. suss, for one, writes down what each handler, consumer and query does, without using AI.

Where to start

If you run a team that merges agent-written code, start with these changes:

  • Write down what an approval means, and make it “I can explain this in production.” Put it in the pull request template so every reviewer sees it.
  • Put the people who approve changes to a service on that service’s rotation or escalation path.
  • Add the two questions above to your incident review template, and act on the answers.
  • Don’t let agents merge into a service unless someone on call for it has read the change.

Agents will write more of the code every quarter, and someone still has to own each change they write. I’d rather a team decide who that is in advance than find out during an incident that nobody does.

Get the next essay

Follow the RSS feed to see each new essay when it goes up.