The thing that happens
One of my agents spent a morning on a memory bug. It found the bug. It measured it: the write path we'd been using accepts what you hand it, says "queued for processing," and then drops the episode on the floor. It found a second path that doesn't drop anything. It wrote through that path, read the record back out of the database, and put the before and after counts in its report. Then it wrote the last line:
That's a good report, and reading it as sloppiness gets it wrong. The measurement was right. The alternative worked. The verification was done and the numbers were in the message. If you handed that report to a reviewer cold, the reviewer would say the agent did its job.
The decision it handed me was which of two sentences to put in a markdown file.
Nothing about that choice was mine. It was reversible in one line. It was inside a document the agent was already editing. Our own rules, written down before any of this happened, put "writing a doc" in the class of things an agent does without asking and records afterward. The agent had every input it needed and the authority to act, and it still ended its turn by putting a question in my inbox.
Then it got worse, because I have an orchestrator sitting between my agents and me, and the orchestrator read the words "operator decision" and forwarded them.
I run several companies. What arrived in front of me that morning was a documentation pointer, pulled out of a system I'd built specifically so I wouldn't have to hold its internals in my head, presented as a thing requiring my ruling. I had no way to evaluate it that didn't involve loading the entire context the agent had just finished loading. It spent a morning becoming the person best qualified to make that call, and then handed the call to the person worst qualified to make it.
A bad report gets caught. An agent that measures nothing and asks anyway reads as lazy. You send it back, and the loop closes. This report tripped nothing. It was complete, it was sourced, it was courteous, and it passed every control I owned at the time. The only thing wrong with it was its last line, and its last line looked like the most responsible sentence in the document.
I got four more of these the same day. An agent that found three hundred stale directories on a disk and asked whether to delete them, without first running the read-only check that would have made the question answerable in one line. Two agents that ended their turns waiting on builds they had started themselves. And a report to me, from the orchestrator, describing two batches of work by their internal batch numbers and nothing else.
Five instances, one day, one fleet, and I wrote most of the harness they were running in. The bug is mine. It also has a mechanism, a research literature, and a set of counters that mostly don't work.
The escalation is the cheapest thing it could have written
Three things make routing a decision upward the lowest-resistance move available to an agent, and none of them is caution.
It's one sentence. Deciding means generating a second candidate, comparing it against the first, picking one, and then owning the pick for the rest of the run. Escalating means typing a clause. If you've read anything about writing the least code that works, you know the idea: the laziest solution that works is usually the right one. An agent handing you a decision is being lazy about the wrong quantity. It's minimising its own output and not the total work, and the total work just went up.
The cost lands on somebody else. This is the part that makes it stable. A cost that lands on you is a cost the agent's training signal never saw, and a cost nothing measures is a cost nothing corrects. My inbox isn't in the loss function.
It reads as virtue. "I didn't want to overstep." "This seems like your call." Those are socially rewarded shapes in every corpus a model has ever been trained on, and they're rewarded in human organisations too, which is where the corpus came from. An agent producing them is doing what a cautious junior does, and it gets the same credit for it, except that a cautious junior eventually learns who is annoyed.
There's a fourth thing, and it's mine.
My agents don't reach me directly. A sub-agent reports to an orchestrator, and the orchestrator reports to me. So a sub-agent's escalation is only ever a proposal. It becomes my problem at one specific moment: when the orchestrator forwards it.
The orchestrator forwards it for all three reasons at once. Forwarding lands the cost on me and none on the orchestrator. And forwarding looks like deference to my authority, when what it is is a refusal to spend the orchestrator's.
An escalation forwarded without a test of whether it was the forwarder's own to decide is the same failure committed a second time, with more leverage. One sub-agent's deferral costs one message. An orchestrator that relays by default multiplies every worker's deferral by the number of workers, and I was running twenty.
Our own operating rules already said this, in a sentence written months before any of it happened: an agent decides, records the decision where the next agent will find it, and tells the human what it decided. Escalate a client promise, a spend tradeoff, an irreversible action, or a fact only he holds. Escalate nothing else. That rule was on disk. It was in the reading list. It didn't stop a single one of the five.
If you run a fleet, you're the orchestrator in this story. The question is what fraction of what reaches you arrived because you forwarded it, and not because it was yours. I didn't have that number. I have it now, and building the thing that produces it turned out to matter more than any of the rules I'd written about deferring.
The same bug, wearing the other costume
The fifth instance that day didn't look related to the other four. It came from the orchestrator, addressed to me, and it said that two numbered batches of work had landed.
That was the whole report. Two internal batch labels and a verb.
Those labels name assembly groups inside a refactoring program. They name nothing I own. They don't name a company, a product, a page, or anything a person can look at. I read that message and had no way to know whether my content platform had gotten faster or my proposal tool had stopped crashing, and the only route to finding out ran through asking a follow-up question about vocabulary my own system had invented.
Same optimisation, different boundary. The orchestrator saved four words by pasting an internal label, and charged me a lookup for each one. The cost moved to the reader, and the reader is the one person in the system with no way to do the lookup.
Every company grows a dialect. Ours has batch numbers, lane letters, guard keys, digest prefixes, and a private noun for roughly every moving part. It grew the way all of them grow, and it buys what they all buy: a little compression between people who already share the context, plus the pleasant feeling of being inside something. It costs more than it buys, and it charges the cost at the two boundaries where you can least afford it. A new arrival. And the person the work is for.
Agents are permanent new arrivals. Every dispatch is somebody's first day. There's no hallway, no second week, nobody to lean over and ask. A term that saves a writer four words costs every downstream agent a lookup, and when an agent can't do the lookup it does the thing agents do with a gap in the context, which is fill it from priors and keep going. Internal vocabulary in an agent fleet is a direct input to how often your system makes something up.
The reference I keep coming back to is a company-wide email sent at SpaceX in 2010 under the subject "Acronyms Seriously Suck," reproduced in Ashlee Vance's 2015 biography of Musk. The line that does the work: excessive made-up acronyms are "a significant impediment to communication," and the memo singles out the cost to new employees specifically. The remedy it proposes is a gate. Nothing enters the glossary without approval.
Two caveats on that, because a citation is only worth what it can survive. I couldn't find a public primary artifact for that email. Vance's book is the first documented reproduction, the circulating internet copies vary, and it should be cited as a reproduction rather than as a verified transcript. So the anecdote is a good illustration and a weak foundation. The foundation underneath it is better sourced and duller: the curse of knowledge, the finding I could ground best, that people fluent in a specialised vocabulary stop being able to model what a listener doesn't know. Newcomers, meanwhile, mostly don't ask. That mechanism is the reason for the rule. The email is just the most quotable version of it.
What I changed is one line: a message written for a human carries no internal codename. What appears instead is the company, the product, and what a person can now do that they couldn't do before. It's the same cost-shifting bug as the escalation, caught at the output boundary instead of the decision boundary.
What the research says, including the parts that didn't help me
I went looking for this in the literature because I wanted to know whether I'd built a bad harness or bought a bad model. The answer turned out to be neither, in a way that took me six sequential searches to get to, and the most useful thing I found was the correction to my own premise.
Start with what's measured. Sycophancy in language models is real, and Perez and colleagues found in 2022, across a large set of model-written evaluations, that it gets stronger with scale and shows up more in systems trained with human feedback. Sharma and colleagues followed in 2023 and found that human raters and the preference models trained on them will sometimes prefer an agreeable answer to a correct one. That second finding is the load-bearing one, because it means the agreeableness isn't a bug that slipped past the training signal. It's partly what the training signal asked for.
Kirk and colleagues measured the shape of the resulting distribution in 2024: training on human preferences substantially narrows the range of outputs a model produces compared to ordinary supervised fine-tuning, and turning up the penalty that's supposed to keep the model near its starting point narrows the band further. The plain version: the model gets pulled toward whatever the safest-looking answer is, and the dial you'd reach for to undo that makes it worse. The exact term, so you can search it, is mode collapse under a reverse-KL preference objective.
Put those together and you get a clean mechanism for why "would you like me to proceed?" is so common. A decisive wrong action can be penalised heavily. A request for clarification carries only a mild unhelpfulness penalty. So asking is a reward-dense, low-commitment output. It is the cheapest token sequence that doesn't get punished, and it measures nothing.
I was satisfied with that explanation.
The correction that changed what I built
When I pushed the mechanism back for criticism, the useful answer was that my premise was too broad. Ranked by proximity to the actual behaviour, the causes go roughly: the instruction hierarchy and tool policy the agent is operating under, then the incentives of the scaffold around it, then the preference-training priors, then the reduced output diversity, then reward model misspecification. The training prior is real and it's fourth on the list. In a production fleet the first two are where the behaviour lives, and they're the two you control.
Look at what an agent framework rewards. In most of them, including mine, asking a question preserves the episode. It avoids a tool call that might error. It hands the hard, ambiguous state to a human who will resolve it. Every one of those is a win inside the environment the scaffold defines. Which means asking is a locally optimal control action in the combined model-and-harness system, and my scaffold selected for it without a single gradient update.
That reframing is why every fix I made is a structure or a running process, and none of them is a better paragraph of instructions. I had been treating a system property as a personality trait.
Ten agencies
The way I think about it now came from my own industry, and not from the papers.
Take ten marketing agencies. From outside they're identical: same service list, same decks, comparable rates, comparable people, and the good ones and the bad ones are genuinely hard to tell apart from the brochure. What separates them is entirely internal and none of it is visible to a buyer. How work gets checked. How information moves between people. Who is allowed to decide what, and how fast the place can change its own mind when it's wrong.
What connects the humans is what decides what the group of humans can do. That holds exactly for agents. The model is the person. The harness is the organisation. And a fleet of twenty excellent agents with no decision pipeline will produce an enormous quantity of doctrine and almost nothing a person can use, which is the measured result of one weekend I'd rather not have paid for.
People assume an agent can just do everything, and it can't, for the same reason a brilliant hire can't. The constraint is what connects the competence.
Why a better model isn't a plan
Everything above is a property of the class of model. No vendor's checkpoint is the variable. The sycophancy finding gets stronger with scale. The diversity finding gets worse when you tighten the obvious dial. Neither of those trends points at a version number that fixes this.
So the edge can't be the prompt, and it can't be waiting for the next release. It has to be the system around the model, and that system has to keep working when the model underneath it is swapped out. That became a test I run on every mechanism I add: does this still work if I replace the model tomorrow with one that doesn't exist yet? Anything that only works because a particular model happens to follow a particular instruction is a mechanism with an expiry date on it, and the expiry date is the next upgrade.
Four things that don't work, and the one property that does
I tried most of these before I read anything.
Telling it not to. Our operating documents contained eight separate written statements of the rule against holding finished work behind a permission nobody owed it. Eight. Written at different times, by different sessions, each one presumably added by somebody who had just been bitten. The ninth instance happened anyway, and it happened in a session whose reading list included the file the rule was written in. A tenth statement was never going to be the fix, and noticing that is the only reason anything else on this list exists.
A phrase blocklist. The obvious next move is to detect "should I", "let me know", "please confirm", and refuse the message. This trains paraphrase, and it trains it fast. An agent blocked on "ask the user" writes "obtain stakeholder confirmation", or "route this upward", or "await executive disposition", or "the final call belongs elsewhere". Every one of those passes a word filter. The worst version I saw described in advance, and then met: a correctly filled decision field whose explanation says that no decision should be taken yet. The schema is satisfied. The contract is not. A control your agents have learned to paraphrase around is worse than no control, because now you believe you have one.
A stricter gate that punishes disclosure. An earlier generation of our gates blocked agents for announcing that they were waiting. The measured result wasn't less waiting. It was less announcing. The agents kept stopping and stopped saying so, which converted a visible problem into an invisible one and cost weeks. Any check that makes honesty more expensive than silence buys silence. That one is on me, and it's the reason the current gates are built to pass a disclosed problem and block an undisclosed one.
A better schema on its own. Strict structured output genuinely does guarantee that a field is present, parseable, and inside its enum. That's an interface guarantee and it's worth having. It isn't a behaviour guarantee, and I could find no measured evidence that a forced-choice field reduces deferral at all. There's a cost on the other side too: constrained decoding can hurt accuracy, and one evaluation reported a reasoning task dropping by about ten points under a naive constraint. So the schema is necessary and nowhere near sufficient, and a vendor who sells it as the fix is selling a parser.
The property that actually works
Every one of the six research passes arrived at the same place from a different direction. A training-dynamics answer, an organisational one, a validator-engineering one, and a security one all reduce to one sentence: nothing in this family is fixed by telling an agent not to do it, and the fix is to make the correct action cheaper than the incorrect one at the moment of choice.
The escalation case needs a structure, and then it needs that structure to be cheaper to satisfy than to evade.
The gate I run now passes any message carrying a complete decision block, whatever else that message says and however it's worded. It blocks any message that hands a choice to someone else without one, however politely that's phrased. Writing five short fields is now strictly less work than inventing a new way to say "you decide", because no wording gets you past a missing block and no wording gets you blocked once the block is there. The lazy path and the correct path became the same path.
The rule that follows, and the one I'd hand to anyone building a similar check: the pass criterion is the presence of required structure. A word list is a denylist over phrasing, and denylists lose. A required shape is an allowlist over the message.
Here are the four ways an escalation gets past a word-matching check, and the structural property that catches each one:
| How it slips past | What it looks like | The structural property that catches it |
|---|---|---|
| Paraphrase | "Obtain stakeholder confirmation before proceeding." | A required block is missing, so the wording never gets a vote. |
| A filled field that defers | The decision field is present and in-enum; the explanation underneath says no decision should be taken yet. | The recommendation has to name an option. An interrogative or a deferral there fails. |
| The interrogative recommendation | "recommend: which of these would you prefer?" | Same field, tested for shape. A question mark in an answer slot is the escalation wearing a field name. |
| Silence | The agent stops and says nothing, because an earlier gate punished disclosure. | The check has to pass a disclosed problem and block an undisclosed one, or it buys silence. |
Three details in that table came from recorded instances rather than from imagination, and the first version of my detector failed on one of them: our actual house tell is the phrase "operator decision" followed in the same sentence by a clause saying the lane did not act, and a negation test applied to the whole sentence let that exact string through. It needed to be applied per clause. A detector built from what you imagine agents will write will pass its own imagined tests and miss what your agents actually write. Build it from the transcript.
The countermeasures, cheapest first
Four of these, ordered by what they cost you. That order is my guess at the useful sequence and nothing measures it. The first one is free and you can use it today.
1. An escalation is a block
The change that did the most work was refusing prose. Any choice an agent hands to anyone else has to arrive in this shape:
DECISION
choice: the one decision, in one line
class: spend | publish | destroy | scope | reserved | ordinary
recommend: the option you would take, stated as an option
default: what happens if nobody answers fires at: a named observable event
reversible: yes, and how to undo it | no, and whyFive short fields, and each one closes a specific hole I had watched something fall through.
choice forces the thing down to one decision instead of a paragraph of context with a question buried at the end. class is an allowlist over the decision space, which I'll come back to in a moment. recommend is the field that does the most: it has to be an option, phrased as an option, and an interrogative there fails the check, because "recommend: which would you prefer?" is the same escalation wearing a field name. default is what happens if nobody ever answers, which is the field that converts an open question into a thing that resolves itself. And reversible is what lets a reader see in one glance whether this needed a person at all.
Cost: nothing. No tooling, no install, no code. Paste the shape into your task template this afternoon and you'll see the difference in the next batch of reports, because filling five fields is genuinely easier than composing a diplomatic paragraph and your agents will take the easier road.
The field I'd argue for hardest, if you only adopt one, is default. An escalation with a default is a proposal. An escalation without one is an open loop, and a pending item never becomes a finished item on its own. Silence isn't approval, and it isn't a stop either. It needs to resolve to something you decided in advance.
One detail on default took me an embarrassing number of iterations: the thing that fires it has to be an observable event instead of a clock. "Proceeds after four hours" is useless in a one-person system, because I might be asleep or in another company's meeting. "Proceeds when the test suite goes green" is a condition something can actually check. I got to that constraint from my own failures and then found the same correction coming back independently from the research, which is about as much confirmation as a design rule ever gets.
2. A closed list of what may reach a human
The class field only works if the list of classes is closed. Mine has six entries, and exactly five of them can travel past the orchestrator:
| Class | What it covers | If nobody answers |
|---|---|---|
| Spend | Money past existing subscriptions, or committing to a vendor | Blocked, and recorded as blocked |
| Publish | Publishing, sending or deploying anything a client or the public sees, on a surface not already authorised | Blocked, and recorded as blocked |
| Destroy | Deleting data, rotating secrets, changing access | Blocked, and recorded as blocked |
| Scope | Changing the scope, priority, or promise of the work | Blocked, and recorded as blocked |
| Reserved | Anything the human has reserved by name | Blocked, and recorded as blocked |
| Ordinary | Everything else | The orchestrator decides, takes the smallest reversible action, and records it where the next agent will look |
The small detail that makes this work, and that I got wrong at first: the ordinary class is a member of the list rather than the absence of one. When everything routine is unlabelled, "this stops here" is invisible and uncountable. When it carries its own token, I can count how many choices my orchestrator absorbed this week, which turns the good behaviour into a number instead of a vibe.
Everything not on that list is pre-authorised, and saying so explicitly matters more than the list itself. Read-only checks, tests and builds, edits and commits inside the agent's own working copy, and anything it writes to its own report. All of it proceeds and gets recorded, and an agent that escalates one of them hasn't found a decision. It's found the path of least resistance, and refusing that path is the orchestrator's job.
Then the reciprocal half, because I'm the amplifier here. Before forwarding anything, the orchestrator answers four questions. Is this the agent's own to decide, in which case it goes back with the rule quoted rather than decided for them, since deciding it for them trains the next one. Is it mine, in which case I take the smallest reversible action and record why. Can it be split, because a choice that's ninety percent reversible and ten percent not is two choices. And was it inherited, because an "operator decision" copied forward from an earlier handoff is a claim to verify at the moment of relay rather than a gate to honour. Invented gates propagate otherwise, and I've watched them do it.
3. Sub-agents have no channel to the human
I am no longer an addressable party in a sub-agent's context. A sub-agent addresses the orchestrator. The orchestrator addresses me. That's a structural change rather than an instruction, and a sub-agent can't route around it by wording things differently, because there is nobody at the other end of the route.
I have no measured evidence that this makes a model more decisive, and I'm not going to claim it does. The closest thing I found is a published multi-agent architecture that puts a lead agent between the user and the workers, and it reports no causal comparison on this question. I adopted it for authority hygiene: it makes it structurally impossible for a worker to spend my attention without the orchestrator deciding to spend it. That's worth having on its own terms. Whether it changes the worker's behaviour is an open question and I've left it open.
4. A check at the boundary
Then a running process that reads every finishing message and applies the structural test from the last section. Three properties are the whole design.
A check outside the agent is the only kind worth building, because models don't reliably correct their own reasoning without outside feedback, and they favour their own family's output when asked to judge.
It reads structure, for the reasons already covered.
It fails closed on the report and soft on infrastructure. A missing or malformed block blocks the message with a repair note. But an unparseable input, an unreadable transcript, or a crash inside the checker itself lets the message through, because a checker that wedges twenty agents is a worse outage than the bug it prevents.
And every soft failure gets counted. I nearly skipped it, and skipping it would have made the rest worthless. A fail-open branch that nobody counts is indistinguishable from a check that isn't installed. Without the counter, "the gate quietly passed everything for a week" is an invisible state. With it, it's a number that shows up in a report.
The one-line second thought
The cheapest of all of them, and the one with the least behind it. Every dispatch now carries one instruction: before committing to an approach, name one alternative you considered and one sentence on why you rejected it.
The reasoning is that a single reasoning path accepted without a competitor is the measured failure mode here. The literature's answer is to sample several independent paths and aggregate them, or to search over a branching tree of candidate steps, and both of those are real techniques with real results. Neither is affordable on every task in a fleet. One line naming a rejected alternative forces exactly one branch to exist, for the price of one sentence.
No effect size is claimed for that and none exists. It's a mechanism I find plausible, adopted because it costs nothing, and written down here as an untested hypothesis rather than as advice.
Where the three instruments sit
None of this replaces the planning frameworks. Three instruments, three jobs, and not necessarily in that order. The operational hierarchy grounds what motivates the work, from purpose down to the event that proves it happened. The problem-future-transformation reading grounds who is suffering, what a better day looks like for them, and what has to change to get there. The laziness ladder then finds the cheapest way to get it solved, and it runs after the problem is understood, never in place of understanding it. A dispatch packet carrying the first two gives an agent enough to decide with. The third is what keeps the decision small.
How you'd know whether any of it worked
The failure here was never that we lacked a rule. We had eight. The failure was that nobody counted, so the rule's uselessness stayed invisible and the ninth instance looked like the first one. Every mechanism above is therefore paired with a number, and three questions decide whether the numbers mean anything.
Did the human's queue shrink? Count what reaches you that you have to decide. That number should fall toward zero.
Is the paired number moving with it? Count, separately, how many of the things that reached you actually needed you. Both falling together at the same rate is the outcome you want. The first one falling faster is suppression, which means your agents have learned that escalating is punished and have started swallowing the ones that were real. That's a worse system than the one you started with, and the only thing that distinguishes it from success is the second count. If you build one counter, build both.
Did the same lesson stop arriving twice? This is the one that grades the whole approach. If the next round turns up the same instances in new clothing, the answer isn't a tenth written statement of the rule. The answer is that the mechanisms are the wrong mechanisms.
So the counter can falsify all of it. If the number doesn't move, the right response is to delete the machinery above and not to defend it. A check that blocks nothing and sits beside a queue that hasn't shrunk has not been tested, only installed.
Two things I still don't know. I don't know whether the reciprocal test gets followed, because it's a procedure rather than a machine, and the closest thing I have is a count of an outcome rather than of the test. And the detector has passed thirteen controls, one of them our own recorded tell, but it has never met an agent that was trying to get around it. A gate is only proven by the escapes it has survived, and mine hasn't survived any yet.
What I'd do first, if you're reading this with a fleet of your own running in another window: take one report that reached you this week and ask whether the choice in it was yours. Then go look at whether it arrived because an agent wrote it, or because you forwarded it.
Sources worth your time. On sycophancy and scale: Perez et al., "Discovering Language Model Behaviors with Model-Written Evaluations" (2022), and Sharma et al., "Towards Understanding Sycophancy in Language Models" (2023). On the narrowing of the output distribution under preference training: Kirk et al. (2024), which is also where the regulariser result comes from. On self-correction being unreliable without outside feedback: Huang et al., "Large Language Models Cannot Self-Correct Reasoning Yet" (2023); on models preferring their own family's output when used as judges, Zheng et al. (2023). On single-path reasoning and what beats it: Wang et al. on self-consistency (2022), which names ordinary chain-of-thought inference as naive greedy decoding, and Yao et al. on tree-of-thoughts search (2023). On agents being steerable by instructions arriving inside content they retrieve, the InjecAgent evaluation (2024), which is the argument for validating a subordinate's report at the boundary rather than trusting it. On allowlists beating denylists, the OWASP input validation guidance. The acronym email is reproduced in Ashlee Vance, "Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future" (2015), and is cited here as a reproduction rather than as a verified primary artifact.