Know the Blast Radius Before You Say Yes

Give an agent room to act and eventually it will do something you wish it had asked about first. The natural fix is to make the agent more careful: better instructions, a stricter prompt, a more capable model. But the failures that actually hurt are rarely failures of care. The agent did exactly what it was told, competently, and the thing it did reached further, or bit harder, than anyone realized when they let it proceed.

That is the shape of the accidents that matter. Not a confused agent fumbling a hard task, but a confident agent doing a small-looking task whose consequences were much larger than its appearance. Which means the question that should decide whether an action needs a human is not how the agent feels about it. It is how far the action reaches and whether it can be undone.

Reach, and the way failures travel

The engineer Charles Perrow spent a career studying disasters in nuclear plants, chemical refineries, and aircraft. In Normal Accidents he argued that in tightly coupled systems, catastrophe does not require a large initial fault. A small failure in one component travels through the system's connections and interacts with other parts in ways nobody anticipated, and the damage is a property of the coupling, not the size of the first mistake.

An agent-native company is a tightly coupled system. A price is not just a number on a page. It is wired to the checkout, the billing webhook, the invoices that go out next week, and the customers who will notice. Editing an internal draft reaches nothing. Changing that price reaches everything downstream of it. The two edits can take the same agent the same thirty seconds and look equally routine, and they are not remotely the same action, because one is coupled to the rest of the company and the other is not.

The first half of an action's blast radius is its reach: how many systems, people, and downstream steps move if it does.

Reversibility, and the line you cannot walk back

The second half is whether the damage can be undone. Nassim Taleb draws a hard line between a loss and ruin. A loss is a setback you recover from and carry lessons out of. Ruin crosses what he calls an absorbing barrier, a state you cannot come back from, and once it is crossed the whole track record that preceded it stops mattering. His warning is not to judge a risk by its average outcome, because the average is meaningless if one tail outcome ends the game. Never wade a river that is on average four feet deep.

Actions sort the same way. Sending five thousand emails is not reversible, and neither is a public post or a payment. A staging change or a draft is. Deleting the backups is ruin. An action's reversibility decides not how bad a mistake is on average, but whether a single bad instance is survivable.

Blast radius is these two together: how far an action reaches, and whether you can take it back.

Judge the action, not the actor

Once you frame it this way, the usual gates look misplaced. Gating on the agent's confidence is backwards, because the dangerous actions are the ones an agent is most sure about. Gating on whether a task is novel is backwards too, because the highest-radius actions are usually the most routine-looking. What deserves a human is fixed by the action, so that is what to grade.

The bottom-right corner is the one that ends companies, and it is also the one most often waved through, because the action that lands there rarely announces itself. A few small-looking edits sit in it far more often than people expect: anything touching billing or payments, anything customer-facing at volume, deleting or overwriting data, changing who can access what, publishing in public, spending money, and anything with legal or compliance weight. A useful habit is to treat every one of those as high-radius by default and make the agent prove otherwise.

Reliability does not buy a pass into the corner

This is where blast radius corrects a tempting mistake about autonomy. It is right that an agent should earn more freedom as it proves reliable. We have made that case. But reliability earns freedom only inside the recoverable region. It should never extend into the irreversible one.

Taleb's point is exactly why. In the ruin corner the average does not apply. An agent that has handled a thousand mass sends flawlessly has told you nothing about whether the thousand-and-first should go without a human, because the cost of the one that is wrong is not one thousandth of anything. It is the whole account. Earned autonomy expands the reversible zone an agent may work in alone. The high-reach, irreversible zone stays gated on principle, not on performance.

Decide on the plan, not the wreckage

All of this only helps if the blast radius is known before the action, not discovered after it. So the practice is to make the agent declare it up front: write down what a run will touch and how far it reaches, and let that assessment decide whether the work proceeds or waits for a person. The decision to gate is then made against a stated plan, while the action is still cheap to hold.

The honest limit is that a blast radius is an estimate, and estimates miss. Perrow's whole argument is that couplings surprise you, so an action will sometimes reach further than anyone mapped. The response is not to pretend the estimate is exact. It is to bias toward caution wherever reach or reversibility is uncertain, and to make reach visible enough that the surprises grow rare. The opposite failure is real too: gate every change with any reach at all and you have thrown away the leverage. The line worth drawing is at genuine reach and irreversibility, not at any change whatsoever.

Never surprised by what a run reached

This is why Task Machine has an agent assess a run's blast radius before it starts, scoring how far the change reaches, and routes high-reach or irreversible work to a person while routine, recoverable work simply proceeds. The autonomy an agent earns on recoverable work does not quietly extend into the actions it can never take back. You approve what reaches far, you let the small things run, and you are not surprised after the fact by what a run touched.

Reach and reversibility decide what needs you. Continue with Earned Autonomy, which decides how much of the rest an agent can be trusted to handle alone.