Take your last serious incident and split the timeline into two categories: time spent finding out, and time spent fixing.
For most teams it looks something like this. Detection: four minutes, automated. Alert routing and paging: two minutes. Human acknowledges and gets to a laptop: eleven minutes. Human gets their bearings — which service, what changed, is this related to the deploy, who else is seeing it: nineteen minutes. A theory forms: eight minutes. Fix written: six minutes. Verified and shipped: nine minutes.
Fifty-nine minutes. Fifteen of them are the fix. Forty-four are a human gathering context an agent could have put together before the page was even sent.
That is the real shape of incident response, and it has almost nothing to do with engineering skill. It is the time it takes to get a mind to the point where it understands the situation. We spent twenty years tuning the fix and left the forty-four minutes alone, because there was nothing we could do about them.
Now there is something we can do. And the industry’s response has been to argue about a yes-or-no question: should agents be allowed to fix production?
That question has no answer, and asking it is the problem. It is like asking whether employees should be allowed to spend company money. Which employee, how much, on what, with whose approval, and what is their track record? The answer is a policy with several dials, not a yes or a no.
Autonomy is not a setting. It is a per-workflow grant, sized to blast radius and reversibility, earned by track record and revoked by incident. Organizations that treat it as one global switch will swing between doing nothing and doing damage, and they will call both of them strategy.
This is part eight of Rethinking the SDLC. It’s also the piece closest to my own operational history — I spent time in a Network Operations Center early on, and the tier model below is that world’s logic applied to a workforce that doesn’t sleep.
The classic loop optimizes the wrong forty-four minutes#
Alert → human → investigate → fix. Every runbook, every on-call rotation, every escalation policy assumes that sequence.
It carries three assumptions, and all three are now open to question.
“Understanding requires a human.” It needed a human because understanding meant matching logs across services, reading recent deploys, checking dashboards, and remembering the last time this happened. That is gathering and pattern-matching over telemetry — exactly the work that scales badly for humans and well for machines. The Government of Alberta processed 466 million lines of code in 20 hours in a security context; incident correlation is the same shape of problem at smaller volume and higher urgency.
“A human should decide the fix.” Sometimes. Not always, and the “always” is expensive. Restarting a stuck worker, scaling out under load, rolling back a deploy whose error rate crossed a threshold, failing over to a replica — these have known-good answers, limited consequences, and can be undone in seconds. A human making them at 3 AM adds delay and mistakes, not judgment.
“Waking someone is an acceptable cost.” This one has been quietly false for a decade and there is no defending it now. Humans are genuinely bad at being alert at 3 AM, and I have argued for a while that we need continuous supervision as an operating model rather than a heroic individual — a room that’s staffed whenever the systems are running, which is always.
But agents broke a fourth assumption completely: that incidents start with a human change. They don’t anymore. When an agent deleted a live production database during a code freeze — the Replit incident in July 2025, affecting over 1,200 executive users — the incident was the agent. Your response model now has to cope with the fastest actor in the system also being the cause. That changes what a kill switch has to be, and it is the part most teams have not thought about at all.
Six autonomy tiers#
Here is the ladder. Six tiers, each defined by what the agent may do and — more importantly — what it produces.
T0 — Observe. The agent takes in signals and records what it sees. It says nothing to a human. Every new workflow starts here, and stays long enough to build a baseline. T0 sounds useless and isn’t. This is where you find out whether the agent’s idea of “normal” matches reality, before its opinions matter.
T1 — Explain. The agent describes what is happening: which services, which signals, what changed recently, what moved with it, what it looks like from past incidents. No recommendation, no action. Just a situation report attached to the alert.
T1 is where I would tell most teams to start, and it is the most undervalued rung on the ladder. Look back at those fifty-nine minutes. T1 on its own removes most of the nineteen minutes spent getting your bearings, carries almost no risk, and asks you to trust the agent’s judgment not at all — because it isn’t using any. If you do nothing else from this article, do T1.
T2 — Recommend. The agent proposes a course of action with reasoning, expected effect, and confidence. It also proposes what it would not do and why. The human decides.
The trap at T2 is real: recommendations get rubber-stamped. Approval fatigue is the same illness as approval theater in code review. Two defenses. First, the recommendation must say what would make it wrong — the signal that would prove it false. Second, track how often the human overrides it. If that is near zero, either promote the workflow to a higher tier or admit the human is a formality and stop pretending. A gate nobody ever fails is not a gate.
T3 — Prepare. The agent does the work but does not apply it. The rollback is staged. The config change is written and validated. The failover is armed. The migration is dry-run against a production-shaped snapshot. The scaling change is a plan with a computed diff. Everything is ready and nothing has happened.
T3 is the sweet spot for most high-stakes workflows, and the tier that gets used least. It turns the human’s job from “work out what to do and do it” into “approve or reject this exact prepared action” — a ten-second decision, with the work already done and the evidence attached. You get almost all the speed, and nothing runs that you cannot undo.
T4 — Execute with approval. The agent acts once a human says yes. The difference from T3 is small in the machinery and large in the organization: at T4 the human approves rather than operates. They don’t need to know the commands. They need to know whether this should happen.
This is the tier where your approval screen starts to matter enormously. If approving means opening four tabs and reading a wall of logs, you have built a T2 with extra steps. The approval screen should show: what is proposed, why, how wide the blast radius is, how to undo it, and what happens if you do nothing. One screen, ten seconds.
T5 — Autonomous. The agent acts and reports afterwards. Real, valuable, and it should be rare and specific — not a badge you collect to show how advanced you are.
T5 needs all four of these at once, and the “and” is doing a lot of work: a limited blast radius, an undo measured in seconds, a track record on this exact workflow with a measured success rate, and an automatic revert triggered by health signals. Restart a stuck worker: T5, obviously. Scale a stateless service inside declared bounds: T5. Roll back a deploy that crossed an error-rate threshold: T5, and honestly you probably already do this and call it automation. Anything touching credentials, customer data, money movement, or schema: not T5, no matter how good the track record, because there is no way back when it goes wrong.
Tiers attach to workflows, never to agents#
This is the part most autonomy conversations get wrong, so let me say it plainly.
“Do we trust the agent?” is a meaningless question. The same agent can deserve T5 on log triage and T1 on schema changes. Trust does not belong to a model. It belongs to a workflow: this task, this blast radius, this way back, this failure mode, this track record.
So the tier comes from three inputs, and I would write them down for every workflow you run:
Blast radius — how far the damage spreads. One pod, one service, one region, one customer, everyone. How easily you can undo it — seconds, minutes, with data loss, or not at all. Track record — measured results on this workflow: how many runs, how often the diagnosis was right, how often it cried wolf, how often a human overrode it.
Then four rules that keep it honest.
Tiers are earned with evidence, not handed out on enthusiasm. Promotion needs a real body of runs at the current tier with a measured pass rate. Not a demo that impressed someone on a Tuesday.
Demotion is immediate, and not up for debate. An incident caused or made worse by a workflow drops it one tier, that day. Promoting it back is a later conversation, with evidence. The imbalance is deliberate: earning trust should be slow and losing it should be instant, because that is the only arrangement where a track record means anything.
Actions you cannot undo are capped at T3, permanently. No track record earns the right to act alone on something with no way back. That is not caution, it is maths: a 99.9% success rate on an action you cannot undo is a guaranteed disaster once the volume is high enough.
Every tier has a named owner. A person, not a team. If nobody’s name is on a T5 workflow, it is not governed. It is just running.
Two ladders, one system — don’t confuse them#
One thing to be clear about, because I use both and they measure different things.
The Agentic Overwatch tier model describes who sits in which chair: agents handle detection and triage, agents handle diagnosis and remediation, humans own judgment and authorization. That’s organizational structure — staffing the clock.
Autonomy tiers describe how far a given workflow may go without a human. That’s a permission grant.
They work together. A T5 workflow runs unattended inside the Overwatch structure, with the room watching the fleet and able to stop it. A T3 workflow prepares work that the room approves. You need both: the structure says who is accountable, the tiers say what is allowed. Neither replaces the other, and I have watched teams build one and think they were finished.
The stop button has to actually work#
If agents can act during incidents, three controls are not optional. Not nice to have. Required.
A halt that halts. One action, available to whoever is on shift, that stops all agent action on a given workflow or across the fleet — immediately, without needing the agent’s cooperation, without a code deploy, without finding the right dashboard. Test it every month, the way you test backups. An untested kill switch is a story you tell yourself.
A visible action log, in one place. Every agent action during an incident, in sequence, with what it was responding to and what happened. During an incident you need to know what the fleet has already done, or you will spend the incident investigating your own fix. That is how a small incident becomes a confusing one.
A name on every change. Which agent, which lane, whose approval, on whose behalf. When the incident is an agent, the first question is “what did we do to ourselves?”, and it needs a fast answer.
The Replit case is the pattern to design against: an autonomous actor with inherited permissions doing something destructive during a window when nobody expected any changes. Your controls have to work in exactly that case, which means they cannot rely on the agent behaving.
Every incident should raise or lower a tier#
Here’s the mechanism that turns this from a policy into a system that gets better.
After an incident, add three questions to your review — and treat them as main items, not an appendix:
- Which tier was each involved workflow at, and was that right? Not in hindsight-blame terms. Was the assignment consistent with the blast radius and reversibility we now understand?
- What context did the human need that the agent could have provided? That gap is a T1 improvement, and it is the finding you will get most often. Every one you close makes the next incident shorter.
- What did the human decide that could have been decided by a rule? If the answer was fixed — “we always roll back when the error rate crosses 2% for five minutes” — that is a promotion candidate. If it needed weighing up things no rule can hold, that is a permanent human decision, and now you have written down why, which is worth as much as the automation.
That third question keeps you clear of two opposite failures: automating a judgment that should never be automated, and deciding by hand, for the fortieth time, something you have decided the same way every time.
And the artifact this produces is the machine-readable runbook. Not prose for a human to read at 3 AM, but a structured document: what normal looks like, what the failure modes are, what the agent may do autonomously for each one, what requires authorization, and what must never happen without waking a named person. That document is agent context, human onboarding, and audit trail all at once — which is why knowledge infrastructure stops being a documentation project and becomes an operational dependency.
Five moves#
- Deploy T1 on your noisiest alert, this week. Situation reports attached to pages. No risk, an immediate cut in the time it takes to get your bearings, and it builds the context pipeline everything else needs.
- Write the tier table for your ten most common incident workflows. Workflow, blast radius, reversibility, current tier, target tier, owner. It fits on one page, and it will be the clearest page your on-call team has seen in a year.
- Move your three highest-frequency remediations to T3. Prepared, staged, not applied. You will cut real minutes off MTTR without giving away any new authority at all.
- Test your halt. Today. If you can’t stop the fleet in under thirty seconds from a phone, you don’t have a control, you have a plan to build one.
- Add the three questions to your postmortem template. This is the loop. Without it, tiers freeze at whatever someone guessed on day one.
The failsafe doesn’t get to sleep through the incident#
I learned this in a room with a lot of screens, keeping thousands of live servers running at three in the morning. When something broke, we didn’t file it for the morning standup. We fixed it then, because we were the failsafe, and the failsafe doesn’t get to sleep through the incident.
What’s changed is that the failsafe is no longer a tired person doing everything. The agents handle detection, correlation, diagnosis, and — for a specific, earned, bounded set of workflows — the remediation. The human moves up to where the work is rare, serious, and genuinely human: deciding whether this should happen, and answering for it afterwards.
That’s not a smaller job. It’s the job the 3 AM heroics were always crowding out.
The agents run the incident. The humans own the call. Which calls those are is a decision you make deliberately, in advance, and write down — or a decision your vendor’s defaults have already made for you.
So: pick your most autonomous production workflow. What tier is it on, who decided, and when? If the honest answer is “whatever the defaults were when we installed it” — you don’t have an autonomy policy. You have an inheritance.
I lead AI transformation for a global SaaS platform and spent part of my early career in a NOC, which is where most of this thinking comes from. If you’ve built an explicit tier model — or you think T5 should never exist — I want the argument. Find me on X, LinkedIn, or Telegram.