Your pipeline does the same thing to every change.
A one-character typo fix in a log message and a schema migration that rewrites a tenant-quota table get the same treatment: same test suite, same stages, same runtime, same gates, same ritual. The pipeline has no idea which one it is looking at. Nobody ever asked it to know.
That was a deliberate and correct design choice. Determinism was the product: the same input gives the same output, every time, so you can reason about it. That is why CI/CD worked, and I am not going to stand here and call it a mistake. It is the reason we stopped shipping software by hand.
But look at the assumption underneath it: a human understood this change before the pipeline ran, and the pipeline’s job is to mechanically confirm what that human already believed. The pipeline was never the thinking part. It was the part that made sure the thinking got checked.
Remove the human who understood the change. Or keep them, but hand them forty changes an hour. Now what is the pipeline confirming?
A pipeline that can’t tell a typo from a migration spends the same verification budget on both. So it is either wasting money on one or under-checking the other. Under agent volume it does both, all day, and reports success.
This is part six of Rethinking the SDLC. The previous two pieces established what evidence a change must arrive with and what counts as proof. This one is about the machine that produces it — and why continuous integration has to become continuous verification.
The three ways deterministic pipelines fail under volume#
Not “get slow.” Fail, in specific ways, by design.
1. A fixed cost per change meets a change count that can grow without limit. Your pipeline’s runtime was tuned when a team produced 30 PRs a week. At 300, a 25-minute suite is 125 hours of compute a week and a queue nobody can drain. The instinct is to make the suite faster, which means making it weaker — and that is how organizations quietly trade verification for throughput without ever writing it down.
The money side moved too. GitHub moved every Copilot plan to usage-based billing for exactly this reason: a chat question and a multi-hour autonomous session cannot cost the same. Your pipeline has the same problem one layer down. It spends the same on verification for changes that are nothing alike. Nobody would run their cloud that way.
2. Red builds stop being events. In a healthy human-scale pipeline, a red build is a signal — someone looks. At agent volume, with flaky tests and repeated agent attempts, red becomes background noise. Then the muting starts: retry on failure, quarantine the flaky test, continue-on-error on the stage that is always noisy. Each decision on its own is reasonable. Together they build a pipeline that reports green because it has been trained not to fail.
I would bet money that if you audit your CI config today, you will find at least one gate that has been letting everything through for months, and that nobody knows about it.
3. Failures don’t get investigated, they get re-rolled. A human whose build fails reads the log and forms a theory. An agent whose build fails just tries something else. That is a sensible response, and a disastrous one at scale, because the failure carried information — a real defect, a broken contract, an environment that drifted — and the retry threw it away. Multiply that across a fleet and your pipeline becomes a slot machine that eventually pays out green.
Better scripts fix none of these. All three come from the same missing ability: the pipeline cannot reason about what it is looking at.
The four capabilities an agentic pipeline needs#
Here is the model. Four capabilities, in the order they depend on each other. Each one is worth building on its own, and each one makes the next possible.
1. Comprehension — the pipeline knows what this change is#
Before it chooses what to run, the system builds a structured picture of the change: which modules and services, whether contracts or schemas moved, whether IAM or config changed, what depends on the paths it touched, what the declared intent was, whether the diff matches that intent, what the change class is, and how wide the blast radius goes.
You can work most of this out today from things you already have — the diff, the dependency graph, ownership metadata, and the intent package. It is not magic. It is just work nobody gave the pipeline, because the pipeline was never meant to have an opinion.
Comprehension is the base. Nothing below it works without it, and it pays off straight away on its own. Even a pipeline that only sorts changes into classes and reports the result hands you the review routing from the previous article for free.
2. Strategy selection — the pipeline chooses verification by risk#
Once the system knows what a change is, running one fixed suite makes no sense.
A dependency bump with no API surface change: build, unit tests on affected modules, dependency-vulnerability scan, done in four minutes. A change to the entitlement path: full suite, property tests on the quota invariants, mutation threshold, production-traffic replay, migration dry-run against a production-shaped snapshot, blast-radius report, two-person approval. Forty minutes, and worth every second.
Two things fall out. The obvious one is cost: you stop spending a Class C budget on Class A changes, which are most of your changes. The less obvious one matters more: you can now afford verification you previously couldn’t. Nobody runs mutation testing and traffic replay on every commit — it’s too slow and too expensive. Run them only on the 8% of changes that deserve it and suddenly you can afford them. Choosing strategy by risk is not a way to verify less. It is the only way to verify the important things more.
And this is where DORA’s finding gets interesting. Their ROI model projects roughly $11.6M in first-year returns on $8.4M invested for a 500-person org — with one condition attached: the maths only holds for organizations with strong platforms and CI. Everyone else gets worse before they get better. That condition is the real headline. Strong CI is verification capacity, and verification capacity is what converts cheap generation into shipped software. It’s the difference between the 180% more code that ships 30% more software and a system that actually delivers.
3. Investigation — the pipeline diagnoses its own failures#
This is the capability that changes daily life on a team, and the easiest one to justify to a doubtful finance partner.
A failure arrives. Instead of a log dump and a Slack ping, the system does this. It finds which assertion failed and why. It re-runs with controlled changes to see whether the failure repeats or is flaky. It bisects to the commit that caused it when the history is unclear. It tells apart “the code is wrong” from “the test is wrong” from “the environment drifted” from “an upstream contract changed”. It checks whether this failure has appeared before and what fixed it. Then it writes a finding, with a likely cause and how confident it is.
Then the routing rule that makes it safe: the pipeline may investigate autonomously and propose freely, but it may not decide that a failure is irrelevant. Retry-on-failure without a diagnosis is banned. Quarantining a test is a change, and it goes through review like any other change, with an owner and an expiry date. The system’s job is to turn a red build into a finding, not to make red builds go away.
The scale case for this is already public. The Government of Alberta scanned 466 million lines of code in 20 hours — analysis work that no human team could have staffed at all. Investigating failures is exactly that shape of problem: huge volume, mostly mechanical, occasionally serious, and today limited by human attention.
4. Evidence production — the pipeline’s output is a case, not a status#
The last capability is the one that connects the pipeline to everything else in this series. The pipeline stops producing a yes/no and starts producing an evidence package: what was claimed, what was run and why those things, what passed, what the mutation score was, what the traffic replay showed, how wide the blast radius is, what is still unverified, and what a human is being asked to decide.
That artifact is what a reviewer rules on instead of reading a diff. It is what an auditor reads. It is what gets attached to the deploy record, so that six months later, during an incident, someone can answer “how did this get in” with a document instead of an archaeology project.
Continuous integration answered did it break the build. Continuous verification answers what do we know about this change, how do we know it, and what don’t we know.
What this does not mean#
Three things to be clear about, because “the pipeline becomes an agent” is easy to hear wrong.
Determinism doesn’t die — it moves. Running a chosen strategy must stay perfectly repeatable. What becomes flexible is choosing the strategy and investigating the failure. And the choice must be auditable: the pipeline records why it picked this strategy for this change, and someone can review that record. A pipeline that makes choices it cannot explain is worse than a dumb one.
Escalation is always up. If the pipeline is unsure what it is looking at — an unfamiliar path, a class it can’t pin down, a diff that doesn’t match its stated intent — it routes to the strictest strategy, not the cheapest. Doubt must never resolve in favour of speed. Get this rule wrong and everything else in the system is decoration.
The pipeline never approves its own consequential deploys. It can prepare, verify, stage, and produce the case. Promoting anything in a serious change class into production stays a human decision. The pipeline’s job is to make that decision cheap, informed, and fast — ten seconds, with the full case attached — not to remove it. This is the same tier structure I’ve argued for in Agentic Overwatch, applied to delivery instead of operations.
The pipeline becomes the control surface#
Here is the strategic point, and it is why I would spend platform money here before almost anywhere else.
Every governance question in an agentic organization needs a place to be enforced. Which changes need human eyes. What proof is required for the auth path. Which agent lanes may touch production. What the blast radius limit is. Who authorized this. What evidence existed at the time.
Those cannot live in a policy document, and they cannot live in the habits of individual engineers. They have to live somewhere every change must pass through. There is exactly one such place, and it is the pipeline.
So the pipeline stops being plumbing and becomes the most important governance system your organization has. That changes how you staff it. A platform team that sees itself as maintaining build infrastructure will optimize for uptime and speed. A platform team that knows it owns the control surface for the whole agent workforce will optimize for something else — and will need people who can reason about risk, not just about caching.
I have made a version of this argument before: when CI/CD speaks human, it stops being a wall between developers and production. The agentic version is stronger. When the pipeline can understand, choose, investigate, and give evidence, it stops being a wall or a rubber stamp. It becomes the thing that lets you raise autonomy without raising risk.
Five moves, in order#
- Audit your gates for fail-open behavior this week. Every
continue-on-error, every automatic retry, every quarantined test with no owner and no expiry date. Make the list, publish it, and put an expiry date on each one. This is the most valuable hour available to you, and it costs nothing. - Build understanding before you change behavior. Have the pipeline classify every change and report the class, while still doing exactly what it does today. Run that for two weeks and compare it with how humans would have classified. You will find your assumptions about your change mix are wrong.
- Split one suite by risk. Pick your slowest stage. Define a fast path for Class A and a strict path for Class C. Measure the compute you save and put it straight back — openly, where people can see it — into mutation testing or traffic replay on the strict path. Same budget, far better coverage of what matters.
- Ban blind retry. Require a finding. Any failure that gets retried must produce a diagnosis first, even a weak one. This one rule turns your pipeline from a slot machine into an instrument.
- Make the evidence package the pipeline’s real output. Structured, attached to the change, kept, and searchable later. Once it exists, review routing, audit, incident forensics, and the metrics in a later piece all get easy. Without it, every one of them stays manual.
The pipeline was always where the discipline lived#
We’ve done this before, and it’s worth remembering how it went.
Twenty-five years ago the bottleneck was manual QA. Releases queued behind human test cycles, and the industry’s first instinct was to demand more discipline from testers. That instinct lost. We built CI, and verification stopped being a virtue we asked people for and became something the system enforced. Nobody today thinks that made us reckless.
We are at exactly the same point, one level up. The bottleneck is understanding and verifying changes nobody typed. The current instinct is to demand more care from reviewers. That instinct will lose too, for the same reason: it doesn’t scale, and asking humans to be the throughput has never once worked in the history of this field.
The answer is the same answer. Build the machine that does it, put the judgment calls at the top where humans belong, and make the machine explain itself.
Your pipeline is already the only thing every change touches. It might as well be the thing that understands them.
So here’s the check I’d run before your next planning cycle. Take the last change that caused an incident. Walk it through your pipeline on paper and find the stage that should have caught it. Then ask why that stage didn’t run, or ran and was ignored.
I’ve done this exercise a lot. The answer is almost never “we lacked the technology.” It’s that the pipeline applied the same twenty-five minutes to that change as it did to a typo, and nobody had ever told it the difference.
I lead AI transformation for a global SaaS platform, where the delivery system is the main place governance actually gets enforced. If you’re building risk-based verification or automated failure investigation, I’d like to compare what worked. Find me on X, LinkedIn, or Telegram.