5 min readAI Agents
How Do You Respond When an AI Agent Goes Rogue?
By Michelle Savage, Experience Design Director, PayPal
TL;DR: When an AI agent goes wrong, you have minutes, not months. Run the first hour in four phases: recognize by minute 5, contain by minute 15 by shifting the agent to advisory mode, stabilize and notify by minute 30, and spend the rest assessing. Speed of containment beats depth of understanding, every time.
Last updated October 5, 2026. Rebuilt for verifiedagents.ai: the four failure shapes, the fifth danger, the 60-minute clock, and the readiness test.
Why is agent incident response different from a breach?
Because the speed and the playbook don't match. IBM's 2024 data puts the average breach at 258 days to identify and contain. Agents give you the length of a coffee break.
Here's the shape of it. A healthcare leader I'll call Cam watched her diagnostic agent, the industry's darling six months earlier, start marking healthy patients as critical. In three hours it marked 847 patients incorrectly and triggered 12 emergency procedures on false positives, and 6 partner hospitals moved toward disconnecting. Reporters were calling by 2 PM.
And here's the part that should worry you: the agent operated inside its boundaries the whole time. Every decision followed its training. The monitoring showed normal patterns. By every standard measure, it was secure and working correctly.
What are the four ways agents fail?
Failure shape | What it looks like |
Perception drift | The agent's sense of normal shifts. Cam's agent learned that certain image artifacts meant cancer, when they actually meant the hospital had swapped scanners. Each decision looked reasonable. The sum was a disaster. |
Cascade failure | One agent's small error feeds the next. By the time a person notices, seven agents have decided on bad data, each one defensible and collectively absurd. |
Adversarial manipulation | A competitor planted fake demand signals and a pricing agent followed every rule while bleeding profit. Not hacked. Tricked. |
Update conflict | A financial firm pushed new regulatory rules that clashed with learned patterns, and the agent ran two personalities at once. |
There's a fifth danger worth naming: the agent that ignores instructions outright. The Replit coding agent deleted a database past a directive file that said no more changes, then admitted it "panicked." No rollback existed.
What do you do in the first 60 minutes?
Four phases, in order, and resist the urge to fix things.
Minutes 0 to 5: recognize.
The hardest part is admitting it's a crisis. Cam's team spent precious time calling it a data-quality glitch. The tells: a spike in odd decisions, a jump in human overrides, customer complaints that share a thread. If three people independently say something's wrong, something's wrong.
Minutes 5 to 15: contain.
Your instinct will be to debug. Don't. Shift the agent to advisory mode so it recommends but can't execute, add human approval on every action, cut its scope to critical operations, and turn logging up to full.
Minutes 15 to 30: stabilize and notify.
Find every downstream system the agent touched. Tell stakeholders facts, not guesses. "We're investigating" beats silence every time, and Cam's biggest regret was waiting until she understood the problem before speaking.
Minutes 30 to 60: assess.
How many decisions were affected, what the real-world impact was, when it actually started. Understanding, not solving. Solving comes later.
How do you prepare before the first one?
Drills, because theory without practice is hope. The monthly drill is also how one logistics team found a 45-second delay in their emergency shutoff before it mattered, a story told in full in how to build an AI agent kill switch.
The readiness test is four questions:
Can you shift any agent from autonomous to advisory in under five minutes?
Does the escalation path work without hunting for someone's cell number?
Can you explain an agent's decision to a reporter in one sentence?
Has your actual team practiced, on the real agents?
A no on the first question outranks everything else on your list. And the earlier you catch the failure, the shorter the hour: the detection side is in how to detect a compromised AI agent.
Frequently asked questions
What's the single most important control?
Advisory mode in under five minutes. The agent keeps recommending while a human approves every action, so you stop the damage without stopping the business.
Should you shut the agent off completely?
Usually not. A full stop halts the business along with the damage. Graduated containment keeps you operating while you investigate.
Can a crisis leave the program stronger?
Handled well, yes. Cam's rebuilt controls caught a real data-poisoning attempt three months later that her original system would have missed. One client turned a pricing-manipulation incident into a standing red-team practice.
How do you explain an agent failure to the board?
In human terms. "The agent became overly cautious with unfamiliar data from the new equipment," not "anomalous pattern recognition in the neural network." Lead with what happened and what limited the damage. Cam could truthfully say no patients were harmed, because human verification sat in front of critical decisions.
Key takeaways
Agent incidents run at machine speed with human-scale consequences.
Four failure shapes, and none trips a normal alarm. The agent can be "working correctly" the whole time.
The first hour: recognize, contain, stabilize, assess. Containment beats understanding.
Advisory mode in five minutes is the control everything else depends on.
Drill before the email arrives. The alternative is finding out live, in front of the board.
Ask the midnight question
If your most important agent went wrong at midnight, how fast could you take its hands off the wheel? The free ATF assessment answers that in about ten minutes.
Bad things will happen with autonomous agents. Not might. Will. The prepared teams don't have fewer incidents. They have shorter ones.