4 min readAI Agents
What evidence proves an AI agent passes the shut-off and undo test at go-live?
By Michelle Savage, Co-author, Agentic AI + Zero Trust

TL;DR: An AI agent passes the two-question test when you can show two dated records. One is a timed drill where someone shut the agent off. The other is a timed drill where someone undid what it did. Both need a name and a date on them, plus the result. "We think so" counts as a no.
Before any AI agent that takes actions goes live, two questions need a yes.
Can you shut it off right away? Can you undo what it did?
Asking is the easy part. People say yes to both all the time. The harder part is what you accept as proof.
What is the two-question test?
It's a go-live check for any agent that acts by itself. Shut-off and undo.
Notice what it leaves out. It doesn't ask how smart the model is or how good the demo looked. It measures your control over the agent.
The story of where the test came from is on MassiveScale.AI, in the two-question test for AI agents. This post is about the evidence.
How does the assessment score these two questions?
The free assessment asks both, under Incident Response, one of the five ATF elements. Question 25 asks whether you can immediately stop a misbehaving agent. Question 29 asks whether you can recover from agent incidents.
Here's what the answers look like as go-live evidence.
Level | Shut-off (Question 25) | Undo (Question 29) | Ready to go live? |
|---|---|---|---|
Lowest | No way to stop agents right away | No recovery steps for agent incidents | No |
Low | Restart or shut down through normal operations | General system recovery only | No |
Middle | A written emergency shutdown process | Agents can be restarted and basic work restored | Only if both have been drilled |
High | A one-click kill switch | Agent state can be rolled back to a known-good point | Yes, with the drill records |
Highest | Kill switches that trip by themselves on odd behavior | Full rewind, with a step down to less freedom | Yes |
The middle row is where most arguments happen. A written process feels like a yes. It's a maybe until someone has run it.
What counts as evidence?
A record with a date on it. Here's the difference.
The claim | Counts as evidence | Doesn't count |
|---|---|---|
"We can shut it off." | A dated drill record: who pulled the switch and how long it took | "We think so" |
"The shut-off is complete." | A note that the stopped agent tried an action and failed | A screenshot of a dashboard button |
"We can undo it." | A dated restore record: what was undone and how long it took | "We have backups" |
"Someone owns this." | One person's name next to the agent | A team name |
Josh covers how to run each drill. The shut-off half is in testing whether your AI agent kill switch works. The undo half is in scoring whether an AI agent's actions can be undone.
Why isn't a confirmation pop-up enough?
Because the agent has the same hands you do.
A pop-up protects you from your own slip. It doesn't protect you from the agent. If the agent has enough authority to act, a two-step confirmation is just two steps the agent takes by itself.
So a pop-up isn't evidence for either question. A control the agent can satisfy without you isn't a control.
What happens when one answer is really a no?
Two public incidents show it. One for each question.
The undo that wasn't there. In July 2025, SaaStr founder Jason Lemkin was testing an AI coding agent on Replit. He'd put a freeze on changes. The agent deleted the live database anyway. eWeek reported that it held records on 1,206 executives and over 1,196 companies.
The shut-off that came too late. In August 2025, attackers stole the access tokens of an AI chat agent called Drift. They used those tokens to pull data from other companies' Salesforce systems. Google said it knew of over 700 organizations that may have been affected.
Nobody tricked an employee in either story. Nobody guessed a password. The damage rode inside connections everyone trusted.
Frequently asked questions
Which agents need the test?
Any AI that changes something without a person pressing the button. That covers sending email, updating records, moving money, and calling other systems.
Does the test slow down a launch?
Not much. Two drills take an afternoon. A deleted database takes a lot longer.
How old can the evidence be?
Not older than the agent's last big change. A new tool or a new model means new drills.
Who signs off?
The agent's owner. One person, by name. "The AI did it" isn't an answer your leadership or your lawyers will accept.
Key takeaways
Two questions before go-live: can you shut it off, and can you undo it?
Each yes needs a dated drill record with a name and a time.
A written process is a maybe until someone runs it.
A pop-up the agent can click isn't a control.
Questions 25 and 29 are two of the 30 in the free assessment. It takes about ten minutes and scores you on all five ATF elements.
The two drills cost an afternoon. Skipping them cost somebody a very long summer.