verifiedagents.ai
All posts

4 min readAI Agents · Cybersecurity

How do you test whether your AI agent kill switch works?

By Josh Woodruff, Founder & CEO, MassiveScale.AI | Founding Chair, Agentic Trust Framework at the CSAI Foundation

Hero: How do you test whether your AI agent kill switch works?

TL;DR: Test a kill switch by pulling it. Run a timed drill on a real agent in a safe window. Clock how long the stop takes, then have the stopped agent try one action. That action should fail everywhere the agent works. Pass means advisory mode in under five minutes, pulled by someone other than the builder.

A kill switch you've never pulled is a theory.

Most teams I ask say they have one. Then I ask when they last used it. The answer is usually never, and that means nobody knows if it works.

What counts as a kill switch?

It's a fast, reliable way to take an agent's hands off the wheel. When the agent starts making bad calls, you pull it out before the next hundred go through.

The setting you'll use most is advisory mode. The agent keeps reading and recommending. Nothing runs without a human yes. That stops the harm without stopping the work.

A full shutdown is for true emergencies. It stops the damage and the business at the same time. The design choices are on MassiveScale.AI, in what a kill switch for AI agents is and how to build one.

How does the assessment score this?

Question 25 of the free assessment asks whether you can immediately stop a misbehaving agent. It belongs to Incident Response, one of the five ATF elements. A timed drill sorts the five answers fast.

Answer

What it says

What the drill clock shows

A

No ability to stop agents instantly.

The clock keeps running while people look for a way.

B

We can restart or shut down through normal operations.

Many minutes, and a change ticket.

C

We have a documented emergency shutdown process.

A few minutes, if the right person is awake and can find the document.

D

We have a one-click kill switch.

Seconds, once a person decides to pull it.

E

We have automated kill switches triggered by anomalies.

Seconds, with no person in the loop.

Answer E has its own test, which I cover in testing whether a runaway AI agent gets stopped automatically.

How do you run the drill?

Pick a quiet hour. Tell the agent's owner it's coming.

  1. Choose one real agent. Start with the one that can do the most.

  2. Hand the switch to someone who didn't build it. Use the on-call person.

  3. Start the clock and call the drill.

  4. Have them move the agent to advisory mode. Stop the clock when it's done.

  5. Now have the agent try one action. Pick something harmless, in each system it can reach.

  6. Write down the time and what got through.

What's the pass bar?

Four checks. All four have to hold.

  • Speed. Advisory mode in under five minutes. A full stop takes seconds.

  • Reach. The test action fails in every system the agent works in.

  • People. Someone other than the builder pulled it, without hunting for a phone number.

  • Fallback. A person can do the agent's job by hand while it's off.

Reach is where drills surprise people. A switch can stop the agent on the main dashboard and miss one connection. The only way to know is to try an action after the stop.

Why does speed get its own check?

Because agents don't wait. A delay that's fine for a person is thousands of actions for an agent.

And a stop that needs you at the keyboard isn't fast. In February 2026, Meta's Director of AI Alignment told an AI agent to suggest what to delete from her inbox and to wait for her approval. It started deleting. She told it to stop and it kept going. She had to get to her computer and shut it down by hand. The OECD's AI incident record has the details.

Her job is AI safety. If stopping an agent took her a run to her computer, don't assume yours is faster. Time it.

Frequently asked questions

Is turning off the model vendor account a kill switch?

No. That stops many systems at once, and it may not cut one agent's local keys. A real switch targets one agent and leaves the rest running.

Do I need a switch for every agent?

Yes. Any agent that can act can act wrongly. What grows with risk is how hard you drill, not whether the stop exists.

How often should I run the drill?

Every quarter, with the people who'd be on call. Run it again after the agent gets a new tool, because that's a new place the switch has to reach.

Should the switch be tested before go-live?

Yes. It's half of the shut-off and undo test an agent should pass before go-live.

Key takeaways

  • Test a kill switch by pulling it, on a real agent, with a clock running.

  • After the stop, have the agent try an action. It should fail everywhere.

  • Pass means advisory mode in under five minutes, pulled by someone who didn't build the agent.

  • A switch that needs one specific person fails the drill.

Question 25 is one of 30 in the free assessment. It takes about ten minutes and scores you on all five ATF elements.

If your most important agent went wrong right now, you'd learn in seconds whether your switch is real. Learn it in a drill.

See where your agents stand.

The free assessment takes ten minutes and scores you on the five elements of the Agentic Trust Framework.