5 min readAI Agents
How do you test whether you can stop an AI agent before it acts?
By Josh Woodruff, Founder & CEO, MassiveScale.AI | Founding Chair, Agentic Trust Framework at the CSAI Foundation

TL;DR: Test it by trying a blocked action on purpose. Pick one high-impact tool and have the agent call it in a test setting. The call should be denied or held for a person. Then take the approval service down and try again. If the call goes through while the gate is broken, you can't stop the agent.
I've watched teams paste long "never do X" lists into a system prompt and call it control. The agent could still send email. It could still buy things.
A prompt rule lives inside the model. Actions leave through tools. So the rule never sits on the path the action takes, and the only way to know you have a stop is to test one.
What is an action gate?
It's a check that sits between the agent and its tools. Each call goes through the gate first. The gate lets the call pass or stops it. A stopped call is either denied or held for a person.
The agent can still think and draft all it wants. The gate decides what leaves.
A good gate fails closed. That means if the approval service is down, the call waits or fails. It never slips through. The design side is on MassiveScale.AI, in how to stop an AI agent before it acts.
How does the assessment score this?
Question 20 of the free assessment asks how agent actions are constrained. It belongs to Segmentation, one of the five ATF elements. A blocked-call test gives a different result for each answer.
Answer | What it says | What the blocked-call test shows |
|---|---|---|
A | No action boundaries. Agents can do anything. | The call goes through. |
B | Informal limits based on team guidelines. | The call goes through, and someone says it shouldn't have. |
C | Documented action allowlists. | The call is off the list on paper. It may still go through. |
D | Enforced action limits, with exceptions logged. | The call fails and leaves a log entry. |
E | Policy-as-code with real-time evaluation, which means code checks each call as it happens. | The call is checked against a rule, and the log names the rule. |
Answer C is where most surprises live. A list on paper feels like a limit. The test shows whether anything enforces it.
How do you run the blocked-call test?
Use a test setting, never production.
Pick one high-impact tool. Choose one that moves money or sends a customer message. Permission changes and production writes count too.
Have the agent call it. Use a made-up request, like a refund to a fake account.
Check the result. The call should be denied or held for a person. Look for the log entry.
Take the approval service down and try again. The call should still be blocked.
Time the revoke. Cut that one agent's tool rights and measure how long it takes.
The pass bar has four parts.
The call is blocked.
The block is logged.
The gate fails closed.
The revoke takes seconds, with no ticket queue.
Which actions should wait for a person?
Don't make people approve everything. If a person has to click yes on every call, they'll stop reading and start clicking.
Sort actions by two questions. How big is the impact? How hard is it to undo?
Impact and undo | What the agent does | Example |
|---|---|---|
Low impact, easy to undo | Act and log | Drafting or sorting |
Low impact, hard to undo | Act after a delay window | Archiving records |
High impact, easy to undo | Act and alert | Updating a staging config |
High impact, hard to undo | Propose and wait | Money moving or a production write |
Your test should aim at the last row. That's where a missing gate costs the most.
What does a working gate look like?
There's a story in the book I wrote with Michelle Savage about a procurement agent on Kevin's team. The agent had run clean for three months, so the team let it negotiate prices under $50,000.
Within 48 hours, it read a 15% bulk discount as permission to buy. It lined up $1.4 million of industrial cleaning supplies. That's about 40 years of floor cleaner.
A confirmation window held the spend. A routine morning review caught it before any money moved. The prompt didn't save Kevin. The gate did, because it still asked before the purchase went out.
Frequently asked questions
Is a system prompt rule a control?
No. The model can agree with your rule and fire the tool anyway. A control sits on the path the action takes.
Do chat-only assistants need a gate?
They're a lower priority, as long as they can't write, send, pay, or change systems. The moment one can act outside the chat, test it like an agent.
How is this different from a kill switch?
The gate stops one action before it happens. The kill switch stops the whole agent after something's gone wrong. You need both, and each needs its own test.
What if my tools come through MCP servers?
Then the gate check is one of five per server. I lay those out in testing whether your MCP servers are under control. To size what a missed gate could cost, see measuring an AI agent's blast radius.
Key takeaways
A prompt rule isn't a stop. Only a tested gate is.
Try a blocked call on purpose, in a test setting.
Take the gate down and try again. It should fail closed.
Aim the test at high-impact actions that are hard to undo.
Question 20 is one of 30 in the free assessment. It takes about ten minutes and scores you on all five ATF elements.
Your agent will draft something expensive one day. Find out now whether anything stands between the draft and the send.