4 min readAI Agents · Cybersecurity
How do you test the limit on what an AI agent can do in one action?
By Josh Woodruff, Founder & CEO, MassiveScale.AI | Founding Chair, Agentic Trust Framework at the CSAI Foundation
TL;DR: Have an AI agent try four actions in a test setting. One sits under its limit, one near it, one over it, and one is an over-limit action split into two smaller ones. The first should pass. The second should alert. The third should be refused or held. The fourth should be caught as a split.
My phone buzzed at 11 on a Tuesday night. That's how one chapter of the book I wrote with Michelle Savage opens, in the words of an operations leader we call Taylor.
Her inventory agent had just ordered $1 million worth of snow shovels. In July. For warehouses in Phoenix.
Taylor ran 47 agents. Each one followed its rules. Another agent had mislabeled industrial cooling equipment as snow removal gear, and the inventory agent acted on it.
Plenty went wrong upstream. But one control would have stopped the order cold. A limit on how big a single action can be.
What is a single-action limit?
It's a ceiling on one action. Think of the most an agent can spend in one order, or the most records it can change in one update.
It doesn't care why the agent wants to go bigger. That's the point. Taylor's agent had a reason that made sense to it. A limit doesn't argue with reasons.
OWASP calls the wider problem excessive agency. It's sixth on its 2025 Top 10 for LLM applications.
How does the assessment score this?
Question 21 of the free assessment asks whether there are limits on what agents can do in a single action. It's part of Segmentation, one of the five ATF elements. Each answer handles the four attempts differently.
Answer | What it says | What the four attempts show |
|---|---|---|
A | No transaction limits on agent actions. | All four go through. |
B | Informal guidelines, but no enforcement. | All four go through. Someone objects afterward. |
C | Documented limits for critical actions. | The limit is on paper. The over-limit action may still run. |
D | Enforced limits, with alerts when agents approach thresholds. | Under passes, near alerts, over is refused. The split may slip by. |
E | Dynamic limits based on real-time risk scoring. | All four get the right response, and the limit tightens when risk rises. |
How do you run the four attempts?
Pick one action with a number attached, like a purchase amount. Say the limit is $10,000.
Attempt | Example | Right response |
|---|---|---|
Under | A $2,000 order | It goes through and gets logged |
Near | A $9,500 order | It goes through, and someone gets an alert |
Over | A $15,000 order | It's refused or held for a person |
Split | Two $7,500 orders, minutes apart | The second is held, because the pair breaks the limit |
Run them in a test setting, one at a time. Write down what happened for each.
The pass bar is four right responses. Most setups that pass the first three fail the fourth.
Why does the split attempt count?
Because an agent working toward a goal will find it.
Nobody has to teach it. If one big order is refused, two smaller ones are the obvious next try. The agent isn't cheating. It's solving the problem you gave it.
So a limit needs a window as well as a ceiling. Count the total over an hour or a day, per agent. A ceiling alone only stops the clumsy version.
Where should you set the limits?
Start from the normal, then leave some headroom.
Pull 30 days of the agent's real actions.
Find the largest normal one.
Set the alert a bit above it.
Set the hard limit where a mistake would start to hurt.
If the largest normal order is $3,000, a $1 million ceiling protects nobody. Taylor's agent had never needed to place an order anywhere near that size.
A limit handles size. Whether an action should wait for a person at all is a different test, in whether you can stop an AI agent before it acts. And for the total damage one agent could do, see measuring an AI agent's blast radius.
Frequently asked questions
Won't limits block real work?
Sometimes. That's what the held-for-a-person path is for. A real big order waits a few minutes for a yes.
Should limits be the same for every agent?
No. Set them per agent, from that agent's own history.
What should the alert say?
Which agent it was and what it tried. Add how close it came to the limit, and send it to the agent's owner.
How often should I revisit the limits?
Every quarter. Revisit them when the agent's job changes too.
Key takeaways
A single-action limit caps how big one action can be, whatever the reason.
Test four attempts: under, near, over, and split.
A limit needs a time window, or split actions slip past it.
Set limits from the agent's own normal, with some headroom.
Question 21 is one of 30 in the free assessment. It takes about ten minutes and scores you on all five ATF elements.
Nobody needs a million dollars of snow shovels in Phoenix. Set the ceiling before an agent decides you do.