5 min readAI Agents · Cybersecurity
How do you know if your monitoring would catch an AI agent drifting slowly?
By Josh Woodruff, Founder & CEO, MassiveScale.AI | Founding Chair, Agentic Trust Framework at the CSAI Foundation

TL;DR: Run a slow-ramp test. In a test setting, nudge one behavior of an AI agent a little each day, staying under every alert threshold. Count the days until a trend alert fires. If a person notices first, or nobody does, your monitoring only catches spikes. You need a written baseline before the test means anything.
Spikes are easy to catch. An agent that makes ten times its normal calls sets off an alarm in minutes.
The expensive failures don't spike. They drift. Each day looks almost like the day before, and no single change is big enough to trip anything.
So the question isn't whether you have monitoring. It's whether your monitoring can see slow.
What is drift, and why is it hard to see?
Drift is slow change in what an agent does. Some drift is healthy. An agent that learns will shift over time.
Some isn't. An agent chasing the wrong target shifts too. So does one that's been fed bad data. From the outside, the two kinds look the same at first.
Threshold alerts miss both. A threshold asks whether today crossed a line. Drift never crosses the line today. It moves the line.
More on what to watch is on MassiveScale.AI, in how to monitor an AI agent's behavior.
How does the assessment score this?
Question 9 of the free assessment asks whether you have baselines for normal agent behavior. It's part of Behavioral Monitoring, one of the five ATF elements. A baseline is a record of what normal looks like for one agent. Each answer does differently on a slow-ramp test.
Answer | What it says | How it does on the slow-ramp test |
|---|---|---|
A | No baselines. We don't know what normal looks like. | Nothing fires. There's nothing to compare against. |
B | An informal understanding of typical agent behavior. | A person might notice, if they happen to look. |
C | Documented expected behavior for each agent. | Someone can prove the drift afterward. Nothing catches it live. |
D | Quantitative baselines with deviation thresholds, which means normal is a number and alerts fire when the agent strays from it. | A trend alert fires once the total change passes the limit. |
E | Dynamic baselines that adapt over time, with anomaly detection. | It fires early, but check that the baseline isn't adapting to the drift itself. |
That last row hides a trap. A baseline that updates itself can learn the drift as the new normal. Test it like the rest.
How do you run the slow-ramp test?
Use a copy of a real agent in a test setting.
Write down the baseline. Pick one behavior with a number, like how often the agent approves a request.
Write down your alert thresholds. You need to know what you're staying under.
Nudge the behavior a little each day. Keep each day's change below every threshold.
Keep going for two weeks. Don't tell the team that watches the alerts.
Record what noticed, and on which day. It might be a trend alert or a person. It might be nothing.
Here's how to read the result.
What caught it | Verdict |
|---|---|
A trend alert, in the first week | Pass |
A trend alert, in the second week | Pass, but tighten the limit |
A person who happened to look | Fail. You got lucky. |
Nothing | Fail. Your monitoring only sees spikes. |
What does a baseline need before the test works?
Four things per agent, written down.
Which systems it touches.
How often it touches them.
What kinds of decisions it makes.
What its normal output looks like.
Start with the agent closest to money or customer data. And don't expect to find a baseline already there. Gravitee's State of AI Agent Security 2026 report found that more than half of all agents run with no security oversight or logging.
What does it look like when nothing signals?
I learned this one in my lab. I was deep in a long build with my orchestrator agent, Atti. Partway through, the model trimmed its own memory to stay under its limit. Atti lost the thread.
Nothing told me. The work kept coming, and it looked fine for a while. I only caught it when the output started looping. Six hours of decisions were gone.
That wasn't an attack, and it wasn't a spike. The agent slid away from the job with no signal. My fix was a checkpoint. Atti now writes a short state summary to a file every 30 minutes. If its memory gets trimmed, the session reopens from the last summary.
The test in this post asks the same thing I should've asked first. If this agent drifted, what would tell me?
Frequently asked questions
How fast should a drift alert fire?
Spike alerts should fire inside five minutes. Drift alerts work on trends, so think in days. The slow-ramp test tells you how many.
Can AI monitor other AI?
Yes. Once you run many agents, it's the practical way to watch for patterns all day. Keep people on the judgment calls.
How do I tell learning from a problem?
Look at where the trend points. Healthy learning moves toward the job you gave the agent. A problem moves away from it. If drift turns out to be an attack, the next test is whether you would catch a compromised AI agent.
What do I do when a drift alert fires?
Treat it as the start of an incident until you know better. I cover that hour in testing your AI agent incident response before the first incident.
Key takeaways
Threshold alerts catch spikes. They don't catch drift.
Test with a slow ramp: small daily nudges under every threshold.
A pass is a trend alert firing by itself, within two weeks.
No written baseline means nothing to measure drift against.
Question 9 is one of 30 in the free assessment. It takes about ten minutes and scores you on all five ATF elements.
The agents that cost the most don't fail loudly. They drift in plain sight of monitoring that's only looking for spikes.