verifiedagents.ai
All posts

5 min readAI Agents · Cybersecurity

What evidence passes each of the five AI agent audit questions?

By Josh Woodruff, Founder & CEO, MassiveScale.AI | Founding Chair, Agentic Trust Framework at the CSAI Foundation

Hero: What evidence passes each of the five AI agent audit questions?

TL;DR: An auditor's AI agent review comes down to five questions, one for each ATF element. Each passes on a specific kind of evidence: a named owner record, a written baseline, a source trace, a blast radius score, and a timed stop drill. A policy document passes none of them. Build one evidence folder per agent.

At the RSAC Conference, a Fortune 500 CISO stopped me in a hallway. Three of her teams were piloting AI agents. She couldn't tell her board who owned any of them.

She wasn't worried about a breach. She was worried about the question. Somebody was going to ask, and she had nothing to hand over.

That's what an audit is. Someone asks, and you hand something over. So the useful thing to know is what to have in your hand.

What are the five audit questions?

The Agentic Trust Framework (ATF) asks five things about any agent. Who are your agents? What are they doing? What data flows through them? Where can they go? What if they go rogue?

Each question belongs to one ATF element. And the free assessment has six scored questions under each element, so you can see where your evidence would stand before an auditor does.

The auditor's side of this is on MassiveScale.AI, in what auditors want to know about AI agents. This post is the evidence list.

What evidence passes each question?

Audit question

ATF element

Evidence that passes

What doesn't pass

Who is it?

Identity Management

A record with the agent's own identity and one named owner

A shared service account, or a team name

What is it doing?

Behavioral Monitoring

A written baseline of normal, and one alert that fired against it

A dashboard nobody reads

What data moves through it?

Data Governance

One output traced back to its sources

A list of data sources with no link to any output

Where can it go?

Segmentation

A blast radius score for the agent, with the controls behind it

A statement that least privilege is followed

What if it goes rogue?

Incident Response

A dated record of a timed stop drill

A kill switch nobody has pulled

Look at the right-hand column. Every item there is a claim. Every item in the evidence column is something you can open and read.

How does the assessment line up with each question?

One assessment question sits closest to each audit question. Here's the answer you'd need to give for your evidence to pass.

Audit question

Closest assessment question

The answer that produces passing evidence

Who is it?

Question 6: Who is responsible for each AI agent?

Each agent has both technical and business owners documented.

What is it doing?

Question 9: Do you have baselines for normal agent behavior?

Quantitative baselines with deviation thresholds.

What data moves through it?

Question 18: Can you trace where agent data came from and how it was used?

Full transformation logging that shows how data was processed.

Where can it go?

Question 24: How do you limit damage if an agent is compromised?

Technical controls actively limit each agent's blast radius.

What if it goes rogue?

Question 25: Can you immediately stop a misbehaving agent?

A one-click kill switch.

Those are the fourth of five answers in each case. The lower answers describe intentions. The fourth is where proof starts to exist.

How do you build the evidence for each one?

Each piece of evidence comes out of a test you can run this month.

  1. Owner record. Run the five-agent spot check in what evidence proves every AI agent has one accountable owner.

  2. Baseline. Write one, then try it against slow change, in whether your monitoring would catch an AI agent drifting slowly.

  3. Source trace. Run the ten-minute trace in whether you can trace what your AI agent treats as true.

  4. Blast radius score. Score five controls per agent, in measuring an AI agent's blast radius.

  5. Stop drill. Pull the switch with a clock running, in testing whether your AI agent kill switch works.

Save what each test produces. Put the five results in one folder per agent, with dates.

How do you score your own readiness?

Pick your highest-impact agent. Give yourself 30 minutes to pull all five pieces of evidence for it.

Score one point for each piece you can open. Five is a pass. A piece that exists but took a day to find scores zero, because an auditor won't wait a day.

Then do the same for the next agent. The score per agent is more honest than one number for the company.

Where does Zero Trust fit?

Zero Trust is the foundation an auditor will expect. It verifies every connection. Never trust, always verify.

Agents need one more check on top. A verified connection can still carry a bad instruction. So the five questions look at what the agent does, as well as who the agent is.

Frequently asked questions

Who audits AI agents today?

Internal audit and your existing outside auditors, mostly. The review usually rides along with access control audits, with new questions added for agents.

Does a policy count as evidence?

No. A policy says what should happen. Evidence shows what did happen, with a date on it.

How old can evidence be?

No older than the agent's last big change. A new model or a new tool means the drill record is about a different agent.

What if we can't answer the first question?

Then start there. Without an owner, nobody can produce the other four pieces.

Key takeaways

  • Five audit questions, one per ATF element.

  • Each passes on evidence you can open: a record, a baseline, a trace, a score, and a drill.

  • A policy doesn't pass. Neither does a dashboard nobody reads.

  • Keep one dated evidence folder per agent.

  • Score readiness per agent, with a 30-minute clock.

The free assessment asks 30 questions, six for each element. It takes about ten minutes and shows where your evidence would stall.

The CISO in the hallway didn't need a better answer. She needed something to hand over. Build the folder before somebody asks for it.

See where your agents stand.

The free assessment takes ten minutes and scores you on the five elements of the Agentic Trust Framework.