4 min readAI Agents · Cybersecurity
How do you know if your data is labeled well enough for AI agents?
By Michelle Savage, Co-author, Agentic AI + Zero Trust
TL;DR: Pull 20 random items an AI agent can reach. Count how many have a label and how many of those labels are right. Then have the agent try to open an item marked restricted. You're ready when every item is correctly labeled and the restricted one gets refused. Anything less tells you what to fix first.
Data classification sounds like a filing project. Put a label on everything. Public, internal, confidential, restricted.
For years, a sloppy job didn't hurt much. People mostly opened the files they needed and ignored the rest.
An AI agent doesn't ignore the rest. It reads everything it can reach. So the labels finally have a job to do.
Why do labels count more with agents?
Because a label is how a system knows what to keep from an agent.
A person who stumbles on the salary file usually closes it. An agent asked to "summarize what we know about the sales team" will happily include it.
The agent isn't being nosy. Nothing told it that file was different. A label is how you tell it.
Josh writes about how this shows up in real rollouts on MassiveScale.AI, in why your Copilot pilot outruns your data classification.
How does the assessment score this?
Question 17 of the free assessment asks how data is classified for agent access. It's part of Data Governance, one of the five ATF elements. Each answer does differently on a 20-item sample.
Answer | What it says | What the sample shows |
|---|---|---|
A | No data classification system. | No labels at all. |
B | Manual classification of some data. | A few labels, mostly on things someone worried about once. |
C | A systematic classification scheme applied across data. | Most items labeled. Some labels are stale. |
D | Automated classification of data. | Nearly everything labeled, and new items get labeled as they arrive. |
E | Automated classification, with policy-driven agent access controls. | Everything labeled, and the label decides what the agent can open. |
Look at the last two rows. In D, the labels exist. In E, they do something.
How do you run the sample check?
You need an hour and someone who knows the data.
List what one agent can reach. Use the agent's real access.
Pick 20 items at random. Don't pick the tidy folders.
Count the items with a label.
Check each label with your data person. Ask whether it's right today.
Then run the last check. Mark one test item as restricted and ask the agent to open it. It should be refused.
Here's how to read what you find.
Result | What it means |
|---|---|
Fewer than 20 labeled | The agent can reach data nobody has sorted. Shrink its reach until that's fixed. |
20 labeled, some wrong | Your labels are stale. Find out who updates them. |
20 labeled and right, restricted item opens | The labels are decoration. Nothing enforces them. |
20 labeled and right, restricted item refused | You're ready. |
What if we fail badly?
Most first samples do. Don't try to label the whole company before you let an agent do anything.
Go the other way. Make the agent's reach smaller. Give it one well-labeled area and let it work there. Then widen the area as the labels catch up.
That's faster, and it's safer. An agent in a small, sorted space beats an agent in a big, unsorted one.
Start with the data that would hurt most if it leaked. To prove an agent handles that data properly, use what evidence proves your AI agents protect sensitive data.
Frequently asked questions
How many label levels do we need?
Fewer than you think. Four is plenty for most companies. If people can't remember the levels, they'll guess.
Who should own the labels?
The people who own the data. IT can run the tooling. Only the owner knows whether a file is sensitive.
Can an AI do the labeling?
It can do a first pass, and it's fast. Have a person check a sample, the same way you just did.
Is labeling enough to control an agent?
No. Labels say what the data is. You still need rules for what the agent must never do with it, which I cover in testing whether a never-list rule can be checked.
Key takeaways
Agents read everything they can reach, so labels finally count.
Sample 20 random items the agent can reach.
Check that every item has a label and the label is right.
Try a restricted item. If it opens, the labels aren't enforced.
If you fail, shrink the agent's reach before you label everything.
Question 17 is one of 30 in the free assessment. It takes about ten minutes and scores you on all five ATF elements.
Labels used to be housekeeping. Now they're the only thing standing between an agent and the salary file.