Governing a predictive model means making sure its numbers stay valid. Governing generative AI and agents is different: here the system writes on the company’s behalf and takes actions on its systems. The risk is no longer an estimation error — it’s an email sent, an order created, or data that left through the wrong door.
It’s the newest governance layer and the one being built fastest, almost always after the technology is already in the building.
Start with what’s already happening without permission
Before writing any policy, look at reality. In most companies, people are already using generative AI tools on their own, with personal accounts, pasting in internal documents. Banning it by memo doesn’t work — it just makes it invisible. Useful governance starts by offering an approved path that’s better than the shadow one, and measuring how much activity goes through each.
The six controls
What data can leave, and where to. A minimal classification — public, internal, confidential, personal — and an explicit rule for which category can be processed by which service. Without it, no policy can actually be enforced.
What an agent can execute. An inventory of permitted actions, scoped permissions, and a clear line between what runs on its own and what needs human sign-off: payments, deletions, external communications. This is the part built into the harness.
Spending limits. A cap per run and per day, with alerts. One unexpected loop can burn through a quarter’s budget overnight.
Audit trails and retention. What gets logged for each interaction, how long it’s kept, and who can access it. Without a trail, you can’t investigate an incident or respond to a complaint.
Measured quality, not assumed quality. A generative system degrades when instructions or models change, even if nobody touches the code. Evaluation and observability turn “it seems to work” into data.
Transparency with people. When users are told they’re talking to a system, how generated content is labeled, and how they can request human review. It’s a regulatory requirement — and, before that, a matter of trust.
How it differs from the other two layers
| Data governance | Machine learning governance | Generative AI and agent governance | |
|---|---|---|---|
| Question | Is what I have reliable? | Is what I infer still valid? | What is it saying and doing on my behalf? |
| Typical risk | Decisions made on dirty data | A model drifting with nobody noticing | Data leaks and unwanted actions |
| Key control | Quality, lineage, and ownership | Validation, drift, and retraining | Permissions, limits, and audit trails |
| Cadence | Continuous | Review cycles | Every run |
Each layer rests on the one below: there’s no reliable agent on top of ungoverned models, and no reliable model on top of ungoverned data.
Where it meets regulation
The EU AI Act sets transparency obligations for systems that interact with people or generate content, and much heavier obligations if the system takes part in high-risk decisions. Add the GDPR as soon as personal data enters the context. The timeline and details are in AI compliance and governance.
When to set it up
Now, if there’s any use of generative AI in your company — approved or not — or any agent with access to real systems. Of the three governance layers, this is the one that can almost never wait, because the risk isn’t accuracy but exposure.
It can wait only if use is confined to a closed pilot, with no real data and no actions on systems.
How we approach it
We start by mapping actual use — including unapproved use — and by setting a minimal data classification, because without it everything else is just good intentions. Then we define the boundary of permitted actions and the limits, and we instrument audit trails and evaluation. The written policy comes last, once we know what can actually be complied with.
Frequently asked questions
Isn’t an AI usage policy enough? No. A policy nobody can enforce or verify doesn’t change behavior. Technical controls — what leaves, what runs, what gets logged — are what hold the policy up.
Should we ban personal tools? Banning without an alternative pushes usage where you can’t see it. It works better to offer a convenient approved path and measure the split between the two.
Who should be accountable for this? Someone in senior management, not just IT. Decisions about what data leaves and what an agent can do on the company’s behalf are business and risk decisions.