AI Agent Governance Framework: From Policy to Runtime Controls
An AI agent governance framework that maps ownership, inventory, authority, runtime controls, evidence, incident response and review to measurable tests.
An AI agent governance framework defines who owns each agent, which work it may perform, how that authority is enforced, what evidence is retained and how the organization stops or changes it. Model review alone is insufficient because an agent's risk also comes from tools, credentials, delegated authority, live context and downstream systems.
Intelliger's canonical page for this subject is Agent Trust. This framework turns governance principles into operating controls.
Use four linked functions
NIST's AI Risk Management Framework organizes work around Govern, Map, Measure and Manage. Apply those functions to the agent as an operational system:
| Function | Agent question | Required artifact |
|---|---|---|
| Govern | Who is accountable and who may approve change? | owner, risk tier, RACI, policy exception |
| Map | What can it reach and affect? | agent, tool, data, authority and dependency inventory |
| Measure | Does the control behave under attack and failure? | evals, negative tests, denial and evidence metrics |
| Manage | How is risk accepted, reduced or stopped? | release gate, incident runbook, revocation and review |
Do not run these as separate documents. A material action in the inventory should point to its policy, tests, approval rule, evidence schema and incident owner.
Create a control register
agent: ap-worker-3
owner: finance-automation
risk_tier: material-write
actions:
- supplier-payment.create
authority:
max_amount_minor: 2500000
currency: EUR
destinations: verified-registry-only
controls:
- exact-request-policy
- independent-approval-over-100000
- atomic-idempotency
- outcome-reconciliation
evidence:
schema: action-receipt-v1
retention_class: finance-7y-review-required
kill_switch: revoke-agent-and-block-service-identity
The example is illustrative. Retention and approval rules must come from the organization's legal and risk owners.
Measure control effectiveness
Count deployed agents only with context. A board needs to know what fraction of material actions are behind enforced policy, how quickly authority can be revoked, whether evidence is complete and how many exceptions have expired without remediation.
Test the control, not the prose. Change an approved payment destination, replay a tool call, remove the policy bundle and call the upstream directly. Record whether the protected operation occurred.
Apply the framework to one action
Start with a single consequential verb, such as supplier-payment.create, rather than trying to govern an entire assistant at once. Map who may request it, which agent may prepare it, which system owns supplier truth, where policy runs, when approval is required and which system can confirm settlement. This exposes missing ownership faster than a general risk questionnaire.
Next, trace every execution route. A policy enforced in the primary gateway is ineffective if a scheduled job, internal load balancer or old service credential can call the payment adapter directly. The framework should name the protected service and the control that makes the approved route mandatory.
Finally, sample evidence in both directions. Select a domain-system transaction and find its authority and policy decision. Then select a decision record and find the protected operation and later outcome. One-way traceability often hides missing provider or system-of-record evidence.
Review the framework after a new tool, model, data source or delegation path is introduced. The risk can change even when the agent's name and stated purpose remain the same.
The governance operating model assigns ownership. Governance dashboard metrics defines formulas. OWASP's Agentic Security Initiative provides agent-specific threat material for the test library.
Intelliger provides developer-preview schemas and reference enforcement paths, not legal compliance certification. Review the live platform overview for the enterprise control model and obtain independent advice for regulated use.
Governance framework questions
Does every agent need the same governance process?
No. Use a common inventory and ownership standard, then select controls by impact, autonomy, data sensitivity and reversibility. A public research assistant may need source and privacy review. A payment agent needs transaction authority, exact approval, idempotency and reconciliation. Record why the lighter profile applies so teams cannot self-classify material work as an experiment indefinitely.
How is an agent different from a model in the risk register?
The model is one component. The agent also has tools, memory, credentials, orchestration, data sources, policies and downstream effects. Register the operational agent system and link its component versions. A model update can trigger review, but so can a new tool, broader service account or changed supplier-data feed.
What is the minimum governance artifact?
At minimum, record purpose, accountable owner, lifecycle state, risk class, reachable tools and data, delegated authority, enforced policy, test evidence, incident path and review date. For a material action, add approval, idempotency, outcome and retention rules. The artifact should resolve to live system identifiers rather than a presentation slide.
Who approves the framework itself?
The organization should name an accountable risk authority and consult business, legal, security, platform, audit and records owners. Domain teams define material facts and acceptable failure. The central group defines common vocabulary, mandatory gates and escalation. Review the framework after incidents and major regulatory or architecture changes.
How does the framework handle third-party agents?
Inventory the external operator, identity, hosting boundary, tools, data, credentials, subcontractors, change process and exit plan. Require evidence for runtime controls rather than accepting a questionnaire alone. Decide how the organization revokes access, exports audit records and investigates an incident when it does not operate the runtime.
What proves governance is active?
Evidence includes recent control tests, resolved ownership, enforced policy coverage, sampled transaction reconstruction, exception expiry and containment drills. Meeting frequency and policy count can support governance but do not show runtime effect. Compare protected system actions with gateway and evidence records to find unknown execution paths.
The framework should also record disagreement. If the business owner accepts a failure mode that security recommends blocking, preserve the exact action, affected dependency, compensating control, approving authority and expiry. Removing that tension from the document makes governance look cleaner while leaving operators without a decision during an outage. An explicit, expiring exception is safer than an unwritten understanding that only appears when the primary path fails.