AI Agent Governance Dashboard: Metrics Boards Can Use
Build an AI agent governance dashboard around authority coverage, control tests, evidence completeness, exceptions, incidents and containment time.
An AI agent governance dashboard should show exposure and control performance, not how many agents the company launched. Board metrics need denominators, risk tiers, trends and named remediation. "98% governed" is meaningless unless governed has a testable definition.
Agent Trust describes the controls behind the dashboard. The metric set below is designed for decisions: where exposure exists, which evidence is missing and who must act.
Start with five board measures
| Metric | Formula | Board question |
|---|---|---|
| material-action control coverage | material actions behind tested enforced policy / total material actions | Which actions can bypass policy? |
| evidence completeness | sampled material actions with required evidence / sample | Can we reconstruct what occurred? |
| authority hygiene | active grants with owner, expiry and review / active grants | Where is standing authority unmanaged? |
| expired exception backlog | critical expired exceptions still open | Which accepted risks are now unauthorized? |
| containment time | detection to enforced revocation, p50 and p95 | How quickly can we stop an agent? |
Report numerator and denominator. Split material writes from low-risk reads so a large read workload cannot hide an ungoverned payment action.
Define metrics in code
metric: material_action_control_coverage
population: production agent actions classified material-write
pass_when:
- enforced_gateway: true
- policy_test_age_days <= 30
- direct_route_test: passed
owner: enterprise-agent-platform
refresh: daily
target: 100_percent
Include freshness. A control that passed nine months ago before three tool changes should not count as current coverage.
Add operator metrics below the board view
Track denial reason distribution, approval mutation failures, replay attempts, stale registry records, policy-bundle age, receipt export lag, uncertain execution backlog and reconciliation age. A sudden drop in denials can indicate a bypass or missing telemetry, not safer behavior.
OpenTelemetry's log data model supports trace correlation and structured attributes. It does not define business authority or evidence completeness; create those semantics explicitly.
Prevent dashboard gaming
Sample source records, reconcile gateway counts with domain-system actions, and show unknown data. Do not silently remove an agent from the denominator because its owner field is blank. The absence is the risk.
Set thresholds that trigger action
A dashboard without escalation rules becomes reporting theatre. For material actions, control coverage and evidence completeness may need a 100 percent target because one uncovered route is the exposure. Other measures, such as median approval latency, can use bands. The risk owner should document why each threshold is tolerable.
Show both current value and oldest unresolved item. Ten recent uncertain payments may be less concerning than one that has been unreconciled for two weeks. Similarly, report p95 containment time alongside the median; a fast routine case can hide a slow regional gateway or cached credential.
Use an explicit unknown category. Missing telemetry should never be counted as pass or fail without evidence. Unknown results need an owner and an expiry for investigation.
Board reporting should connect metrics to decisions. If action coverage falls after a new integration, the report should name the affected workflow, interim restriction and remediation date. If an incident reveals a new bypass class, change the denominator and restate prior periods where practical rather than presenting an artificial improvement.
The governance framework defines the control register. The agent inventory schema provides denominator data. NIST's AI RMF supports the continuing measurement and management cycle.
Intelliger's public materials demonstrate evidence and control concepts in developer preview. Dashboard targets and regulatory interpretations belong to the deploying organization's risk owners. Review AI agent audit trails before defining evidence coverage.
Dashboard design questions
Should the dashboard count agents or actions?
Use both, but make protected actions the exposure denominator. One agent may have a harmless read tool and a material payment tool. An agent-level green status can hide the latter. Group actions by risk, owner and enforced path, then let users drill back to the agent and service.
How should denied actions be interpreted?
Denials can show that policy works, users misunderstand boundaries or attackers are probing them. Track reason, action class and trend. A decrease is not automatically good; missing telemetry or a bypass can also reduce denials. Compare with domain-system operations and gateway traffic.
What is evidence completeness?
Define required events for each action class, such as verified identity, authority, policy decision, dispatch and outcome. Sample completed domain actions and test whether every required record resolves and matches. A receipt with an unresolved provider outcome may pass integrity but fail outcome completeness.
How are exceptions shown?
Show open count, risk, oldest age, expiry, owner and affected action coverage. Separate accepted current exceptions from expired or unowned ones. Do not bury a critical bypass exception in a total dominated by low-risk documentation gaps.
Which containment measure matters?
Measure from accountable decision to enforced denial at the last protected service, including cached grants, queued jobs and child agents. Report median, p95 and slowest unresolved case. Administrative UI response time is not a containment measure unless it proves execution stopped.
How should metrics be audited?
Version each definition and query, preserve source lineage and sample underlying records. Reconcile independent populations where possible. When a definition changes, explain the impact and restate history or mark the break. Dashboard access and edits should also be logged because the view informs risk decisions.
Add a short narrative only where the number cannot explain the decision. If control coverage fell from 100 percent to 92 percent because one newly discovered service bypasses the gateway, say which service and what restriction is active. Avoid model-generated summaries that smooth over unknowns or infer causes from correlation. The metric owner should sign off on the explanation and link the incident, exception or remediation record.
Keep the raw denominator visible in exports. A percentage can improve because agents were retired, telemetry failed or the definition narrowed. Showing counts by risk class lets a reviewer tell whether exposure fell or the measurement population changed.