What to Log for AI Agent Activity

9 minute read · Security, privacy and audit review required

Log enough AI agent activity to reconstruct who acted, under which authority, on what exact request, through which policy, and with what observed outcome. Do not put access tokens, payment credentials, private prompts or full sensitive documents into the log. Use stable identifiers, minimal structured fields, digests and controlled evidence references.

The distinction between logs and signed evidence is covered in AI Agent Audit Trails. What follows is the operational event model: the fields engineers need during a denial, timeout or incident.

Use a structured event envelope

type AgentActionEvent = {
  eventId: string;
  eventType: string;
  timestamp: string;
  observedTimestamp: string;
  tenantId: string;
  principalId: string;
  agentId: string;
  transactionId: string;
  traceId?: string;
  action: string;
  resource: string;
  requestDigest: string;
  authorityDigest?: string;
  policyDigest?: string;
  decision?: 'allow' | 'deny' | 'approval_required';
  reasonCodes?: string[];
  executionState?: string;
  externalReference?: string;
  evidenceRefs: string[];
};

OpenTelemetry's log data model defines useful timestamp, severity, trace and attribute concepts. Domain fields such as authority and execution state need explicit local semantics.

Separate facts by source

A model transcript can explain why the agent proposed something. It should not overwrite these system observations.

Minimize sensitive data

Log a destination ID instead of bank details, an invoice digest instead of the document, and a controlled evidence reference instead of raw customer data. Redaction must happen before export. Hashing a low-entropy secret does not make it safe because attackers can guess candidate values.

Test that tokens in headers, tool arguments, model context and provider errors never reach the log sink. Test tenant filtering at query time and storage time.

Define event quality tests

Create fixtures for allow, deny, approval required, dispatch, provider timeout and later reconciliation. Each fixture should include the expected fields and explicitly forbidden data. Validate that all events in one transaction carry the same tenant and transaction identifiers.

Test partial failure. If the policy decision is recorded but the adapter event is missing, the monitoring system should report an evidence gap rather than infer that execution did not happen. Reconcile domain-system writes against gateway events to find unobserved routes.

Use controlled cardinality. Raw prompt text, full URLs with identifiers and arbitrary error messages make logs expensive and unsafe. Put stable reason codes and action names in indexed attributes; place protected details behind an evidence reference with stricter access.

Document clock behavior. Event time and observed time can differ when a worker buffers or exports late. Preserve both and measure source clock offsets for incident reconstruction.

Use audit trail architecture for the pipeline and audit retention and privacy for lifecycle decisions. The OATI Receipt documentation describes portable evidence concepts in developer preview.

Logging does not prove that no bypass existed or that every source claim was true. Preserve those assurance limits in dashboards and audit reports.

Operational logging review

Decide whether prompts belong in the logging system at all. They can help debug planning but may contain personal data, secrets and hostile documents. Transaction evidence should rely on the structured request, source references and enforced decision. If prompts are captured, keep them in a restricted store with a shorter, explicit purpose and retention period rather than the general operational index.

Give each event a unique ID and connect it to tenant, transaction and trace. Use a stable action name and schema version. Record denied requests as well as successful ones, including verified caller, request digest, policy digest and protected reason code. Denials reveal probing, broken clients and working controls. Public error text can remain narrower than the internal decision event.

Maintain a prohibited-field test suite for tokens, private keys, payment credentials, authorization codes and raw sensitive payloads. Run it against headers, tool arguments, model output and provider errors before release. Redaction should occur before export. A downstream filter cannot undo exposure to collectors, alerting tools or temporary buffers that already received the secret.

Measure completeness from the domain system backward. Define required event families for each action class, sample completed operations and confirm identity, authority, decision, dispatch and outcome records resolve. Missing events create an owned evidence gap. Counting messages from the same failing collector does not prove the protected action population is complete.

Use stable, controlled attributes for indexed search and place detailed protected material behind access-controlled references. Arbitrary error messages, full URLs and prompt text create sensitive high-cardinality logs. Preserve event and observed timestamps because queues and exports can arrive late. Measure clock offset where incident ordering matters.

A log becomes a receipt only when it follows a stable evidence schema, binds the protected request and authority, and carries integrity and issuer semantics understood by an independent verifier. Signing an arbitrary log line does not supply business meaning. Keep operational telemetry and portable evidence linked but distinct.

Test the event model with a request that never executes. An agent proposes a payment, policy requires approval, the user rejects it and the workflow ends. The trail should show verified identities, the exact proposal, policy decision and rejection without inventing a dispatch or outcome. This case catches schemas that assume every transaction ends in success or failure at an external provider.

Then test the opposite gap: a protected service changes state but no matching gateway dispatch exists. The reconciliation job should surface an unknown execution path. It should not fabricate a trace ID or quietly attach the write to the nearest agent conversation. That unknown population is how logging helps find bypass, legacy automation and missing instrumentation.

Review log access with the same care as action access. A support or analytics role that can search prompts, identifiers and denial reasons may learn more than the agent itself could access. Partition views, mask protected attributes and record sensitive queries. Evidence is useful only if collecting it does not create a second uncontrolled data path.