Just-in-Time Access for Production AI Agents
Use just-in-time access to run production AI agents without standing credentials, with scoped mandates, deterministic policy and short-lived capabilities.

Just-in-time access lets a production AI agent act without holding a standing credential. An incident agent can read an alert, inspect a deployment and propose a repair in seconds. The dangerous part starts when it can also reach a cluster with a reusable credential.
Most teams try to make that credential safer. They put it in a secret store, rotate it, hide it from logs and tell the model not to reveal it. Those controls matter, but they do not fix the basic design error. A general agent session still has a reusable capability that may outlive the decision that justified its use.
A better design keeps the production credential outside the agent. The agent proposes an operation. A deterministic enforcement path checks identity, delegated authority, ticket context, policy and approval. Only then does a credential broker obtain a short-lived capability for the exact target. The gateway executes the approved request and records what happened.
This article develops that pattern for a Kubernetes remediation agent. The same architecture fits cloud administration, secret rotation, database maintenance and managed-service access.
Start just-in-time access with a transaction, not a session
Suppose an alert says that payments-api has entered a crash loop in the production cluster. An agent investigates and proposes a rollback to image digest sha256:4c2... under incident INC-4821.
The system should not ask a vague question such as "May this agent administer production?" It should decide on a concrete transaction:
{
"transactionId": "txn_01J8K7V3T6",
"agentId": "agent:ops:remediator-7",
"action": "kubernetes.deployment.rollback",
"resource": "cluster/prod-eu/namespace/payments/deployment/payments-api",
"parameters": {
"imageDigest": "sha256:4c2...",
"replicaCount": 6
},
"purpose": "incident-remediation",
"ticket": "INC-4821",
"requestedAt": "2026-08-10T09:42:11Z"
}
That envelope gives the policy engine something stable to evaluate. It also prevents a common security failure: approving a human-readable intention, then executing a materially different API call.
Canonicalise the envelope before signing or hashing it. Bind the approval, temporary credential and final receipt to the same digest. If the image, namespace, replica count or ticket changes, the digest changes and the earlier decision cannot be reused.
Separate proposal, authorisation and execution
The agent belongs in the proposal path. It can interpret alerts, retrieve runbooks, compare recent deployments and explain why a rollback may help. It should not make the final access decision or mint its own credentials.
Use three distinct components:
- The agent runtime builds a proposed transaction from approved tools and observations.
- A gateway and policy decision point validate the transaction against deterministic constraints.
- A credential broker obtains a temporary capability and an executor calls the production API.
Alert and ticket
|
v
Investigation agent
|
| proposed transaction, no production secret
v
Gateway -> identity + mandate + policy + approval
|
| authorised request digest
v
Credential broker -> short-lived capability
|
v
Executor -> Kubernetes or cloud API
|
v
Result + signed action receipt
Keep these trust domains separate even if the first implementation runs them in one cluster. The agent process should not have network access to the secret store's administrative interface. The credential broker should accept only an authorised, signed request from the gateway. The executor should reject a capability whose target or lifetime does not match the operation.
Express authority as a bounded mandate
Identity tells you which agent made the request. It does not tell you why that agent may alter payments-api.
Represent delegated authority in a short-lived mandate. The following YAML is illustrative rather than an OATI schema definition:
mandateId: mandate_inc_4821
principal: org:example:platform-operations
subject: agent:ops:remediator-7
purpose: incident-remediation
validFrom: 2026-08-10T09:30:00Z
expiresAt: 2026-08-10T11:00:00Z
constraints:
actions:
- kubernetes.deployment.get
- kubernetes.pod.logs.read
- kubernetes.deployment.rollback
resources:
- cluster/prod-eu/namespace/payments/deployment/payments-api
ticketIds:
- INC-4821
maintenanceWindow:
start: 2026-08-10T09:30:00Z
end: 2026-08-10T11:00:00Z
maxReplicaCount: 8
allowedImageDigests:
- sha256:4c2...
maxExecutions: 1
delegation:
allowed: false
The mandate should be narrow enough that a stolen copy has little value. It should also be machine-checkable. Phrases such as "take reasonable corrective action" belong in a runbook, not in the final authorisation object.
If one agent delegates investigation to another, derive the child authority by intersecting every constraint set with the parent. Missing constraints must not mean unlimited access. A child mandate may shorten expiry, remove actions and narrow resources. It may never add a namespace, action, destination or budget.
Put deterministic policy on the execution path
The policy engine receives verified identity, the mandate, the canonical transaction and current environmental facts. It returns a structured result such as allow, deny, transform or approval_required.
For example:
deny if passport.status != "active"
deny if mandate.expiresAt <= now
deny if transaction.purpose != mandate.purpose
deny if transaction.ticket not in mandate.ticketIds
deny if transaction.action not in mandate.actions
deny if transaction.resource not in mandate.resources
deny if transaction.parameters.imageDigest not in mandate.allowedImageDigests
deny if transaction.parameters.replicaCount > mandate.maxReplicaCount
deny if executionState.reserve(transaction.transactionId, transaction.digest) fails
approval_required if resource.environment == "production"
approval_required if change.riskScore >= configuredThreshold
allow otherwise
The risk score, if you use one, may inform routing. Do not let an opaque model score override hard constraints. The safest policy result is explainable in terms of the input facts and a versioned rule bundle.
Apply transformations before execution and include them in the final digest. A gateway may clamp a timeout, remove an unapproved field or replace a human-readable resource alias with a resolved identifier. If the transformation changes the action's meaning, route it back for a new decision instead of quietly continuing.
Bind human approval to the exact operation
"Approve remediation" is not a sufficient approval record. The approver should see the target, action, parameter diff, evidence, policy version, expiry and request digest.
{
"approvalId": "apr_01J8K8A7H2",
"transactionDigest": "sha256:ab91...",
"decision": "approved",
"approver": "user:platform:oncall-12",
"role": "production-change-approver",
"expiresAt": "2026-08-10T10:05:00Z",
"reason": "Rollback matches incident runbook and last known good digest"
}
The executor must reject the approval if the transaction digest differs, if the approval expired or if the approver was also the principal who issued the authority when separation of duties applies. This blocks a subtle context-swap attack in which an agent obtains approval for a harmless read and attaches it to a write.
Mint a capability the agent cannot reuse
After authorisation, the gateway asks a credential broker for a capability. The broker may call a cloud security token service, a Vault-like secret system or an internal credential service. The implementation varies, but the contract should stay small:
type CredentialRequest = {
transactionDigest: string;
subject: string;
audience: string;
action: string;
resource: string;
expiresInSeconds: number;
proofKeyThumbprint?: string;
};
type TemporaryCapability = {
handle: string;
expiresAt: string;
audience: string;
resource: string;
proofBinding?: string;
};
Prefer an opaque handle over returning a bearer token to the agent runtime. The executor can redeem the handle once, inside a restricted network boundary. If the upstream supports proof of possession, bind the credential to an executor-held key through DPoP or mTLS. A copied token is then insufficient on its own.
The capability lifetime should reflect execution time, not the length of the incident. Five minutes may be appropriate for one API call while the mandate remains valid for ninety minutes. Those are different controls with different purposes.
Never put the upstream credential in model context, tool output, traces or a receipt. Store a credential class, issuer, audience, expiry and non-secret reference if investigators need to reconstruct how execution occurred.
Constrain the executor too
The executor is a security boundary, not a generic HTTP client. Give it explicit adapters for approved operations. A rollback adapter might accept only a deployment identifier and image digest. It should not accept arbitrary Kubernetes manifests or shell commands.
The reservation moves through explicit states such as reserved, executing and completed. An approval wait does not consume it. A failed broker call can release it according to policy, while an ambiguous upstream result stays reserved until reconciliation.
async function executeRollback(tx: AuthorizedTransaction) {
assert(tx.action === "kubernetes.deployment.rollback");
assert(tx.approval.transactionDigest === tx.digest);
assert(await executionState.begin(tx.id, tx.digest));
const credential = await broker.redeemOnce(tx.credentialHandle, tx.digest);
// `kubernetes` is an internal typed adapter, not a Kubernetes client API.
const result = await kubernetes.rollback({
deployment: tx.resource,
imageDigest: tx.parameters.imageDigest,
replicaCount: tx.parameters.replicaCount,
credential,
});
const normalised = normaliseResult(result);
await executionState.complete(tx.id, normalised.operationReference);
return normalised;
}
Do not expose a general run_shell(command) tool and hope policy catches every dangerous string. Typed adapters reduce the number of meanings a request can acquire between approval and execution.
Network policy should reinforce the application controls. The agent runtime can reach the gateway, but not the Kubernetes API. The executor can reach the relevant cluster endpoint, but not unrelated infrastructure. The broker can reach the credential issuer, but does not need general outbound access.
Record the result without overstating it
After execution, create a signed action receipt that binds the agent, organisation, mandate, transaction digest, policy decision, approval, executed operation, response digest, external provider reference, time and idempotency data.
For a one-company deployment, this is unilateral evidence. It proves what the deploying organisation's enforcement system recorded and signed. It does not prove that Kubernetes reported the truth, that the service recovered, or that another company agreed with the record. If the counterparty or target system later signs the result, the assurance level can rise, but the distinction must remain explicit.
Outcome verification belongs after the action. A rollback returning HTTP 200 is not the same as restored service. Run separate checks against health, error rate and traffic recovery. Link those observations to the transaction without rewriting the original receipt.
Design the failure path before the happy path
Production agents become interesting when dependencies fail halfway through a transaction.
The control plane is unavailable
Keep signed policy bundles and short-lived trust state near the gateway. Material actions should fail closed if the gateway cannot validate revocation, policy or replay state according to the configured freshness limits. Do not turn a central outage into an implicit bypass.
The replay store is unavailable
Fail the write. A one-time mandate or transaction cannot be enforced if the system cannot claim and consume its identifier. Retrying should reuse the same idempotency key and retrieve the prior result where possible.
The approval arrives after the mandate expires
Deny it. Approval does not extend delegated authority. Issue a new mandate and evaluate a new transaction.
The credential is minted but execution times out
Treat the result as unknown, not failed. Query the target system using the idempotency key or operation reference before retrying. A blind retry can turn a network timeout into a duplicate change.
The target mutates between decision and execution
Use resource versions or preconditions where the upstream supports them. If the deployment revision changed after approval, stop and build a new transaction. The old decision described a different state.
The model proposes an action outside the runbook
Deny it predictably and return a structured reason. The agent can ask for a broader human-authored mandate, but it cannot rewrite policy through prompt text.
Test attacks, not only workflows
A useful test suite should attempt to:
- change the namespace after approval;
- replace the approved image digest;
- replay the same transaction identifier;
- redeem a credential from another executor;
- use a capability after expiry;
- bypass the gateway and reach the target directly;
- substitute another ticket with a similar description;
- execute during a trust or replay-store outage;
- inject instructions through alert text or pod logs;
- approve and execute with the same principal where separation of duties is required.
Assert both the denial and the evidence. A blocked request should still produce a decision record with a reason code, policy version and correlation identifier, while excluding secrets and unnecessary payload data.
Implementation checklist
- Model every consequential request as a canonical transaction.
- Keep production credentials outside the agent process and model context.
- Use short-lived mandates with explicit actions, resources, purpose and expiry.
- Evaluate hard constraints in deterministic code or policy.
- Bind approvals to the exact transaction digest.
- Broker task-specific capabilities only after an allow decision.
- Prefer one-time opaque handles and proof-bound credentials.
- Use typed executors instead of general shell or HTTP tools.
- Enforce the path with network controls so agents cannot bypass the gateway.
- Claim transaction identifiers before execution and preserve idempotency across retries.
- Treat timeouts as unknown outcomes until reconciled.
- Verify operational outcomes separately from API success.
- Sign a receipt that states its assurance level.
- Test revocation, outages, stale state, replay and context substitution.
Related engineering guides
- Define degraded behavior with fail-open vs fail-closed rules for enterprise AI agents.
- Preserve the complete control path with an auditable AI agent transaction architecture.
Where OATI fits
OATI models this flow as Passport, Mandate, Transaction, Policy, Approval, Credentials, Connect and Receipt responsibilities. The public developer framework already includes core objects, canonicalisation, signatures, deterministic evaluation, Receipt tooling and a reference Envoy enforcement path. It remains a developer preview pending independent protocol and implementation review and additional production gates.
The hardened gateway fleet, full credential-broker operations, durable evidence service and customer outage exercises are target commercial capabilities, not completed production claims. That boundary is deliberate. The useful architectural idea does not depend on a product promise: an enterprise remediation agent should receive authority for one transaction, while the credential and final decision stay outside the model.