operator runbooks

Intended Documentation

Incident Response

Contain and investigate an authority incident — query the SHA-256 audit chain, inspect and verify an Ed25519 decision token, and use the emergency-control actions (tenant/environment halt, token revoke) to stop new approvals. No fictional kill-switch or incident CLI.

9 min read

Incident Response#

This runbook covers containing and investigating an authority incident: reading the audit chain, inspecting the decision token behind a specific outcome, and using the emergency-control actions to stop new approvals. Containment first; investigation second.

Danger

During an active incident, prioritize containment over investigation. If agents are acting in ways they should not, halt the affected scope first (tenant or environment), then investigate. The emergency-control actions are the real stop mechanism — see Step 3.

What does and does not exist

There is no intended emergency kill/lift/status, no intended incident create/report/update/list, and no intended audit trace. Real verbs: audit-query and audit-export (CLI), inspect-token and verify (CLI), and the emergency-control API POST /admin/emergency-controls/:action. Decision tokens are Ed25519 (EdDSA) JWTs, not RS256 or ES256.

Severity and first action#

Severity	Description	First action
Critical	Unauthorized or dangerous execution in production	`environment_halt` immediately, then investigate.
High	Authorization failures blocking critical services	Check recent deploys; roll back if correlated.
Medium	Unexpected decision pattern (e.g. deny/escalate spike)	Query the audit chain for the window.
Low	Audit anomaly, non-blocking	Investigate within the working day.

Step 1 — Query the audit chain#

Every decision, approval and reported outcome writes to a per-tenant, SHA-256 hash-chained audit log. Start every investigation by querying the affected window.

$intended audit-query

Query the tenant's hash-chained audit entries.

--tenantstring *

Tenant id (or configure tenantId / use --session).

--correlationstring

Filter by correlationId.

--event-typestring

Filter by event type (e.g. AUTHORITY_DECISION).

--limitstring

Max entries (default 100).

--offsetstring

Pagination offset.

bash

# Via the API
curl "https://api.intended.so/tenants/tenant_acme_prod/audit?tenantId=tenant_acme_prod&eventType=AUTHORITY_DECISION&limit=100" \
  -H "Authorization: Bearer $INTENDED_API_KEY" \
  -H "x-tenant-id: tenant_acme_prod"

# Or via the CLI
intended audit-query --tenant tenant_acme_prod --event-type AUTHORITY_DECISION --limit 100

Confirm the chain is intact#

A run of unexplained denials, or any suspicion of tampering, warrants a chain-integrity check. The verifier recomputes each entry's content hash and its linkage to the prior entry.

bash

curl "https://api.intended.so/tenants/tenant_acme_prod/audit/chain-verification?tenantId=tenant_acme_prod" \
  -H "Authorization: Bearer $INTENDED_API_KEY" \
  -H "x-tenant-id: tenant_acme_prod"
# → { "valid": true, "totalEntries": 12840 }  (or "brokenAt" if linkage fails)

Tip

A deny/escalate spike clustered on one policy shortly after a deploy almost always means a policy regression. Cross-reference GET /policy/packs/:pack/versions and roll back — see Deploy and Rollback.

Step 2 — Inspect the decision token#

When a specific intent was APPROVED, the runtime minted an Authority Decision Token — an Ed25519 JWT, 300-second TTL (default and max), single-use via a unique nonce. Inspect it to see exactly what was authorized, and verify it to confirm it is genuine and unexpired.

A token is minted only on APPROVED, carries the tenant, adapter, action and a single-use nonce, is verified against the tenant's Ed25519 public key by kid, and is consumed exactly once by the service that verifies it before acting. Inspection decodes its claims; verification checks the Ed25519 signature, expiry, and tenant/adapter binding. — The decision token in an incident

Decode the claims#

inspect-token decodes the JWT payload without verifying — fast for reading what a token claims.

$intended inspect-token

Decode an Authority Decision Token's claims (no signature check).

--tokenstring *

The Ed25519 JWT to decode.

bash

intended inspect-token --token "$DECISION_TOKEN"
# → { intentId, tenantId, adapterId, adapterTarget, targetSystem,
#     proposedAction, decision: "APPROVED", issuedAt, expiresAt, nonce,
#     iss: "intended-authority", aud: "<adapterId>", iat, exp }

Verify it cryptographically#

To confirm the token is genuine, unexpired, and bound to the expected tenant/adapter, verify it against the tenant's Ed25519 public key with kid pinning.

$intended verify

Verify an Authority Decision Token (Ed25519, kid-pinned) against a public key.

--tokenstring *

The JWT to verify.

--keystring *

Path to the tenant public key PEM.

--kidstring

Expected key id to pin.

--tenantstring

Expected tenant id.

--adapterstring

Expected adapterId claim (decision-only for /v1/agent-intent decisions).

bash

intended verify \
  --token "$DECISION_TOKEN" \
  --key ./tenant-public.pem \
  --kid "tenant_acme_prod-1717761600-a1b2" \
  --tenant tenant_acme_prod
# exit 0 → valid; exit 2 → invalid (bad signature, expired, or binding mismatch)

Note

Intended never runs the action itself; the service your agent called is expected to have verified the token before acting, failing closed on a bad signature, expiry, tenant binding or reused nonce. Re-verifying during an incident tells you whether the token itself was sound, separating a policy problem from a token problem.

Step 3 — Contain with emergency controls#

The real stop mechanism is the emergency-control API, POST /admin/emergency-controls/:action. It requires the emergency:invoke permission (held only by owner and admin), a required reason recorded to the event ledger, and the usual tenant-match. Full reference: Emergency Controls.

Halt the affected scope

bash

# Stop a whole environment
curl -X POST https://api.intended.so/admin/emergency-controls/environment_halt \
  -H "Authorization: Bearer $INTENDED_API_KEY" \
  -H "x-tenant-id: tenant_acme_prod" \
  -H "Content-Type: application/json" \
  -d '{
    "tenantId": "tenant_acme_prod",
    "environmentKey": "production",
    "reason": "Suspected unauthorized execution (INC-2026-0342)"
  }'

Action	Use when
`tenant_halt`	The whole tenant must stop.
`environment_halt`	Contain one environment (needs `environmentKey`).
`token_revoke`	A credential is compromised (needs `targetRef` = credential id).

Revoke a compromised credential

bash

curl -X POST https://api.intended.so/admin/emergency-controls/token_revoke \
  -H "Authorization: Bearer $INTENDED_API_KEY" \
  -H "x-tenant-id: tenant_acme_prod" \
  -H "Content-Type: application/json" \
  -d '{
    "tenantId": "tenant_acme_prod",
    "targetRef": "cred_123",
    "reason": "Credential compromise suspected (INC-2026-0342)"
  }'

Resume once contained

After the root cause is remediated, lift the halt with the matching resume action (environment_resume, tenant_resume). The handler is fail-closed: if the mutation fails, the event is still recorded with status: failed.

bash

curl -X POST https://api.intended.so/admin/emergency-controls/environment_resume \
  -H "Authorization: Bearer $INTENDED_API_KEY" \
  -H "x-tenant-id: tenant_acme_prod" \
  -H "Content-Type: application/json" \
  -d '{ "tenantId": "tenant_acme_prod", "environmentKey": "production", "reason": "Root cause remediated; prod-release-controls rolled back to v0." }'

Danger

A halt denies every new intent across the scope, including legitimate operations. Use it when the risk of further approvals outweighs the impact of a full stop, scope as narrowly as the incident allows (prefer environment_halt over tenant_halt when one environment is at fault), and always supply a meaningful reason — it is required and lands in the event ledger.

Step 4 — Investigate and resolve escalations#

If the incident involved intents that were ESCALATED, work the escalation queue rather than leaving them pending.

bash

intended escalations-list --tenant tenant_acme_prod
intended escalations-reject --id esc_123 --reason "Tied to INC-2026-0342; denying pending review"

Use escalations-approve only for escalations you have confirmed are legitimate and unrelated to the incident.

Step 5 — Evidence and post-mortem#

Export a self-contained evidence bundle for any intent central to the incident. The bundle's integrityHash is a SHA-256 over its events; its signature is an HMAC-SHA256 keyed off the tenant's own secret — so it is tamper-evident and verifiable by anyone holding that tenant secret, not publicly/asymmetrically verifiable. State that precisely in your write-up.

bash

intended audit-export --tenant tenant_acme_prod --intent intent_abc123
# → EvidenceBundle { intentId, events[], integrityHash, signature }

Root cause	Indicators	Resolution
Policy regression	Deny/escalate spike after a deploy	Roll back, fix the draft, re-simulate, redeploy.
Credential compromise	Unexpected actor in the audit chain	`token_revoke`, rotate, tighten scope.
Token problem	`verify` fails on a token that ran	Investigate the signer/key rotation; check `kid`.

Incident records

There is no built-in incident-ticket store or intended incident … command. Track the incident in your own system of record and attach the exported evidence bundle and the emergency-control event log.

Incident checklist#

[ ] Assess severity; notify on-call.
[ ] If critical: environment_halt (or narrower) immediately.
[ ] Query the audit chain for the window; verify chain integrity.
[ ] Inspect and verify the decision tokens behind anomalous outcomes.
[ ] Cross-reference recent policy deploys; roll back if correlated.
[ ] Contain: emergency control + rollback as needed.
[ ] Work the escalation queue.
[ ] Export evidence bundles; write the post-mortem.
[ ] Resume the halted scope once remediated; verify the runtime is healthy.

Next steps#

Emergency Controls — the full containment-action reference.
Deploy and Rollback — roll back the deploy behind a regression.
Decision Token Model — what the Ed25519 token proves and why.