Risk Scoring: Beyond Binary Allow/Deny

The Problem with Binary Decisions

Traditional authorization systems return one of two answers: allow or deny. You either have permission or you do not. This model works for simple access control. Can this user read this file? Yes or no.

AI agent governance is different. An agent's action might be low-risk in one context and high-risk in another. Deploying code to staging is routine. Deploying the same code to production during a peak traffic period is high-risk. Reading a customer record for support resolution is expected. Reading 10,000 customer records in an hour is anomalous.

Binary decisions force you to choose between two bad options. You can set permissive policies that allow most actions, accepting the risk of bad outcomes. Or you can set restrictive policies that deny borderline cases, sacrificing the productivity gains that agents are supposed to deliver.

Risk scoring gives you the middle ground. Instead of allow or deny, you get a risk score that drives graduated responses: allow, allow with conditions, escalate for human review, or deny. The decision is proportional to the risk.

The Eight Risk Dimensions

Intended evaluates risk across eight independent dimensions. Each dimension captures a different aspect of the action's risk profile, and each is scored independently before being combined into a composite score.

1. Scope of Impact

How many systems, users, or resources are affected by this action? An action that modifies a single configuration value has narrow scope. An action that modifies a network-wide security policy has broad scope. An action that deploys code to all production instances has maximum scope.

Scope is evaluated based on the resource type, the action type, and the resource metadata from the organization's asset inventory. A database query scoped to a single table scores lower than a query scoped to an entire database. A deployment to one service scores lower than a deployment to the entire infrastructure.

2. Reversibility

Can this action be undone? Some actions are trivially reversible: changing a configuration value back, rolling back a deployment, restoring from backup. Others are irreversible: deleting data without backup, sending an email, executing a financial transaction.

Reversibility is determined by the action type and the target system. Delete operations on systems without soft-delete score high. Create operations score low (you can delete what you created). Update operations score based on whether the system maintains version history.

3. Data Sensitivity

What data does this action access, modify, or expose? Actions involving PII, financial data, health records, or credentials score higher than actions involving non-sensitive operational data.

Data sensitivity is determined by the resource classification in the organization's data catalog. If the organization has tagged their databases and APIs with sensitivity levels, Intended uses those tags. If not, Intended applies default sensitivity rules based on resource type and naming conventions.

4. Environment Classification

Is this action targeting production, staging, development, or a sandbox environment? The same action in different environments has fundamentally different risk profiles. Dropping a test database in development is routine maintenance. Dropping a database in production is a potential disaster.

Environment classification comes from the connector configuration. Each connected system is tagged with its environment. The Authority Engine checks the environment tag against the intent's target resource to determine the environment context.

5. Regulatory Implications

Does this action have compliance implications? Actions that touch regulated data (GDPR, HIPAA, PCI-DSS, SOX) score higher on the regulatory dimension. Actions that generate evidence of compliance (audit exports, control testing) score lower.

Regulatory implications are determined by the domain pack configuration. Each domain pack includes regulatory context for its action categories. The financial operations domain pack, for example, flags actions involving transaction processing as having SOX implications.

6. Historical Patterns

Has this agent performed this action before? An agent executing a routine action it performs daily scores lower than an agent executing an action for the first time. A sudden change in an agent's behavior pattern is a risk signal.

Historical patterns are evaluated using the agent's action history. Intended maintains a rolling window of each agent's recent actions, including action types, frequencies, success rates, and escalation rates. Actions that deviate significantly from the historical baseline score higher on this dimension.

7. Velocity

How quickly is this agent performing actions? An agent executing one action per minute is operating normally. An agent executing 100 actions per minute might be malfunctioning, compromised, or operating outside its intended parameters.

Velocity is measured as actions per unit time, segmented by action type and domain. Each agent has a baseline velocity derived from its historical patterns. Actions that exceed the baseline by more than two standard deviations score higher on this dimension.

8. Agent Trust Level

What is this agent's established trust level? New agents start with a lower trust level and build trust over time through consistent, appropriate behavior. Agents with a history of escalations or policy violations have lower trust levels. Agents with long track records of compliant operation have higher trust levels.

Trust level is a persistent attribute of each agent, updated after every governance decision. Approved actions with no escalations increase trust. Escalated actions that are ultimately approved by humans have no effect. Denied actions and policy violations decrease trust.

Composite Scoring

The eight dimension scores are combined into a composite risk score using a weighted average. The weights are configurable per organization, per domain, and per policy. An organization with strict data protection requirements might weight data sensitivity at 2x. An organization operating in a heavily regulated industry might weight regulatory implications at 3x.

The default weights are equal across all dimensions, but few organizations use the defaults. During onboarding, Intended works with each customer to calibrate weights based on their risk priorities.

The composite score is a value between 0 and 100. The score maps to governance outcomes through configurable thresholds. The default thresholds are: 0-25 maps to allow (green zone), 26-50 maps to allow with conditions (yellow zone), 51-75 maps to escalate for human review (orange zone), and 76-100 maps to deny (red zone).

Conditions and Constraints

The "allow with conditions" outcome is unique to risk-based governance. When an action falls in the yellow zone, it is approved but with constraints attached to the authority token.

Common conditions include time-boxing (the action must be completed within a shorter TTL), scope narrowing (the action is approved for a subset of the requested resources), notification (a human is notified that the action was approved), and enhanced audit (additional detail is captured in the audit record).

Conditions are encoded in the authority token and enforced at the execution perimeter. The agent cannot strip conditions from the token because the conditions are part of the signed payload.

Dynamic Recalibration

Risk scores are not static formulas. Intended continuously recalibrates scoring based on outcomes. If actions with scores in the 40-50 range are consistently approved by human reviewers without modification, the system learns that those actions are lower-risk than the score suggests. The weights adjust to reflect observed reality.

This feedback loop means the risk scoring model improves over time. Early in deployment, the model is conservative and escalates frequently. As it accumulates data about the organization's actual risk tolerance, it calibrates to match, reducing unnecessary escalations while maintaining protection against genuine risk.