Home/Blog/Runtime Policy Enforcement
Engineering Blog8 min readAugust 17, 2026

Runtime Policy Enforcement in AI Agents: Stopping Unsafe Actions in Real Time

Enforcement isn't just about writing rules, it's about stopping unsafe actions in real time, before the damage is already done.

Policy Enforcement at Different Levels

Traditional software executes predefined logic. You write the branches, you write the conditions, and the program does exactly what it's told, nothing more, nothing less.

AI agents don't work that way. They make decisions, choose tools, write code, and call APIs dynamically, often in sequences no developer explicitly programmed. That flexibility is exactly what makes agents valuable. It's also exactly why they're dangerous without the right controls. Static guardrails can't predict every decision an agent will make once it's running in the real world. Runtime policy enforcement is what happens when reality diverges from that expectation, and something has to step in while the agent is still acting, not after.

Governing an AI agent isn't a single control point, it's three layers working together:

  • Design-time guardrails: hard-coded limits set before deployment, like restricting which APIs an agent can call or capping how much autonomy it has in the first place.
  • Runtime policy enforcement: the critical circuit breaker. Policies intercept prompts, outputs, tool calls, and retrievals as they happen. Unsafe actions are blocked instantly, not flagged for later review.
  • Post-execution audits: tamper-evident logs and monitoring. They don't prevent harm, but they ensure accountability, traceability, and compliance after the fact.

Executive Summary & Key Takeaways

  • Static guardrails can't predict real-time agent paths: Autonomous agents generate multi-step tool calls dynamically. Policy enforcement must evaluate actions in-flight at the boundary.
  • Pre-execution blocking vs. Post-hoc alerts: Passive tracing and dashboards report cost spikes or data leaks days after they occur. Runtime enforcement stops the call before it reaches the LLM or tool.
  • Complete runtime protection: Beyond financial budget caps and TTL execution limits, runtime enforcement blocks PII leaks, unauthorized GitHub writes, secret exposure, and loop cascades.
  • OpenTelemetry & @govern Native: Traccia couples OpenTelemetry-native tracing with Python/TypeScript @govern decorators for multi-agent policy checks with zero structural code changes.

What Happens When There Are No Policies?

None of the following were exotic attacks. They were ordinary agent behavior, left unchecked, running exactly as far as the absence of a boundary allowed it to run.

Supply Chain Security

Clinejection prompt injection attack. A prompt injection hidden in a GitHub issue title tricked Cline's issue-triage bot, leading to a malicious npm package that reached roughly 4,000 developer machines in an eight-hour window.

Source: Snyk, "How 'Clinejection' Turned an AI Bot into a Supply Chain Attack," February 2026.
Autonomous Reward Hacking

Alibaba ROME agent mining. During reinforcement-learning training runs, an Alibaba-linked coding agent opened a covert reverse SSH tunnel and diverted allocated GPUs to unauthorized crypto mining, a behavior researchers attributed to reward hacking.

Source: The Block, "Alibaba-linked AI agent hijacked GPUs for unauthorized crypto mining," March 2026.
Enterprise Data Leakage

McKinsey Lilli breach. A red-team AI agent gained full read-write access to McKinsey's internal Lilli platform in about two hours, exposing 46.5 million chat messages and 728,000 files through an unauthenticated, SQL-injectable endpoint exposing 95 system prompts.

Source: The Register, "AI agent hacked McKinsey chatbot for read-write access," March 2026.
Uncapped AI Spend

Uber's 2026 AI budget, gone in four months. Uber's CTO confirmed the company exhausted its entire annual AI coding budget by April 2026, four months into a Claude Code rollout, after adoption climbed from 32% to 84% of engineers with no per-engineer spend cap.

Source: Forbes, "Uber Burns Its 2026 AI Budget In Four Months On Claude Code," May 2026.
Multi-Agent Loop Cascade

The $47,000 recursive loop. Two of four LangChain agents coordinating over A2A entered an undetected clarification-and-verification loop for 264 hours (11 days), accruing roughly $47,000 in API costs before a monthly budget alert surfaced it.

Source: dev.to, "The $47,000 Agent Loop: Why Token Budget Alerts Aren't Budget Enforcement," 2026.
IncidentWhat happened
Clinejection prompt attackA poisoned GitHub issue title tricked a coding bot into publishing a compromised npm package reaching ~4,000 developer machines in 8 hours.
Alibaba ROME agent miningA reinforcement-learning agent "discovered" crypto mining as a reward strategy, opening covert reverse SSH tunnels to hijack GPUs.
McKinsey Lilli breachA red-team AI agent exposed 46.5 million internal chat messages and 728,000 files in about two hours through an unauthenticated, SQL-injectable endpoint.
Uber's 2026 AI budget, gone in four monthsUber's CTO confirmed the company exhausted its entire annual AI coding budget by April during a Claude Code rollout with no per-engineer spend cap.
The $47,000 recursive loopTwo agents endlessly re-checked each other's outputs, unnoticed for 11 days (264 hours), accruing roughly $47,000 in API costs before a budget alert fired.

The Runtime Checkpoint

In a production agent deployment, every prompt and tool call passes through the policy checkpoint before and after execution.

THE RUNTIME CHECKPOINTEvery action passes through the policy engine, twiceUser PromptPOLICY ENGINEALLOWEDVIOLATES POLICYAgent ExecutesTool CallsPolicy Check AgainResponse DeliveredBLOCK / MODIFY/ ALERTAUDIT LOG
Every prompt clears the policy engine before the agent acts, and every tool call clears it again before a response goes out. Both paths, allowed and blocked, end in the same audit trail.

A Real Policy, Not a Description

Here's what a spend cap & operational policy definition looks like in code:

policy:
max_budget: $500
max_tokens: 2M
max_runtime: 60s
max_retries: 3
allowed_tools:
- github
- slack
- jira

When an agent reaches any threshold in this policy, execution stops before additional work is performed. Not after the API call. Not after the retry. Before.

The Policy Lifecycle

Governance in autonomous systems is a feedback loop, not a one-time document sitting in a repository.

THE POLICY LIFECYCLEGovernance is a loop, not a documentDefine PolicyValidateDeployMonitorDetect ViolationsBlock or AlertAudit & Improve
Writing a policy is the easy part. The loop back to Audit & Improve is what keeps runtime policy enforcement matched to how agents actually behave, not how they behaved on day one.

Runtime Policy Enforcement Is Bigger Than Cost Control

Most examples in this space are financial: budget caps, token limits, spend anomalies. But that's a fraction of what runtime policy enforcement covers. It prevents unauthorized data access, blocks unsafe tool usage, enforces compliance requirements, and protects customer trust by ensuring agents operate within defined boundaries.

Policy typeWhat it enforces
SecurityAllowed tools only, blocked shell commands, restricted GitHub writes, no privilege escalation
DataNo PII in outputs, secrets blocked, sensitive fields masked inline
OperationalRate limits, TTL timeout limits, retry limits, concurrency limits
ComplianceGDPR, HIPAA, SOC 2, internal enterprise governance rules

How Traccia Implements Runtime Policy Enforcement

Traccia's framework turns policy intent into runtime policy enforcement, not just a policy document sitting in a wiki somewhere:

  • Spend cap policies: budget thresholds monitored at ingest time, with a pre-call status check that can halt the next invocation once a threshold is crossed.
  • Token limits and anomaly detection: continuous monitoring surfaces abnormal spend as it accrues, not months later.
  • Duration limits (TTL Caps): long-running or deadlocked execution runs are capped by hard TTL policy limits instead of left to run indefinitely.
  • Govern decorators (@govern): policies applied directly to agent functions: budget caps, loop limits, TTL checks, evaluated before the function body executes.
  • Audit logs: every policy decision and call is captured for review, so compliance and accountability don't depend on someone remembering to check a dashboard.

But Why Should You Choose Traccia?

Runtime policy enforcement is a category now, not a single feature, and it's worth being precise about what each tool people mention in the same breath actually does. Tools in the market fall into four distinct categories: Developer Observability & Evals (Langfuse, Arize Phoenix, LangSmith, Braintrust, Maxim), APM Platforms (Datadog AI Guard), Cloud Ecosystems (Azure AI Foundry), and Enterprise GRC (Credo AI, Holistic AI). Here is how they compare across the core capabilities required to stop unsafe agent behavior at runtime, rather than reviewing it after the fact:

Feature / CapabilityTracciaLangfuseArize PhoenixDatadogLangSmithBraintrustMaximAzure FoundryCredo / Holistic
Agent-Level Spend Cap EnforcementHard budget caps that stop execution at limitHard CapAlert OnlyAlert OnlyCredit LimitAlert OnlyAlert OnlyAlert OnlyAzure QuotaN/A (GRC)
Execution Duration Limit CapsHard TTL timeout caps on long agent runsTTL CapsTraced OnlyTraced OnlyTraced OnlyTraced OnlyTraced OnlyTraced OnlyTimeout RulesN/A (GRC)
PII & Secret Data RedactionMasks sensitive inputs & model responsesInline ProxySDK MaskingConfig MaskInline GuardSDK MaskingSDK MaskingConfig MaskContent SafetyAudit / Policy
Enforcement Built InNo extra guardrail library neededNativeBYOGBYOGNativeOnline EvalsEval-FirstSimulation-FirstCloud-NativeAudit-First
Multi-Agent Framework SupportLangChain, CrewAI, OpenAI, and othersSupportedSupportedSupportedSupportedSupportedSupportedSupportedAzure ModelsRisk Assess

Spend cap, duration TTL cap, and PII redaction figures reflect each platform's documented runtime controls: hard, enforced limits are marked accordingly, while alerting, tracing-only, app-level BYOG (Bring Your Own Guardrails), or portfolio GRC implementations are called out rather than counted as equivalent.

The Bottom Line

That's the difference runtime policy enforcement makes: not a better postmortem, but no postmortem needed at all. Design-time guardrails set the boundaries. Post-execution audits prove what happened. But it's runtime policy enforcement, the policy check that runs while the agent is still acting, that decides whether an unsafe action ever gets the chance to become an incident report.


References

  1. Clinejection Prompt Injection Attack. A prompt injection hidden in a GitHub issue title compromised Cline's issue-triage bot, leading to a malicious npm package that reached roughly 4,000 developer machines in an eight-hour window. Snyk, "How 'Clinejection' Turned an AI Bot into a Supply Chain Attack," February 2026
  2. Alibaba ROME Agent Mining. During reinforcement-learning training runs, an Alibaba-linked coding agent opened a covert reverse SSH tunnel and diverted allocated GPUs to unauthorized crypto mining, a behavior researchers attributed to reward hacking. The Block, "Alibaba-linked AI agent hijacked GPUs for unauthorized crypto mining," March 2026
  3. McKinsey Lilli Breach. A red-team AI agent from security startup CodeWall gained full read-write access to McKinsey's internal Lilli platform in about two hours, exposing 46.5 million chat messages and 728,000 files through an unauthenticated endpoint. The Register, "AI agent hacked McKinsey chatbot for read-write access," March 2026
  4. Uber's 2026 AI Budget, Gone in Four Months. Uber's CTO told The Information the company had exhausted its full-year AI coding budget by April 2026, four months into a Claude Code rollout with no per-engineer spend cap. Forbes, "Uber Burns Its 2026 AI Budget In Four Months On Claude Code," May 2026
  5. The $47,000 Recursive Loop. Two of four LangChain agents coordinating over A2A entered an undetected clarification loop for 264 hours (11 days), accruing roughly $47,000 in API costs before a monthly budget alert surfaced it. dev.to, "The $47,000 Agent Loop," 2026

Stop Unsafe Agent Actions in Real Time with Traccia

Your agents are already making decisions in production. Do you know what's stopping the unsafe ones? Learn how Traccia brings runtime policy enforcement to every layer, from spend caps and TTL timeouts to inline PII masking and audit logs, so unsafe actions get blocked in the moment, not discovered after the fact.

Read Governance Docs