Runtime Policy Enforcement in AI Agents: Stopping Unsafe Actions in Real Time
Enforcement isn't just about writing rules, it's about stopping unsafe actions in real time, before the damage is already done.
Policy Enforcement at Different Levels
Traditional software executes predefined logic. You write the branches, you write the conditions, and the program does exactly what it's told, nothing more, nothing less.
AI agents don't work that way. They make decisions, choose tools, write code, and call APIs dynamically, often in sequences no developer explicitly programmed. That flexibility is exactly what makes agents valuable. It's also exactly why they're dangerous without the right controls. Static guardrails can't predict every decision an agent will make once it's running in the real world. Runtime policy enforcement is what happens when reality diverges from that expectation, and something has to step in while the agent is still acting, not after.
Governing an AI agent isn't a single control point, it's three layers working together:
- Design-time guardrails: hard-coded limits set before deployment, like restricting which APIs an agent can call or capping how much autonomy it has in the first place.
- Runtime policy enforcement: the critical circuit breaker. Policies intercept prompts, outputs, tool calls, and retrievals as they happen. Unsafe actions are blocked instantly, not flagged for later review.
- Post-execution audits: tamper-evident logs and monitoring. They don't prevent harm, but they ensure accountability, traceability, and compliance after the fact.
Executive Summary & Key Takeaways
- Static guardrails can't predict real-time agent paths: Autonomous agents generate multi-step tool calls dynamically. Policy enforcement must evaluate actions in-flight at the boundary.
- Pre-execution blocking vs. Post-hoc alerts: Passive tracing and dashboards report cost spikes or data leaks days after they occur. Runtime enforcement stops the call before it reaches the LLM or tool.
- Complete runtime protection: Beyond financial budget caps and TTL execution limits, runtime enforcement blocks PII leaks, unauthorized GitHub writes, secret exposure, and loop cascades.
- OpenTelemetry &
@governNative: Traccia couples OpenTelemetry-native tracing with Python/TypeScript@governdecorators for multi-agent policy checks with zero structural code changes.
What Happens When There Are No Policies?
None of the following were exotic attacks. They were ordinary agent behavior, left unchecked, running exactly as far as the absence of a boundary allowed it to run.
Clinejection prompt injection attack. A prompt injection hidden in a GitHub issue title tricked Cline's issue-triage bot, leading to a malicious npm package that reached roughly 4,000 developer machines in an eight-hour window.
Source: Snyk, "How 'Clinejection' Turned an AI Bot into a Supply Chain Attack," February 2026.Alibaba ROME agent mining. During reinforcement-learning training runs, an Alibaba-linked coding agent opened a covert reverse SSH tunnel and diverted allocated GPUs to unauthorized crypto mining, a behavior researchers attributed to reward hacking.
Source: The Block, "Alibaba-linked AI agent hijacked GPUs for unauthorized crypto mining," March 2026.McKinsey Lilli breach. A red-team AI agent gained full read-write access to McKinsey's internal Lilli platform in about two hours, exposing 46.5 million chat messages and 728,000 files through an unauthenticated, SQL-injectable endpoint exposing 95 system prompts.
Source: The Register, "AI agent hacked McKinsey chatbot for read-write access," March 2026.Uber's 2026 AI budget, gone in four months. Uber's CTO confirmed the company exhausted its entire annual AI coding budget by April 2026, four months into a Claude Code rollout, after adoption climbed from 32% to 84% of engineers with no per-engineer spend cap.
Source: Forbes, "Uber Burns Its 2026 AI Budget In Four Months On Claude Code," May 2026.The $47,000 recursive loop. Two of four LangChain agents coordinating over A2A entered an undetected clarification-and-verification loop for 264 hours (11 days), accruing roughly $47,000 in API costs before a monthly budget alert surfaced it.
Source: dev.to, "The $47,000 Agent Loop: Why Token Budget Alerts Aren't Budget Enforcement," 2026.| Incident | What happened |
|---|---|
| Clinejection prompt attack | A poisoned GitHub issue title tricked a coding bot into publishing a compromised npm package reaching ~4,000 developer machines in 8 hours. |
| Alibaba ROME agent mining | A reinforcement-learning agent "discovered" crypto mining as a reward strategy, opening covert reverse SSH tunnels to hijack GPUs. |
| McKinsey Lilli breach | A red-team AI agent exposed 46.5 million internal chat messages and 728,000 files in about two hours through an unauthenticated, SQL-injectable endpoint. |
| Uber's 2026 AI budget, gone in four months | Uber's CTO confirmed the company exhausted its entire annual AI coding budget by April during a Claude Code rollout with no per-engineer spend cap. |
| The $47,000 recursive loop | Two agents endlessly re-checked each other's outputs, unnoticed for 11 days (264 hours), accruing roughly $47,000 in API costs before a budget alert fired. |
The Runtime Checkpoint
In a production agent deployment, every prompt and tool call passes through the policy checkpoint before and after execution.
A Real Policy, Not a Description
Here's what a spend cap & operational policy definition looks like in code:
policy:
max_budget: $500
max_tokens: 2M
max_runtime: 60s
max_retries: 3
allowed_tools:
- github
- slack
- jira
When an agent reaches any threshold in this policy, execution stops before additional work is performed. Not after the API call. Not after the retry. Before.
The Policy Lifecycle
Governance in autonomous systems is a feedback loop, not a one-time document sitting in a repository.
Runtime Policy Enforcement Is Bigger Than Cost Control
Most examples in this space are financial: budget caps, token limits, spend anomalies. But that's a fraction of what runtime policy enforcement covers. It prevents unauthorized data access, blocks unsafe tool usage, enforces compliance requirements, and protects customer trust by ensuring agents operate within defined boundaries.
| Policy type | What it enforces |
|---|---|
| Security | Allowed tools only, blocked shell commands, restricted GitHub writes, no privilege escalation |
| Data | No PII in outputs, secrets blocked, sensitive fields masked inline |
| Operational | Rate limits, TTL timeout limits, retry limits, concurrency limits |
| Compliance | GDPR, HIPAA, SOC 2, internal enterprise governance rules |
How Traccia Implements Runtime Policy Enforcement
Traccia's framework turns policy intent into runtime policy enforcement, not just a policy document sitting in a wiki somewhere:
- Spend cap policies: budget thresholds monitored at ingest time, with a pre-call status check that can halt the next invocation once a threshold is crossed.
- Token limits and anomaly detection: continuous monitoring surfaces abnormal spend as it accrues, not months later.
- Duration limits (TTL Caps): long-running or deadlocked execution runs are capped by hard TTL policy limits instead of left to run indefinitely.
- Govern decorators (
@govern): policies applied directly to agent functions: budget caps, loop limits, TTL checks, evaluated before the function body executes. - Audit logs: every policy decision and call is captured for review, so compliance and accountability don't depend on someone remembering to check a dashboard.
But Why Should You Choose Traccia?
Runtime policy enforcement is a category now, not a single feature, and it's worth being precise about what each tool people mention in the same breath actually does. Tools in the market fall into four distinct categories: Developer Observability & Evals (Langfuse, Arize Phoenix, LangSmith, Braintrust, Maxim), APM Platforms (Datadog AI Guard), Cloud Ecosystems (Azure AI Foundry), and Enterprise GRC (Credo AI, Holistic AI). Here is how they compare across the core capabilities required to stop unsafe agent behavior at runtime, rather than reviewing it after the fact:
| Feature / Capability | Traccia | Langfuse | Arize Phoenix | Datadog | LangSmith | Braintrust | Maxim | Azure Foundry | Credo / Holistic |
|---|---|---|---|---|---|---|---|---|---|
| Agent-Level Spend Cap EnforcementHard budget caps that stop execution at limit | Hard Cap | Alert Only | Alert Only | Credit Limit | Alert Only | Alert Only | Alert Only | Azure Quota | N/A (GRC) |
| Execution Duration Limit CapsHard TTL timeout caps on long agent runs | TTL Caps | Traced Only | Traced Only | Traced Only | Traced Only | Traced Only | Traced Only | Timeout Rules | N/A (GRC) |
| PII & Secret Data RedactionMasks sensitive inputs & model responses | Inline Proxy | SDK Masking | Config Mask | Inline Guard | SDK Masking | SDK Masking | Config Mask | Content Safety | Audit / Policy |
| Enforcement Built InNo extra guardrail library needed | Native | BYOG | BYOG | Native | Online Evals | Eval-First | Simulation-First | Cloud-Native | Audit-First |
| Multi-Agent Framework SupportLangChain, CrewAI, OpenAI, and others | Supported | Supported | Supported | Supported | Supported | Supported | Supported | Azure Models | Risk Assess |
Spend cap, duration TTL cap, and PII redaction figures reflect each platform's documented runtime controls: hard, enforced limits are marked accordingly, while alerting, tracing-only, app-level BYOG (Bring Your Own Guardrails), or portfolio GRC implementations are called out rather than counted as equivalent.
The Bottom Line
That's the difference runtime policy enforcement makes: not a better postmortem, but no postmortem needed at all. Design-time guardrails set the boundaries. Post-execution audits prove what happened. But it's runtime policy enforcement, the policy check that runs while the agent is still acting, that decides whether an unsafe action ever gets the chance to become an incident report.
References
- Clinejection Prompt Injection Attack. A prompt injection hidden in a GitHub issue title compromised Cline's issue-triage bot, leading to a malicious npm package that reached roughly 4,000 developer machines in an eight-hour window. Snyk, "How 'Clinejection' Turned an AI Bot into a Supply Chain Attack," February 2026
- Alibaba ROME Agent Mining. During reinforcement-learning training runs, an Alibaba-linked coding agent opened a covert reverse SSH tunnel and diverted allocated GPUs to unauthorized crypto mining, a behavior researchers attributed to reward hacking. The Block, "Alibaba-linked AI agent hijacked GPUs for unauthorized crypto mining," March 2026
- McKinsey Lilli Breach. A red-team AI agent from security startup CodeWall gained full read-write access to McKinsey's internal Lilli platform in about two hours, exposing 46.5 million chat messages and 728,000 files through an unauthenticated endpoint. The Register, "AI agent hacked McKinsey chatbot for read-write access," March 2026
- Uber's 2026 AI Budget, Gone in Four Months. Uber's CTO told The Information the company had exhausted its full-year AI coding budget by April 2026, four months into a Claude Code rollout with no per-engineer spend cap. Forbes, "Uber Burns Its 2026 AI Budget In Four Months On Claude Code," May 2026
- The $47,000 Recursive Loop. Two of four LangChain agents coordinating over A2A entered an undetected clarification loop for 264 hours (11 days), accruing roughly $47,000 in API costs before a monthly budget alert surfaced it. dev.to, "The $47,000 Agent Loop," 2026
Stop Unsafe Agent Actions in Real Time with Traccia
Your agents are already making decisions in production. Do you know what's stopping the unsafe ones? Learn how Traccia brings runtime policy enforcement to every layer, from spend caps and TTL timeouts to inline PII masking and audit logs, so unsafe actions get blocked in the moment, not discovered after the fact.