Establishing Control Boundaries for AI Agents: The Difference Between Prompts and Policy Enforcement

Two Categories, Not One Guardrail

When an AI agent can touch a customer's record, send a message in someone's name, move money, or alter production data, a confident-sounding system prompt is not a control. It's a suggestion the model may or may not follow, and the moment that agent has real capability, hope stops being an acceptable strategy.

This is the problem Traccia is built around, and it's worth being precise about what that means in practice, because "AI agent governance" gets used to describe a lot of different things. Traccia organizes agent policy enforcement into two categories, borrowed from a much older discipline in security and audit, but applied to the mechanics of AI agents:

• Preventive policy is about stopping an agent from running again once it's shown it shouldn't be trusted to.
• Detective policy is about knowing, reliably, what an agent actually did, whether that's a guardrail firing, a policy rule getting violated, or a gap in coverage nobody noticed.

It's worth being honest about the shape of each of these today, because they're not identical to how "prevention" and "detection" get used in other security contexts. Below is how we think about it, one layer at a time.

What Happens When There's No Real Boundary?

A line that exists only in text is easy to cross without noticing. Here's what crossing an invisible boundary looks like in practice: no exploit, no jailbreak, no malicious prompt. It only requires an agent doing exactly what it was told to do, one authorized step at a time.

Every individual step looks fine in isolation. The read is authorized. The email tool is permitted. Nothing about step 4 is inherently wrong. The violation lives in the path connecting steps 2 and 5, not in any single action. A system that only checks "is this tool call allowed right now" will miss it completely, because it's evaluating requests one at a time instead of evaluating the trajectory the agent is actually on.

This is exactly why instructions like "never send customer data to an external address," buried in a system prompt, aren't enforcement. They can shape a model's behavior, but they don't guarantee an email actually gets stopped. Real enforcement has to sit outside the model.

Preventive Policy: How it Actually Works

Traccia's preventive mechanism is built around two SDK-level functions: observe, which is observability only, and govern, which adds a runtime policy check on top of observability and requires the Traccia platform.

govern calls the Traccia API to check an agent's status before the wrapped function body runs:

from traccia import init, govern
from traccia.governance import AgentBlockedError

init(api_key="...", endpoint="https://api.traccia.ai/v2/traces")

@govern(agent_id="my-agent", fail_open=False, name="run_agent")
def run_agent(prompt: str) -> str:
    return call_llm(prompt)

On the platform side, you define policies as declarative rules: metric, operator, threshold, and, for aggregate rules, a time window and an aggregation function.

Here's the part that matters most for understanding what "preventive" really means in the product today: policy evaluation happens after a trace is ingested: the trace is already stored by the time a rule runs. And Warn or Block don't interrupt the run that triggered them. They arm the next @govern() call. If that next call has fail_open=False, Traccia raises AgentBlockedError before the function body executes, stopping that invocation from starting. Ingest itself never rejects a trace because of a policy.

Two things are useful to know if you're setting this up:

• Simulate Last 7 Days: lets you backtest a draft policy against the last week of traces before activating it, so you can see how many runs would have matched without turning on enforcement blind.
• fail_open on @govern(): is what determines fail-closed behavior. Set it to False if you want a status-check failure or a Block to actually stop the next run, rather than let it through.

Guardrail Detection: A Separate, Passive Layer

Alongside policy enforcement, Traccia ships a guardrail detection engine that runs as an OTel span processor. It's important to be clear about what this is: passive detection, zero runtime enforcement. It doesn't block anything. It classifies signals already present in your traces and tells you what it found, and what it thinks is missing.

Findings come in three tiers, each with different confidence:

Seeing something that looks like a denial isn't the same as having a guardrail, which is why Tier C never counts toward coverage on its own.

When a trace's root span ends, a missing-guardrail evaluator infers what the agent's capabilities were (does it call an LLM, handle user text, use tools) and reports which guardrail categories you'd expect but didn't detect, each with its own confidence level. You can suppress specific categories for agents where they genuinely don't apply, like internal batch pipelines.

Detective Policy: Violations, Not Automated Remediation

On the detective side, when a policy rule is exceeded, Traccia records a violation, not an automatic remediation action.

A later compliant run doesn't retroactively close an earlier open one. That's deliberate. Every evaluation, matched or not, is written to a Decision Log, including quiet "No Match" rows, so you can confirm a rule is actually running rather than silently broken.

To be clear about what's not here yet: there's no automated credential revocation, data quarantine, or rollback built into the product today. Remediation, once a violation is Open, is a human workflow: review it in the Governance Hub, acknowledge or resolve it, and act outside the system if the situation calls for it.

Where This Sits in the Broader Platform

All of the above is one piece of a larger idea: the same trace that tells you what an agent did should be the thing that makes it governable, not a separate system bolted on afterward. That's what we mean when we call Traccia an agent control plane: a single layer of visibility, control, and evidence sitting under every agent you run. Traccia is organized around four pillars that build on one another:

The SDK itself is open source and OpenTelemetry-native, so it works standalone against your own backend (Jaeger, Grafana Tempo, Zipkin, or any OTLP-compatible collector), with the hosted platform layered on top when you want enforcement, evaluations, and governance evidence.

The Bottom Line

Guardrail Detection tells you what protections actually fired, or should have existed but didn't, inside a single run. Policy enforcement tells you when an agent's behavior, aggregated or per-run, has crossed a line you defined, and can stop that agent from running again until someone looks at it.

Neither one, by itself, gives you a system that inspects and blocks an individual tool call mid-flight based on its specific arguments. If that's the kind of control your use case genuinely needs, it's worth knowing that today, going into a design.

That's what Traccia gives you today: Guardrail Detection surfaces what's protected and what's missing inside a run, and Preventive policy, through govern and @govern(), gives you a circuit breaker to stop the next invocation once a violation has been recorded. It's not a black box you only open after something's gone wrong. The same trace data drives detection, policy evaluation, and the evidence trail behind both. The goal isn't confidence in the model. It's confidence in the system that's actually watching it.