Stop A Dangerous Tool

Some tools should not run just because a model decided to call them. High-Risk Tool denies a function by its name, such as delete_account. Dangerous Shell Commands denies a listed tool when the command text contains a string you wrote, such as rm -rf.

Both run before the function body. A denied call shows up in Decision Log as Denied, from the SDK, in Block. The trace is still saved.

Two Different Checks

What You Set

High-Risk Tool needs the function names. The agent sends the function name. Arguments are not read. Dangerous Shell Commands needs the shell function names and the text patterns. The agent sends command, cmd, argv, or code. If that text is missing, the rule is skipped and the tool still runs.

Turn It On

1. Open Policies and click Create Policy.
2. For a function that should never run unsupervised, choose High-Risk Tool and type the name, such as delete_account.
3. For a shell helper, choose Dangerous Shell Commands. Under Block If Command Contains, add rm -rf. Under Applies To, list the tool name, usually shell.
4. Set the mode to Block, then Activate Policy.
5. Wrap the agent with govern() and mark those functions as tools. Both cards say SDK. They do not apply on the Gateway.

If a shell call has no command text, that rule is skipped and the tool still runs. High-Risk Tool does not need arguments. The name is enough to deny.

A Small Ops Agent

This function never reaches a real shell. The policy denies the call before the body runs. Get the API key from app.traccia.ai.

from traccia import init, govern, observe
from traccia.governance import AgentBlockedError

init(
    api_key="...",
    endpoint="https://api.traccia.ai/v2/traces",
    agent_id="ops-shell",
)

@observe(name="shell", as_type="tool")
def shell(command: str) -> dict:
    return {"ok": True, "command": command}

@govern(fail_open=False, name="ops_shell")
def handle_ticket(command: str) -> dict:
    return shell(command)

try:
    handle_ticket("rm -rf /tmp/cache")
except AgentBlockedError as exc:
    print(exc)

The same wrap denies delete_account when a High-Risk Tool policy lists that name. lookup_user is a different name, so it runs.