Unapproved Model Alert

Your team agreed on two models. Then someone swaps in a new one to test something and forgets to change it back. Unapproved Model Alert checks the model on every trace and opens a violation when it is not on your list.

The template starts with an allowed list of gpt-4o and gpt-4o-mini, with Critical severity. You can change every value.

What Happens

This runs after the trace is accepted, so the run that triggered it is never undone. Observe records the violation. Warn also warns the next governed run. Block stops the next governed run. Ingest never rejects a trace because of a policy.

Set It Up

1. In the app, open Policies and click Create Policy.
2. Choose Unapproved Model Alert under After Run.
3. Give the policy a name and pick its scope: the organization, a workspace, or one Agent ID.
4. Leave Metric on LLM Model Name. Under Condition, Not In List means the models you type are the ones you allow, and In List means the models you type are the ones you want flagged. Type the model names separated by commas, for example gpt-4o, gpt-4o-mini.
5. Pick a mode. Observe only records. Warn and Block act on the next governed run.
6. Turn on Email Alerts if you want mail when a violation opens.
7. On Review, run Simulate Last 7 Days to see how often it would have matched, then click Activate Policy.

Good To Know

• The check compares the model name recorded on the trace, so write names the way your provider reports them.
• Unapproved Model Alert tells you after the fact. To deny the call before it reaches the provider, use Model Boundary.
• This is a Per Trace policy. There is no window, and every trace is checked on its own.