Traccia emits OpenTelemetry (OTLP) metrics alongside traces so you can measure token usage, latency, cost, and agent behavior. These metrics work across plain OpenAI/Anthropic calls, LangChain, CrewAI, and the OpenAI Agents SDK.

Two main layers of metrics

LLM/client metrics – one data point per LLM call (tokens, cost, duration, errors).
Agent metrics – one data point per agent run (runs, execution time, error rate).

Enabling metrics

Metrics are enabled by default when you call init(). You can control the endpoint and sampling via traccia.tomlor parameters to init().

traccia.toml

toml

[tracing]
endpoint = "http://localhost:4318/v1/traces"

[metrics]
enable_metrics = true
# When talking directly to OTLP/HTTP (e.g. trace-injection-service),
# the SDK defaults to {traces_base}/v2/metrics.
# For OTEL Collector + Prometheus, point at the collector metrics endpoint:
metrics_endpoint = "http://localhost:4318/v1/metrics"
metrics_sample_rate = 1.0

main.py

python

from traccia import init

# Simple: traces + metrics to the same OTLP collector
init(
    endpoint="http://localhost:4318/v1/traces",
    enable_metrics=True,
    metrics_endpoint="http://localhost:4318/v1/metrics",
    metrics_sample_rate=1.0,
)

Built-in LLM metrics

For each LLM call, Traccia emits OTLP metrics using the gen_ai.client.* naming convention. These are populated by the OpenAI/Anthropic monkey patches, LangChain callback, CrewAI LLM wrapper, and OpenAI Agents SDK integration.

Token usage

gen_ai.client.token.usage (OTLP histogram) – prompt and completion tokens per call.

Latency

gen_ai.client.operation.duration (OTLP histogram, seconds) – end-to-end LLM call duration.

Cost

gen_ai.client.operation.cost (OTLP histogram, USD) – cost per call based on model pricing.

Exceptions

gen_ai.client.completions.exceptions (OTLP counter) – number of failed LLM calls.

Prometheus metric names

When you export metrics through the OTEL Collector's Prometheus exporter, these names appear with underscores and suffixes. For example:

gen_ai_client_token_usage_bucket, _sum, _count
gen_ai_client_operation_duration_seconds_bucket, _sum, _count
gen_ai_client_operation_cost_usd_bucket, _sum, _count

Agent metrics

On top of raw LLM calls, Traccia emits higher-level gen_ai.agent.* metrics for frameworks that expose an explicit agent lifecycle (e.g. CrewAI and OpenAI Agents SDK).

Agent runs

gen_ai.agent.runs (OTLP counter) – one increment per agent/crew run.

Execution time

gen_ai.agent.execution_time (OTLP histogram, seconds) – wall-clock time per agent run.

Framework coverage

Framework	LLM metrics	Agent metrics
Plain OpenAI / Anthropic	Yes – via OpenAI / Anthropic instrumentation.	No built-in agent concept (you can still add custom metrics).
LangChain	Yes – via LangChain callback handler (plus underlying OpenAI/Anthropic patches).	Only if you build an agent abstraction on top (not emitted by default).
CrewAI	Yes – LLM calls wrapped under agent / crew spans.	Yes – agent / crew runs and execution time via CrewAI integration.
OpenAI Agents SDK	Yes – via `llm.openai.responses` spans and OTLP metrics.	Yes – agent runs and execution time via the Agents tracing processor.

Custom metrics

You can emit your own counters and histograms using traccia.metrics.

Counters

Use record_counter() to track how often something happens (cache hits, tool failures, etc.).

metrics_counters.py

python

from traccia.metrics import record_counter

def handle_tool_call(tool_name: str, success: bool) -> None:
    # Count every tool invocation
    record_counter(
        name="tool_invocations",
        value=1,
        attributes={"tool_name": tool_name},
    )

    # Separate success/failure counters
    record_counter(
        name="tool_results",
        value=1,
        attributes={"tool_name": tool_name, "status": "success" if success else "error"},
    )

Histograms

Use record_histogram() to measure distributions like latency, payload size, or custom scores.

metrics_histogram.py

python

import time
from traccia.metrics import record_histogram

def run_rag_search(query: str) -> str:
    start = time.perf_counter()
    result = do_search(query)
    duration = time.perf_counter() - start

    record_histogram(
        name="rag_search_latency",
        value=duration,
        attributes={"index": "kb_docs"},
        unit="s",
    )
    return result

Metric types

Counters only ever increase (events per label set).
Histograms record sampled values into buckets so you can compute averages and quantiles later.

Prometheus & Grafana examples

When exporting through the OTEL Collector's Prometheus exporter, you can use queries like:

prometheus.queries

promql

# Total cost (USD)
sum(gen_ai_client_operation_cost_usd_sum)

# Total tokens
sum(gen_ai_client_token_usage_sum)

# LLM calls over time (5m buckets)
sum(increase(gen_ai_client_operation_duration_seconds_count[5m]))

# Agent runs by name (from Agents / CrewAI integrations)
sum(gen_ai_agent_runs_total) by (gen_ai_agent_name)

# P95 LLM latency (seconds, last 24h)
histogram_quantile(
  0.95,
  sum(increase(gen_ai_client_operation_duration_seconds_bucket[24h])) by (le)
)

Sampling and sparsity

If your traffic is low, prefer increase(metric[1h]) over short windows like rate(metric[5m]) so that histograms have enough data for accurate quantiles.