Cost Attribution & Optimization

Both

Reduce LLM costs with Traccia insights and SDK features.

LLM costs can add up quickly in production. Agent pricing is calculated locally without outbound API calls, powered by a continuously updated remote registry of 2,500+ models. Traccia records cost as an OpenTelemetry metric independently of trace sampling.

Step 1: Understand Your Costs

Use Traccia's automatic cost tracking to identify where money is being spent:

enable_costs.py
python
from traccia import init


# Enable cost calculation (enabled by default)
init(
    enable_token_counting=True,
    enable_costs=True
)


# Costs are automatically calculated and added to spans as:
# - llm.usage.prompt_tokens
# - llm.usage.completion_tokens
# - llm.usage.total_tokens
# - llm.cost.usd (in USD)

Key Questions to Answer:

  • • Which agents cost the most?
  • • Which models are most expensive?
  • • Are there unexpectedly high token counts?
  • • Do costs spike at certain times?

Step 2: Reduce Trace Volume with Sampling

Sampling reduces observability costs by only exporting a percentage of traces. This doesn't reduce LLM costs, but it reduces backend and storage costs.

sampling.py
python
from traccia import init


# Development: Capture everything
init(sample_rate=1.0)


# Production: Sample 10% (good for high-volume systems)
init(sample_rate=0.1)


# Critical agents: Higher sampling
init(sample_rate=0.5)
10% sampling:Good for high-volume production systems (10,000+ traces/day)
50% sampling:Good for moderate volume with decent coverage
100% sampling:Good for development and low-volume production

Sampling trade-offs

Lower sampling rates reduce storage and backend costs but may miss rare errors or edge cases. Balance cost savings against observability needs.

Step 3: Control Peak Load with Rate Limiting

Rate limiting caps the maximum spans exported per second, protecting your backend and controlling costs during traffic spikes:

rate_limiting.py
python
from traccia import init


# Limit to 100 spans/second
init(
    max_spans_per_second=100,
    max_block_ms=100,  # Block up to 100ms before dropping
    max_queue_size=5000  # Max buffered spans
)
traccia.toml
toml
[rate_limiting]
max_spans_per_second = 100.0
max_block_ms = 100
max_queue_size = 5000

When to use rate limiting

Use rate limiting when you have unpredictable traffic spikes or want to guarantee a cost ceiling on tracing infrastructure.

Step 4: Use Custom Pricing Tables

Traccia includes default pricing for common models, but you can override with your negotiated rates or custom pricing:

bash
# Set custom pricing via environment variable
export AGENT_DASHBOARD_PRICING_JSON='{
  "gpt-4": {"input": 0.025, "output": 0.050},
  "gpt-3.5-turbo": {"input": 0.0005, "output": 0.0015},
  "claude-3-opus": {"input": 0.015, "output": 0.075}
}'

Or via programmatic override:

custom_pricing.py
python
from traccia import start_tracing


custom_pricing = {
    "gpt-4": {"input": 0.025, "output": 0.050},
    "gpt-3.5-turbo": {"input": 0.0005, "output": 0.0015},
}


start_tracing(
    pricing_override=custom_pricing,
    pricing_refresh_seconds=3600  # Refresh pricing every hour
)

Optimization Strategies

1. Model Selection

Use cheaper models for simple tasks. GPT-3.5-turbo is ~10x cheaper than GPT-4.

python
# Expensive: GPT-4 for everything
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": simple_query}]
)


# Optimized: GPT-3.5 for simple tasks
response = client.chat.completions.create(
    model="gpt-3.5-turbo",  # 10x cheaper
    messages=[{"role": "user", "content": simple_query}]
)

2. Prompt Optimization

Shorter prompts = fewer tokens = lower costs. Use Traccia to identify verbose prompts.

python
# Check token counts in traces to find verbose prompts
# Look for llm.token_count.prompt > 2000


# Before: Verbose system prompt (500 tokens)
system_prompt = """
You are an expert assistant with deep knowledge...
[500 words of instructions]
"""


# After: Concise prompt (100 tokens)
system_prompt = "You are a helpful assistant. Be concise."

3. Response Length Limits

Set max_tokens to prevent unexpectedly long completions.

python
# Without limit: Could generate 4000 tokens ($0.24)
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": query}]
)


# With limit: Max 500 tokens ($0.03)
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": query}],
    max_tokens=500  # Cap completion length
)

4. Caching and Deduplication

Cache responses for common queries to avoid repeated LLM calls.

python
from functools import lru_cache
from traccia import observe


@observe()
@lru_cache(maxsize=1000)
def cached_agent(query: str) -> str:
    # Repeated queries hit cache instead of LLM
    return call_expensive_llm(query)


# First call: Goes to LLM ($0.02)
result1 = cached_agent("What is AI?")


# Second call: Cache hit ($0.00)
result2 = cached_agent("What is AI?")

5. Disable Features Selectively

If you don't need token counting or cost tracking in certain environments, disable them to reduce overhead:

python
# Development: Keep all features
init(
    enable_token_counting=True,
    enable_costs=True
)


# Production with external cost tracking: Disable
init(
    enable_token_counting=False,  # Save ~1-5ms per LLM call
    enable_costs=False
)

Real-World Optimization Example

Here's how one team reduced their monthly LLM costs from $5,000 to $1,200:

1

Identified top 3 expensive agents

Using the Costs dashboard, found 3 agents responsible for 70% of spend

2

Switched to GPT-3.5 for simple tasks

Analysis showed 60% of queries were simple enough for GPT-3.5 (saved $2,500/month)

3

Reduced prompt verbosity

Trimmed system prompts from 800 to 200 tokens (saved $700/month)

4

Added caching for FAQ queries

40% of queries were duplicates (saved $600/month)

Result: 76% cost reduction

$5,000/month → $1,200/month

SDK-Level Cost Controls

Sampling for High-Volume Systems

high_volume.py
python
from traccia import init


# Production config: Sample 5% to control costs
init(
    api_key="tr_live_xxxxxxxxxxxx",
    sample_rate=0.05,  # Only 5% of traces exported
    max_spans_per_second=50  # Cap at 50 spans/sec
)


# Even with 100,000 agents runs/day, you'll only export:
# - 5,000 traces (5% sampling)
# - This reduces backend costs by 95%

Conditional Instrumentation

conditional.py
python
from traccia import observe
import os


# Only trace in development
is_dev = os.getenv("ENV") == "development"


if is_dev:
    from traccia import init
    init(enable_console_exporter=True)
    
    @observe()
    def my_function():
        pass
else:
    # No tracing overhead in production
    def my_function():
        pass

Monitoring Costs Over Time

Use the file exporter to track costs locally and create custom reports:

cost_tracking.py
python
from traccia import init
import json


# Export to file for analysis
init(
    enable_file_exporter=True,
    file_exporter_path="costs.jsonl"
)


# After running your agents, analyze costs
def analyze_costs(file_path="costs.jsonl"):
    total_cost = 0
    cost_by_agent = {}
    cost_by_model = {}
    
    with open(file_path) as f:
        for line in f:
            span = json.loads(line)
            attrs = span.get("attributes", {})
            
            cost = attrs.get("llm.cost.usd", 0)
            agent = attrs.get("agent.id", "unknown")
            model = attrs.get("llm.model", "unknown")
            
            total_cost += cost
            cost_by_agent[agent] = cost_by_agent.get(agent, 0) + cost
            cost_by_model[model] = cost_by_model.get(model, 0) + cost
    
    print(f"Total: ${total_cost:.2f}")
    print(f"By agent: {cost_by_agent}")
    print(f"By model: {cost_by_model}")


analyze_costs()

Next Steps

Costs Dashboard

Use the platform for visual cost analytics

Set Up Cost Policies

Create alerts for cost thresholds

Advanced SDK Features

Sampling, rate limiting, and more

© 2026 Traccia.