How to Trace GitHub Copilot Sessions with Traccia?

Why Trace GitHub Copilot? Lessons From Real Sessions

Public incidents and research point to four lessons for any team that lets Copilot act on its code. None of them depends on which observability tool you use, so each one ends with what a trace adds.

1. An agent's summary can leave out what it did

In October 2026, Adversa AI published research on the GitHub Copilot CLI running in autopilot mode. Instructions shipped as encrypted text inside content the agent fetched got it to decrypt them in its own shell, read a .env.prod file and send the contents to a remote endpoint. The chain took 28 seconds and never asked for confirmation. The agent's closing summary described the outbound request as a harmless check, and the transcript never named the host that received the data. GitHub validated the behavior but did not classify it as a vulnerability, because the user had granted autonomous permissions and asked the agent to fetch that content.

The model mattered as well. In Adversa's tests, Microsoft's mai-code-1.1-flash completed the chain in half of its runs while two GPT-5.6 models refused, and with Auto model selection the user does not pick which model answers.

2. Approval prompts can be switched off without anyone noticing

In August 2025, security researcher Johann Rehberger showed that prompt injection in a source file or web page could get Copilot's agent mode in VS Code to add "chat.tools.autoApprove": true to the workspace's .vscode/settings.json. That setting turns off confirmations, so the commands that followed ran without asking, and the edit was written to disk immediately. Microsoft fixed it as CVE-2025-53773 in that month's Patch Tuesday release.

The Copilot CLI had its own version in March 2026. Under CVE-2026-29783, commands that hid code inside bash parameter expansion, such as ${var@P}, passed the CLI's read-only check and ran without approval. Copilot CLI 0.0.423 fixed it.

3. How fast it feels is a poor measure

In GitHub's 2022 controlled study, 95 developers were split into two groups to write an HTTP server in JavaScript, and the group using Copilot finished 55% faster. In 2025, METR ran a randomized trial with 16 experienced open-source developers on 246 real tasks in their own repositories, mostly using Cursor with Claude models. With AI allowed, they took 19% longer. They had expected to be 24% faster, and afterwards still believed they had been about 20% faster.

Both studies are narrow, and METR says its results reflect early-2025 tools and should not be generalized. The lasting lesson is the gap between feeling and measurement: those developers misjudged their own speed by about 40 percentage points. Your own sessions are better evidence than either study.

4. Developers already distrust the output

In the 2025 Stack Overflow Developer Survey, 84% of respondents used or planned to use AI tools, yet more distrusted their accuracy (46%) than trusted it (33%). The most common frustration, cited by 66%, was AI answers that are almost right but not quite, and 45% said debugging AI-generated code takes longer. Asked about AI agents, 81% agreed they have concerns about the security and privacy of their data.

Questions every team should be able to answer

Taken together, the lessons come down to six questions about any Copilot session. Here is where Traccia answers each one:

Cost raised the stakes in 2026. Since June 1, Copilot usage draws down GitHub AI Credits that are metered on tokens, so the context a long agent session re-sends counts against your plan. The same reasoning applies to agents you build yourself; see What Is AI Agent Observability?

How GitHub Copilot Telemetry Reaches Traccia

Copilot ships with an OpenTelemetry exporter. You turn it on and point it at Traccia's OTLP base endpoint, https://api.traccia.ai/otel, with your ingestion key. Copilot appends /v1/traces, /v1/metrics and /v1/logs itself, and Traccia accepts the payloads directly over HTTPS, so there is no collector or proxy to run.

The part that trips people up is that VS Code runs Copilot in two places, configured in two different ways:

Most real work in VS Code happens in agent mode, so set up both. With only the settings, you will see Copilot Chat's own small background calls and no agent turns. Using other coding agents as well? Traccia also traces Claude Code and Cursor.

Before you start: create an ingestion key

VS Code and the CLI start the same way. Sign in at app.traccia.ai, open Settings, then API Keys, and create an ingestion key. Copy it; it starts with tr_live_. Every snippet below uses YOUR_TRACCIA_API_KEY as a placeholder for it. Then follow the VS Code steps, the CLI steps, or both.

Trace GitHub Copilot in VS Code

Allow about five minutes. You need VS Code with GitHub Copilot signed in and the ingestion key from above. Working in the terminal instead? Go to Trace the GitHub Copilot CLI.

Step 1: Trace Copilot Chat with User settings

Open the Command Palette (Ctrl+Shift+P, or Cmd+Shift+P on macOS), run Preferences: Open User Settings (JSON) and add these lines inside the existing object:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "otlp-http",
  "github.copilot.chat.otel.otlpEndpoint": "https://api.traccia.ai/otel",
  "github.copilot.chat.otel.headers": {
    "Authorization": "Bearer YOUR_TRACCIA_API_KEY"
  },
  "github.copilot.chat.otel.captureContent": true
}

Save the file and run Developer: Reload Window. Three details matter:

• Use User settings. These are application-level settings, so a workspace .vscode/settings.json is ignored.
• The headers setting stores your key in plain text. Keep it out of settings you sync or share.
• If your version of Copilot Chat has no github.copilot.chat.otel.headers setting, remove that line. The environment variables in Step 2 supply the key.

Step 2: Set the environment variables for agent mode

In our tests (VS Code 1.142, with agent mode running in the Agent Host), agent mode ignored the settings above and read only environment variables. Microsoft also documents a Local harness that does read these settings; its traces start at invoke_agent copilot instead of vscode.agent_host.session. The variables work in both cases. The Copilot CLI reads the same four, so setting them once covers both. Set them in a terminal:

export COPILOT_OTEL_ENABLED=true
export OTEL_EXPORTER_OTLP_ENDPOINT=https://api.traccia.ai/otel
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20YOUR_TRACCIA_API_KEY"
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

$env:COPILOT_OTEL_ENABLED = "true"
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://api.traccia.ai/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer%20YOUR_TRACCIA_API_KEY"
$env:OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT = "true"

Note the %20 after Bearer: the OpenTelemetry specification requires the space in a header value to be encoded. When these variables are set, they take precedence over the VS Code settings, so keep both pointed at the same Traccia endpoint and key.

Variables set this way last only as long as that terminal. To keep them:

• Windows: save them as user environment variables with setx, shown below. setx does not reach programs that are already running, so open a new terminal and restart VS Code afterwards. Once saved, VS Code picks them up however you launch it, and so does any other app that reads the standard OTEL_EXPORTER_OTLP_* variables.
• macOS and Linux: add the four export lines to ~/.zshrc or ~/.bashrc. VS Code opened from the Dock or an app launcher does not read your shell profile, so start it with code from a terminal.

setx COPILOT_OTEL_ENABLED "true"
setx OTEL_EXPORTER_OTLP_ENDPOINT "https://api.traccia.ai/otel"
setx OTEL_EXPORTER_OTLP_HEADERS "Authorization=Bearer%20YOUR_TRACCIA_API_KEY"
setx OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT "true"

Step 3: Restart VS Code and use agent mode

This is the step most setups miss. Quit VS Code completely: close every window, and on macOS also quit the app with Cmd+Q, because closing windows leaves it running. Then start VS Code from the same terminal where you set the variables:

code .

If any VS Code window is still open, code . hands the folder to that running instance, and Copilot's runtime keeps the environment it started with, without your variables. Once VS Code is up, open Copilot Chat, switch to Agent mode and give it a real task that reads files and runs a command.

Step 4: Find your first trace

Copilot exports spans in batches every few seconds, and a turn is complete once it finishes. In Traccia, open Traces and filter by the agent GitHub Copilot (id github-copilot). The first trace usually appears within seconds of a turn completing.

Nothing there, or only tiny one-call traces? Go to Troubleshooting.

Trace the GitHub Copilot CLI

The GitHub Copilot CLI runs Copilot's agent in your terminal. It plans a task, reads files, runs shell commands and edits code, and it exports the same OpenTelemetry spans as VS Code. It is configured only through environment variables, which it reads when copilot starts. If you already set them for agent mode in VS Code, skip to Step 3.

Step 1: Install the CLI and sign in

You need an active GitHub Copilot subscription. Install the CLI with npm (Node.js 22 or later), or use brew install --cask copilot-cli on macOS and Linux or winget install GitHub.Copilot on Windows. Then sign in once:

npm install -g @github/copilot
copilot login

copilot login signs you in to GitHub through OAuth. If you skip it, the CLI asks you to run /login the first time you start it.

Step 2: Point the CLI at Traccia

Set four environment variables in the terminal you will start copilot from, using your ingestion key:

export COPILOT_OTEL_ENABLED=true
export OTEL_EXPORTER_OTLP_ENDPOINT=https://api.traccia.ai/otel
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20YOUR_TRACCIA_API_KEY"
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

$env:COPILOT_OTEL_ENABLED = "true"
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://api.traccia.ai/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer%20YOUR_TRACCIA_API_KEY"
$env:OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT = "true"

What each of the four variables does:

• COPILOT_OTEL_ENABLED: turns on Copilot's OpenTelemetry export.
• OTEL_EXPORTER_OTLP_ENDPOINT: Traccia's OTLP base URL. The exporter adds the signal path, such as /v1/traces.
• OTEL_EXPORTER_OTLP_HEADERS: your ingestion key. The space after Bearer must be encoded as %20.
• OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT: true sends prompts, responses, and tool inputs and outputs. false sends metadata only.

The CLI exports over OTLP/HTTP, as JSON by default or as protobuf with OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf. It has no gRPC option, so don't point it at Traccia's gRPC port, 4317.

Variables set this way last only as long as that terminal. To keep them, add the four export lines to ~/.zshrc or ~/.bashrc, or on Windows save them with setx as shown in the VS Code steps. Saved variables also reach VS Code agent mode and take precedence over its settings, so keep both pointed at the same endpoint and key.

Step 3: Start copilot and give it real work

Start the CLI from the same terminal. If a copilot session was already running before you set the variables, exit it and start a new one, because a running session will not pick them up.

copilot

Ask for something that touches your code: read a file, run the tests, fix a failure. Each prompt you send becomes its own trace, rooted at an invoke_agent span. Inside it you get a plan span when the CLI plans the task, a chat span for each model call and an execute_tool span for each tool call, including the shell commands it runs. Copilot exports spans in batches every few seconds, so a prompt's trace appears shortly after the prompt finishes.

Step 4: Find the CLI session in Traccia

Open Traces and filter by the agent GitHub Copilot. CLI sessions carry a CLI badge and VS Code sessions an IDE badge, so you can tell them apart under the same agent. Turn on Group by session to fold a conversation into a single row: ten prompts are ten traces but one session, because every prompt carries the conversation's gen_ai.conversation.id. To add to an earlier conversation, start the CLI with copilot --resume and pick it; the new prompts join the same Traccia session.

Trace scripted and CI runs with copilot -p

copilot -p runs one prompt without the interactive session and exits when it finishes, which is how teams script the CLI or run it in a pipeline. With the same variables set, the run is exported like any other prompt. Grant only the tools the task needs:

copilot -p "Run the unit tests and summarize any failures" --allow-tool='shell(npm:*)'

GitHub's guide advises against broad options such as --allow-all outside a sandbox. In GitHub Actions, the CLI signs in with a token in COPILOT_GITHUB_TOKEN. Set the Traccia variables in the same step, keep the ingestion key in a repository secret, and set an agent.id such as copilot-ci through OTEL_RESOURCE_ATTRIBUTES so pipeline runs are easy to filter. A separate ingestion key for CI keeps them in their own environment. Traccia's integration guide covers interactive use, so check that one scripted run reaches Traccia before you depend on it.

How CLI traces differ from VS Code

Git details depend on what Copilot reports. In our test runs on the same repository, the CLI session showed the branch and the VS Code agent-mode session showed none. To group traces by team or project either way, tag them yourself: OTEL_RESOURCE_ATTRIBUTES, covered under Teams and Privacy, sets agent.id and agent.name for every prompt sent from that terminal.

Read a GitHub Copilot Trace

Click a trace to open it. Cards at the top sum up its duration, estimated cost, LLM calls, tokens and tool calls, and the execution timeline below places every span on a time axis. Copilot uses the same span names in VS Code and the CLI but groups them differently, and that changes what one trace means:

Subagents appear as nested spans under the same GitHub Copilot agent, not as separate agents. Each session is also tagged IDE or CLI (Figure 5), and shows the repository and branch when Copilot reports them.

Read the timeline top to bottom: what you asked, which model answered, which tool it ran and where the time went. In a longer session, this is where you see Copilot re-reading the same file or running a failing test again.

Sessions, Turns and Tool Retries

Copilot tags every turn with a stable gen_ai.conversation.id. Each prompt in the same conversation is a new turn with that ID, and Traccia groups those turns into one session. On the Traces page, turn on Group by session to collapse a conversation into one row:

• Session row: total tokens, wall time and estimated cost across all turns.
• Turn rows: each prompt, expanded underneath.
• Badges: the number of turns and tool retries at a glance.
• Open: a session page with every turn in order, plus GitHub Copilot and IDE or CLI badges.

The number of traces depends on where you worked. A CLI conversation with ten prompts produces ten traces and one session. The same conversation in VS Code agent mode, in our Agent Host runs, is a single trace with ten turns. Either way, the session tells you what the whole conversation used, without adding up traces by hand.

A failed tool call does not fail the turn

A turn is marked failed only when the agent turn itself errors. If a tool call fails inside a turn, the error stays on that execute_tool span, and the turn can still show Success with a tool retry badge. The session list can look green while a tool failed underneath, so treat the badge as your cue to open the turn.

A command that runs but reports a failure is different. When Copilot runs npm test and a test fails, the tool call itself succeeded, so the span is not marked as an error and no retry badge appears. In both of our test runs, the failing test run looked like any other tool call. Open the span and read its output, as in Figures 3 and 4, to see what the command actually reported.

Tokens, AI Credits and What Copilot Costs

How GitHub bills Copilot now

On June 1, 2026, GitHub moved Copilot to usage-based billing. Every plan includes a monthly allotment of GitHub AI Credits, and paid plans can buy more. Usage is calculated from tokens, counting input, output and cached tokens, at each model's listed API rate. Code completions and Next Edit suggestions stay included and do not use credits. One exception: annual Copilot Pro and Pro+ subscribers keep premium-request pricing until their plan expires.

What Traccia shows

To split that usage by team or agent, set the identity attributes described in the next sections; the ingestion key sets the environment. For the wider picture, see AI Agent Cost Attribution.

Alert on Copilot Usage, and What Traccia Does Not Do

After Run policies check traces once they land, so they can watch Copilot sessions with no change on the developer's machine. Scope each one to the Agent ID github-copilot, or to your own agent.id. Four templates fit Copilot well; test each on your own sessions before you rely on it:

• Cost Spike Alert. The sum of cost over an hourly, daily or weekly window, starting at $100 a day. Copilot totals are estimates, and Unpriced calls add nothing to them.
• Tool Storm Alert. Too many tool calls in one run, starting at 25. In the Agent Host, a VS Code Copilot trace is a whole chat, so size the limit for a chat rather than a single prompt.
• Input Token Limit. Prompts growing past a token budget, starting at an average of 8,000 input tokens, checked daily. Long agent sessions can pass that on their own, so start from your own baseline.
• Unapproved Model Alert. A run used a model outside your list. The template allows only gpt-4o and gpt-4o-mini, so change the list to the models your team uses before you turn it on. If your team uses Auto model selection, this is how you learn which models Auto actually picked.

On the Review step, run Simulate Last 7 Days to replay the draft against last week's traces and see how many would have matched. Then activate it with Email Alerts on, because alerts default to Off. The email usually follows within about 30 seconds of a violation opening.

Traccia cannot stop a Copilot command

Warn and Block on an After Run policy act on the next run that passes through Traccia's govern(). Copilot never does, so for Copilot a policy records a violation and sends an email, and that is all. To stop a command before it runs, use Copilot's own approval and permission settings. Traccia enforces policy only where it sits in the path of a call, which is how an AI agent control plane works; AI Observability vs AI Governance explains the difference.

Roll Out to a Team Without Sending Source Code

Split usage by team or environment

By default every turn, from VS Code or the CLI, rolls up under one agent, GitHub Copilot. To tell teams apart, set your own agent.id and agent.name as resource attributes. In VS Code User settings:

"github.copilot.chat.otel.resourceAttributes": {
  "agent.id": "copilot-backend",
  "agent.name": "copilot-backend"
}

For the Copilot CLI, set the same values as an environment variable:

export OTEL_RESOURCE_ATTRIBUTES="agent.id=copilot-backend,agent.name=copilot-backend"

The environment Traccia shows comes from the ingestion key, not from these attributes. In our test, a CLI prompt sent with env=staging appeared under its own agent.id but still showed the key's environment. To keep staging, CI and production apart, give each its own ingestion key.

Give each team its own Traccia workspace and an ingestion key scoped to it, so traces stay separate. If your organization manages Copilot's OpenTelemetry settings centrally, those managed settings replace each developer's own, so put the Traccia endpoint and key there.

Send metadata only

With content capture on, Copilot sends prompts, responses, and tool inputs and outputs, which can include source code. Two independent switches control this: github.copilot.chat.otel.captureContent for Copilot Chat's own calls, and OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT for agent mode and the CLI. Turning off only one leaves the other sending content. To send metadata only, set both to false:

"github.copilot.chat.otel.captureContent": false

export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=false

$env:OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT = "false"
# If you saved it with setx, change the saved value too:
setx OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT "false"

Then restart everything that reads them: quit VS Code completely and relaunch it from a terminal that has the new value, and restart any running copilot session. Traccia still receives what you need to run Copilot well:

• Models and token counts
• The span tree, timings and tool names
• Errors, outcomes and tool retry badges

Content capture also records what tools return. If Copilot reads a .env file, its contents can land in a span, the same kind of data the Adversa research showed leaving a laptop. Keep secrets out of the folders Copilot can read, or send metadata only.

Before a wide rollout, open a few traces and read the span attributes to confirm what actually arrives. Traccia truncates very large values on ingest, and in VS Code you can cap them yourself with github.copilot.chat.otel.maxAttributeSizeChars.

Troubleshooting GitHub Copilot Tracing

Frequently asked questions