Traces

A trace is the ordered record of what happened inside one run. For a service, it is spans across components. For an agent, it is the sequence of reasoning steps, tool calls, arguments, results, and data touched, from the request to the final answer.

It is the artifact you need when someone asks what happened. It is also the artifact an auditor will ask for, which makes it the most direct overlap between Observe and Secure.

What a usable agent trace contains

• The triggering input, including who or what supplied it.
• Each model call: the model and version, the resolved prompt, the completion, the token counts, and the latency.
• Each tool call: the tool, the arguments, the result or error, and the identity the call was made under.
• Every document retrieved, with an identifier stable enough to find it again.
• Decision points: where the agent chose between options, and what it had available.
• The terminal state: completed, failed, stopped by a human, or abandoned on a timeout.

The test is whether a competent colleague, with no memory of the incident, can read the trace and say what the system did and why. If they have to guess at a step, that step is missing.

Tool calls are the part that matters

Model reasoning is interesting. Tool calls are consequential. The tool call is where the agent stops producing text and starts changing the world: writing a row, sending a message, opening a ticket, deploying something.

So if you can only instrument one thing, instrument tool calls with full arguments, results, and the identity used. That gives you the answer to “what did it actually do”, which is the question that gets asked in incidents and audits, and it is a shorter path than capturing every token.

Two cautions. Arguments frequently contain credentials or personal data, so redact on the way in rather than at query time. And record the identity the call was made under, not just the tool name, because “the agent updated the record” and “the agent updated the record as the finance service account” are very different sentences.

Sampling, retention, and the honest trade

Full-fidelity traces on every request get expensive quickly, particularly when you are storing prompts and completions.

A workable policy:

• Sample the routine. A small percentage of successful, low-risk sessions.
• Keep everything abnormal. Errors, human interventions, anything that touched production, anything above a cost threshold, anything a user flagged.
• Keep tool calls at full fidelity even when you sample the reasoning. They are small and they are the evidence.
• Set retention by what it contains, not by what it is. A trace holding personal data inherits that data’s retention rules, and pretending it is telemetry does not change that.

Correlation across the boundary

An agent trace that stops at the tool call boundary is half a story. The tool call hit a service, that service hit a database, and the interesting failure is often on the far side.

Propagate a trace identifier from the agent through every tool call into the downstream systems, using whatever standard you already use. It is a small amount of plumbing and it converts two separate investigations into one.