Tracing calls

Evals tell you how the feature does on questions you thought of. Observability tells you what's happening on the questions real users actually send — and the atom of AI observability is the trace: a complete record of one request's journey through your system.

What a trace must capture

For a single LLM call, the trace records everything you'd need to reproduce and judge it later:

• The full prompt sent — system + messages, including the assembled context (retrieved chunks, tool results). Not your template — the actual bytes the model saw. Debugging without this is guessing.
• The full response — text, tool calls, stop reason.
• Metadata: model + version, the prompt/feature version, max_tokens, timestamp, latency, token counts (in/out), and cost.
• Identity: a trace_id for the request and the prompt/feature version that produced it (Module 4's versioning pays off here — 'which prompt said that?' must be answerable).
• Outcome hooks: was it flagged, did a validator fire, did the user thumbs-down it, was it escalated. The links between a trace and what happened next.

Tracing multi-step systems

A RAG query or an agent task is many model calls plus retrievals and tool calls. A flat log of individual calls can't show you the shape. The structure is a tree of spans under one trace: the top span is the whole request; child spans are each retrieval, each model call, each tool execution — each with its own timing and payload. This is exactly how you'll debug the agent from Agentic AI: the trace tree is the trajectory, now captured in production instead of printed in a lab.

import time, uuid, json, contextlib, contextvars

# Illustrative prices, USD per 1M tokens — check current pricing at
# docs.anthropic.com/en/docs/about-claude/pricing before you trust a bill.
PRICE = {"claude-sonnet-5": {"in": 3.00, "out": 15.00}}

def call_cost(model, tokens_in, tokens_out):
    p = PRICE[model]
    return tokens_in / 1e6 * p["in"] + tokens_out / 1e6 * p["out"]

_current_span = contextvars.ContextVar("current_span", default=None)

class Tracer:
    def __init__(self): self.spans = []
    @contextlib.contextmanager
    def span(self, name, **meta):
        s = {"span_id": uuid.uuid4().hex[:8], "name": name,
             "parent_span_id": _current_span.get(),   # None at the root
             "start": time.time(), "meta": meta}
        token = _current_span.set(s["span_id"])       # this span is now the parent
        try:
            yield s
        finally:
            _current_span.reset(token)                # pop back to the parent
            s["duration_s"] = round(time.time() - s["start"], 3)
            self.spans.append(s)
    def emit(self, trace_id):
        # spans carry parent_span_id, so a reader can rebuild the tree.
        json.dump({"trace_id": trace_id, "spans": self.spans},
                  open(f"traces/{trace_id}.json", "w"))

# usage inside the feature — nesting span() calls builds the tree:
#   with tracer.span("request"):                 # root span
#       with tracer.span("llm_call", model="claude-sonnet-5") as s:
#           resp = client.messages.create(...)
#           ti, to = resp.usage.input_tokens, resp.usage.output_tokens
#           s["meta"].update(tokens_in=ti, tokens_out=to,
#                            cost=call_cost("claude-sonnet-5", ti, to))

Two things to notice. The span tree is expressed by parent_span_id: nesting span() calls means each child records the id of the span open above it, so a reader rebuilds the trajectory from a flat list. And the cost on each span comes from the token counts times a price table — the PRICE numbers above are illustrative (per 1M tokens); pin the current figures from the pricing docs (docs.anthropic.com) before you report a dollar amount to anyone.