Skip to main content

AI Agent Observability

Instrument AI agents with OpenTelemetry to trace the whole conversation - model calls, tool invocations, agent handoffs, and the downstream API and database work each decision triggers - in a single correlated view.

An agent is not a single LLM call. It is a loop: the model reasons, calls a tool, reads the result, calls another agent, and eventually answers. When that loop misbehaves - a tool times out, a sub-agent returns garbage, a retry storm burns tokens - the LLM is rarely the root cause. The failure lives in the tool call, the handoff, or the downstream service. To debug it you need telemetry that spans the entire agent execution, not just the chat completion.

This guide is the framework-agnostic companion to the language-specific LLM Observability guides. It focuses on the agent layer: the spans, attributes, and conventions that turn a scattered set of LLM calls into a coherent agent timeline. Code samples use the OpenTelemetry Python SDK, but every pattern is portable - the attribute names are the OpenTelemetry GenAI semantic conventions, so the same telemetry works with any OpenTelemetry backend, including base14 Scout.

AI agent timeline showing one conversation across an orchestrator, a billing sub-agent, tool calls, and downstream Stripe and database spans

TL;DR

Bind every span an agent produces - LLM calls, tool calls, sub-agent handoffs, and downstream HTTP and database work - with three attributes: gen_ai.conversation.id, gen_ai.agent.name, and gen_ai.operation.name. Emit invoke_agent spans for agent steps and execute_tool spans for tool calls, and thread the conversation ID through the whole call stack. The result is a single agent timeline in base14 Scout that shows exactly where an agent run spent time and where it failed.

Running this in production

Storing and querying these traces at production volume is what base14 Scout does. Check out Scout LLM Observability.

AI Agent Observability Overview​

This guide shows you how to:

  • Model an agent run as a conversation-level timeline, not isolated LLM calls
  • Use the GenAI agent operation types (invoke_agent, execute_tool, create_agent, chat)
  • Bind every span in a run with gen_ai.conversation.id and propagate it to downstream HTTP, database, and queue work
  • Instrument tool calls with the gen_ai.tool.* attributes and proper error handling
  • Trace Model Context Protocol (MCP) tool calls as execute_tool spans
  • Trace multi-agent orchestration and handoffs so each agent gets its own swim lane
  • Record agent metrics and evaluation results for cost, latency, and quality
  • Handle content capture, PII, and the experimental semantic-convention opt-in

Who This Guide Is For​

This documentation is designed for:

  • AI engineers: building tool-using or multi-agent systems and needing to see why a run was slow, expensive, or wrong
  • Backend developers: embedding agents into existing services and wanting agent spans correlated with HTTP and database telemetry
  • Platform teams: standardizing agent observability across frameworks (LangGraph, custom orchestration, MCP servers) on one open standard
  • DevOps and SRE: operating agents in production with cost, error-rate, and quality alerting

Prerequisites​

This guide builds on the base OpenTelemetry setup covered elsewhere. Before starting, ensure you have:

  • A working OpenTelemetry SDK setup (tracer and meter providers, OTLP exporter). See LLM Observability for the Python bootstrap, or the Rust and Java guides for those stacks.
  • Scout Collector configured and reachable from your application
  • Auto-instrumentation for your web framework and HTTP client, so downstream work joins the trace automatically. See Auto-Instrumentation.
  • Familiarity with GenAI spans for LLM calls (the chat operation). This guide extends that model to the agent and tool layers.

Compatibility Matrix​

ComponentMinimum VersionRecommended
opentelemetry-sdk1.39.01.39+
opentelemetry-api1.39.01.39+
GenAI semantic conv.1.36 (baseline)1.37+
Python (samples)3.123.13+

Last verified 2026-10-07 against two runs on local Ollama models: the ai-kyc-onboarding example on Pydantic AI 2.49.0 and Temporal Python SDK 1.33.0, and the agent-rebooking example on Microsoft Agent Framework 1.23.0 and the MCP C# SDK 2.2.0, each through its own Scout verification, with open-telemetry/semantic-conventions-genai at commit 4f85037 (2026-10-06). The agent conventions are in Development status, so pin each instrumentation to an exact version and re-check the spans after every upgrade. Framework-specific dates are on the framework pages under What's Next.

The Agent Timeline​

The core idea of agent observability is that a GenAI span is not just an LLM call. When an agent decides to call a tool, that tool might query a database, hit a third-party API, or enqueue a background job. All of that work is part of the same agent run and belongs in the same trace.

Three attributes bind an agent run together. Set them on every span the run produces:

AttributePurpose
gen_ai.conversation.idShared ID linking all operations in a run
gen_ai.agent.nameWhich agent produced the span
gen_ai.operation.nameWhat kind of operation the span represents

With those in place, a single agent run reads as one timeline:

Agent timeline for one conversation
POST /chat                                        6.1s  [auto: HTTP]
└─ invoke_agent orchestrator                      6.0s  [agent]
   ├─ chat claude-sonnet-4                          1.2s  [LLM: plan]
   ├─ execute_tool search_orders                    0.9s  [tool]
   │  └─ db.query SELECT orders                     18ms  [auto: DB]
   ├─ invoke_agent billing                          2.4s  [handoff]
   │  ├─ chat claude-sonnet-4                        1.1s  [LLM]
   │  └─ execute_tool issue_refund                   1.2s  [tool]
   │     └─ HTTP POST api.stripe.com                1.1s  [auto: httpx]
   └─ chat claude-sonnet-4                           1.3s  [LLM: final answer]

Every span above carries the same gen_ai.conversation.id. The auto-instrumented db.query and HTTP POST spans join the trace because they run inside the tool-call span's context - so a slow refund shows up as a slow Stripe call under a specific tool under a specific agent, not as an unattributed blip somewhere in the system.

Agent Operation Types​

The GenAI semantic conventions define gen_ai.operation.name values that classify each span. Use them consistently - backends and dashboards group and filter on this attribute.

gen_ai.operation.nameSpan represents
create_agentCreating/registering an agent (e.g. Assistants)
invoke_agentOne agent taking a turn or being handed control
chatAn LLM chat completion
execute_toolA tool or function call made by an agent
embeddingsGenerating vector embeddings
generate_contentA multimodal content-generation call

Most agent frameworks only need invoke_agent, chat, and execute_tool. Use create_agent for remote agent services (OpenAI Assistants, AWS Bedrock Agents) where agent creation is a distinct, observable step.

Instrumenting the Agent Layer​

Framework auto-instrumentation (LangChain callbacks, provider instrumentors) usually owns the chat spans. You own the agent-layer and conversation-layer attributes. A good division of labor:

Let the framework instrumentation own the LLM-layer spans, and you own the agent-layer and conversation-layer attributes.

The invoke_agent Span​

Wrap each agent turn in an invoke_agent span that carries the three binding attributes. This is the parent under which the agent's LLM and tool spans nest:

agent.py - invoke_agent span
from opentelemetry import trace

tracer = trace.get_tracer("gen_ai.agent")


async def run_agent(agent_name, state, conversation_id):
    """Run one agent turn under an invoke_agent span."""
    with tracer.start_as_current_span(
        f"invoke_agent {agent_name}"
    ) as span:
        # The three binding attributes
        span.set_attribute(
            "gen_ai.operation.name", "invoke_agent"
        )
        span.set_attribute("gen_ai.agent.name", agent_name)
        span.set_attribute(
            "gen_ai.conversation.id", conversation_id
        )

        # Optional agent metadata
        span.set_attribute("gen_ai.agent.id", state.agent_id)

        result = await agent_logic(state)

        span.set_attribute(
            "gen_ai.response.finish_reasons",
            [result.finish_reason],
        )
        return result

Threading the Conversation ID​

The conversation ID is only useful if it reaches every span, including downstream HTTP clients, database queries, and queue workers. Because those spans are auto-instrumented and created deep in library code, you cannot set the attribute on them directly. Two portable options:

  1. Set it at the root and read it back. Store the conversation ID on the first span of the run, and have your agent code re-read it and set it on each child span it creates.
  2. Propagate it as baggage. OpenTelemetry baggage travels with the context across async boundaries and process hops. A span processor can then copy it onto every span, including auto-instrumented ones.

The baggage approach keeps the conversation ID flowing without threading it through every function signature:

conversation.py - baggage + span processor
from opentelemetry import baggage, context
from opentelemetry.sdk.trace import SpanProcessor


def set_conversation(conversation_id):
    """Attach the conversation ID to the current context."""
    return context.attach(
        baggage.set_baggage(
            "gen_ai.conversation.id", conversation_id
        )
    )


class ConversationSpanProcessor(SpanProcessor):
    """Copy the conversation ID from baggage onto every span."""

    def on_start(self, span, parent_context=None):
        conversation_id = baggage.get_baggage(
            "gen_ai.conversation.id", parent_context
        )
        if conversation_id:
            span.set_attribute(
                "gen_ai.conversation.id", conversation_id
            )

Register ConversationSpanProcessor on the tracer provider alongside your BatchSpanProcessor. Now the auto-instrumented db.query and HTTP POST spans carry the conversation ID too, and the whole run stays linked.

Note: Baggage is propagated over the wire via the baggage HTTP header. Do not put secrets or PII in baggage - treat the conversation ID as an opaque correlation key.

Tool Call Observability​

Tool calls are where most agentic failures live. A model that picks the wrong tool, passes malformed arguments, or mishandles a tool error will fail even when every chat span looks healthy. Instrument tool calls as first-class execute_tool spans.

The execute_tool Span​

tools.py - execute_tool span
import json

from opentelemetry import trace

tracer = trace.get_tracer("gen_ai.tool")


async def call_tool(tool, arguments, call_id):
    """Run a tool under an execute_tool span."""
    with tracer.start_as_current_span(
        f"execute_tool {tool.name}"
    ) as span:
        span.set_attribute(
            "gen_ai.operation.name", "execute_tool"
        )
        span.set_attribute("gen_ai.tool.name", tool.name)
        span.set_attribute("gen_ai.tool.call.id", call_id)
        span.set_attribute(
            "gen_ai.tool.description", tool.description
        )

        # Arguments and result capture PII - see PII section
        span.set_attribute(
            "gen_ai.tool.call.arguments",
            json.dumps(arguments)[:2000],
        )

        try:
            result = await tool.run(**arguments)
        except Exception as e:
            span.record_exception(e)
            span.set_attribute("error.type", type(e).__name__)
            raise

        span.set_attribute(
            "gen_ai.tool.call.result",
            json.dumps(result)[:2000],
        )
        return result

Tool Call Attribute Reference​

AttributeTypeRequiredDescription
gen_ai.operation.namestringYesAlways "execute_tool"
gen_ai.tool.namestringRecommendedTool/function name
gen_ai.tool.call.idstringRecommendedUnique ID for this call
gen_ai.tool.descriptionstringRecommendedWhat the tool does
gen_ai.tool.typestringRecommendedfunction, extension
gen_ai.tool.call.argumentsstringOpt-inJSON arguments (PII risk)
gen_ai.tool.call.resultstringOpt-inJSON result (PII risk)
error.typestringConditionalException class on failure

Setting error.type and recording the exception is what lets base14 Scout highlight the failing tool in the timeline and lets you query for the tools that fail most often.

MCP Tools​

The Model Context Protocol (MCP) is a common way for agents to reach external tools. An MCP tools/call is just a tool invocation, so model it as an execute_tool span with the same gen_ai.tool.* attributes, adding MCP-specific context:

mcp_tools.py - MCP call as execute_tool
with tracer.start_as_current_span(
    f"execute_tool {tool_name}"
) as span:
    span.set_attribute("gen_ai.operation.name", "execute_tool")
    span.set_attribute("gen_ai.tool.name", tool_name)
    span.set_attribute("gen_ai.tool.type", "function")

    # MCP context
    span.set_attribute("mcp.server.name", server_name)
    span.set_attribute("mcp.method.name", "tools/call")

    result = await mcp_session.call_tool(tool_name, arguments)

Because the MCP client's transport is auto-instrumented, the underlying HTTP or stdio call nests under this span automatically - so a slow or failing MCP server is attributed to the exact tool the agent invoked. If you run your own MCP server with Scout, see the Scout MCP guides for setup and usage.

Multi-Agent Orchestration​

In a multi-agent system, one agent hands control to another: an orchestrator calls a specialist, a planner delegates to a worker. Two rules keep these handoffs readable in the timeline.

Rule 1: The Caller Emits the Handoff Span​

The calling agent emits the invoke_agent span for the agent it invokes - not the agent being called. This makes orchestration explicit: the span sits under the orchestrator, so the timeline shows who delegated to whom.

orchestrator.py - handoff ownership
async def orchestrator(state, conversation_id):
    with tracer.start_as_current_span(
        "invoke_agent orchestrator"
    ) as span:
        span.set_attribute(
            "gen_ai.operation.name", "invoke_agent"
        )
        span.set_attribute(
            "gen_ai.agent.name", "orchestrator"
        )
        span.set_attribute(
            "gen_ai.conversation.id", conversation_id
        )

        # The orchestrator emits the handoff span for "billing"
        with tracer.start_as_current_span(
            "invoke_agent billing"
        ) as handoff:
            handoff.set_attribute(
                "gen_ai.operation.name", "invoke_agent"
            )
            handoff.set_attribute(
                "gen_ai.agent.name", "billing"
            )
            handoff.set_attribute(
                "gen_ai.conversation.id", conversation_id
            )
            return await billing_agent(state)

The billing agent then emits its own chat and execute_tool spans under its own gen_ai.agent.name, so its work appears in the billing swim lane.

Rule 2: Every Agent Gets a Distinct Name​

Give each agent - including sub-agents - a distinct gen_ai.agent.name. Spans that omit it collapse into an Unknown swim lane, and you lose the ability to tell which agent did what or attribute cost per agent. When you fan out to parallel sub-agents, name each one.

Agent Metrics​

Agent runs need the same GenAI metrics as LLM calls, dimensioned by agent and tool so you can answer "which agent is slowest?" and "which tool fails most?". Reuse the metric definitions from the LLM Observability guide and add agent/tool attributes when recording:

MetricTypeKey attributes
gen_ai.client.token.usageHistogramgen_ai.agent.name, model
gen_ai.client.operation.durationHistogramgen_ai.operation.name, agent
base14.gen_ai.costCountergen_ai.agent.name, model
base14.gen_ai.error.countCountergen_ai.tool.name, error.type

The last two are application metrics. No framework records cost or an error count, and gen_ai.* belongs to the conventions, so the examples use the base14. prefix. Recording gen_ai.agent.name on both spans and metrics lets you build sum(base14.gen_ai.cost) by (gen_ai.agent.name) to find your most expensive agent, and sum(base14.gen_ai.error.count) by (gen_ai.tool.name) to find your flakiest tool.

Evaluation and Quality​

Agent quality - did it use the right tool, was the final answer correct, did it hallucinate - is a first-class observability concern. Attach evaluation results as gen_ai.evaluation.result events on the relevant span:

eval.py - evaluation event
span.add_event(
    "gen_ai.evaluation.result",
    attributes={
        "gen_ai.evaluation.name": "tool_selection",
        "gen_ai.evaluation.score.value": 1.0,
        "gen_ai.evaluation.score.label": "correct",
    },
)

Record the same score as a base14.gen_ai.evaluation.score histogram to track quality trends over time. See the evaluation section of the LLM Observability guide for the metric definition and dashboard patterns.

Content Capture, PII and Security​

The richest agent-debugging attributes also carry the most sensitive data:

  • gen_ai.input.messages / gen_ai.output.messages - full prompts and responses
  • gen_ai.system_instructions - your system prompt
  • gen_ai.tool.call.arguments / gen_ai.tool.call.result - tool inputs and outputs

These capture PII and sensitive data by default when enabled. Protect them:

  • Keep content capture off by default. Enable it deliberately, and prefer non-production environments for full capture.
  • Redact at the application layer before setting attributes, and truncate long payloads (2000 characters is a practical limit).
  • Redact at the collector with the redaction or attributes processor as a safety net, so sensitive fields never reach the backend.

Enabling the Latest GenAI Conventions​

The GenAI semantic conventions are still under active development. Newer instrumentations gate the latest attribute shapes behind an environment variable so existing pipelines do not break:

Terminal
export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental

Set this when you want the current message and tool-call attribute formats. Without it, instrumentations default to the transition baseline. Pin your collector and backend expectations to the same convention version.

Known Gaps​

As of the GenAI conventions at commit 4f85037 (2026-10-06), checked 2026-10-07 across the framework pages this guide links, each written from a run:

  • No framework records a wait on a person. Temporal parks a Pydantic AI workflow, and Agent Framework parks a run on RequestInfoEvent, without a span, a metric or a log line. The kyc.await_review span in the KYC example and the base14.approval.requested and decided spans in the rebooking example are application code. See Agent Approval Gates and Pydantic AI on Temporal.
  • gen_ai.conversation.id is never set for you. Pydantic AI takes it as conversation_id on agent.run, Strands through trace_attributes and the OpenAI Agents SDK through RunConfig(group_id=...). Agent Framework and Google ADK set it only from a managed session ID, Mastra maps only threadId, and Claude Code and Codex carry session.id instead. The other examples set it in a span processor.
  • error.type is missing on most frameworks' spans. Pydantic AI, Strands and Google ADK set error status and a recorded exception only; the OpenAI Agents instrumentation sets _OTHER; Mastra reports unknown on a failed model connection. Agent Framework sets it. The examples add it in a span exporter.
  • A failed tool call rarely fails the run. execute_tool carries the error and invoke_agent stays clean when the agent recovers: an unknown booking in the rebooking example, a failed ranking fetch in the filing analyst, a sanctions lookup retried under Temporal. Filter on tool spans as well as run spans.
  • A nested agent hides its failures from the caller. Strands ends the outer execute_tool without error when a tool inside the inner agent fails, and Agent Framework handoffs have no span at all. Read nesting from gen_ai.agent.name on each invoke_agent span.
  • Metrics differ by framework. Pydantic AI records gen_ai.client.token.usage but no duration; Strands records strands.* metrics; Vercel AI SDK 7 and Mastra record none, so their examples derive gen_ai.client.* from spans in the collector; Claude Code and Codex export their own counters. Google ADK and Agent Framework record both conventional histograms.
  • Cost is a price-table lookup, not a framework feature. Pydantic AI prices only models in its own table; no other framework records cost. Every example adds base14.gen_ai.cost from a shared price table, zero and flagged simulated on local models.
  • OTEL_SEMCONV_STABILITY_OPT_IN reaches only some instrumentations. OpenTelemetry's own packages, Strands and OpenClaw read it; Pydantic AI, Vercel AI SDK 7 and the Agent Framework .NET packages do not. Set it anyway, so every instrumentation that does read it writes the same attribute shapes.
  • OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT is the same. The OpenAI Agents instrumentations read it; Pydantic AI, Strands and Agent Framework need the application to map it onto their own switch. With capture on, Agent Framework stores each message on the chat span and again on the invoke_agent span above it.
  • Content survives where telemetry does not reach. Temporal history keeps signal payloads and activity inputs whatever the capture setting says.

What to Look For in Scout​

Find every span of one run​

Filter on gen_ai.conversation.id. In the KYC example it is the case ID, so one filter returns the extraction and assessment runs across every document round. Where a framework does not set it, the example's own ID does the same job: base14.run.id in the rebooking example.

Separate failed runs from runs that recovered​

Filter invoke_agent spans on status = Error and group by error.type for runs that failed outright: pydantic_ai.exceptions.UsageLimitExceeded when a KYC run hits its request budget. Then filter execute_tool spans the same way for runs that recovered: an unknown booking in the rebooking example fails tools/call lookup_booking and execute_tool lookup_booking while the run completes.

Follow a handoff or a nested agent​

Group a trace's invoke_agent spans by gen_ai.agent.name. A nested agent sits under the caller's execute_tool span, as invoke_agent ranking does under execute_tool rank_among_filers in the filing analyst. An Agent Framework handoff is two invoke_agent spans in sequence, triage then rebooking, with the handoff tool listed in gen_ai.tool.definitions on the first.

See where a run waited for a person​

Read the wait span's duration: kyc.await_review in the KYC example, with the base14.kyc.review.wait histogram by base14.kyc.review_decision; or base14.agent.approval.wait.duration by base14.approval.outcome in the rebooking example, where the requested and decided spans sit in different traces and are joined by a span link.

Read tokens and cost per agent​

Sum gen_ai.usage.input_tokens and gen_ai.usage.output_tokens on chat spans grouped by gen_ai.agent.name, which Pydantic AI and Agent Framework set on chat spans as well as on invoke_agent. Read cost from base14.gen_ai.cost on the same spans, and treat base14.gen_ai.cost.simulated=true as a placeholder, not a price.

Production Configuration​

Agents send telemetry to a local OpenTelemetry Collector, which authenticates to base14 Scout and forwards traces, metrics, and logs. Point your application at the collector with OTEL_EXPORTER_OTLP_ENDPOINT, and configure the collector to export to Scout:

otel-collector-config.yaml
extensions:
  health_check:
    endpoint: 0.0.0.0:13133
  zpages:
    endpoint: 0.0.0.0:55679
  oauth2client:
    client_id: ${SCOUT_CLIENT_ID}
    client_secret: ${SCOUT_CLIENT_SECRET}
    token_url: ${SCOUT_TOKEN_URL}
    endpoint_params:
      audience: b14collector
    timeout: 10s

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 512
    spike_limit_mib: 128
  redaction:
    allow_all_keys: true
    blocked_values:
      - "[0-9]{16}" # credit card numbers
      - "[\\w.]+@[\\w.]+" # email addresses
  batch:
    timeout: 10s
    send_batch_size: 1024
    send_batch_max_size: 2048

exporters:
  otlphttp/b14:
    endpoint: ${SCOUT_ENDPOINT}
    auth:
      authenticator: oauth2client
    compression: gzip
    timeout: 30s
    retry_on_failure:
      enabled: true
  debug:
    verbosity: basic

service:
  extensions: [health_check, zpages, oauth2client]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, redaction, batch]
      exporters: [otlphttp/b14, debug]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/b14, debug]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, redaction, batch]
      exporters: [otlphttp/b14, debug]

The redaction processor in the traces and logs pipelines strips sensitive values from captured content before it leaves your environment. For the full collector reference and deployment options, see OpenTelemetry Collector Setup.

Troubleshooting​

Common Issues​

Issue: Spans show an "Unknown" agent​

A span is missing gen_ai.agent.name, so the backend cannot assign it to a swim lane.

Solutions:

  1. Set gen_ai.agent.name on every invoke_agent, chat, and execute_tool span the agent produces
  2. For sub-agents, give each a distinct name - do not reuse the parent's
  3. Verify the attribute is set before the span ends, not after

LLM and tool spans appear, but downstream HTTP and database spans are not part of the same conversation.

Solutions:

  1. Confirm gen_ai.conversation.id is set on the root agent span
  2. If using baggage, verify ConversationSpanProcessor is registered on the tracer provider and that baggage is attached to the active context
  3. Ensure downstream calls run inside the agent span's context so they inherit the trace

Issue: Tool spans are missing​

Tool calls do not appear as execute_tool spans.

Solutions:

  1. Wrap tool execution explicitly - framework auto-instrumentation covers LLM calls but often not custom tools
  2. Check that the tool wrapper runs inside the invoke_agent span's context
  3. For MCP tools, ensure the transport (HTTP/stdio) instrumentation is enabled so the underlying call nests under the execute_tool span

Performance Considerations​

Agent instrumentation overhead is negligible next to model and tool latency. A tool call or LLM completion takes hundreds of milliseconds to seconds; span creation adds microseconds.

  • Span creation: ~1-5 microseconds per span
  • Attribute setting: ~0.5 microseconds per attribute
  • Baggage propagation: negligible; a single small header
  • Content capture: the main cost - large argument/result payloads inflate span size, so truncate and gate capture behind a flag

FAQ​

What is the difference between LLM observability and agent observability?​

LLM observability traces individual model calls - model, tokens, cost, latency. Agent observability traces the whole run: the loop of LLM calls, tool calls, and sub-agent handoffs, plus the downstream work each triggers, all bound by a conversation ID. Agent failures usually live in the tool calls and handoffs, not the model call - which is exactly what the agent layer makes visible.

Do I need an agent framework to get agent observability?​

No. The conventions are framework-agnostic. Whether you use LangGraph, another framework, or hand-rolled orchestration, you emit invoke_agent and execute_tool spans with the GenAI attributes. Frameworks with OpenTelemetry support give you some spans for free; you still own the conversation ID and agent names.

How do I trace multi-agent handoffs?​

The calling agent emits the invoke_agent span for the agent it delegates to, and every agent uses a distinct gen_ai.agent.name. This produces one swim lane per agent and makes the delegation structure explicit in the timeline.

Are MCP tool calls traced?​

Yes. Model an MCP tools/call as an execute_tool span with the gen_ai.tool.* attributes plus MCP context (mcp.server.name, mcp.method.name). The auto-instrumented transport nests the underlying call under that span.

Can I see the arguments an agent passed to a tool?​

Yes, via gen_ai.tool.call.arguments and gen_ai.tool.call.result. These capture PII by default, so redact and truncate them, keep capture off in production unless needed, and add a collector redaction processor as a backstop.

How do I find the most expensive or slowest agent?​

Record gen_ai.agent.name on both spans and metrics, then query sum(base14.gen_ai.cost) by (gen_ai.agent.name) for cost and histogram_quantile over gen_ai.client.operation.duration grouped by agent for latency.

What's Next?​

Language-Specific Guides​

Framework Guides​

Grouped by how much telemetry the framework emits on its own.

Native OpenTelemetry, enabled by configuration:

  • Agent Approval Gates - Microsoft Agent Framework and the MCP C# SDK, with a human approval gate on top
  • OpenClaw - the gateway's diagnostics-otel plugin, run, model and tool spans, metrics and trace-correlated logs
  • Claude Code - Claude Code and the Claude Agent SDK, interaction, model request and tool spans, cost and token metrics and per-event logs
  • Codex CLI - the OpenAI Codex CLI, turn, model request and command spans, codex.* metrics and per-event logs
  • Google ADK Instrumentation - agent, model and tool spans, gen_ai.* metrics and inference events
  • Microsoft Agent Framework Instrumentation - agent, chat and tool spans, metrics and message events, on by default in Python and enabled per agent in .NET

Built in behind a flag or a package:

Traced through the model SDK packages, with spans of your own where needed:

Runnable Examples​

Scout Platform Features​

References​

Last updated on
Was this page helpful?