Agent Evaluation: Trajectories, Tool Calls, and Multi-Step Correctness

Your customer support agent just processed 10,000 tickets. The outcome metrics look fine — task completion rate 91%, no hard errors logged, average response time 2.1 seconds. Then a human QA sweep on a random 50-ticket sample finds something alarming: in 23% of cases, the agent is issuing service credits by hallucinating that the customer mentioned a billing error, when the conversation actually contained no such mention. The final action (issue credit) sometimes aligned with what the customer wanted anyway, so the end-to-end success metric missed it entirely. You shipped a hallucinating agent that happens to give correct answers often enough to look fine in aggregate.

This is not a hypothetical. It is the standard failure mode of agents evaluated only on outcomes, and it gets worse as chains get longer.

Why outcome-only evaluation fails

End-to-end task success measures one thing: did the final output satisfy the goal? That is a legitimate and necessary signal. It is not sufficient for three reasons.

First, wrong paths can produce correct outputs by accident. A research agent that finds the right answer by calling a web search tool, ignoring the results, and hallucinating a plausible conclusion from its training data will pass every outcome eval while being entirely unreliable on anything that falls outside its training distribution. The test passed; the agent is broken.

Second, correct paths can degrade without changing outcomes. An agent that used to take 3 tool calls and now takes 7 to accomplish the same task is getting worse — it is burning more tokens and adding four more chances to fail — but if the final answer is still correct, outcome metrics show a flat line.

Third, and most practically: the compound error problem means per-step reliability has to be very high. If each step of a 10-step agent succeeds 90% of the time, the agent completes without error on roughly 35% of runs. To hit 90% clean end-to-end, you need each step at 99% reliability. You cannot optimize toward that target if you only observe the final output.

{ "type": "error-compound", "accuracy": 0.9, "steps": 10, "title": "Why per-step reliability has to be high" }

The practical implication: agent evaluation and why agents fail in production is the foundational question, and the answer requires opening up the trajectory, not just reading the final score.

The three evaluation levels

Think of agent evaluation as three concentric circles, each one catching failures the outer circle misses.

Level 1 — Outcome: Did the task complete successfully? Was the final output correct and complete? This is a binary or rubric score on the final deliverable. It is the cheapest to compute and the easiest to explain to non-technical stakeholders. Use an LLM-as-judge with a task-specific rubric rather than pure binary pass/fail — a refund that was issued for the right reason with the right amount is not the same as one issued for the wrong reason by chance. For how to calibrate your LLM judge, see LLM-as-Judge: Calibrating Your Automated Evaluator.

Level 2 — Trajectory: Was the path the agent took to reach that outcome sound? This is where most of the diagnostic value lives. A trajectory evaluation looks at:

• Which tools were called, in which order
• Whether tool calls were necessary (or redundant loops)
• Whether tool parameters were correctly populated from the task context
• Whether intermediate reasoning steps that were verbalized were consistent with the tool outputs received
• Whether the agent abandoned a valid approach prematurely or persisted past the point of useful information

Level 3 — Component: Which specific step, tool, or sub-agent broke? In a multi-agent system, this means attributing failure to the orchestrator, a specific worker agent, or a specific tool invocation. This is the level that produces actionable fixes — not "the agent is bad" but "the search_orders tool call at step 3 consistently receives a malformed date parameter when the user input is ambiguous."

Capturing agent traces

You cannot evaluate a trajectory you have not captured. The emerging standard for this is OpenTelemetry with GenAI semantic conventions — the gen_ai.* span attribute namespace that stabilized in 2025 and added MCP tool-call observability in OTel v1.39.

A gen_ai.* span captures, at minimum:

• gen_ai.system — which model provider (openai, anthropic, etc.)
• gen_ai.operation.name — the operation type (chat, tool_call, embeddings)
• gen_ai.request.model — the model identifier
• gen_ai.usage.input_tokens and gen_ai.usage.output_tokens
• For tool calls: the tool name, input arguments, and returned output

Here is what instrumenting an agent with the Anthropic SDK looks like using a lightweight OTel wrapper:

from anthropic import Anthropic
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
import json

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(
    endpoint="https://your-langfuse-or-arize-endpoint/v1/traces"
)))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("agent.eval")

client = Anthropic()

MAX_STEPS = 20  # the loop guard we tell everyone to add and then forget

def run_agent_with_tracing(task: str, tools: list[dict]) -> dict:
    with tracer.start_as_current_span("agent_run") as root_span:
        root_span.set_attribute("gen_ai.system", "anthropic")
        root_span.set_attribute("gen_ai.operation.name", "agent_run")
        root_span.set_attribute("agent.task", task)

        messages = [{"role": "user", "content": task}]

        for step in range(MAX_STEPS):
            with tracer.start_as_current_span(f"agent_step_{step}") as step_span:
                response = client.messages.create(
                    model="claude-sonnet-4-5",
                    max_tokens=4096,
                    tools=tools,
                    messages=messages,
                )
                step_span.set_attribute("gen_ai.request.model", "claude-sonnet-4-5")
                step_span.set_attribute("gen_ai.usage.input_tokens", response.usage.input_tokens)
                step_span.set_attribute("gen_ai.usage.output_tokens", response.usage.output_tokens)
                step_span.set_attribute("agent.step_index", step)

                if response.stop_reason == "tool_use":
                    tool_results = []
                    for tool_use in (b for b in response.content if b.type == "tool_use"):
                        with tracer.start_as_current_span("tool_call") as tool_span:
                            tool_span.set_attribute("gen_ai.tool.name", tool_use.name)
                            tool_span.set_attribute("gen_ai.tool.call.id", tool_use.id)
                            tool_span.set_attribute(
                                "gen_ai.tool.input", json.dumps(tool_use.input)
                            )
                            # Execute the actual tool here
                            result = execute_tool(tool_use.name, tool_use.input)
                            tool_span.set_attribute("gen_ai.tool.output", json.dumps(result))
                            tool_results.append({
                                "type": "tool_result",
                                "tool_use_id": tool_use.id,
                                "content": json.dumps(result),
                            })

                    messages.append({"role": "assistant", "content": response.content})
                    messages.append({"role": "user", "content": tool_results})
                    continue

                # "end_turn", "max_tokens", "refusal" — anything that is not a
                # tool request ends the run. Record why, so the eval layer can
                # distinguish a clean finish from a truncated one.
                final_text = next(
                    (b.text for b in response.content if b.type == "text"), ""
                )
                root_span.set_attribute("agent.final_answer", final_text)
                root_span.set_attribute("agent.stop_reason", response.stop_reason)
                return {
                    "answer": final_text,
                    "steps": step,
                    "stop_reason": response.stop_reason,
                }

        root_span.set_attribute("agent.stop_reason", "max_steps_exceeded")
        return {"answer": None, "steps": MAX_STEPS, "stop_reason": "max_steps_exceeded"}

This trace now contains every tool call with its inputs and outputs, token counts per step, and the final answer — everything needed for all three evaluation levels.

Scoring trajectories

Once you have traces, you need to score them. The scoring strategy differs by level.

Outcome scoring

Use an LLM judge with a task-specific rubric. The rubric should be derived from your product specification rather than constructed ad hoc — a formal spec makes rubrics consistent across product versions.

OUTCOME_RUBRIC = """
You are evaluating a customer support agent's response to a refund request.

Score the final answer on a scale of 1-5:
5 — Correct action (approve/deny), correct amount if approved, correct reason cited,
    polite tone, references correct policy section.
4 — Correct action and amount, minor policy citation error or slight tone issue.
3 — Correct action, but wrong amount OR wrong policy cited.
2 — Wrong action (approved when should deny or vice versa) but with coherent reasoning.
1 — Wrong action with no coherent reasoning, or rude/confusing response.

Task: {task}
Agent response: {final_answer}
Expected outcome: {expected_outcome}

Think step by step. Cite specific evidence from the response. Output your score on the last line as "Score: N".
"""

Trajectory scoring: picking a matching strategy

Trajectory matching is not one algorithm — there is a menu, ordered by strictness, and the choice changes what your eval can catch:

Strategy	How it matches	What it catches	What it misses	Cost
Exact match	Full trajectory identical to reference: same tools, same order, same params	Everything, in principle	Nothing — but flags every legitimate alternative path as a failure	Cheap (string compare)
In-order match	Reference tools appear in order; extra calls allowed	Wrong tools, wrong ordering	Redundant/unnecessary calls, param errors	Cheap
Any-order match	Reference tools all called, order ignored	Missing tool calls	Ordering bugs (canceling before verifying), extra calls	Cheap
Precision/recall on tool calls	Set overlap between predicted and reference calls	Both missing and unnecessary calls, quantified	Ordering, param correctness	Cheap
LLM-judge semantic match	Judge scores whether the path was reasonable given the task	Equivalent-but-different valid paths, param semantics	Nondeterministic; needs calibration	~$0.01–0.05/trajectory

My actual recommendation: start with in-order matching for tool selection plus an LLM judge for parameters — which is what the code below implements. Exact match generates so many false failures on legitimate path variation that teams stop trusting the eval within a month, and any-order match will happily bless an agent that cancels an order before checking it exists. Add precision/recall as a second signal once you care about trajectory efficiency, not just correctness.

Trajectory scoring: tool selection accuracy

For each step in the reference trajectory (the gold-standard path), check whether the agent called the expected tool:

def compute_tool_selection_accuracy(
    predicted_trajectory: list[dict],
    reference_trajectory: list[dict],
) -> float:
    """
    Each entry: {"step": int, "tool": str, "inputs": dict}
    Returns fraction of steps where the correct tool was called.
    Reference may be shorter (agent found a shorter path) or longer.
    We align by step index and penalize missing steps.
    """
    if not reference_trajectory:
        return 1.0

    correct = 0
    for ref_step in reference_trajectory:
        idx = ref_step["step"]
        pred_step = next(
            (s for s in predicted_trajectory if s["step"] == idx), None
        )
        if pred_step and pred_step["tool"] == ref_step["tool"]:
            correct += 1

    return correct / len(reference_trajectory)

Trajectory scoring: tool parameter accuracy

Tool parameter accuracy is a separate metric. An agent that calls search_orders every time but always passes customer_id instead of order_id has 100% tool selection accuracy and 0% parameter accuracy.

def compute_tool_parameter_accuracy(
    predicted_trajectory: list[dict],
    reference_trajectory: list[dict],
    judge_client,  # Anthropic or OpenAI client
) -> float:
    """
    Use an LLM judge to score parameter correctness.
    Pure string matching breaks on equivalent values (datetime formats, etc.).
    """
    scores = []

    for ref_step in reference_trajectory:
        idx = ref_step["step"]
        pred_step = next(
            (s for s in predicted_trajectory if s["step"] == idx), None
        )
        if not pred_step or pred_step["tool"] != ref_step["tool"]:
            scores.append(0.0)
            continue

        prompt = f"""
Tool: {ref_step['tool']}
Expected parameters: {json.dumps(ref_step['inputs'])}
Actual parameters: {json.dumps(pred_step['inputs'])}

Are the actual parameters semantically equivalent to the expected parameters?
Consider format differences acceptable (ISO date vs. unix timestamp for the same moment).
Score 1.0 if equivalent, 0.5 if partially correct (some fields right, some wrong),
0.0 if materially incorrect. Output only the numeric score.
"""
        response = judge_client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=10,
            messages=[{"role": "user", "content": prompt}],
        )
        scores.append(float(response.content[0].text.strip()))

    return sum(scores) / len(scores) if scores else 1.0

Building a trajectory golden dataset

A golden dataset for agent evaluation has a different structure than one for RAG or chat — each example is a full trajectory, not just an (input, expected output) pair.

Each example should contain:

• Task description: the user's actual request
• Reference trajectory: the gold-standard sequence of tool calls (tool name, inputs, expected output range)
• Expected final outcome: the correct final answer or action, with rubric
• Failure mode tags: which known failure modes this example is designed to surface (wrong tool selection, hallucinated parameter, unnecessary loop, premature termination)

On dataset sizing: 50 examples detects major regressions. Getting statistical confidence on smaller shifts — say, a 5% drop in tool parameter accuracy after a prompt change — requires 150–200 trajectories. For the argument on dataset construction methodology, see Golden Datasets: Building the Ground Truth Your Evals Depend On.

Adversarial trajectories deserve special attention. These are tasks explicitly designed to trigger known failure modes: ambiguous parameters that should cause the agent to ask for clarification rather than hallucinate, tool results that return empty sets (does the agent handle this or hallucinate a result?), and contradictory information across tool outputs. Random production sampling will not generate enough of these; you have to construct them deliberately.

# Example adversarial trajectory for "hallucinated parameter from ambiguous input"
ADVERSARIAL_EXAMPLE = {
    "task": "Cancel my order from last week",
    "task_type": "order_cancellation",
    "failure_mode": "hallucinated_parameter",
    "expected_behavior": "Agent must call get_recent_orders to find the order_id before "
                         "calling cancel_order. Should NOT call cancel_order with a "
                         "guessed or hallucinated order_id.",
    "reference_trajectory": [
        {
            "step": 0,
            "tool": "get_recent_orders",
            "inputs": {"customer_id": "<from_context>", "days_back": 14},
            "acceptable_output_fields": ["order_id", "status", "date"],
        },
        {
            "step": 1,
            "tool": "cancel_order",
            "inputs": {"order_id": "<from_step_0_output>"},
        },
    ],
    "red_flags": [
        "cancel_order called before get_recent_orders",
        "cancel_order called with order_id not present in get_recent_orders output",
    ],
}

Task-adaptive rubrics with AdaRubric

When your agent handles heterogeneous tasks — customer support, code generation, data analysis, scheduling — a single fixed evaluation rubric performs poorly. The rubric calibrated for a refund task will be wrong for a code-writing task, and vice versa.

AdaRubric (2025) addresses this by auto-generating per-task-type evaluation criteria. The idea: given a task description and the agent's trajectory, generate a rubric tailored to what correctness means for this specific task, then score against that rubric. This is not the same as letting the judge make up evaluation criteria on the fly — the rubric generation step produces explicit, inspectable criteria before the scoring step runs.

def generate_task_rubric(task: str, task_type: str, judge_client) -> str:
    """
    Step 1 of AdaRubric: generate the evaluation criteria.
    Inspect this output before using it in production — garbage in, garbage out.
    """
    prompt = f"""
Task type: {task_type}
Task: {task}

Generate a scoring rubric for evaluating whether an AI agent completed this task correctly.
The rubric should list 4-6 specific, binary-checkable criteria relevant to THIS task type.
Do not include generic criteria like "is the response polite?" unless relevant to the task.
Format each criterion as a yes/no question.
"""
    response = judge_client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=512,
        messages=[{"role": "user", "content": prompt}],
    )
    return response.content[0].text


def score_with_adaptive_rubric(
    task: str, task_type: str, trajectory: dict, judge_client
) -> dict:
    rubric = generate_task_rubric(task, task_type, judge_client)
    # Step 2: score the trajectory against the generated rubric
    scoring_prompt = f"""
Rubric:
{rubric}

Agent trajectory:
{json.dumps(trajectory, indent=2)}

For each criterion in the rubric, answer yes (1) or no (0).
Output as JSON: {{"criteria": [{{"question": "...", "met": true/false}}]}}
"""
    response = judge_client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": scoring_prompt}],
    )
    return json.loads(response.content[0].text)

One practical caution: the rubric generation step can itself fail — producing vague criteria, or missing the actual correctness requirement entirely. Inspect generated rubrics for the first 20–30 task examples in each task type before trusting them in CI.

Wiring evaluation into CI

The evaluation loop only prevents regressions if it runs automatically on every change. The structure is:

sequenceDiagram
    participant DEV as Developer
    participant CI as CI pipeline
    participant EVAL as Eval runner
    participant GOLD as Golden dataset
    participant GATE as Quality gate

    DEV->>CI: Pull request (prompt change / tool change)
    CI->>EVAL: Run agent on all 50-200 golden tasks
    EVAL->>GOLD: Load reference trajectories
    EVAL-->>CI: Outcome scores + trajectory scores per task
    CI->>GATE: Aggregate: tool_selection_acc, param_acc, outcome_score
    GATE-->>DEV: PASS if all metrics above threshold
    GATE-->>DEV: BLOCK if any metric dropped > N%
    Note over DEV,GATE: Blocking thresholds: outcome -3%, tool_selection -5%, param_acc -5%

The blocking thresholds require calibration. Start permissive — block only on catastrophic drops (> 10%) — and tighten as your dataset grows and your baselines stabilize. The worst outcome is a CI gate so strict that every PR fails and developers learn to ignore it or work around it.

There is a second noise source the thresholds have to respect: the agent itself is stochastic. Run the identical agent on the identical 100-task golden set twice and the pass rate will move — at a true 80% success rate, single-run sampling noise is roughly ±4 percentage points (binomial standard error, √(0.8 × 0.2 / 100)). A 3–5% blocking threshold on single runs is inside that noise band, which means your gate will flap on PRs that changed nothing. The standard mitigation, popularized by τ-bench's pass^k metric, is multiple trials: run each golden task k = 3–5 times and gate on the mean score, or on pass^k if you specifically care about consistency (an agent that passes a task 3-of-5 times is a different animal from one that passes 5-of-5). Yes, this multiplies the eval cost by k — the ~$2.63 run below becomes ~$8–13 — which is still cheaper than one flapping-gate investigation. And size your thresholds from data: measure the run-to-run standard deviation of each metric on your own dataset over a week of no-op runs, then set the block line at 2–3σ rather than a percentage that felt reasonable in a planning meeting.

On cost: running 100 trajectories through a 5-step agent at Claude Sonnet pricing (~$3 per million input tokens, ~$15 per million output tokens as of mid-2026, illustrative) with an average of 500 input tokens and 200 output tokens per step:

100 tasks × 5 steps × 500 input tokens  = 250,000 input tokens  → ~$0.75
100 tasks × 5 steps × 200 output tokens = 100,000 output tokens → ~$1.50
LLM-judge scoring: ~50 tokens/step × 500 steps = 25,000 tokens → ~$0.38

Total per CI run: ~$2.63

At 20 PRs/week, that is ~$52/week.
A single engineer-hour to investigate a production regression that eval would have caught
costs more than a month of CI eval runs.

What breaks

Agent trajectory evaluation has its own failure modes that are easy to overlook.

Reference trajectory brittleness. Your gold-standard paths assume a specific API response shape. If a tool returns results in a different order or format, the reference step fails even when the agent's behavior is correct. Use semantic equivalence checks (via LLM judge) rather than exact-match for tool outputs.

Trajectory length mismatch. The agent finds a shorter path than the reference. Is that a success (efficient reasoning) or a failure (skipped a necessary verification step)? You need explicit criteria for whether shorter paths are acceptable, and they differ by task type. Cancelling an order in 1 step when 2 are expected might mean the agent skipped confirming the order exists first.

LLM judge drift in trajectory scoring. The same judge model version applied to the same trajectory can return different scores across runs. This is the non-determinism problem documented in LLM-as-Judge: Calibrating Your Automated Evaluator. For trajectory scoring, run each trace through the judge with temperature=0 and at least 2 passes; flag any disagreements for human review. Over a 100-trajectory dataset, expect 10–15% of scores to flip between runs with a single judge pass.

Overfitting the golden dataset. Once developers know which tasks are in the CI eval set, there is an incentive — usually unconscious — to tune agents and prompts specifically to pass those tasks. The dataset needs to be refreshed with new tasks periodically, and ideally a held-out subset should be unknown to the team. This is the agent equivalent of the benchmark contamination problem.

Missing tool execution in test environments. Running agents in CI often means mocking tool calls. A mocked search_orders that always returns a well-formatted response does not test the agent's robustness to empty results, malformed outputs, or timeout errors. Include failure-injection scenarios in your golden dataset where tools return errors or unexpected shapes.

The "correct by accident" flip. The most dangerous failure: the agent's tool parameter accuracy drops from 85% to 70% across a code change, but the final outcomes are still correct because the tasks in your golden set are forgiving. This is why tracking trajectory metrics separately from outcome metrics is not optional — the outcome metric will miss this regression, and you will discover it only when you deploy to production and hit the 30% of tasks where the hallucinated parameter matters.

Multi-agent systems: attribution

When multiple agents collaborate — an orchestrator routing tasks to specialist workers — evaluation needs to attribute failures to the right component. A wrong final answer could be the orchestrator's fault (wrong routing), a worker agent's fault (wrong tool use), or a tool's fault (returning bad data).

flowchart LR
    ORCH[Orchestrator\norchestrator_agent] -->|routes task| WA[Worker A\nresearch_agent]
    ORCH -->|routes task| WB[Worker B\ndata_agent]
    WA -->|calls| T1[web_search tool]
    WA -->|calls| T2[summarize tool]
    WB -->|calls| T3[query_db tool]
    WB -->|calls| T4[format_report tool]
    WA -->|result| ORCH
    WB -->|result| ORCH
    ORCH -->|final output| OUT[Final answer]

    style ORCH fill:#a855f7,color:#fff
    style WA fill:#00e5ff,color:#0a0a0f
    style WB fill:#00e5ff,color:#0a0a0f
    style T1 fill:#0e7490,color:#fff
    style T2 fill:#0e7490,color:#fff
    style T3 fill:#0e7490,color:#fff
    style T4 fill:#0e7490,color:#fff

The OTel trace tree already gives you this structure — each agent creates a child span under the parent, and each tool call creates a child span under the agent that invoked it. The evaluation layer needs to score each node in the tree independently, not just the root.

Practically: assign a component_id to each agent and tool, score each independently, and aggregate to a component-level accuracy table. When you see a regression in the final output score, you can immediately look up which component's score dropped rather than running a debugging investigation from scratch. This is the same reasoning behind the observability tooling discussed in Tracing and Observability Tooling: LangSmith, Langfuse, Braintrust, and Arize Compared.

The evaluation pipeline as infrastructure

{ "type": "eval-pipeline", "title": "Agent evaluation as a layered safety net" }

The five-stage eval pipeline applies to agents exactly as it does to simpler LLM applications, but the agent-specific additions at each stage are:

• Dev unit tests: mock tool execution, test that the agent calls the right tool for a specific input pattern, assert against expected intermediate steps.
• Pre-release adversarial: run the agent against deliberately tricky trajectories — ambiguous parameters, contradictory tool outputs, deliberately empty result sets.
• CI regression: full golden dataset with trajectory + outcome scoring; quality gate blocks the merge on metric drops.
• Production sampling: 5–10% of live traffic, score with LLM judge, feed interesting failures (especially "correct output from broken trajectory") back into the golden dataset.
• Runtime guardrails: detect infinite tool-call loops (> N iterations), parameter hallucination patterns in real-time, and escalate to a fallback response. This is complementary to the offline vs online evaluation split.

The feedback loop from production sampling to the golden dataset is where the compounding value is. Every production failure that gets de-identified and annotated becomes a test case that prevents the same failure on the next deployment. This is the data flywheel applied to agent quality.

The decision in practice

Agent evaluation is genuinely more expensive and operationally heavier than chat or RAG evaluation. The case for it is simple: the failure modes are also more expensive. A hallucinating chat assistant gives a user a wrong answer. A hallucinating agent with write access to your systems takes a wrong action — issues a refund, cancels an order, sends an email, modifies a record.

Start with outcome scoring on a 50-task golden dataset. Add trajectory scoring once you see the first "correct output from wrong reasoning" failure in your CI runs — and you will. Roll component attribution as your agent becomes multi-agent and you need to triage failures quickly.

The teams that treat agent quality like infrastructure — the same CI gating, the same dataset management practices described in Golden Datasets: Building the Ground Truth Your Evals Depend On, the same regression-alerting discipline they would apply to a production database — are the ones running agents at scale without weekly regression incidents. The teams that ship on vibe and hope the outcome metrics look good are the ones discovering production outages in their support queue.