Your First Production Agent

Your demo agent looked great. It answered the question, called the right tool, formatted the output. You shipped it. Three days later, a support ticket: the agent looped 47 times on a malformed input and spent $11 in API calls before timing out at the infrastructure layer. Nobody set a budget. Nobody defined done.

That's the real first lesson of production agents. The loop is the easy part.

An agent is structurally simple: an LLM, a set of tools, and a while-loop that runs until some stop condition fires. What's hard is everything around the loop — the stop conditions, the budgets, the retry logic, the gates that prevent irreversible actions from running without a human in the chain. Build those first and the agent is almost an afterthought.

This module builds a real agent, step by step, across working Python. By the end you'll have a functioning research assistant agent and the mental model to know when agents are the right choice at all.

The mental model: reason → act → observe

Every agent, regardless of framework, executes the same cycle. The ReAct paper from 2022 named it Thought → Action → Observation; Anthropic's building effective agents post describes the same cycle. The name doesn't matter; the structure does.

flowchart TD
    S[Start: user task] --> R[Reason\nLLM reads context, decides next step]
    R --> D{Tool call\nor done?}
    D -- tool call --> A[Act\nhost executes tool]
    A --> O[Observe\nappend tool result to context]
    O --> B{Budget\nexceeded?}
    B -- no --> R
    B -- yes --> FAIL[Terminate: budget hit]
    D -- done --> ANS[Final answer]
    style FAIL fill:#ff2e88,color:#111
    style ANS fill:#a855f7,color:#fff
    style S fill:#00e5ff,stroke:#00e5ff,color:#0a0a0f

The model never "does" anything. It reads the current state of the conversation — which includes every prior tool result — and returns either a tool call or a final answer. Your code does the doing. This separation is not just an API detail; it's load-bearing for safety. The LLM proposes; you control whether it executes.

Three things make this loop go wrong:

1. No stop condition other than "model says done." The model will say done when it hallucinates success. Your hard cap is the actual stop condition.
2. No budget. Each iteration costs tokens. A stuck loop at $0.002/turn is $0.002. A stuck loop at $0.15/turn with a 50-turn timeout is $7.50, repeated across concurrent users.
3. No verification. The model's self-assessment of whether a step succeeded is unreliable. Your code needs to validate tool results.

The function calling contract

Before writing the loop, understand what function calling actually does at the API level.

You pass the model a tools list — an array of JSON schemas, one per tool. Each schema has a name, a description, and a parameters object that's a JSON Schema describing the tool's inputs. When the model decides to call a tool, it returns a response with stop_reason: "tool_use" (Anthropic) or finish_reason: "tool_calls" (OpenAI), and the response body contains one or more tool call objects with the tool name and the arguments it wants to pass.

Your application then executes the tool with those arguments, appends the result to the conversation as a tool_result message, and calls the model again. The model never touches your code, your database, or your APIs directly. It only emits structured requests and reads the results you feed back.

Here's the bare-minimum Python loop against Anthropic's SDK:

import anthropic

client = anthropic.Anthropic()

tools = [
    {
        "name": "search_web",
        "description": "Search the web for current information on a topic. Use when you need facts that may be more recent than your training data.",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "The search query. Be specific."
                }
            },
            "required": ["query"]
        }
    }
]

def run_agent(task: str, max_iterations: int = 10, max_tokens_total: int = 20000) -> str:
    messages = [{"role": "user", "content": task}]
    total_tokens = 0

    for iteration in range(max_iterations):
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            tools=tools,
            messages=messages
        )

        total_tokens += response.usage.input_tokens + response.usage.output_tokens

        # Model finished — return the answer even if this response nudged
        # us past the budget; we already paid for it
        if response.stop_reason == "end_turn":
            for block in response.content:
                if hasattr(block, "text"):
                    return block.text
            return "[No text output]"

        # Enforce the budget before the next API call, not after the answer
        if total_tokens > max_tokens_total:
            return f"[BUDGET EXCEEDED after {iteration+1} iterations, {total_tokens} tokens]"

        # Model wants to call a tool
        if response.stop_reason == "tool_use":
            messages.append({"role": "assistant", "content": response.content})

            tool_results = []
            for block in response.content:
                if block.type == "tool_use":
                    result = execute_tool(block.name, block.input)
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": result
                    })

            messages.append({"role": "user", "content": tool_results})
            continue

        # Unknown stop reason — fail fast
        return f"[Unexpected stop_reason: {response.stop_reason}]"

    return f"[MAX ITERATIONS ({max_iterations}) REACHED without final answer]"

Two things worth noticing. max_iterations=10 is a hard cap, not a suggestion. And max_tokens_total is tracked across all iterations — without cross-iteration token accounting, a long task just quietly burns money. The budget ceiling is pre-agreed with whoever owns the bill.

{"type": "agent-loop", "scenario": "weather-trip", "title": "The reason → act → observe loop"}

Tool design is the highest-return thing you control

The model cannot call a tool it doesn't know how to describe. And the description isn't documentation — it's a prompt. The model reads the description and decides whether to call the tool, when to call it, and what arguments to pass. A vague or overlapping description is a broken contract.

Tool description quality directly determines agent routing accuracy. Tool hallucination — the model calling a tool with parameters it invented — is far more common when agents are given irrelevant or ambiguous tool schemas. That's not a model failure; that's a schema design failure.

Four rules that cover most of the damage:

1. Few, sharp tools over many overlapping ones. If you have search_documents and search_knowledge_base and query_docs, the model has to guess which one to use for any given query, and it will get it wrong. One well-scoped search tool beats three fuzzy ones. Token overhead is also real: each tool definition adds 300–800 tokens to every request on Anthropic, even if the tool is never called.

2. The description tells the model when — not just what. Bad: "search_web": "Searches the web." Good: "search_web": "Search the web for current information on a topic. Use when you need facts that may be more recent than your training data, or when the user asks about events after 2024." The when is the part that determines correct tool selection.

3. Parameters should be specific about format. If a parameter expects an ISO 8601 date string, say that. If it expects a numeric ID, say "type": "integer" not "type": "string". The model will try to conform to your schema, but ambiguous types give it room to hallucinate values that seem plausible but are structurally wrong.

4. Return human-readable results, not raw UUIDs. Tool results go back into the context as text. If you return {"record_id": "a3f8-bc12-..."}, the model has to reason about an opaque identifier. If you return {"customer": "Acme Corp", "status": "active", "record_id": "a3f8-bc12-..."}, it can actually use the result to make a decision.

Here's a real tool schema for a simple research agent, written to these rules:

tools = [
    {
        "name": "search_web",
        "description": (
            "Search the web for current information. Use when you need facts "
            "published after 2024 or when the user asks about recent events. "
            "Do NOT use for information likely in your training data — "
            "prefer reasoning from what you know first."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "Specific search query. Prefer narrow queries over broad ones."
                },
                "max_results": {
                    "type": "integer",
                    "description": "Number of results to return. Default 3, max 10.",
                    "default": 3
                }
            },
            "required": ["query"]
        }
    },
    {
        "name": "get_page_content",
        "description": (
            "Fetch the full text of a web page by URL. "
            "Use after search_web when you need the full content of a specific result."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "url": {
                    "type": "string",
                    "description": "The full URL to fetch, including https://"
                }
            },
            "required": ["url"]
        }
    }
]

Two tools. One searches; one fetches. The model knows exactly when to use each. This beats a single do_research mega-tool and it beats six specialized variants.

The compound accuracy problem

Here's the number that should make you nervous about long-running agents.

If each step in your agent has a 95% chance of producing a correct result, and you chain 10 steps, the end-to-end success rate is 0.95¹⁰ = 59%. Drop per-step accuracy to 90% and ten steps gives you 0.90¹⁰ = 35%. This is not a theoretical concern — it's the math, and the math is merciless.

{"type": "error-compound", "accuracy": 0.95, "steps": 10, "title": "Why per-step accuracy collapses over 10 steps"}

The implication is not "make the model more accurate." The implication is: keep loops short, validate intermediate results, and treat any task requiring 15+ steps as a design smell that usually means the task should be decomposed and handed off to a human checkpoint.

This also reframes when agents are appropriate at all. The compounding error rate makes agents riskier than single-shot LLM calls for tasks that are long or where each step has real consequences. Short, verifiable loops with human checkpoints at branching points are dramatically more reliable than fully autonomous long chains.

Termination criteria and the budget

"The model says it's done" is not a termination criterion. The model will say it's done when it hallucinates a successful search, when it gets confused and starts looping, and when it genuinely completes the task. You cannot tell those apart without your own checks.

The three real termination conditions are:

flowchart LR
    L[Loop iteration] --> C1{stop_reason =\nend_turn?}
    C1 -- yes --> DONE[Return answer]
    C1 -- no --> C2{Iterations >=\nmax_iterations?}
    C2 -- yes --> ITFAIL["Fail: iteration cap"]
    C2 -- no --> C3{"Tokens >=\ntoken_budget?"}
    C3 -- yes --> TBFAIL["Fail: token budget"]
    C3 -- no --> NEXT[Next iteration]
    style DONE fill:#a855f7,color:#fff
    style ITFAIL fill:#ff2e88,color:#111
    style TBFAIL fill:#ff2e88,color:#111

Setting the budget: for most research or summarization tasks, 5–10 iterations covers the real work. Budget a per-iteration token cost of roughly system_prompt + tools + history + output — with 2 tools of the schema above, that's around 1,500–2,000 tokens overhead per call before any content. On claude-sonnet-4-5 at illustrative mid-2026 pricing, a 10-iteration research agent costs roughly $0.08–$0.25 depending on content length. Set a total-token budget of 20,000–40,000 tokens for a single run; anything that needs more is a task design problem.

What to do when you hit the cap: fail loudly. Return a structured failure response, log the full conversation state, and surface it for analysis. Most teams discover that 90% of cap-hitting runs are caused by 2–3 pathological inputs or tool errors that return unhelpful content. Fix those; the cap stops being a common event.

Retries and idempotency

Tool calls fail. APIs time out. Search returns zero results. The agent needs a policy for what to do next.

The safe retry pattern: when a tool returns an error, pass the error back to the model as a tool_result with the error content included. The model will usually try a different approach — different query, different arguments, different tool. If the same error fires twice in a row for the same tool and arguments, stop retrying that tool and let the model decide to either use an alternative or admit it can't complete the task.

def execute_tool(name: str, inputs: dict) -> str:
    """Execute a tool and return a string result (or structured error)."""
    try:
        if name == "search_web":
            results = web_search(inputs["query"], max_results=inputs.get("max_results", 3))
            if not results:
                return "No results found for that query. Try a different search term."
            return format_search_results(results)

        elif name == "get_page_content":
            content = fetch_url(inputs["url"])
            return content[:8000]  # truncate; keep it manageable

        else:
            return f"Unknown tool: {name}"

    except TimeoutError:
        return f"Tool '{name}' timed out. Try a simpler or different query."
    except Exception as e:
        return f"Tool '{name}' failed: {str(e)}"

The model call itself fails too, and in an 8-iteration loop it's the single most likely thing to throw: rate limits (429), overloaded errors (529), network timeouts. One unhandled exception from client.messages.create kills the run and loses the entire conversation state. The Anthropic SDK retries 429s and 5xx errors twice with exponential backoff by default (raise max_retries on the client if your traffic bursts) — but when the retries run out, you still want a structured api_error terminal state that logs the conversation, not a stack trace. The final agent below does exactly that.

Idempotency matters when tools have side effects. A send_email tool called twice sends two emails. A write_record tool called twice may create duplicates or overwrite data. For any tool with side effects, track which tool calls have been executed in the current run. If the loop retries a step, check whether the side-effecting tool already ran successfully before firing it again. The simplest approach: keep a set of executed_tool_ids and skip re-execution for any tool call ID you've already completed.

Human-approval gates for consequential actions

Some actions are not retriable. Email sent is email sent. Record deleted is record deleted. Database updated is database updated.

The pattern: classify your tools into read and write (or safe/unsafe). Read tools execute automatically. Write tools checkpoint for human approval before execution. This is not a framework feature — it's four lines of code:

WRITE_TOOLS = {"send_email", "write_record", "delete_record", "post_to_api"}

def execute_tool_with_gate(name: str, inputs: dict, auto_approve: bool = False) -> str:
    if name in WRITE_TOOLS and not auto_approve:
        # In a real system, surface this to the UI and wait for approval
        # Here we simulate by raising an exception the caller handles
        raise HumanApprovalRequired(tool=name, inputs=inputs)
    return execute_tool(name, inputs)

In a web application, this turns into an async checkpoint: the agent run pauses, the user sees "Agent wants to send this email — approve?" with the full proposed content, the user approves or edits, and the run resumes. This is how every production agent that touches external systems should work. The model proposing an action and a human approving it is not a limitation of current AI — it's the correct architecture for consequential irreversible steps.

Workflows vs agents: the simplest thing that works

This is where Anthropic's framing is directly useful. Building effective agents draws the line between workflows — LLMs and tools orchestrated through predefined code paths — and agents, which dynamically direct their own process and tool usage. Its guidance boils down to: find the simplest solution that works, and add autonomy only when a fixed pipeline genuinely can't express the task.

A workflow is a fixed sequence of LLM calls and code. You decide the steps upfront; the LLM fills in the content at each step. A prompt chaining pattern — call 1 extracts entities, call 2 looks them up, call 3 writes a summary — is a workflow. It's predictable, debuggable, and the failure modes are enumerable.

An agent is appropriate when:

• The required tools and their sequencing cannot be known until the task is underway.
• The task requires conditional branching that depends on intermediate results.
• The task is long enough that encoding all decision points in a fixed pipeline would be impractical.

Most use cases that feel like agents are actually workflows. A "research and summarize" task where you always do search → fetch top results → summarize is a three-call pipeline, not an agent. The agent version makes sense when the model needs to decide whether to search again, which result to fetch, whether to cross-reference two sources, and when to stop — because those decisions depend on what it finds.

flowchart TD
    Q[Can you enumerate the steps upfront?] -- yes --> WF[Use a workflow\nFixed sequence, predictable]
    Q -- no --> AG[Consider an agent\nDynamic tool selection]
    WF --> W2[Prompt chaining, parallelization\nrouter, evaluator-optimizer]
    AG --> A2[Keep loops short\nAdd budgets and gates]
    style WF fill:#0e7490,color:#fff
    style AG fill:#00e5ff,color:#0a0a0f

The practical heuristic: if you can draw the flowchart before you write the code, it's a workflow. Reach for an agent when the model itself needs to make the flowchart.

Putting it together: a small research agent

Here's the complete, runnable research agent from the pieces above. This is the version you'd push to a staging environment.

import anthropic
import json
from typing import Optional

client = anthropic.Anthropic()

# Tools with side effects get the idempotency guard. This agent's tools
# are both read-only, so the set is empty — but the plumbing is here for
# when you add send_email or write_record.
WRITE_TOOLS: set = set()

TOOLS = [
    {
        "name": "search_web",
        "description": (
            "Search the web for current information. Use when you need facts "
            "published after 2024 or when the user asks about recent events. "
            "Do NOT use for information in your training data."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Specific search query."},
                "max_results": {"type": "integer", "default": 3}
            },
            "required": ["query"]
        }
    },
    {
        "name": "get_page_content",
        "description": "Fetch the full text of a web page. Use after search_web for specific results.",
        "input_schema": {
            "type": "object",
            "properties": {
                "url": {"type": "string", "description": "Full URL including https://"}
            },
            "required": ["url"]
        }
    }
]

SYSTEM_PROMPT = """You are a research assistant. When given a question:
1. Determine if you need current information (search) or can answer from knowledge.
2. Use search_web to find relevant sources, then get_page_content for the most useful one.
3. Synthesize a clear, accurate answer with source citations.
4. Stop when you have enough information. Don't over-search.

Be direct. Cite your sources. If you cannot find reliable information, say so."""

def run_research_agent(
    question: str,
    max_iterations: int = 8,
    token_budget: int = 30_000
) -> dict:
    messages = [{"role": "user", "content": question}]
    total_tokens = 0
    executed_tools = set()

    for i in range(max_iterations):
        try:
            response = client.messages.create(
                model="claude-sonnet-4-5",
                max_tokens=2048,
                system=SYSTEM_PROMPT,
                tools=TOOLS,
                messages=messages
            )
        except anthropic.APIError as e:
            # The SDK already retried 429/5xx with backoff. If it still
            # failed, return a terminal state instead of crashing mid-run.
            return {
                "status": "api_error",
                "iterations": i + 1,
                "tokens": total_tokens,
                "error": str(e),
                "answer": None
            }

        total_tokens += response.usage.input_tokens + response.usage.output_tokens

        if response.stop_reason == "end_turn":
            answer = next(
                (b.text for b in response.content if hasattr(b, "text")), ""
            )
            return {
                "status": "complete",
                "iterations": i + 1,
                "tokens": total_tokens,
                "answer": answer
            }

        # Budget check after the end_turn check: a finished answer always
        # returns, and the budget stops us before the next API call
        if total_tokens > token_budget:
            return {
                "status": "budget_exceeded",
                "iterations": i + 1,
                "tokens": total_tokens,
                "answer": None
            }

        if response.stop_reason == "tool_use":
            messages.append({"role": "assistant", "content": response.content})
            tool_results = []

            for block in response.content:
                if block.type != "tool_use":
                    continue

                # Idempotency guard: only for side-effecting tools, and
                # only for calls that already SUCCEEDED — a failed write
                # must stay retriable. Read tools re-run freely.
                call_key = f"{block.name}:{json.dumps(block.input, sort_keys=True)}"
                if block.name in WRITE_TOOLS and call_key in executed_tools:
                    result = "Already executed this call. Use the previous result."
                else:
                    result = execute_tool(block.name, block.input)
                    if block.name in WRITE_TOOLS and not result.startswith("Tool error"):
                        executed_tools.add(call_key)

                tool_results.append({
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": result
                })

            messages.append({"role": "user", "content": tool_results})
            continue

        return {"status": "unexpected_stop", "stop_reason": response.stop_reason}

    return {
        "status": "max_iterations",
        "iterations": max_iterations,
        "tokens": total_tokens,
        "answer": None
    }


def execute_tool(name: str, inputs: dict) -> str:
    """Stub — replace with real implementations."""
    try:
        if name == "search_web":
            # Replace with Tavily, Brave, or SerpAPI call
            return f"[STUB] Search results for: {inputs['query']}"
        elif name == "get_page_content":
            # Replace with httpx/playwright fetch
            return f"[STUB] Content of: {inputs['url']}"
        return f"Unknown tool: {name}"
    except Exception as e:
        return f"Tool error: {str(e)}"

This hundred-ish lines is a real agent. It has a budget, a hard iteration cap, an idempotency guard scoped to side-effecting tools, structured return values for every terminal state — including provider-side API failures — and clean separation between the loop and the tool execution. The stubs are the only things that need replacing.

What breaks in production

The failure modes are not what most people expect. The model quality is usually fine. The operational stuff is what kills agents.

Context rot. As the conversation grows, the model's effective attention on earlier content degrades. A tool result from iteration 1 may be effectively ignored by iteration 8. The mitigation is aggressive context management — summarize intermediate results into the system prompt rather than leaving raw tool outputs in the message history. See context window management for agents for the mechanics.

Tool result poisoning. If a web search returns a page that itself contains instruction-looking text ("Ignore previous instructions and..."), that text lands in the model's context as a tool_result. This is indirect prompt injection, and it's the most common security vulnerability in agents that browse the web. Sanitize tool results before they enter the context. The guardrails and security module covers this in depth.

Silent wrong answers. The model completes successfully, returns a confident final answer, and the answer is wrong. This is worse than a crash because nothing in your error handling catches it. You need output validation — a separate check (deterministic rule, second LLM call, or human review for high-stakes outputs) that catches plausible-sounding but incorrect answers. Eval infrastructure before you deploy is not optional; it's the whole next module.

The runaway loop. Nothing technically errors, but the model keeps searching for better information, never deciding it has enough. This is "context rot" plus "no hard stopping criterion." The fix is both the iteration cap and explicit stopping guidance in the system prompt: "Stop when you have enough information to answer the question. Do not search more than 3 times for the same topic."

Token budget surprises. Each tool schema is injected into every request. Two tools at 400 tokens each means 800 overhead tokens on every call. At 8 iterations, that's 6,400 tokens before any actual content — roughly $0.02–$0.04 at the same illustrative pricing, just for the schema overhead. Track this in your cost model. Caching tool schemas using prefix caching (both OpenAI and Anthropic support it) eliminates most of this overhead for long-running agents; the prompt caching article has the mechanics.

Where to go next

• The agent loop in depth — the five-stage perceive→reason→plan→act→observe cycle with termination patterns and circuit breakers.
• Designing tool schemas that models understand — the description is the prompt; this covers the schema mistakes that most commonly cause tool hallucination.
• The tool-use round trip — parallel tool calls, stop reasons, and what stop_reason: "tool_use" means at the API level.
• Context window management for agents — how to keep 8-iteration history from growing into 80,000 tokens of noise.
• Eval first: building the safety net — you cannot ship an agent without catching silent wrong answers; this is the module that tells you how.
• Guardrails and security: production is an attack surface — indirect prompt injection through tool results is the primary attack vector for web-browsing agents.
• Reliable tool-using systems: patterns from production — what 1,200 production deployments agree on, including DoorDash's "budgeting the loop" pattern.