# The Agent Loop: How LLMs Become Actors > The perceive→reason→act→observe cycle that turns a static LLM call into an autonomous process — plus the hard-stop guardrails that keep it from spinning forever. Canonical: https://ironclad.academy/ai-engineering/articles/agent-loop-perceive-reason-act | Difficulty: beginner | Topics: Agents, Tool Use ## Key takeaways - The agent loop is perceive→reason→act→observe repeated until an explicit termination condition fires — not until the model decides it is done. - Single-agent loops burn ~4× the tokens of a chat call; multi-agent systems ~15×. Cost budgeting is not optional. - 10 steps at 85% per-step accuracy yields only ~20% end-to-end success — never evaluate agents at single-step granularity. - Tool calls are JSON the host application executes; the model never touches external systems directly. That separation is what makes agents auditable. - Hard iteration caps, token budgets, and no-progress detectors are the three mandatory loop guards. The model's own stop signal is advisory only. - Build and validate a single-agent loop before adding multi-agent complexity — Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027, often because single-step reliability was never measured. > **In one line:** The agent loop is the perceive→reason→act→observe cycle that turns a static LLM into an autonomous process — and the hard-stop guardrails are the part that keeps it from spinning forever and billing you into next quarter. **The idea.** A regular LLM call is stateless: one prompt in, one response out, done. An agent is what happens when you wrap that call in a loop. The model reads its context, decides on an action (usually a tool call), the host application executes it, the result gets appended to the conversation, and the model runs again. Repeated N times, this is how an LLM books flights, writes and runs code, queries databases, and coordinates with other models. The loop is conceptually simple. Getting it to terminate correctly, stay within budget, and recover from tool failures is the part nobody warns you about until the bill arrives. **Key points.** - The five stages: Perceive (read context), Reason (think through the problem), Plan (decide the next action), Act (emit a tool call or final answer), Observe (append the tool result). - Tool calls are structured JSON the host application executes — the model never directly contacts APIs or runs code. - Hard termination guards are mandatory: iteration cap, token budget, no-progress detector. - Per-step accuracy compounds: 85% per step × 10 steps ≈ 20% end-to-end success. - Single-agent loops cost ~4× a chat call in tokens; multi-agent systems ~15×. ```mermaid flowchart LR S([Start]) --> P["Perceive\n(read context)"] P --> R["Reason\n(think + plan)"] R --> A["Act\n(tool call or answer)"] A -->|"tool_use"| EX["Host executes\ntool"] EX --> O["Observe\n(append result)"] O --> G{"Termination\nguards"} G -->|"continue"| P G -->|"cap hit / done"| END([End]) A -->|"end_turn"| END style S fill:#a855f7,color:#fff style END fill:#a855f7,color:#fff style EX fill:#0e7490,color:#fff style G fill:#ffaa00,color:#0a0a0f ``` **Key design decisions.** | Decision | Choice | Why | |---|---|---| | Termination | Hard iteration cap + token budget + no-progress detector | Model self-assessment of "done" is unreliable; explicit guards are not | | Tool execution | Host application, not the model | Keeps the model auditable; enables validation before execution | | Context growth | Append-only per iteration | Full history needed for coherent multi-step reasoning; compress if window fills | | Error handling | Return structured error to model; retry read-only tools up to N times, gate side-effecting ones | Model can adapt; halting on first error wastes partial progress — but a blind retry can double-execute a refund | | Cost control | Prompt caching on static segments; token budget per loop | ~4× token multiplier makes budgeting mandatory | **If you have 60 seconds, say this.** "An agent is a loop. The model reads its current context — system prompt, tool definitions, conversation history, and any previous tool results — then either emits a tool call or a final answer. If it emits a tool call, my application executes the tool, appends the result, and calls the model again. I put three guards on every loop: a hard iteration cap, a token budget checked before each call, and a no-progress check that fires if the last few tool calls are identical. The model's own stop signal is advisory — I always verify the actual output. Single loops cost about 4× a chat call in tokens; if I'm coordinating multiple agents it's closer to 15×. I cache the static parts of the prompt and set explicit budgets before deployment." A startup ran their customer support agent in production for three days before the pager went off. The agent had been looping on a single order lookup — the inventory API was returning a 429, the agent kept retrying, and nobody had set an iteration cap. The bill for those three days was $4,200. The task itself, if it had succeeded, would have cost about $0.20. This is the failure mode that the agent loop tutorial never shows you. ## What the loop actually is An **agent** is any system where an LLM output drives the choice of the next action, and that action's result feeds back as input to the next LLM call. That's it. No exotic architecture required. The minimal implementation is about 20 lines of Python. The loop has five conceptual stages that every framework eventually reduces to: **Perceive** — the model reads its full current context: system prompt, tool definitions, conversation history, prior tool results. Everything it knows about the task lives here. There is no other memory by default. **Reason** — the model generates a response. For a capable model on a hard problem, this includes internal reasoning (visible as thinking tokens in models like Claude's extended thinking mode) before it commits to an action. **Plan** — the model decides what to do next. In practice this is embedded in the reasoning step: the output is either a tool call or a final answer, and that choice *is* the plan. **Act** — the model emits its decision. If it's a tool call, the output is a structured JSON object. If the task is done, the output is a text response. The model itself does nothing further: **the host application is responsible for executing any tool calls**. **Observe** — the host application runs the tool, gets a result, and appends both the tool call and its result to the conversation. The updated context becomes the input to the next Perceive step. ```mermaid sequenceDiagram participant U as User / Orchestrator participant M as LLM participant H as Host App participant T as Tool (API / DB / Code) U->>H: task + initial context loop Agent iteration H->>M: messages array (system + history) M-->>H: response (tool_use OR end_turn) alt tool_use H->>T: execute tool call T-->>H: tool result H->>H: append tool_use + tool_result to messages else end_turn H-->>U: final answer end end ``` The model never touches `T` directly. It never runs a network request. It emits JSON; your code runs the network request. This separation is not incidental — it is what makes agent behavior auditable and what lets you validate or reject tool calls before they execute. ## The tool call in detail When the model decides to call a tool, it emits a structured block. On Anthropic the stop reason is `tool_use`; on OpenAI it's `tool_calls` in the response. The payload looks like this: ```python # What the model returns (Anthropic SDK) { "type": "tool_use", "id": "toolu_01XNrGbp...", "name": "get_order_status", "input": { "order_id": "ORD-4892", "include_shipping": True } } ``` Your host application sees the stop reason, extracts this block, runs `get_order_status(order_id="ORD-4892", include_shipping=True)`, and appends both the call and the result back to the conversation: ```python # Minimal agent loop (Anthropic SDK) import anthropic client = anthropic.Anthropic() def run_agent(task: str, tools: list, tool_executor: callable, max_iterations: int = 10) -> str: messages = [{"role": "user", "content": task}] for iteration in range(max_iterations): response = client.messages.create( model="claude-sonnet-4-5", max_tokens=4096, tools=tools, messages=messages, ) # Append assistant response messages.append({"role": "assistant", "content": response.content}) if response.stop_reason == "end_turn": # Extract text content from the final response for block in response.content: if block.type == "text": return block.text return "" if response.stop_reason == "tool_use": tool_results = [] for block in response.content: if block.type == "tool_use": result = tool_executor(block.name, block.input) tool_results.append({ "type": "tool_result", "tool_use_id": block.id, "content": str(result), }) messages.append({"role": "user", "content": tool_results}) return "Max iterations reached without completing task." ``` This is the complete loop. Everything else — planning strategies, memory, multi-agent coordination — is built on top of this basic mechanism. The [ReAct, Reflexion, and Planning Patterns](/ai-engineering/articles/react-reflexion-planning-patterns) article covers the prompting strategies that change *how the model reasons* inside this loop; the loop mechanics are the same. ## Termination: the part that actually matters The most common production mistake is treating `stop_reason == "end_turn"` as authoritative. It is not. The model believes it is done. It might be wrong. It might have hallucinated a success without actually completing the task. It might have gotten a cached tool result and assumed the task was complete. You need three independent guards, enforced by your application, not the model: **1. Hard iteration cap.** Set it before deployment. For most tasks 8-15 iterations is enough; anything requiring more is probably a sign the task is poorly scoped or the tools are inadequate. Cap it at 20 for complex workflows and treat any run that hits the cap as a failure to investigate, not a success. **2. Token budget.** Check the remaining token budget *before* each LLM call. A loop that has consumed 80% of a 128k context window is increasingly unreliable — the model is attending to a degraded, overlapping history. Stop early and surface the partial result rather than letting the context rot. ``` Back-of-envelope: token cost per agent run System prompt + tool definitions: ~2,000 tokens (static, cacheable) Per-iteration overhead (avg): ~800 tokens (tool call + result) Each API call re-bills the full growing prefix, so 10 iterations bill: (2,000 + 800) + (2,000 + 1,600) + ... + (2,000 + 8,000) = ~56,000 input tokens total, not 10,000 At claude-sonnet-4-5 pricing (mid-2026, illustrative): Input: 56,000 × $3/1M = ~$0.17 Output: ~1,500 × $15/1M = ~$0.02 Total per run: ~$0.19 uncached With prompt caching (0.1× reads on the shared prefix): ~$0.06–0.07 Multi-agent (4 subagents, orchestrator), ~15× a chat turn: ~$0.70–0.80 per task uncached At 100,000 tasks/day: $70,000–80,000/day uncached Caching the shared, growing prefix cuts the input bill ~3× (~$0.12 saved per single-agent run = ~$12,000/day at 100k tasks) ``` **3. No-progress detector.** Compare the last N tool calls. If they are identical (same tool name, same inputs), the agent is oscillating. Stop it. If the last N tool results all contain error strings, the agent is stuck on a broken tool. Stop it and escalate. ```python def check_no_progress(messages: list, window: int = 3) -> bool: """Returns True if the last `window` tool calls are identical.""" tool_calls = [ (block.name, str(block.input)) for msg in messages if msg["role"] == "assistant" for block in (msg["content"] if isinstance(msg["content"], list) else []) if hasattr(block, "type") and block.type == "tool_use" ] if len(tool_calls) < window: return False return len(set(tool_calls[-window:])) == 1 ``` ## Why error rates compound so badly Here is the number that makes every agent demo look better than it is. [Interactive visualizer on the original page: error-compound — Per-step accuracy compounds fast] At 85% per-step accuracy — which is good, better than many production systems — a 10-step workflow succeeds end-to-end only about 20% of the time. The math is `0.85^10 ≈ 0.197`. At 95% accuracy per step, a 10-step workflow succeeds about 60% of the time. To hit 90% end-to-end success on a 10-step task, you need 99% per-step accuracy. This is why the single most important investment in agent reliability is not picking a better model for the orchestrator — it's shortening the chains. Three 5-step pipelines with checkpoints between them will reliably outperform one 15-step autonomous chain, because you get to validate the intermediate results and abort early if something is wrong. The [agent evaluation article](/ai-engineering/articles/agent-evaluation-and-failure-modes) covers the measurement methodology; the loop design principle is: keep chains short and verify at boundaries. ## What actually goes in the context The **context window** is the agent's only working memory within a loop. On each iteration, the model can see exactly what is in its messages array and nothing else. This sounds obvious but has non-obvious consequences. [Interactive visualizer on the original page: context-window — Agent context budget across iterations] At the start of a 10-iteration run, your context looks roughly like this: | Segment | Tokens (approx) | Notes | |---|---|---| | System prompt | 500–2,000 | Static; prime candidate for caching | | Tool definitions | 290–800 per tool set | 290-800 tokens just for Anthropic's tool-use system prompt injection | | User task | 50–500 | | | Per-iteration history | ~800 per iteration | Tool call + result, growing linearly | | Output reserve | 1,000–4,096 | Must keep headroom for the next response | After 10 iterations at 800 tokens each, you have consumed roughly 2,000 (static) + 8,000 (history) + 1,000 (output reserve) = ~11,000 tokens minimum. On a 128k context model that's fine. On a 32k model with verbose tool results, you'll hit the window in fewer than 20 iterations. The [context window management article](/ai-engineering/articles/context-window-management-agents) covers compression strategies; the budget math above tells you when to start caring. The static segments — system prompt, tool definitions — are the prime candidates for prompt caching. Tool definitions especially: they don't change between iterations, but they get re-sent on every call. On Anthropic, a cached prefix costs 0.1× the normal input price after the first call. On a 100k-task/day deployment, caching the 2,000-token static prefix saves real money (see the cost block above). ## The tool call round-trip in practice A few mechanics that bite people in production: **Parallel tool calls.** Both Anthropic and OpenAI allow the model to emit multiple tool calls in a single response. Execute them concurrently and send all results back in one turn — do not send each result as a separate turn. The model expects all the results it requested before it reasons again. ```python # Parallel execution of multiple tool calls import asyncio async def execute_tools_parallel(tool_calls, executor): tasks = [executor(tc.name, tc.input) for tc in tool_calls] results = await asyncio.gather(*tasks, return_exceptions=True) return [ { "type": "tool_result", "tool_use_id": tc.id, "content": str(r) if not isinstance(r, Exception) else f"Error: {r}", "is_error": isinstance(r, Exception), } for tc, r in zip(tool_calls, results) ] ``` **Tool errors should go back to the model, not crash your application.** When a tool fails, set `is_error: true` in the result and include a structured error message. The model will often adapt — try a different parameter, use a fallback tool, or ask for clarification. Hard-crashing on first tool failure wastes partial progress and produces worse user experience than letting the model recover. **But "let the model retry" only applies to read-only tools.** `get_order_status` can run five times with no harm; `issue_refund` cannot. Classify every tool as read-only or side-effecting before wiring it into a loop. For side-effecting tools, three rules: give each call an idempotency key so a retried request is deduplicated server-side; never auto-retry on a timeout, because the first call may have succeeded — check the resulting state instead; and gate irreversible actions (refunds, outbound emails, deletes) behind a human or policy approval step. Host-side execution exists precisely so you can hold a `tool_use` block, validate or approve it, and only then run it — use that hook. **Refusals are structured objects, not empty strings.** On OpenAI's API, when the model declines to produce a structured output, it returns a `refusal` field in the message. Code that blindly parses `message.content` will crash. Always check `message.refusal` (OpenAI) or inspect `stop_reason` (Anthropic) before extracting content. ## What breaks Agent loops fail in a small number of ways, most of them predictable: **Oscillation.** The agent calls the same tool with the same arguments repeatedly. Usually caused by: the tool returns an error the model doesn't understand, the model's belief about the task state is wrong, or the tool result contradicts what the system prompt implies is possible. Fix: no-progress detector (above) + structured error messages from tools. **Context rot.** After many iterations, the prompt contains so much overlapping and conflicting information that the model's responses degrade. Symptoms: the model starts ignoring earlier instructions, "forgets" constraints stated in the system prompt, or produces responses inconsistent with the conversation history. Fix: compress or summarize earlier turns when the context exceeds ~60% of the window. **Silent wrong answers.** The model emits `end_turn` with a confident-sounding answer that is factually wrong or incomplete. No exception is raised. No error is logged. The user gets garbage. This is the hardest failure mode to catch because it is invisible to your application. Fix: output validation on the final response — either a Pydantic/Zod schema check, a secondary LLM evaluator, or explicit post-task assertions specific to your task. **Runaway cost.** No iteration cap, verbose tool results, a task that keeps spawning subtasks. Fix: iteration cap + token budget enforced before each LLM call, not as a soft preference but as a hard gate. **Tool hallucination.** The model calls a tool that does not exist, or passes parameters that are not in the schema. This is rarer with good schema design but does happen. Fix: validate tool call names and parameters against your registered tool list before execution; return a structured error ("unknown tool") if the model invents one. For a comprehensive taxonomy of failure modes and the evaluation methodology to catch them systematically, see [Agent Evaluation and Why Agents Fail in Production](/ai-engineering/articles/agent-evaluation-and-failure-modes). ## How the agent loop connects to everything else The loop described here is the foundation. Every other concept in this section is a layer on top of it: [**Tool and function calling**](/ai-engineering/articles/tool-function-calling-structured-output) covers how tool schemas are designed and how the model uses them to decide what to call — the mechanism that turns "Act" into a structured API call. [**Agent Memory**](/ai-engineering/articles/agent-memory-short-long-term) covers what to do when the context window isn't enough — episodic stores, semantic vector databases, and procedural caches that persist knowledge across loop restarts. [**ReAct, Reflexion, and Planning Patterns**](/ai-engineering/articles/react-reflexion-planning-patterns) covers the prompting strategies — ReAct's interleaved thought-action traces, Reflexion's self-critique loop, and Plan-and-Execute's upfront decomposition — that change what the model does in the Reason and Plan stages. [**Multi-Agent Orchestration**](/ai-engineering/articles/multi-agent-orchestration-patterns) covers what happens when you run multiple agent loops in coordination — the orchestrator-worker pattern, dynamic handoffs, and the state management that keeps context from getting lost between agents. [**Model Context Protocol**](/ai-engineering/articles/model-context-protocol-mcp) covers the standardized protocol (adopted across OpenAI, Anthropic, Google, and Microsoft by mid-2025) that separates tool servers from agent logic, so the same tool integrations work across different agent frameworks. ```mermaid flowchart TD LOOP["Agent Loop\n(this article)"] TOOLS["Tool Calling\n& Schemas"] MEMORY["Agent Memory\n(4 tiers)"] PLANNING["ReAct / Reflexion\n/ Planning Patterns"] MULTI["Multi-Agent\nOrchestration"] MCP["Model Context\nProtocol"] EVAL["Agent Evaluation\n& Failure Modes"] LOOP --> TOOLS LOOP --> MEMORY LOOP --> PLANNING LOOP --> MULTI TOOLS --> MCP MULTI --> EVAL LOOP --> EVAL style LOOP fill:#a855f7,color:#fff style TOOLS fill:#0e7490,color:#fff style MEMORY fill:#0e7490,color:#fff style PLANNING fill:#0e7490,color:#fff style MULTI fill:#0e7490,color:#fff style MCP fill:#0e7490,color:#fff style EVAL fill:#ff2e88,color:#fff ``` ## The decision in practice Should you build an agent loop, or just chain a few LLM calls? The honest answer: chain LLM calls first. A fixed three-call pipeline — classify the request, retrieve relevant data, generate the response — is predictable, cheap, and easy to test. Add a loop when the task requires a number of steps that you cannot enumerate upfront, or when the sequence depends on the output of prior steps in a way you can't hardcode. When you do build a loop, start with a single agent before coordinating multiple. Anthropic's explicit guidance is to add agent complexity only when simpler solutions fall short. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, and in practice most multi-agent failures trace back to the same root cause: single-step reliability was never measured before multi-agent complexity was added. The agent loop is not magic. It's a while loop with an LLM inside it. The model is good at reasoning about what to do next; it is not good at knowing when to stop, managing its own costs, or recovering from tool failures without explicit structure. Your job as the engineer is to provide that structure. Set the guards. Measure the per-step accuracy before you measure anything else. And get the termination logic right before you worry about which planning strategy the model uses. The loop is simple. The guards are what make it work. ## Frequently asked questions Q: What is the agent loop in AI systems? A: The agent loop is the repeating perceive→reason→act→observe cycle that transforms a single LLM call into an autonomous process. On each iteration the model reads its current context (perceive), generates reasoning and a next action (reason/plan), the host application executes that action — typically a tool call (act), and the result is appended to context (observe). The loop repeats until an explicit termination condition is met or a hard cap is reached. Q: How much do agents cost compared to a regular chat call? A: Single-agent loops consume roughly 4× the tokens of a standard chat turn, because each iteration appends tool definitions, tool call output, and the growing conversation history to the prompt. Multi-agent systems that coordinate several loops consume approximately 15× more tokens than a single chat turn. These multipliers make token budgeting and prompt caching mandatory, not optional, before any production deployment. Q: What causes an agent loop to run forever or get stuck? A: Infinite loops happen when there is no explicit termination criterion and the model is left to decide when it is done. In practice, three failure modes cause most stuck loops: the agent emits repeated identical tool calls (oscillation), reaches a dead end where no available tool can make progress (stalling), and hallucinates a success message without actually completing the task (silent wrong answer). Hard iteration caps and token budget limits are the standard guards — not relying on the model's own judgment. Q: What is the difference between ReAct and a basic agent loop? A: The basic agent loop is the infrastructure: context in, reasoning + tool call out, tool result appended, repeat. ReAct is a prompting strategy that runs on top of this infrastructure: it asks the model to interleave Thought steps (explicit reasoning traces) with Action steps before emitting tool calls. On ALFWorld, ReAct beat an act-only baseline by roughly 25 points of absolute success rate and imitation-learning baselines by 34 points (Yao et al., 2022), but it is a prompting technique layered on the same loop mechanics. The agent loop article covers the loop itself; the ReAct article covers the planning strategies you choose to run inside it. Q: What is a tool call and how does it work in an agent loop? A: A tool call is a structured JSON object the model emits instead of a text response when it decides an external capability is needed. The host application inspects the stop reason (tool_use on Anthropic, tool_calls on OpenAI), extracts the JSON, executes the named function, and appends the result back to the conversation. The model never directly runs code or contacts APIs — the host application does. This separation is what makes agents auditable and controllable. Q: How do I know when to stop an agent loop? A: Never leave termination to the model alone. Set three independent guards: a hard iteration cap (8-15 steps for most tasks), a token budget (checked before each LLM call), and a no-progress detector (if the last N tool calls are identical or return identical errors, halt). A fourth optional guard is a wall-clock timeout. The model's own stop signal (end_turn / finish_reason: stop) is a soft signal that indicates the model believes it is done — treat it as advisory, not authoritative, and always verify the actual task output independently.