Code Mode: why agents write code instead of calling tools

Attach a dozen MCP servers to your agent and print the first request. Before the user has typed anything, you are carrying tens of thousands of tokens of JSON schemas — every parameter description for every tool on every server, on every single turn. Now give it a small task: "attach the transcript of yesterday's customer call to the Salesforce lead." The Drive tool returns the transcript — call it 20,000 tokens — into context. Then the model passes it to the Salesforce tool the only way it can: by re-emitting the entire transcript, token by token, as a JSON argument. At output-token prices. At decode speed, that's several minutes of a frontier model retyping a document from one side of its own context window to the other. Nothing failed. That's the problem.

In the fall of 2025, engineering teams at Cloudflare and then Anthropic published the same conclusion within six weeks of each other: stop making the model call tools one JSON blob at a time. Make it write code against the tools instead. Anthropic's post calls it code execution with MCP; Cloudflare calls it Code Mode. The numbers they report are not subtle, and the architecture change that produces them is worth understanding precisely — including where it makes things worse.

The two taxes on JSON tool calling

The standard agentic loop — model emits a tool_use block, runtime executes it, result gets appended to the transcript, model runs again — charges two separate taxes, and they compound.

Tax one: O(number-of-tools) context, paid every turn. Each tool definition is a name, a description, and a JSON Schema for its parameters. A well-designed schema with real descriptions runs 300-1,000 tokens. MCP made connecting tool servers nearly free for developers, which means agents now routinely carry five servers with twenty tools each: 100 definitions, somewhere between 30K and 70K tokens, resident in context on every turn whether the task needs one tool or none. The pathological end of this curve is real: Cloudflare's own API has over 2,500 endpoints, and exposing each as an MCP tool would cost about 1.17 million tokens — several times more than most models' entire context window, before any conversation happens.

Tax two: O(steps) round trips, with data transiting the model. Every intermediate result enters the context window as a tool result. If the model needs to pass that data to another tool, it exits the model as generated output — the retyping problem from the opening. Each step is a full model inference over an ever-growing transcript, so cumulative prefill grows quadratically with step count. Prompt caching blunts the bill and most of the prefill latency, but nothing blunts the context growth itself: a 10,000-row query result occupies context forever after, degrading attention over everything that follows.

{ "type": "context-window", "window": 200000, "title": "Where tool schemas and tool results sit in your context budget" }

Parallel tool calls help with latency — the model can emit several tool_use blocks in one turn, and a good runtime executes them concurrently. But every result still lands in context, and the model still has to read all of it. Concurrency shortens the wall clock. It does not shrink the bill.

The same task, twice

Make it concrete. Three tools: billing.list_failed_charges, crm.get_customer, slack.post_message. The task: summarize yesterday's 120 failed charges with customer emails, post to #billing-alerts.

The JSON tool-calling version looks like this on the wire:

sequenceDiagram
    participant M as Model
    participant R as Runtime
    participant B as Billing API
    participant C as CRM API

    M->>R: tool_use list_failed_charges
    R->>B: GET /charges?status=failed
    B-->>R: 120 records
    R-->>M: ~18K tokens of JSON into context
    loop 120 times — one full inference each
        M->>R: tool_use get_customer(id)
        R->>C: GET /customers/id
        C-->>R: customer record
        R-->>M: result appended to context
    end
    M->>R: tool_use post_message(summary)
    Note over M,R: every byte transited context — twice for anything passed onward

Each iteration of that loop is a model inference: prefill the whole growing transcript, decode one small tool call. The model is being used as a for loop, which is roughly the most expensive for loop money can buy.

Illustrative token math (3 tools, 120 records):

JSON tool calling
  tool schemas (3 verbose tools)               ~1,800 tokens resident
  list_failed_charges result: 120 × ~150       ~18,000 tokens into context
  120 × get_customer (call + result ~180)      ~21,600 tokens
  final summary re-emitted as Slack argument    ~1,000 output tokens
  ------------------------------------------------------------------
  context by the final step                    ~42,000 tokens
  model inferences                             123
  wall clock at ~2 s per inference             ~4 minutes
  cumulative prefill (no caching)              123 × ~21K avg ≈ 2.6M input tokens

Code execution
  API files read on demand (3 files)           ~600 tokens
  generated script                             ~300 output tokens
  logged result                                ~150 tokens
  ------------------------------------------------------------------
  context total                                ~1,000-2,000 tokens
  model inferences                             1-2
  wall clock                                   API latency + ~1 s

The code version: the agent reads three small TypeScript files that were generated from the same tool schemas, then writes one script.

import * as billing from './servers/billing';
import * as crm from './servers/crm';
import * as slack from './servers/slack';

const charges = await billing.listFailedCharges({ since: '2026-07-17' });

const lines: string[] = [];
for (const charge of charges) {
  try {
    const customer = await crm.getCustomer({ id: charge.customerId });
    lines.push(`${customer.email}: $${(charge.amount / 100).toFixed(2)} — ${charge.failureReason}`);
  } catch (err) {
    lines.push(`${charge.customerId}: lookup failed (${(err as Error).message})`);
  }
}

await slack.postMessage({
  channel: '#billing-alerts',
  text: `${charges.length} failed charges yesterday:\n${lines.join('\n')}`,
});

console.log(`Posted summary: ${charges.length} charges, ${lines.length} lines.`);

The 120 customer records never enter the model's context. The try/catch handles the three customers that have been deleted from the CRM. The only thing that flows back to the model is the one-line console.log. One inference to write the script, maybe a second to confirm the result. That's the whole trade.

{ "type": "agent-loop", "scenario": "refund-support", "title": "The classic loop this replaces: one inference per step, results into context" }

Where the tokens go: progressive disclosure

The context savings come from two mechanisms, and it pays to keep them separate because they fail separately.

The first is on-demand tool discovery. Instead of injecting every schema into the system prompt, Anthropic's design presents servers as a filesystem: servers/google-drive/getDocument.ts, servers/salesforce/updateRecord.ts, each file a typed async function generated from the MCP schema. The agent lists directories to see what exists and reads only the two or three files the task needs. Their worked example — a workflow that previously loaded all tool definitions and shuttled intermediate results through context — dropped from 150,000 tokens to 2,000 tokens, a 98.7% reduction. For very large tool surfaces you add a search_tools step with a detail-level parameter (names only, names plus descriptions, or full schemas), so even discovery is metered. This is the same principle behind progressive disclosure in agent skills: the context window is not a config file, and nothing belongs there by default.

The second is data staying in the execution environment. Cloudflare's Code Mode server-side variant makes the fixed-footprint claim explicit: two umbrella tools, search() and execute(), backed by a typed representation of their OpenAPI spec. The model writes JavaScript to discover endpoints and then more JavaScript to call them — pagination, response checks, and multi-endpoint chaining all inside one execution. Their published comparison: ~1,000 tokens of context for a 2,500+ endpoint API, versus 1.17M tokens as raw tool schemas — 99.9% less, and the footprint stays flat no matter how many endpoints get added.

flowchart TD
    A["Agent lists ./servers/"] --> B["Reads the 3 tool files it needs (~600 tokens)"]
    B --> C["Writes one TypeScript script"]
    C --> D["Sandbox executes: loop, retries, joins"]
    D --> E["billing / crm / slack APIs"]
    E --> D
    D --> F["console.log — summary only"]
    F --> G["Model sees ~150 tokens, not ~40K"]
    style A fill:#0e7490,color:#fff
    style C fill:#00e5ff,color:#111
    style D fill:#15803d,color:#fff
    style E fill:#374151,color:#fff
    style G fill:#0e7490,color:#fff

There is a privacy corollary that is easy to miss: if data never enters the model's context, it never reaches the model provider either. Anthropic's post describes the MCP client tokenizing PII in flight — emails become [EMAIL_1], phone numbers [PHONE_1] — so real values flow from Google Sheets to Salesforce through the sandbox while the model only ever sees placeholders. For teams with data-residency constraints, that is a structural guarantee, not a policy promise.

Control flow moves into the code

The subtler shift is architectural: the agent loop stops being the only control-flow mechanism.

In classic tool calling, every loop and every retry is an agent-loop iteration — a model inference. Polling Slack every thirty seconds for a deploy notification means one inference per poll, each one re-reading the whole transcript to decide "not yet, check again." In code mode, that's a while loop with a sleep in the generated script, and the model hears about it exactly once, when the condition fires. Error handling migrates the same way: the try/catch in the script above replaces what would otherwise be a failed tool result entering context, the model reasoning about it, and another round trip for the retry.

And models are genuinely better at this. Cloudflare's argument for why is blunt: LLMs have trained on millions of real programs in public repositories, versus a comparatively tiny, synthetic diet of tool-call transcripts. Asking a strong coding model to express a workflow as TypeScript plays to the deepest part of its training distribution; asking it to express the same workflow as a novel JSON dialect does not. The evidence predates both blog posts: the CodeAct paper (ICML 2024) had agents act through executable Python instead of JSON or text formats and measured up to 20% higher absolute success rates across 17 LLMs on API-Bank and M3ToolEval, with fewer interaction turns — the gap widening on exactly the multi-tool composition tasks where control flow matters. Hugging Face's smolagents library turned that paper into the most accessible implementation: its default CodeAgent writes Python actions, and it is the fastest way to feel this pattern work in an afternoon — as long as you treat its local executor as a demo and use a sandboxed executor for anything real.

Practitioner reports on the MCP community discussion point the same direction, with more texture: one benchmark on a 5,205-issue dataset saw code execution succeed where direct MCP calls hit context overflow and failed outright, and on a small 45-issue dataset the code path was ~3.5× faster and ~67% cheaper. Not a controlled study — but consistent with the mechanism, which is what you'd expect: the wins concentrate where intermediate data is large and steps are many.

The sandbox is now load-bearing

Here is the bill for all of this. In classic tool calling, the model chooses from a fixed menu of actions you defined, and your runtime executes each one deterministically. In code mode, the model emits arbitrary programs. Combine that with the fact that any tool result — a webpage, a support ticket, a calendar invite — can carry injected instructions, and the threat model changes category: a successful prompt injection is no longer "the model calls a tool you didn't want." It is "the model writes exfiltration code and something runs it."

So the execution environment stops being a convenience and becomes a security boundary. Anthropic's post is direct about the cost side: running agent-generated code requires sandboxing, resource limits, and monitoring — operational overhead that direct tool calls simply don't have. Cloudflare's answer to the overhead problem is the most interesting engineering in either post: instead of containers, each script runs in a fresh V8 isolate — the same primitive as a browser tab — which starts in a handful of milliseconds and costs a few megabytes of memory. Isolates are cheap enough to be disposable per-script, and the sandbox is built with no filesystem, no environment variables, and outbound fetches disabled by default. Its only capabilities are explicit bindings to the connected MCP servers.

flowchart LR
    UNTRUSTED["Model-written code (untrusted)"] --> ISO["V8 isolate — no fs, no env vars, fetch disabled"]
    ISO -->|"binding"| MCP1["MCP server A"]
    ISO -->|"binding"| MCP2["MCP server B"]
    ISO -.->|"blocked"| NET["Open internet"]
    style UNTRUSTED fill:#dc2626,color:#fff
    style ISO fill:#15803d,color:#fff
    style MCP1 fill:#374151,color:#fff
    style MCP2 fill:#374151,color:#fff
    style NET fill:#374151,color:#fff

Notice what that diagram does not solve: the bindings themselves are the remaining exfiltration channel. If the sandbox can read from your CRM and write to Slack, injected code can move CRM data to Slack without ever touching the open internet. Capability design — which tools a given execution may bind, and what egress each binding permits — is where the real security work lives, and it is the same least-privilege discipline covered in sandboxing AI agents. A sandbox with generous bindings is a nicer-looking hole.

When direct tool calls still win

The token math is lopsided enough that it's tempting to declare tool calling dead. It isn't, and the cases where it wins are not edge cases.

Per-call auditability and approval. A tool_use block is a discrete, typed statement of intent: this tool, these arguments, right now. You can log it, diff it against policy, and put a human approval gate in front of exactly the dangerous ones. A 40-line generated script is a blob — a reviewer approves all of it or none of it, and static analysis of model-written code at approval time is a research problem, not a control. If your agent deletes production resources only after a human clicks yes, that click needs to be per-action, and that means tool calls (or a hybrid where specific bindings pause the sandbox and surface an approval).

Permissioning. Per-tool allowlists map one-to-one onto tool calling. In code mode, the equivalent is per-binding capabilities plus egress rules, which is more expressive and much easier to get wrong.

Low-latency single calls. For "what's the weather in Kyiv," authoring a program is pure overhead — more output tokens than the tool call it wraps, plus sandbox spin-up. One round trip, one small result: the classic path is faster and cheaper. The break-even arrives with either lots of tools in context or lots of data in motion, not before.

Tools that talk to the model, not past it. The MCP community pushback captured in discussion #1780 is worth taking seriously: plenty of MCP servers return guidance — instructions, elicitations, next-step hints — that only works if the model reads it. Routing those results around the model breaks the server's contract. Related: MCP tool output schemas are optional in the spec and unevenly adopted as of mid-2026, so generated code is often written against guessed response shapes. JSON tool calling degrades gracefully there (the model reads whatever comes back); code crashes.

Dimension	JSON tool calling	Code execution
Context cost of N tools	O(N), ~300-1,000 tokens each, every turn	~Fixed; definitions read on demand
K-step workflow	K inferences, quadratic cumulative prefill	1-2 inferences
Intermediate data	Enters context; re-emitted if passed onward	Stays in sandbox; model sees logs only
Auditability	Per-call, typed, gateable	Whole-script blob
Permissioning	Per-tool allowlists	Per-binding capabilities + egress rules
Single-call latency	One round trip, minimal output	Script authoring + sandbox overhead
Infrastructure	The agent loop you already have	Mandatory sandbox, codegen, monitoring
Injection blast radius	Bounded by tool menu	Arbitrary code; bounded only by sandbox

What breaks

Code written against imagined response shapes. The script above assumes charge.customerId exists. If the billing server actually returns customer_id, the script throws, the error goes back to the model, and it tries again — burning the round trips you were saving. Without enforced output schemas, this is the most common failure in practice, and it is why the codegen step (schemas → typed API files) matters as much as the sandbox: types turn silent shape-guessing into visible compile-time errors.

Debugging blind. Intermediate data staying out of context is the headline feature, and it is also the failure mode. When the script produces a wrong answer rather than an exception — filtered the wrong rows, joined on the wrong key — the model has no visibility into the values that flowed. Its natural recovery is to re-run everything with console.log on the raw data, which re-inflates the very context you were protecting, sometimes in one shot. Budget for this: give the sandbox a truncating logger, or you will occasionally pay full price plus the cost of the failed attempt.

Error-swallowing scripts. Models write plausible code, and plausible code includes catch blocks that hide failures. The example above degrades politely when a CRM lookup fails — but a slightly different generation swallows the error entirely and posts a summary of 87 charges labeled as 120. In the classic loop, every failed tool result is visible to the model by construction. In code mode, failure visibility is a property of the generated code, which means it is a property of luck unless your executor forces non-zero exits and stderr back into the transcript.

Injection with a compiler. A poisoned document in the transcript-summarization flow used to yield a bad summary or a rogue tool call. Now it can yield code — a loop that pages through the CRM and posts the results to the one place the bindings allow writes. The sandbox contains what the code can touch; it does nothing about what the code does with legitimate access. Treat every binding as an exfil path and scope accordingly.

State and partial completion. A script that dies at iteration 90 of 120 — timeout, resource limit, transient API failure — leaves external systems 75% mutated, and the model may see only "execution failed." The retry it writes next will re-run the first 90 side effects unless the code was idempotent, and nothing made it idempotent. The production patterns for reliable tool use — idempotency keys, checkpointing, reconciliation — don't disappear in code mode; they move into code the model writes, which means your API design has to make the safe pattern the obvious one.

The infrastructure you now own. Schema-to-TypeScript codegen that drifts when servers update. Isolate pools, execution timeouts, memory caps. Monitoring for what ran, per-execution audit records of which bindings were touched. None of it is exotic, all of it is real, and all of it is overhead the JSON path never asks for. This is the honest reading of Anthropic's own caveat: the pattern pays off when the token and latency savings outweigh a permanent new operational surface.

The decision in practice

The heuristic that survives contact with production, as of mid-2026:

Default to direct tool calls when the agent has under ~10 tools, tasks resolve in 1-3 steps, results are small, or any action needs per-call human approval. The schema tax is noise at that scale, and you keep typed intent, trivial permissioning, and free auditability. Most chat-with-tools products live here and should stay here.

Reach for code execution when tool count makes schemas a five-figure token line item, or when workflows are batch-shaped — many records, multiple systems, data flowing between tools that the model has no reason to read. That's where the 98.7% number lives, and where the error compounding of long agent loops hurts most. If you're doing this over MCP, generate typed APIs from the schemas rather than letting the model guess shapes, and treat the sandbox spec as seriously as Cloudflare does: no filesystem, no env, bindings only, fresh isolate per run.

Run both. The architectures compose cleanly: expose read-heavy, data-heavy tools through the code path, and keep destructive or approval-gated actions as direct tools that the sandbox cannot reach. The community argument about whether this belongs in the MCP spec — versus staying a client-side pattern — is still open in discussion #1780, and protocol-level progressive disclosure may eventually shrink the schema tax without any sandbox at all. The round-trip tax is different: no protocol change makes data that transits the model free. That asymmetry is why code execution is not a fad optimization but a durable architecture — the model becomes the control plane, and stops being the data plane. Which is, if you squint, just systems engineering rediscovering itself: we spent decades learning not to route bulk data through the coordinator.