# JSON Mode vs Structured Outputs vs Tool Calling: What Actually Differs
> Three APIs that all return JSON but with very different guarantees. Understand when each breaks, what they cost, and which to reach for in production.
Canonical: https://ironclad.academy/ai-engineering/articles/json-mode-vs-structured-outputs-vs-tool-calling | Difficulty: beginner | Topics: Tool Use, Structured Outputs, Fundamentals, Prompting
Covers: OpenAI, Anthropic
## Key takeaways
- JSON mode guarantees valid JSON syntax only — it does not enforce your schema, and fields can go missing or have wrong types.
- OpenAI Structured Outputs with strict: true is enforced at inference time via constrained decoding; a schema-invalid token is literally impossible to generate.
- Tool calling and structured outputs serve different purposes: tool calling triggers actions; structured outputs format responses. Conflating them leads to overengineered pipelines.
- strict: true and parallel_tool_calls are mutually exclusive on OpenAI — a concrete footgun that silently breaks in production.
- Schema conformance is not semantic correctness: a perfectly valid JSON object can contain a hallucinated order ID, an out-of-range price, or a foreign key that does not exist.
- Anthropic's tool-use system prompt adds 290–804 tokens per request, and each tool definition adds ~50–200 more — trim the tool list to what the current step actually needs.
> **In one line:** JSON mode, Structured Outputs, and tool calling all produce JSON — but they enforce different contracts, trigger different model behaviors, and fail in completely different ways.
**The idea.** JSON mode is a soft instruction: "please output valid JSON." Structured Outputs with strict mode is a hard constraint: the inference engine masks invalid tokens at every decoding step so a schema-violating character literally cannot be produced. Tool calling is something else entirely — the model is not formatting a response, it is deciding to invoke an external action. Mixing up these three leads to either weaker guarantees than you thought, unnecessary complexity, or silent failures that show up three weeks after launch.
**Key points.**
- JSON mode → syntactically valid JSON, no schema guarantee, cheap to use, easy to break downstream parsers.
- Structured Outputs (`strict: true`) → schema-enforced JSON via constrained decoding, requires schema to satisfy strict constraints (`additionalProperties: false`, all fields required).
- Tool calling → action invocation, not response formatting; the model returns `tool_use` blocks your application executes.
- OpenAI `strict: true` and `parallel_tool_calls` are mutually exclusive.
- Schema conformance still leaves semantic validation entirely to you.
```
Back-of-envelope: tool definition token cost on Anthropic
• Base tool-use system prompt injection: 290–804 tokens per request
• A tool set of 20 tools at ~100 tokens each = 2,000 tokens per request
• At $3/M input tokens (illustrative mid-2026 frontier pricing) and 10k req/day:
20 tools × 100 tok × 10,000 req × $3/1M = $60/day just in tool definitions
• Trim to the 3–5 tools the current step needs → ~$9–15/day
```
**Three-way comparison.**
| | JSON Mode | Structured Outputs (`strict: true`) | Tool Calling |
|---|---|---|---|
| Schema enforcement | None | Inference-time, hard | Input schema only |
| Parallel calls | N/A | Incompatible on OpenAI | Yes (with caveats) |
| Use case | Quick extraction | Data extraction, response formatting | Action invocation |
| Failure mode | Missing fields, wrong types | Semantic errors, refusal object | Hallucinated tool name, wrong args |
| Setup cost | Low | Schema compilation on first use | Tool definitions in every request |
```mermaid
flowchart TD
Q["Do you need the model to\ntrigger an action or call an API?"]
Q -->|Yes| TC["Tool Calling\ntool_use block → your code executes"]
Q -->|No| Q2["Do you need schema\nconformance guaranteed?"]
Q2 -->|No| JM["JSON Mode\nvalid syntax only"]
Q2 -->|Yes| SO["Structured Outputs\nstrict: true"]
SO --> Q3["Need parallel tool calls?"]
Q3 -->|Yes| NOTE["Can't have both on OpenAI:\ndrop strict on tools and validate\nargs yourself, or serialize the calls"]
Q3 -->|No| GOOD["All good — compile schema\nand set additionalProperties: false"]
style TC fill:#a855f7,color:#fff
style JM fill:#0e7490,color:#fff
style SO fill:#15803d,color:#fff
style NOTE fill:#ff2e88,color:#111
```
**If you have 60 seconds, say this.** "JSON mode tells the model to output valid JSON — that's it. If a field goes missing or has the wrong type, you find out when your code crashes. Structured Outputs with strict mode compiles your schema into a grammar and masks out invalid tokens during generation, so a schema violation is literally impossible. Tool calling is a different concept: the model returns a structured action request, your code executes it, and you feed the result back. The three are not interchangeable — pick based on whether you need an action (tool calling), guaranteed schema shape (strict structured outputs), or just parseable JSON (mode). And if you're on OpenAI with strict mode, remember that parallel tool calls are off the table — those two features don't coexist."
Your extraction pipeline has been running fine for six weeks. Then a field goes missing in a response — `confidence` is just absent — and three downstream services throw `KeyError` in the same deploy window. No error was logged by the LLM. The JSON was syntactically valid. JSON mode did exactly what it promised: it returned valid JSON. It just didn't promise your schema.
That specific failure is the reason three distinct mechanisms exist. They look similar from the outside — you pass a schema, you get JSON back — but they enforce different contracts, fail in different ways, and carry different costs. Getting this wrong means either weaker guarantees than you assumed, or unnecessary complexity that adds latency and tokens for no benefit.
## What JSON mode actually guarantees
JSON mode is a soft instruction. You tell the model to output JSON, and the model — being a next-token predictor — tries to comply. On strong frontier models with clean prompts, it succeeds almost all the time. "Almost" is the problem.
What JSON mode guarantees: the output will parse without a `json.JSONDecodeError` — provided generation actually finishes. Hit your `max_tokens` limit mid-object and even that fails (more on this in [What breaks](#what-breaks)). That's it. Fields can be absent. Types can be wrong. Required nested objects can be replaced with `null`. The model can add fields you didn't ask for. None of this violates the JSON mode contract.
```python
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": "Extract order data as JSON with fields: order_id (string), amount (float), currency (string)."
},
{
"role": "user",
"content": "Process order #A-2948 for $129.99"
}
]
)
import json
data = json.loads(response.choices[0].message.content)
# data might be {"order_id": "A-2948", "amount": 129.99, "currency": "USD"}
# or it might be {"order_id": "A-2948", "total": 129.99} ← "amount" missing
# or {"order_id": "A-2948", "amount": "129.99", "currency": "USD"} ← wrong type
```
This is not hypothetical sloppiness — it happens under distributional shift (prompts the model hasn't seen variants of), under brevity pressure (the model decided `currency` was obvious from context and omitted it), or when the schema is complex enough that the model loses track of a nested requirement.
JSON mode is fine for exploratory work, for prototypes, and for cases where you control validation downstream and can handle malformed output gracefully. OpenAI itself says in its documentation: use Structured Outputs instead whenever possible.
## Structured Outputs with strict mode
Structured Outputs changes the enforcement layer from "the model tries to follow instructions" to "the inference engine prevents invalid tokens from being generated."
At each decoding step, the server compiles your JSON schema into a grammar (a pushdown automaton or finite-state machine, depending on the implementation), then applies a binary mask over the vocabulary. Tokens that would produce output violating the grammar get zeroed out before sampling. The model samples from what's left. See [structured outputs and constrained decoding](/ai-engineering/articles/structured-outputs-constrained-decoding) for the full mechanism.
The result: a field cannot go missing because the grammar requires it. A string cannot appear where a number is required because `"` is masked out at that position. The output is schema-valid by construction, not by instruction.
```python
from openai import OpenAI
from pydantic import BaseModel
client = OpenAI()
class OrderExtraction(BaseModel):
order_id: str
amount: float
currency: str
response = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06", # strict mode requires this or newer
messages=[
{
"role": "system",
"content": "Extract order data."
},
{
"role": "user",
"content": "Process order #A-2948 for $129.99"
}
],
response_format=OrderExtraction,
)
order = response.choices[0].message.parsed
# order.amount is guaranteed to be a float
# order.currency is guaranteed to be present
# order.order_id is guaranteed to be a string
```
The Python SDK's `.parse()` method handles the Pydantic-to-JSON-Schema conversion and sets `strict: true` automatically. Under the hood the schema sent to the API looks like:
```json
{
"type": "json_schema",
"json_schema": {
"name": "OrderExtraction",
"strict": true,
"schema": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"amount": {"type": "number"},
"currency": {"type": "string"}
},
"required": ["order_id", "amount", "currency"],
"additionalProperties": false
}
}
}
```
Those last two lines — `required` listing every field, `additionalProperties: false` — are not optional when using `strict: true`. They are required. Not because open schemas are impossible to constrain — JSON mode itself is a grammar over arbitrary valid JSON — but because with `additionalProperties` allowed, the grammar can no longer guarantee the output contains only the fields your code expects, which is the entire guarantee the feature exists to provide. So OpenAI only compiles closed schemas it can enforce completely, and rejects schemas that set `strict: true` without these constraints.
### The refusal case
One failure mode that trips up people new to strict mode: the API can return a refusal instead of a parsed object. If the input triggers a content policy refusal, `message.parsed` is `None` and `message.refusal` contains the refusal string. Code that directly accesses `message.parsed` without checking will raise `AttributeError` or `NullPointerException` depending on your language.
```python
if response.choices[0].message.refusal:
# handle refusal — log it, return a fallback, escalate
raise ValueError(f"Model refused: {response.choices[0].message.refusal}")
order = response.choices[0].message.parsed
```
This is not a JSON mode problem — JSON mode just returns text. It is specific to Structured Outputs because the strict API returns a typed response object with a clearly separable refusal path.
## Tool calling: a different concept
Tool calling gets conflated with structured outputs because both return JSON. They are solving different problems.
Structured outputs: "format your response as this schema."
Tool calling: "decide whether to invoke an external capability, and if so, with what arguments."
The distinction matters because tool calling changes the model's reasoning mode. When you provide tools, the model is not just formatting a response — it is deciding whether a function should be called at all, which function among possibly many, and what values to pass. The model returns a `tool_use` block (Anthropic) or `tool_calls` array (OpenAI) that your application code executes. The result comes back to the model as a `tool_result` message, and the model continues.
```mermaid
sequenceDiagram
participant App as Your Application
participant M as Model
participant T as External Tool
App->>M: user message + tool definitions
M-->>App: stop_reason: tool_use
tool_use {name: "get_order", input: {id: "A-2948"}}
App->>T: execute get_order(id="A-2948")
T-->>App: {status: "shipped", eta: "2026-03-25"}
App->>M: tool_result {content: {...}}
M-->>App: stop_reason: end_turn
final answer text
```
The model sees the tool definitions as part of its reasoning context. The descriptions and parameter names are what it uses to decide when to call a tool and how to populate it — they function as a prompt, not just API documentation. This is covered in depth in [designing tool schemas that models actually understand](/ai-engineering/articles/designing-tool-schemas-llm-agents) and [the tool-use round trip](/ai-engineering/articles/tool-use-round-trip-agentic-loop).
For this article, the key point is: if you are trying to extract structured data from text that does not require any external call, tool calling is the wrong mechanism. You would be abusing a function-calling interface to get JSON formatting, which adds schema-in-the-tool-description overhead and forces the model into an action-selection mode when you just want a formatted answer. Use structured outputs.
Use tool calling when:
- The operation has a side effect (write to a database, call an API, send a message).
- The model needs to retrieve information it does not have in context.
- The model must choose between multiple possible actions.
Use structured outputs when:
- You want the model's answer in a specific JSON shape.
- You are extracting entities, classifying content, generating a report.
- No external system needs to be invoked.
[Interactive visualizer on the original page: agent-loop — Tool-use round trip: model decides, app executes, model continues]
## The parallel calls footgun
Both OpenAI and Anthropic support parallel tool calls: the model can return multiple `tool_use` blocks in one response. You execute them concurrently, collect all results, and send them back in a single message. This is good for performance — parallel independent queries run in wall-clock time equal to the slowest one rather than their sum.
On OpenAI, `strict: true` on tool schemas and `parallel_tool_calls: true` are mutually incompatible. The combination either errors or silently falls back. You must set:
```python
response = client.chat.completions.create(
model="gpt-4o",
tools=[...],
tool_choice="auto",
parallel_tool_calls=False, # required when strict: true on any tool
messages=[...]
)
```
Missing this flag is one of the most common production bugs when migrating from JSON mode to strict structured outputs. It does not always error visibly — depending on the model version and the exact tool schema, it may silently disable strict enforcement instead of rejecting the request.
Anthropic's behavior differs. Parallel tool calls work fine with `strict: true` on client tools. The constraint only appears for server tools running in the same parallel group, which require explicit handling of turn boundaries. Check the Anthropic docs for the current server-tool rules, as the behavior has evolved through 2025.
## Token overhead you are paying regardless
Every mechanism carries token costs beyond the content itself. This is not a reason to avoid them — it is a reason to be deliberate.
```
Token overhead comparison (illustrative, mid-2026 models):
JSON mode:
• System prompt: your instructions only
• Schema: must be in the prompt as natural language or examples
• Per-request overhead: low (~100–300 tokens for schema description)
Structured Outputs:
• Schema is injected by the API, not in your prompt
• Per-request overhead: similar to JSON mode; schema compilation is server-side
• No extra user-visible tokens; the schema is metadata
Tool calling (Anthropic):
• Base tool-use system prompt: 290–804 tokens per request (varies by model + tool_choice)
• Each tool definition: ~50–200 tokens depending on description length
• 10 tools at 100 tokens each = 1,000 tokens per request
• At $3/M input tokens and 50k req/day:
1,000 tok × 50,000 req × $3/1M = $150/day in tool definitions alone
Mitigation: pass only the tools the current step needs, not your full tool catalog.
Anthropic's Tool Search Tool (2025) solves this for large catalogs: Claude loads tools on demand.
```
This math is why trimming your tool list matters at scale, and why Anthropic's Programmatic Tool Calling (launched November 2025 under `advanced-tool-use-2025-11-20`) reduces costs: Claude writes Python to orchestrate tool calls itself, so intermediate tool results stay in the code-execution environment instead of round-tripping through the model's context on every turn. On large workflows this cut billed input tokens by roughly 38% in Anthropic's benchmarks. (Loading definitions on demand is the Tool Search Tool's job, not PTC's.)
## What breaks
### Truncation voids the parse guarantee
Constrained decoding works token by token — it cannot conjure the closing braces if generation stops early. When the response hits your `max_tokens` limit, OpenAI returns `finish_reason: "length"` (Anthropic: `stop_reason: "max_tokens"`) and hands you JSON cut off mid-object. `json.loads` throws, strict mode or not. OpenAI's own docs tell you to check `finish_reason` before parsing, and this is a genuinely common production bug: the schema that fit comfortably in tests grows a 40-item array in production and blows past the limit.
```python
choice = response.choices[0]
if choice.finish_reason == "length":
raise ValueError("Response truncated at max_tokens — do not parse")
data = json.loads(choice.message.content)
```
Size `max_tokens` for your worst-case payload, not your median one, and treat a length finish as a hard error rather than attempting to repair the fragment.
### Schema conformance is not semantic correctness
This is the subtler failure and the one that survives strict mode. A perfectly schema-valid JSON object can contain a hallucinated order ID that does not exist in your database, a `confidence` value of `1.7` when your range is 0.0–1.0, or a `currency` of `"Euros"` when you specified ISO 4217 codes.
Constrained decoding enforces structural rules. It has no mechanism to enforce business logic. You need a separate validation layer:
```python
from pydantic import BaseModel, field_validator
class OrderExtraction(BaseModel):
order_id: str
amount: float
currency: str
@field_validator("currency")
@classmethod
def must_be_iso_4217(cls, v):
valid = {"USD", "EUR", "GBP", "JPY", "CAD", "AUD"}
if v not in valid:
raise ValueError(f"currency must be ISO 4217, got {v!r}")
return v
@field_validator("amount")
@classmethod
def must_be_positive(cls, v):
if v <= 0:
raise ValueError(f"amount must be positive, got {v}")
return v
```
When Pydantic raises `ValidationError`, re-invoke the model with the error message appended as a user turn. This typically corrects semantic errors in one retry. Do not re-prompt from scratch — the error context is what helps.
### Hallucinated tool names
When the model invents a tool that does not exist in your definitions, it returns a `tool_use` block with a name your application has no handler for. This is a tool-calling-specific failure that structured outputs do not have.
Guard against it with `tool_choice: {"type": "tool", "name": "..."}` when you know exactly which tool should be called, or `tool_choice: "required"` combined with validating the returned name against your registry before executing.
### Schema rejection at compile time
Schemas that are technically valid JSON Schema but are not compilable to a finite automaton will be rejected by strict mode at request time. Common causes:
- Using `oneOf` or `anyOf` at the top level without a discriminator field.
- Recursive schemas (a `Comment` type that contains an array of `Comment` children).
- `additionalProperties: true` or absent while `strict: true` is set.
The error messages from OpenAI are reasonably descriptive. Test your schemas before deploying — compile them against the API once in your test suite and fail fast.
```mermaid
flowchart LR
S["JSON Schema"] --> C["Grammar compiler\n(server-side, one-time)"]
C -->|success| G["Compiled automaton\n(cached, reused)"]
C -->|failure| E["Schema rejection error\nat request time"]
G --> D["Inference: mask invalid\ntokens at each step"]
D --> O["Schema-valid output\n(structural guarantee only)"]
style E fill:#ff2e88,color:#111
style O fill:#15803d,color:#fff
```
### Context rot in tool loops
Tool calling introduces a loop: model call → tool execution → model call. Each turn appends to the context. After five or ten turns, the context contains the full history of every tool call and result, which can push you toward the context limit or shift the model's attention toward earlier turns and away from the current task.
The [tool-use round trip article](/ai-engineering/articles/tool-use-round-trip-agentic-loop) covers loop management in detail. The short version: cap iterations explicitly, summarize intermediate results when context exceeds a threshold, and never assume an agentic loop will converge on its own.
## A concrete worked example: data extraction vs action invocation
Suppose you are building an order management system. Two tasks:
**Task A:** Parse an email and extract the order details.
Use structured outputs. No external system needs to be called. The model reads the email and formats the answer.
```python
class OrderDetails(BaseModel):
order_id: str
items: list[str]
total_amount: float
requested_delivery_date: str # ISO 8601
response = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role": "user", "content": f"Extract order details from: {email_text}"}
],
response_format=OrderDetails,
)
```
**Task B:** Process the order — check inventory, create the order record, send a confirmation.
Use tool calling. Three tools: `check_inventory`, `create_order`, `send_confirmation`. Each triggers an actual operation with side effects.
```python
tools = [
{
"type": "function",
"function": {
"name": "check_inventory",
"description": "Check if the requested items are in stock. Returns availability status per item.",
"parameters": {
"type": "object",
"properties": {
"item_ids": {
"type": "array",
"items": {"type": "string"},
"description": "List of item SKUs to check"
}
},
"required": ["item_ids"],
"additionalProperties": False
},
"strict": True
}
}
# ... create_order, send_confirmation omitted for brevity
]
```
Notice `strict: True` on the tool schema. This ensures the model's tool arguments always match the schema. Notice `additionalProperties: False` — required for strict. And for this scenario, `parallel_tool_calls=False` if you set `strict=True` on any tool.
The two tasks use different mechanisms for good reason. Task A needs a formatted answer; Task B needs sequenced action invocation with real-world side effects.
## Anthropic-specific behavior
On Anthropic's API, the parallel pattern is `tool_use` content blocks in the assistant message, and `tool_result` blocks in the subsequent user message. The `stop_reason` of `"tool_use"` signals that the model wants tools executed before it continues.
```python
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
tools=[
{
"name": "get_weather",
"description": "Get current weather for a location.",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"}
},
"required": ["location"]
}
}
],
messages=[{"role": "user", "content": "What's the weather in Tokyo?"}]
)
if response.stop_reason == "tool_use":
tool_use_block = next(b for b in response.content if b.type == "tool_use")
tool_result = call_weather_api(tool_use_block.input["location"])
# Continue the conversation
follow_up = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
tools=[...], # same tools
messages=[
{"role": "user", "content": "What's the weather in Tokyo?"},
{"role": "assistant", "content": response.content},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": tool_use_block.id,
"content": str(tool_result)
}
]
}
]
)
```
For structured data extraction on Anthropic without tool calling, the standard approach is to describe the JSON schema in the system prompt (prefilling the assistant turn with `{` helps) — or use tool calling as a formatting trick (pass a single tool named `extract_data` and force its invocation with `tool_choice: {"type": "tool", "name": "extract_data"}`). Anthropic does not have a first-party strict structured output API as of mid-2026 that's exactly equivalent to OpenAI's `response_format: json_schema` with `strict: true`, though their tool `strict: true` mode (2025 beta) gives schema enforcement for tool inputs.
```mermaid
sequenceDiagram
participant App
participant Claude
App->>Claude: user message + tools (stop_reason pending)
Claude-->>App: stop_reason=tool_use
content: [tool_use {id, name, input}]
Note over App: Execute tool synchronously or async
App->>Claude: messages with tool_result
{tool_use_id, content}
Claude-->>App: stop_reason=end_turn
content: [text block]
```
## The MCP angle
The [Model Context Protocol](/ai-engineering/articles/model-context-protocol-mcp) (spec version 2025-06-18) added `outputSchema` to tool definitions. When a server provides `outputSchema`, it declares that its tool's return value will conform to that schema. Clients should validate tool results against it.
This is structured outputs extended to tool results: not just tool inputs (which strict mode already covers), but tool outputs too. It shifts validation left into the protocol layer, so your application doesn't have to write custom validators for every tool response.
```json
{
"name": "get_order_status",
"description": "Returns the current status and estimated delivery of an order.",
"inputSchema": {
"type": "object",
"properties": {
"order_id": {"type": "string"}
},
"required": ["order_id"]
},
"outputSchema": {
"type": "object",
"properties": {
"status": {"type": "string", "enum": ["processing", "shipped", "delivered"]},
"eta": {"type": "string", "format": "date"}
},
"required": ["status"],
"additionalProperties": false
}
}
```
If you are building MCP servers or consuming tools through MCP clients, this is the pattern to follow. It makes the contract machine-readable at both ends of the call.
## The decision in practice
Here is the actual decision process used in production:
**Do you need to call an external API, execute code, read from a database, or take any action with a side effect?** Tool calling. Full stop.
**Do you need the model's answer in a specific JSON structure?** Structured Outputs with `strict: true`. Define a Pydantic model, use the `.parse()` API, add semantic validators on top.
**Are you working with an older model that does not support strict structured outputs, or a self-hosted model without constrained decoding?** JSON mode with mandatory downstream validation. Treat every response as untrusted until validated. See [structured data extraction with LLMs](/ai-engineering/articles/llm-structured-data-extraction) for validation patterns.
**Is your schema too complex to satisfy strict mode's constraints** (recursive types, dynamic field sets, polymorphic unions without discriminators)? Consider splitting the extraction into multiple simpler calls, or use JSON mode with strict Pydantic validation on the output.
One last thing that does not fit neatly into a table: the model's behavior shifts based on which mechanism you use. Tool calling puts the model in an action-selection mode — it is asking itself "should I call something, and if so what?" Structured outputs put it in a formatting mode — "how do I express this answer in the required shape?" For extraction tasks, structured outputs typically give better semantic quality than forcing extraction through a tool call, because the model's reasoning is oriented toward the right goal. For agentic tasks, tool calling is correct because the model needs to reason about whether to act at all.
Use the mechanism that matches the model's task, not just the output format you want.
## Frequently asked questions
Q: What is the difference between JSON mode and Structured Outputs in OpenAI?
A: JSON mode guarantees that the model returns syntactically valid JSON, but makes no promise that the output matches any schema you provide. OpenAI Structured Outputs with strict: true enforces schema compliance at inference time via constrained decoding — the model physically cannot generate a token that would violate the schema. OpenAI explicitly recommends Structured Outputs over JSON mode for any application that depends on a specific field shape.
Q: When should I use tool calling instead of structured outputs?
A: Use tool calling when the model needs to decide whether and which action to take — calling an API, running a query, retrieving a document. The model returns a tool_use block that includes the tool name and arguments, and your application executes the action. Use structured outputs when you want the model's final answer formatted as JSON — data extraction, classification results, report generation. The distinction is: tool calling triggers side effects; structured outputs format the response.
Q: Does strict mode in structured outputs slow down generation?
A: On the first request with a new schema, the provider compiles the schema into a grammar automaton, which adds latency. Subsequent requests reuse the compiled grammar, so the overhead is amortized. OpenAI's documentation notes a one-time compilation cost on schema first-use — typically seconds, and up to a minute for complex schemas — then effectively zero once the compiled grammar is cached. Warm new schemas in CI or at deploy time so production traffic never pays it.
Q: Why does OpenAI strict mode require additionalProperties: false?
A: Constrained decoding works by compiling your schema into a grammar that masks invalid tokens at each generation step. If additionalProperties were allowed, the model could emit field names your schema never declared, and the grammar could no longer guarantee the output contains only the fields your code expects — the guarantee the feature exists to provide. Requiring additionalProperties: false and marking all fields required gives OpenAI a closed schema it can enforce completely.
Q: Can I use parallel tool calls with strict structured outputs on OpenAI?
A: No — these two features are mutually incompatible on OpenAI as of mid-2026. When strict: true is set on a tool schema, you must also set parallel_tool_calls: false, or the API will either error or silently fall back to non-strict mode. This is a documented constraint that trips up many teams moving from JSON mode to strict structured outputs.
Q: What does Anthropic Programmatic Tool Calling do?
A: Programmatic Tool Calling (launched November 2025, beta header advanced-tool-use-2025-11-20) lets Claude write Python to orchestrate tool calls itself rather than returning explicit tool_use blocks for your application to dispatch. On large workflows this reduces billed input tokens by roughly 38%, because intermediate tool results stay in the code-execution environment instead of round-tripping through the model's context on every turn. Loading tool definitions on demand is a separate feature, the Tool Search Tool.