All three major LLM API providers — OpenAI, Anthropic, and Google — now offer structured output capabilities that guarantee model responses conform to a specified JSON Schema. This is a fundamental shift from the early days of prompting models to “please return valid JSON,” which was unreliable and required extensive error handling and retry logic.
Under the hood, structured output APIs use constrained decoding — compiling your JSON Schema into a grammar and restricting token generation during inference so that only schema-valid tokens can be produced. The result is 100% structural validity: every required field is present, every value matches its declared type, and every enum value is valid. No parsing errors, no retries.
However, the three providers differ significantly in their API design, schema support, and implementation approach. Understanding these differences is critical for building portable applications, choosing the right provider for your use case, and avoiding subtle compatibility issues when switching between providers.
OpenAI was the first major provider to launch structured outputs (August 2024). Their implementation uses the response_format parameter with type: "json_schema". As of mid-2025, OpenAI uses Microsoft’s llguidance engine internally for constrained decoding, which expanded their JSON Schema coverage significantly.
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
response_format={
"type": "json_schema",
"json_schema": {
"name": "product_review",
"strict": True,
"schema": {
"type": "object",
"properties": {
"sentiment": {
"type": "string",
"enum": ["positive", "negative", "neutral"]
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"key_points": {
"type": "array",
"items": {"type": "string"}
},
"recommendation": {"type": "boolean"}
},
"required": ["sentiment", "confidence",
"key_points", "recommendation"],
"additionalProperties": False
}
}
},
messages=[{
"role": "user",
"content": "Analyze this review: 'Great product, fast shipping!'"
}]
)
result = json.loads(response.choices[0].message.content)When "strict": true is set, OpenAI enforces the schema via constrained decoding — invalid tokens are masked during generation. Without strict mode, the model is merely instructed to follow the schema but may still produce non-conforming output. Always use strict mode in production.
OpenAI’s implementation requires additionalProperties: false on all objects when using strict mode, and all fields must be listed in required. Optional fields are represented as union types with null: {"type": ["string", "null"]}. This is more restrictive than standard JSON Schema but ensures unambiguous constrained decoding.
Anthropic offers two complementary structured output mechanisms: JSON output format (controlling what Claude says) and strict tool use (controlling how Claude calls tools). Both use constrained decoding under the hood.
import anthropic
client = anthropic.Anthropic()
# Method 1: JSON output format (controls response structure)
response = client.messages.create(
model="claude-sonnet-4-6-20250514",
max_tokens=1024,
messages=[{
"role": "user",
"content": "Analyze: 'Great product, fast shipping!'"
}],
output_format={
"type": "json_schema",
"json_schema": {
"name": "product_review",
"schema": {
"type": "object",
"properties": {
"sentiment": {
"type": "string",
"enum": ["positive", "negative", "neutral"]
},
"confidence": {"type": "number"},
"key_points": {
"type": "array",
"items": {"type": "string"}
}
},
"required": ["sentiment", "confidence", "key_points"]
}
}
}
)
result = json.loads(response.content[0].text)# Method 2: Strict tool use (controls tool call parameters)
response = client.messages.create(
model="claude-sonnet-4-6-20250514",
max_tokens=1024,
tools=[{
"name": "analyze_review",
"description": "Analyze a product review",
"strict": True,
"input_schema": {
"type": "object",
"properties": {
"sentiment": {
"type": "string",
"enum": ["positive", "negative", "neutral"]
},
"confidence": {"type": "number"},
"key_points": {
"type": "array",
"items": {"type": "string"}
}
},
"required": ["sentiment", "confidence", "key_points"]
}
}],
tool_choice={"type": "tool", "name": "analyze_review"},
messages=[{
"role": "user",
"content": "Analyze: 'Great product, fast shipping!'"
}]
)
result = response.content[0].input # Already parsed dictThe tool-based approach has been available longer and is well-integrated with Claude’s agentic capabilities. The JSON output format is the newer addition, providing a cleaner API when you just want structured responses without the tool-calling semantics. Both can be combined in the same request for agentic workflows that need both structured tool calls and structured final responses.
Google enforces structured outputs natively in the Gemini API through a response_schema parameter. The schema is specified using a subset of the OpenAPI 3.0 schema specification, which differs slightly from standard JSON Schema.
import google.generativeai as genai
model = genai.GenerativeModel("gemini-2.0-flash")
response = model.generate_content(
"Analyze: 'Great product, fast shipping!'",
generation_config=genai.GenerationConfig(
response_mime_type="application/json",
response_schema={
"type": "object",
"properties": {
"sentiment": {
"type": "string",
"enum": ["positive", "negative", "neutral"]
},
"confidence": {"type": "number"},
"key_points": {
"type": "array",
"items": {"type": "string"}
}
},
"required": ["sentiment", "confidence", "key_points"]
}
)
)
result = json.loads(response.text)Gemini’s approach is the most straightforward API-wise — just set the MIME type and schema in the generation config. Through Vertex AI, Google also supports controlled generation for enterprise deployments with additional governance features.
OpenAI: Requires additionalProperties: false and all fields in required. Uses llguidance internally. Strict mode is opt-in via the strict flag. Supports deeply nested schemas.
Anthropic: Offers both JSON output format and strict tool use. Tool-based approach returns parsed objects directly. Supports Claude Opus 4.6, Sonnet 4.6/4.5, and Haiku 4.5.
Google: Uses OpenAPI 3.0 schema subset (not standard JSON Schema). Simplest API surface. Set via generation config with MIME type.
Since all three providers support JSON Schema (with minor variations), you can build a thin abstraction layer that normalizes the API differences. Define your schemas as Pydantic models, then convert to each provider’s expected format. Libraries like Instructor and LiteLLM already provide this abstraction, letting you switch providers with a single parameter change.
Structured outputs are ideal for extracting typed data from unstructured text at scale — invoices, resumes, medical records, legal documents. Define the extraction schema once, and process thousands of documents with guaranteed output consistency. No parsing failures means no manual intervention in the pipeline.
When an LLM powers a backend API endpoint, structured outputs guarantee the response matches the API contract. Your API consumers get consistent, typed responses regardless of the prompt content. This eliminates an entire category of production incidents where malformed LLM output breaks downstream systems.
In agentic systems where one LLM call’s output feeds into the next step, structured outputs ensure clean handoffs. The planning step produces a structured plan object, the execution step produces structured action results, and the evaluation step produces a structured assessment — all guaranteed to parse correctly without intermediate validation.
For most applications, the choice comes down to model quality and pricing rather than structured output capabilities, since all three providers now offer comparable schema enforcement. However, consider these edge cases:
$ref, oneOf), OpenAI’s llguidance backend currently supports the broadest subsetresponse_format with json_schema type and requires strict mode for guaranteed compliance; internally powered by llguidanceoutput_format for structured responses and strict: true on tools for validated tool calls, which can be combinedresponse_schema in generation config with OpenAPI 3.0 schema format — the simplest API surface