Structured output comparison across popular LLM providers - OpenAI, Gemini, Anthropic, Mistral and AWS Bedrock
Slightly different APIs require special approach.
Here’s a side-by-side support comparison of structured output (getting reliable JSON back) across popular LLM providers, plus minimal Python examples
We already had a look how to request structured output from the LLM hosted on Ollama. Here we review how to request the same from other providers.
TL;DR matrix
Provider | Native “JSON mode” | JSON Schema enforcement | Typical knob | Notes |
---|---|---|---|---|
OpenAI | Yes | Yes (first-class) | response_format={"type":"json_schema", ...} |
Works via the Responses API or Chat Completions; can also do function calling. |
Google Gemini | Yes | Yes (first-class) | response_schema= + response_mime_type="application/json" |
Returns strictly validated JSON when schema is set. |
Anthropic (Claude) | Indirect | Yes (via Tool Use with JSON schema) | tools=[{input_schema=...}] + tool_choice |
Force the model to “call” your schema-defined tool; yields schema-shaped args. |
Mistral | Yes | Partial (JSON-only; no server-side schema) | response_format={"type":"json_object"} |
Ensures JSON, but you validate against your schema client-side. |
AWS Bedrock (platform) | Varies by model | Yes (via Tool/Converse schema) | toolConfig.tools[].toolSpec.inputSchema |
Bedrock’s Converse API validates tool input against a JSON schema. |
LLM Structured Output - General Information
LLM structured output refers to the ability of large language models (LLMs) to generate responses that strictly adhere to a predefined, specific format or structure rather than producing free-form text. This structured output can be in formats like JSON, XML, tables, or templates, making the data machine-readable, consistent, and easily parsable by software for use in various applications.
Structured outputs differ from traditional LLM outputs, which typically generate open-ended, natural language text. Instead, structured outputs enforce a schema or format, such as JSON objects with defined keys and value types, or specific classes in the output (e.g., multiple-choice answers, sentiment classes, or database row formats). This approach improves reliability, reduces errors and hallucinations, and simplifies integration into systems such as databases, APIs, or workflows.
The generation of structured outputs in LLMs often involves techniques such as:
- Specifying detailed prompt instructions to guide the model to produce output in the desired format.
- Using validation and parsing tools like Pydantic in Python to ensure the output matches the schema.
- Sometimes enforcing decoding constraints based on grammar or finite state machines to ensure token-level compliance with the format.
Benefits of structured LLM outputs include:
- Machine-readability and ease of integration.
- Reduced variability and errors.
- Improved predictability and verifiability for tasks that require consistent data formats.
Challenges include designing effective schemas, handling complex nested data, and potential limitations in reasoning capabilities compared to free-form text generation.
Overall, structured output enables LLMs to be more useful in applications requiring precise, formatted data rather than solely human-readable text.
Structured Output Python examples
All snippets extract event info as JSON: {title, date, location}
. Replace keys/models as you like.
1) OpenAI — JSON Schema (strict)
from openai import OpenAI
import json
client = OpenAI()
schema = {
"name": "Event",
"schema": {
"type": "object",
"properties": {
"title": {"type": "string"},
"date": {"type": "string", "description": "YYYY-MM-DD"},
"location": {"type": "string"}
},
"required": ["title", "date", "location"],
"additionalProperties": False
},
"strict": True
}
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "user", "content": "Extract the event from: 'PyData Sydney is on 2025-11-03 at Darling Harbour.'"}
],
response_format={"type": "json_schema", "json_schema": schema},
)
data = json.loads(resp.choices[0].message.content)
print(data)
OpenAI’s Structured Outputs feature enforces this schema on the server side.
2) Google Gemini — response schema + JSON MIME
import google.generativeai as genai
from google.genai import types
# Configure with your API key
# genai.configure(api_key="your-api-key")
schema = types.Schema(
type=types.Type.OBJECT,
properties={
"title": types.Schema(type=types.Type.STRING),
"date": types.Schema(type=types.Type.STRING),
"location": types.Schema(type=types.Type.STRING),
},
required=["title", "date", "location"],
additional_properties=False,
)
resp = genai.generate_content(
model="gemini-2.0-flash",
contents="Extract the event from: 'PyData Sydney is on 2025-11-03 at Darling Harbour.'",
generation_config=genai.GenerationConfig(
response_mime_type="application/json",
response_schema=schema,
),
)
print(resp.text) # already valid JSON per schema
Gemini will return strict JSON that conforms to response_schema
.
3) Anthropic (Claude) — Tool Use with JSON schema
from anthropic import Anthropic
import json
client = Anthropic()
tool = {
"name": "extract_event",
"description": "Return event details.",
"input_schema": {
"type": "object",
"properties": {
"title": {"type": "string"},
"date": {"type": "string"},
"location": {"type": "string"}
},
"required": ["title", "date", "location"],
"additionalProperties": False
}
}
msg = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=256,
tools=[tool],
tool_choice={"type": "tool", "name": "extract_event"}, # force schema
messages=[{"role": "user", "content":
"Extract the event from: 'PyData Sydney is on 2025-11-03 at Darling Harbour.'"}],
)
# Claude will "call" the tool; grab the args (which match your schema)
tool_use = next(b for b in msg.content if b.type == "tool_use")
print(json.dumps(tool_use.input, indent=2))
Claude doesn’t have a generic “JSON mode” switch; instead, Tool Use with an input_schema
gives you validated, schema-shaped arguments (and you can force its use).
4) Mistral — JSON mode (client-side validation)
from mistralai import Mistral
import json
client = Mistral()
resp = client.chat.complete(
model="mistral-large-latest",
messages=[{"role":"user","content":
"Return JSON with keys title, date (YYYY-MM-DD), location for: "
"'PyData Sydney is on 2025-11-03 at Darling Harbour.'"}],
response_format={"type": "json_object"} # guarantees valid JSON
)
data = json.loads(resp.choices[0].message.content)
print(data)
# Tip: validate `data` against your Pydantic/JSON Schema locally.
Mistral’s json_object
enforces JSON shape (not your exact schema) — validate client-side.
5) AWS Bedrock — Converse API Tool schema (model-agnostic)
import boto3, json
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
model_id = "anthropic.claude-3-5-sonnet-20240620-v1:0"
tools = [{
"toolSpec": {
"name": "extract_event",
"inputSchema": {
"json": {
"type": "object",
"properties": {
"title": {"type": "string"},
"date": {"type": "string"},
"location": {"type": "string"}
},
"required": ["title","date","location"],
"additionalProperties": False
}
}
}
}]
resp = bedrock.converse(
modelId=model_id,
toolConfig={"tools": tools},
toolChoice={"tool": {"name": "extract_event"}}, # force schema
messages=[{"role":"user","content":[{"text":
"Extract the event from: 'PyData Sydney is on 2025-11-03 at Darling Harbour.'"}]}],
)
# Pull toolUse content
tool_use = next(
c["toolUse"] for c in resp["output"]["message"]["content"] if "toolUse" in c
)
print(json.dumps(tool_use["input"], indent=2))
Bedrock validates tool input against your JSON schema and many hosted models (e.g., Claude) support it through Converse.
Practical guidance and Validation
- If you want the strongest server-side guarantees: OpenAI structured outputs or Gemini response schema.
- If you’re already on Claude/Bedrock: define a Tool with a JSON schema and force its use; read the tool’s arguments as your typed object.
- If you use Mistral: enable
json_object
and validate locally (e.g., with Pydantic).
Validation pattern (works for all of them)
from pydantic import BaseModel, ValidationError
class Event(BaseModel):
title: str
date: str
location: str
try:
event = Event.model_validate(data) # `data` from any provider
except ValidationError as e:
# handle / retry / ask model to fix with e.errors()
print(e)
Useful links
- https://platform.openai.com/docs/guides/structured-outputs
- https://ai.google.dev/gemini-api/docs/structured-output
- https://docs.mistral.ai/capabilities/structured-output/json_mode/
- https://aws.amazon.com/blogs/machine-learning/structured-data-response-with-amazon-bedrock-prompt-engineering-and-tool-use
- https://github.com/aws-samples/anthropic-on-aws/blob/main/complex-schema-tool-use/README.md
- https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html
- Howto Request Structured Output from the LLM hosted on Ollama
- Python Cheatsheet
- AWS SAM + AWS SQS + Python PowerTools
- AWS lambda performance comparison: JavaScript vs Python vs Golang