Structured output comparison across popular LLM providers - OpenAI, Gemini, Anthropic, Mistral and AWS Bedrock

Slightly different APIs require special approach.

Page content

Here’s a side-by-side support comparison of structured output (getting reliable JSON back) across popular LLM providers, plus minimal Python examples

colorised bars standing

We already had a look how to request structured output from the LLM hosted on Ollama. Here we review how to request the same from other providers.

TL;DR matrix

Provider Native “JSON mode” JSON Schema enforcement Typical knob Notes
OpenAI Yes Yes (first-class) response_format={"type":"json_schema", ...} Works via the Responses API or Chat Completions; can also do function calling.
Google Gemini Yes Yes (first-class) response_schema= + response_mime_type="application/json" Returns strictly validated JSON when schema is set.
Anthropic (Claude) Indirect Yes (via Tool Use with JSON schema) tools=[{input_schema=...}] + tool_choice Force the model to “call” your schema-defined tool; yields schema-shaped args.
Mistral Yes Partial (JSON-only; no server-side schema) response_format={"type":"json_object"} Ensures JSON, but you validate against your schema client-side.
AWS Bedrock (platform) Varies by model Yes (via Tool/Converse schema) toolConfig.tools[].toolSpec.inputSchema Bedrock’s Converse API validates tool input against a JSON schema.

LLM Structured Output - General Information

LLM structured output refers to the ability of large language models (LLMs) to generate responses that strictly adhere to a predefined, specific format or structure rather than producing free-form text. This structured output can be in formats like JSON, XML, tables, or templates, making the data machine-readable, consistent, and easily parsable by software for use in various applications.

Structured outputs differ from traditional LLM outputs, which typically generate open-ended, natural language text. Instead, structured outputs enforce a schema or format, such as JSON objects with defined keys and value types, or specific classes in the output (e.g., multiple-choice answers, sentiment classes, or database row formats). This approach improves reliability, reduces errors and hallucinations, and simplifies integration into systems such as databases, APIs, or workflows.

The generation of structured outputs in LLMs often involves techniques such as:

  • Specifying detailed prompt instructions to guide the model to produce output in the desired format.
  • Using validation and parsing tools like Pydantic in Python to ensure the output matches the schema.
  • Sometimes enforcing decoding constraints based on grammar or finite state machines to ensure token-level compliance with the format.

Benefits of structured LLM outputs include:

  • Machine-readability and ease of integration.
  • Reduced variability and errors.
  • Improved predictability and verifiability for tasks that require consistent data formats.

Challenges include designing effective schemas, handling complex nested data, and potential limitations in reasoning capabilities compared to free-form text generation.

Overall, structured output enables LLMs to be more useful in applications requiring precise, formatted data rather than solely human-readable text.

Structured Output Python examples

All snippets extract event info as JSON: {title, date, location}. Replace keys/models as you like.

1) OpenAI — JSON Schema (strict)

from openai import OpenAI
import json

client = OpenAI()

schema = {
    "name": "Event",
    "schema": {
        "type": "object",
        "properties": {
            "title": {"type": "string"},
            "date":  {"type": "string", "description": "YYYY-MM-DD"},
            "location": {"type": "string"}
        },
        "required": ["title", "date", "location"],
        "additionalProperties": False
    },
    "strict": True
}

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": "Extract the event from: 'PyData Sydney is on 2025-11-03 at Darling Harbour.'"}
    ],
    response_format={"type": "json_schema", "json_schema": schema},
)

data = json.loads(resp.choices[0].message.content)
print(data)

OpenAI’s Structured Outputs feature enforces this schema on the server side.


2) Google Gemini — response schema + JSON MIME

import google.generativeai as genai
from google.genai import types

# Configure with your API key
# genai.configure(api_key="your-api-key")

schema = types.Schema(
    type=types.Type.OBJECT,
    properties={
        "title": types.Schema(type=types.Type.STRING),
        "date": types.Schema(type=types.Type.STRING),
        "location": types.Schema(type=types.Type.STRING),
    },
    required=["title", "date", "location"],
    additional_properties=False,
)

resp = genai.generate_content(
    model="gemini-2.0-flash",
    contents="Extract the event from: 'PyData Sydney is on 2025-11-03 at Darling Harbour.'",
    generation_config=genai.GenerationConfig(
        response_mime_type="application/json",
        response_schema=schema,
    ),
)

print(resp.text)  # already valid JSON per schema

Gemini will return strict JSON that conforms to response_schema.


3) Anthropic (Claude) — Tool Use with JSON schema

from anthropic import Anthropic
import json

client = Anthropic()

tool = {
    "name": "extract_event",
    "description": "Return event details.",
    "input_schema": {
        "type": "object",
        "properties": {
            "title": {"type": "string"},
            "date": {"type": "string"},
            "location": {"type": "string"}
        },
        "required": ["title", "date", "location"],
        "additionalProperties": False
    }
}

msg = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=256,
    tools=[tool],
    tool_choice={"type": "tool", "name": "extract_event"},  # force schema
    messages=[{"role": "user", "content":
        "Extract the event from: 'PyData Sydney is on 2025-11-03 at Darling Harbour.'"}],
)

# Claude will "call" the tool; grab the args (which match your schema)
tool_use = next(b for b in msg.content if b.type == "tool_use")
print(json.dumps(tool_use.input, indent=2))

Claude doesn’t have a generic “JSON mode” switch; instead, Tool Use with an input_schema gives you validated, schema-shaped arguments (and you can force its use).


4) Mistral — JSON mode (client-side validation)

from mistralai import Mistral
import json

client = Mistral()

resp = client.chat.complete(
    model="mistral-large-latest",
    messages=[{"role":"user","content":
        "Return JSON with keys title, date (YYYY-MM-DD), location for: "
        "'PyData Sydney is on 2025-11-03 at Darling Harbour.'"}],
    response_format={"type": "json_object"}  # guarantees valid JSON
)

data = json.loads(resp.choices[0].message.content)
print(data)
# Tip: validate `data` against your Pydantic/JSON Schema locally.

Mistral’s json_object enforces JSON shape (not your exact schema) — validate client-side.


5) AWS Bedrock — Converse API Tool schema (model-agnostic)

import boto3, json

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
model_id = "anthropic.claude-3-5-sonnet-20240620-v1:0"

tools = [{
    "toolSpec": {
        "name": "extract_event",
        "inputSchema": {
            "json": {
                "type": "object",
                "properties": {
                    "title": {"type": "string"},
                    "date": {"type": "string"},
                    "location": {"type": "string"}
                },
                "required": ["title","date","location"],
                "additionalProperties": False
            }
        }
    }
}]

resp = bedrock.converse(
    modelId=model_id,
    toolConfig={"tools": tools},
    toolChoice={"tool": {"name": "extract_event"}},  # force schema
    messages=[{"role":"user","content":[{"text":
        "Extract the event from: 'PyData Sydney is on 2025-11-03 at Darling Harbour.'"}]}],
)

# Pull toolUse content
tool_use = next(
    c["toolUse"] for c in resp["output"]["message"]["content"] if "toolUse" in c
)
print(json.dumps(tool_use["input"], indent=2))

Bedrock validates tool input against your JSON schema and many hosted models (e.g., Claude) support it through Converse.


Practical guidance and Validation

  • If you want the strongest server-side guarantees: OpenAI structured outputs or Gemini response schema.
  • If you’re already on Claude/Bedrock: define a Tool with a JSON schema and force its use; read the tool’s arguments as your typed object.
  • If you use Mistral: enable json_object and validate locally (e.g., with Pydantic).

Validation pattern (works for all of them)

from pydantic import BaseModel, ValidationError

class Event(BaseModel):
    title: str
    date: str
    location: str

try:
    event = Event.model_validate(data)  # `data` from any provider
except ValidationError as e:
    # handle / retry / ask model to fix with e.errors()
    print(e)