Structured Outputs | Flow AI Docs

Structured outputs make the model answer with JSON that follows a JSON Schema you give it. Use them when your code reads the answer: extracting fields, filling a form, classifying, returning data to a UI.

With a schema you get:

answers you can parse every time, with no “Sure! Here’s the JSON:” around them,
the fields you asked for, with the types you asked for,
no made-up fields when additionalProperties is false.

Using `response_format`

Add response_format with type: "json_schema" and your schema:

{
  "model": "openai/gpt-6-luna",
  "messages": [
    { "role": "user", "content": "What's the weather like in London?" }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "weather",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "location": { "type": "string", "description": "City or location name" },
          "temperature": { "type": "number", "description": "Temperature in Celsius" },
          "conditions": { "type": "string", "description": "Weather conditions" }
        },
        "required": ["location", "temperature", "conditions"],
        "additionalProperties": false
      }
    }
  }
}

The answer’s message.content is a JSON string that follows the schema:

{
  "location": "London",
  "temperature": 18,
  "conditions": "Partly cloudy with light drizzle"
}

JSON mode

{"type": "json_object"} only promises valid JSON, in any shape. Describe the shape you want in the prompt, and say “JSON” in it; some models refuse JSON mode otherwise. Prefer json_schema when the model supports it.

Examples

The samples use the OpenAI SDK set up as in the Quickstart.

curl https://ai.hyperflow.fun/v1/chat/completions \
  -H "Authorization: Bearer $HYPERFLOW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-6-luna",
    "messages": [{ "role": "user", "content": "Extract: Lan, 29, lives in Da Nang." }],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "name": { "type": "string" },
            "age": { "type": "integer" },
            "city": { "type": "string" }
          },
          "required": ["name", "age", "city"],
          "additionalProperties": false
        }
      }
    }
  }'

Schemas from your types

The OpenAI SDKs can build the schema from a Pydantic model or a Zod schema, and parse the answer for you:

from pydantic import BaseModel
class Person(BaseModel):
    name: str
    age: int
    city: str
completion = client.chat.completions.parse(
    model="openai/gpt-6-luna",
    messages=[{"role": "user", "content": "Extract: Lan, 29, lives in Da Nang."}],
    response_format=Person,
)
person = completion.choices[0].message.parsed

Streaming

Structured outputs work with stream: true. The JSON arrives in pieces; join the delta.content of every chunk and parse it once the stream ends. Partial JSON can’t be parsed until then.

Which models support it

Not every model can follow a schema. Most recent OpenAI, Google and Anthropic models can, as can many open models. Pick one from the Models page, and test your schema on it before you rely on it.

Writing good schemas

Describe every field. description is the model’s only guide to what goes where.
Use strict: true. The model then follows the schema exactly instead of loosely.
List every field in required and set additionalProperties: false. Strict mode on most models needs both. For an optional field, allow null: {"type": ["string", "null"]}.
Prefer enum for fixed choices. {"type": "string", "enum": ["positive", "neutral", "negative"]} beats explaining the options in the prompt.
Keep it flat and small. Deep nesting and very long schemas cost input tokens and make mistakes more likely.

Troubleshooting

The answer is cut off and won’t parse. The model reached max_tokens; finish_reason is length. Raise max_tokens and always check finish_reason before parsing.

The answer ignores the schema. The model may not support structured outputs. Try another model, or fall back to JSON mode with the shape in the prompt, and validate the result yourself.

400 with a schema error. The schema isn’t valid JSON Schema, or uses a feature the model doesn’t support, such as $ref or oneOf on some models. Simplify it.

Billing

The schema counts as input tokens on every request that sends it, and the JSON answer is output. There’s no extra charge for using structured outputs.

Using response_format