Tool Calling

Tool calls (also called function calls) let a model use things outside itself: your database, an API, a calculator. The model never runs anything. It suggests a call, with the function name and arguments; your code runs it and sends the result back; the model then uses the result to answer.

Flow AI uses the OpenAI tool-calling format for every model that supports tools, so the same code works across labs.

How it works

A tool call takes three steps.

1. Send the request with tools

{
  "model": "openai/gpt-6-luna",
  "messages": [
    { "role": "user", "content": "What are the titles of some James Joyce books?" }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "search_gutenberg_books",
        "description": "Search for books in the Project Gutenberg library",
        "parameters": {
          "type": "object",
          "properties": {
            "search_terms": {
              "type": "array",
              "items": { "type": "string" },
              "description": "Search terms to find books"
            }
          },
          "required": ["search_terms"]
        }
      }
    }
  ]
}

The model answers with finish_reason: "tool_calls" and a tool_calls list instead of text:

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {
      "id": "call_abc123",
      "type": "function",
      "function": {
        "name": "search_gutenberg_books",
        "arguments": "{\"search_terms\": [\"James\", \"Joyce\"]}"
      }
    }
  ]
}

2. Run the tool in your code

arguments is a JSON string. Parse it and call your function:

result = search_gutenberg_books(["James", "Joyce"])

3. Send the result back

Append the assistant message and one tool message per call, with the matching tool_call_id, and send tools again:

{
  "model": "openai/gpt-6-luna",
  "messages": [
    { "role": "user", "content": "What are the titles of some James Joyce books?" },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": { "name": "search_gutenberg_books", "arguments": "{\"search_terms\": [\"James\", \"Joyce\"]}" }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "[{\"id\": 4300, \"title\": \"Ulysses\", \"authors\": [{\"name\": \"Joyce, James\"}]}]"
    }
  ],
  "tools": [ "...the same tools as in step 1..." ]
}

Full example

This searches Project Gutenberg for books. The samples use the OpenAI SDK set up as in the Quickstart.

import json
import requests
MODEL = "openai/gpt-6-luna"
def search_gutenberg_books(search_terms):
    response = requests.get("https://gutendex.com/books", params={"search": " ".join(search_terms)})
    return [
        {"id": book["id"], "title": book["title"], "authors": book["authors"]}
        for book in response.json().get("results", [])
    ]
tools = [{
    "type": "function",
    "function": {
        "name": "search_gutenberg_books",
        "description": "Search for books in the Project Gutenberg library",
        "parameters": {
            "type": "object",
            "properties": {
                "search_terms": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Search terms to find books",
                },
            },
            "required": ["search_terms"],
        },
    },
}]
TOOLS = {"search_gutenberg_books": search_gutenberg_books}
messages = [
    {"role": "system", "content": "You are a helpful librarian."},
    {"role": "user", "content": "What are the titles of some James Joyce books?"},
]
response = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
message = response.choices[0].message
# Keep the assistant's turn so the model sees its own calls.
messages.append(message)
for call in message.tool_calls or []:
    args = json.loads(call.function.arguments)
    result = TOOLS[call.function.name](**args)
    messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
final = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
print(final.choices[0].message.content)

An agent loop

Real tasks often need several calls in a row: search, then fetch details, then check something else. Loop until the model answers without calling a tool, and cap the number of turns so a confused model can’t run up a bill.

MAX_TURNS = 8
for _ in range(MAX_TURNS):
    response = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
    message = response.choices[0].message
    messages.append(message)
    if not message.tool_calls:
        print(message.content)
        break
    for call in message.tool_calls:
        try:
            result = TOOLS[call.function.name](**json.loads(call.function.arguments))
        except Exception as error:
            result = {"error": str(error)}
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})

Return errors to the model as tool results instead of throwing. A clear message like {"error": "No book with id 99999"} lets the model try another approach.

Defining good tools

Use clear, specific names. get_weather_forecast tells the model what the tool does; weather doesn’t.

Describe when and how to use it. The description and each parameter’s description are all the model knows about your tool:

{
  "name": "get_weather_forecast",
  "description": "Current conditions and a 5-day forecast for a place. Accepts a city, a postcode or coordinates.",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "City name, postcode or 'lat,lng'. Examples: 'Hanoi', '10001', '21.03,105.85'"
      },
      "units": {
        "type": "string",
        "enum": ["celsius", "fahrenheit"],
        "description": "Temperature unit"
      }
    },
    "required": ["location"]
  }
}

Use enum and required. They narrow what the model can send and cut down on bad arguments.

Design tools that chain. search_products → get_product_details → check_inventory lets the model work step by step, each call narrowing the next.

Keep the list short. Every tool definition is sent, and paid for, on every request. Send only the tools the current task needs.

Controlling tool use

tool_choice

parallel_tool_calls

Most models can ask for several calls in one turn, for example the weather in three cities at once. Each call has its own id; answer every one before the next request.

Set "parallel_tool_calls": false to get at most one call per turn, when your tools must run in order.

Streaming tool calls

With stream: true, a tool call arrives in pieces: the first delta for a call carries its index, id and name, and later deltas append to arguments. Join them by index, and run the calls once finish_reason is tool_calls.

stream = client.chat.completions.create(model=MODEL, messages=messages, tools=tools, stream=True)
calls = {}
for chunk in stream:
    if not chunk.choices:
        continue  # the final chunk carries only usage
    choice = chunk.choices[0]
    for delta in choice.delta.tool_calls or []:
        call = calls.setdefault(delta.index, {"id": None, "name": "", "arguments": ""})
        if delta.id:
            call["id"] = delta.id
        if delta.function and delta.function.name:
            call["name"] = delta.function.name
        if delta.function and delta.function.arguments:
            call["arguments"] += delta.function.arguments
    if choice.finish_reason == "tool_calls":
        for call in calls.values():
            print(call["name"], json.loads(call["arguments"]))

Reasoning between calls

Reasoning models can think after each tool result before choosing the next call: read the search results, notice a gap, search again. This often gives better multi-step answers, at the cost of more output tokens and slower turns. Use reasoning_effort to set how much they think.

What’s supported

• Function tools only. Tools of any other type, such as a host’s built-in web search or code execution, are removed from the request, because the host bills them separately.
• Models that support tools. Not every model can call tools. Pick one that does from the Models page.
• The OpenAI format. tools, tool_choice, parallel_tool_calls and tool messages work as in the OpenAI API. Other request fields are removed; see the API overview.

Troubleshooting

The model answers in text instead of calling the tool. Make the description say when to use the tool, or set tool_choice to "required" or to the tool’s name.

arguments isn’t valid JSON. It can happen with smaller models, or when the answer is cut off by max_tokens (finish_reason: "length"). Raise max_tokens, and catch the parse error and return it to the model as the tool result.

400 on the second request. Every tool_calls entry in an assistant message needs a tool message with the same tool_call_id, and tools must be sent again.

Billing

Tool definitions are input tokens on every request that sends them, and the model’s tool calls are output tokens. Each round trip in a loop is its own request, held and charged separately. Your own tools run on your side and cost nothing on Flow AI.