> Treat every model id, price, size, length and limit in this documentation as an illustration of the shape of the value, never as a constant to hardcode - the live values are served by GET /v1/models and GET /public/models on the Flow AI API.

# Tool Calling

> Let the model call functions in your code.

Tool calls (also called function calls) let a model use things outside itself: your database, an API, a calculator. The model never runs anything. It **suggests** a call, with the function name and arguments; your code runs it and sends the result back; the model then uses the result to answer.

Flow AI uses the OpenAI tool-calling format for every model that supports tools, so the same code works across labs.

## How it works

A tool call takes three steps.

### 1. Send the request with `tools`

```json
{
  "model": "openai/gpt-6-luna",
  "messages": [
    { "role": "user", "content": "What are the titles of some James Joyce books?" }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "search_gutenberg_books",
        "description": "Search for books in the Project Gutenberg library",
        "parameters": {
          "type": "object",
          "properties": {
            "search_terms": {
              "type": "array",
              "items": { "type": "string" },
              "description": "Search terms to find books"
            }
          },
          "required": ["search_terms"]
        }
      }
    }
  ]
}
```

The model answers with `finish_reason: "tool_calls"` and a `tool_calls` list instead of text:

```json
{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {
      "id": "call_abc123",
      "type": "function",
      "function": {
        "name": "search_gutenberg_books",
        "arguments": "{\"search_terms\": [\"James\", \"Joyce\"]}"
      }
    }
  ]
}
```

### 2. Run the tool in your code

`arguments` is a JSON string. Parse it and call your function:

```python
result = search_gutenberg_books(["James", "Joyce"])
```

### 3. Send the result back

Append the assistant message and one `tool` message per call, with the matching `tool_call_id`, and send `tools` again:

```json
{
  "model": "openai/gpt-6-luna",
  "messages": [
    { "role": "user", "content": "What are the titles of some James Joyce books?" },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": { "name": "search_gutenberg_books", "arguments": "{\"search_terms\": [\"James\", \"Joyce\"]}" }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "[{\"id\": 4300, \"title\": \"Ulysses\", \"authors\": [{\"name\": \"Joyce, James\"}]}]"
    }
  ],
  "tools": [ "...the same tools as in step 1..." ]
}
```

> **Note**
>
> Send `tools` on every request in the conversation, not only the first. The model needs the definitions to read its own earlier calls.

## Full example

This searches [Project Gutenberg](https://gutendex.com) for books. The samples use the OpenAI SDK set up as in the [Quickstart](/quickstart#using-the-openai-sdk).

**`Python`**

```python title="Python"
import json
import requests

MODEL = "openai/gpt-6-luna"

def search_gutenberg_books(search_terms):
    response = requests.get("https://gutendex.com/books", params={"search": " ".join(search_terms)})
    return [
        {"id": book["id"], "title": book["title"], "authors": book["authors"]}
        for book in response.json().get("results", [])
    ]

tools = [{
    "type": "function",
    "function": {
        "name": "search_gutenberg_books",
        "description": "Search for books in the Project Gutenberg library",
        "parameters": {
            "type": "object",
            "properties": {
                "search_terms": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Search terms to find books",
                },
            },
            "required": ["search_terms"],
        },
    },
}]
TOOLS = {"search_gutenberg_books": search_gutenberg_books}

messages = [
    {"role": "system", "content": "You are a helpful librarian."},
    {"role": "user", "content": "What are the titles of some James Joyce books?"},
]

response = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
message = response.choices[0].message

# Keep the assistant's turn so the model sees its own calls.
messages.append(message)

for call in message.tool_calls or []:
    args = json.loads(call.function.arguments)
    result = TOOLS[call.function.name](**args)
    messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})

final = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
print(final.choices[0].message.content)
```

**`TypeScript`**

```typescript title="TypeScript"
import type OpenAI from 'openai'

const MODEL = 'openai/gpt-6-luna'

async function searchGutenbergBooks({ search_terms }: { search_terms: string[] }) {
  const url = `https://gutendex.com/books?search=${encodeURIComponent(search_terms.join(' '))}`
  const data = await (await fetch(url)).json()
  return data.results.map((book: any) => ({ id: book.id, title: book.title, authors: book.authors }))
}

const tools: OpenAI.ChatCompletionTool[] = [{
  type: 'function',
  function: {
    name: 'search_gutenberg_books',
    description: 'Search for books in the Project Gutenberg library',
    parameters: {
      type: 'object',
      properties: {
        search_terms: { type: 'array', items: { type: 'string' }, description: 'Search terms to find books' },
      },
      required: ['search_terms'],
    },
  },
}]
const TOOLS: Record<string, (args: any) => Promise<unknown>> = { search_gutenberg_books: searchGutenbergBooks }

const messages: OpenAI.ChatCompletionMessageParam[] = [
  { role: 'system', content: 'You are a helpful librarian.' },
  { role: 'user', content: 'What are the titles of some James Joyce books?' },
]

const response = await client.chat.completions.create({ model: MODEL, messages, tools })
const message = response.choices[0].message

// Keep the assistant's turn so the model sees its own calls.
messages.push(message)

for (const call of message.tool_calls ?? []) {
  if (call.type !== 'function') continue
  const result = await TOOLS[call.function.name](JSON.parse(call.function.arguments))
  messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(result) })
}

const final = await client.chat.completions.create({ model: MODEL, messages, tools })
console.log(final.choices[0].message.content)
```

## An agent loop

Real tasks often need several calls in a row: search, then fetch details, then check something else. Loop until the model answers without calling a tool, and cap the number of turns so a confused model can't run up a bill.

**`Python`**

```python title="Python"
MAX_TURNS = 8

for _ in range(MAX_TURNS):
    response = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
    message = response.choices[0].message
    messages.append(message)

    if not message.tool_calls:
        print(message.content)
        break

    for call in message.tool_calls:
        try:
            result = TOOLS[call.function.name](**json.loads(call.function.arguments))
        except Exception as error:
            result = {"error": str(error)}
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
```

**`TypeScript`**

```typescript title="TypeScript"
const MAX_TURNS = 8

for (let turn = 0; turn < MAX_TURNS; turn++) {
  const response = await client.chat.completions.create({ model: MODEL, messages, tools })
  const message = response.choices[0].message
  messages.push(message)

  if (!message.tool_calls?.length) {
    console.log(message.content)
    break
  }

  for (const call of message.tool_calls) {
    if (call.type !== 'function') continue
    let result: unknown
    try {
      result = await TOOLS[call.function.name](JSON.parse(call.function.arguments))
    } catch (error) {
      result = { error: String(error) }
    }
    messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(result) })
  }
}
```

Return errors to the model as tool results instead of throwing. A clear message like `{"error": "No book with id 99999"}` lets the model try another approach.

## Defining good tools

**Use clear, specific names.** `get_weather_forecast` tells the model what the tool does; `weather` doesn't.

**Describe when and how to use it.** The description and each parameter's description are all the model knows about your tool:

```json
{
  "name": "get_weather_forecast",
  "description": "Current conditions and a 5-day forecast for a place. Accepts a city, a postcode or coordinates.",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "City name, postcode or 'lat,lng'. Examples: 'Hanoi', '10001', '21.03,105.85'"
      },
      "units": {
        "type": "string",
        "enum": ["celsius", "fahrenheit"],
        "description": "Temperature unit"
      }
    },
    "required": ["location"]
  }
}
```

**Use `enum` and `required`.** They narrow what the model can send and cut down on bad arguments.

**Design tools that chain.** `search_products` → `get_product_details` → `check_inventory` lets the model work step by step, each call narrowing the next.

**Keep the list short.** Every tool definition is sent, and paid for, on every request. Send only the tools the current task needs.

## Controlling tool use

### `tool_choice`

| Value                                                                  | Effect                                             |
| ---------------------------------------------------------------------- | -------------------------------------------------- |
| `"auto"`                                                               | The model decides. The default when `tools` is set |
| `"none"`                                                               | The model answers in text and calls no tool        |
| `"required"`                                                           | The model must call at least one tool              |
| `{"type": "function", "function": {"name": "search_gutenberg_books"}}` | The model must call this tool                      |

### `parallel_tool_calls`

Most models can ask for several calls in one turn, for example the weather in three cities at once. Each call has its own `id`; answer every one before the next request.

Set `"parallel_tool_calls": false` to get at most one call per turn, when your tools must run in order.

## Streaming tool calls

With `stream: true`, a tool call arrives in pieces: the first delta for a call carries its `index`, `id` and `name`, and later deltas append to `arguments`. Join them by `index`, and run the calls once `finish_reason` is `tool_calls`.

**`Python`**

```python title="Python"
stream = client.chat.completions.create(model=MODEL, messages=messages, tools=tools, stream=True)

calls = {}
for chunk in stream:
    if not chunk.choices:
        continue  # the final chunk carries only usage
    choice = chunk.choices[0]
    for delta in choice.delta.tool_calls or []:
        call = calls.setdefault(delta.index, {"id": None, "name": "", "arguments": ""})
        if delta.id:
            call["id"] = delta.id
        if delta.function and delta.function.name:
            call["name"] = delta.function.name
        if delta.function and delta.function.arguments:
            call["arguments"] += delta.function.arguments
    if choice.finish_reason == "tool_calls":
        for call in calls.values():
            print(call["name"], json.loads(call["arguments"]))
```

**`TypeScript`**

```typescript title="TypeScript"
const stream = await client.chat.completions.create({ model: MODEL, messages, tools, stream: true })

const calls = new Map<number, { id?: string; name: string; arguments: string }>()
for await (const chunk of stream) {
  const choice = chunk.choices[0]
  if (!choice) continue // the final chunk carries only usage
  for (const delta of choice.delta.tool_calls ?? []) {
    const call = calls.get(delta.index) ?? { name: '', arguments: '' }
    if (delta.id) call.id = delta.id
    if (delta.function?.name) call.name = delta.function.name
    if (delta.function?.arguments) call.arguments += delta.function.arguments
    calls.set(delta.index, call)
  }
  if (choice.finish_reason === 'tool_calls') {
    for (const call of calls.values()) console.log(call.name, JSON.parse(call.arguments))
  }
}
```

## Reasoning between calls

[Reasoning models](/features/reasoning) can think after each tool result before choosing the next call: read the search results, notice a gap, search again. This often gives better multi-step answers, at the cost of more output tokens and slower turns. Use `reasoning_effort` to set how much they think.

## What's supported

* **Function tools only.** Tools of any other type, such as a host's built-in web search or code execution, are removed from the request, because the host bills them separately.
* **Models that support tools.** Not every model can call tools. Pick one that does from the [Models page](https://ai.hyperflow.fun/models).
* **The OpenAI format.** `tools`, `tool_choice`, `parallel_tool_calls` and `tool` messages work as in the OpenAI API. Other request fields are removed; see the [API overview](/api/overview).

## Troubleshooting

**The model answers in text instead of calling the tool.** Make the description say when to use the tool, or set `tool_choice` to `"required"` or to the tool's name.

**`arguments` isn't valid JSON.** It can happen with smaller models, or when the answer is cut off by `max_tokens` (`finish_reason: "length"`). Raise `max_tokens`, and catch the parse error and return it to the model as the tool result.

**`400` on the second request.** Every `tool_calls` entry in an assistant message needs a `tool` message with the same `tool_call_id`, and `tools` must be sent again.

## Billing

Tool definitions are input tokens on every request that sends them, and the model's tool calls are output tokens. Each round trip in a loop is its own request, held and charged separately. Your own tools run on your side and cost nothing on Flow AI.