Tool calling

Tool calling lets a model ask your application to run a function. The model chooses a tool and supplies arguments; your code validates and executes the request, then sends the result back for the model to turn into a final answer.

The model never executes your function. Treat its arguments as untrusted input, validate them, and keep authorization checks inside your application.

Complete tool loop

This example defines a weather tool, executes every requested call, and sends the results back to the model.

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inquantum.ai/v1",
  apiKey: process.env.PLANCK_API_KEY,
});

const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather for a city",
      parameters: {
        type: "object",
        properties: {
          city: { type: "string", description: "City and country" },
          unit: { type: "string", enum: ["celsius", "fahrenheit"] },
        },
        required: ["city"],
        additionalProperties: false,
      },
    },
  },
];

const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
  { role: "user", content: "What is the weather in Chicago?" },
];

const first = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages,
  tools,
  tool_choice: "auto",
});

const assistant = first.choices[0].message;
messages.push(assistant);

for (const call of assistant.tool_calls ?? []) {
  if (call.type !== "function" || call.function.name !== "get_weather") {
    throw new Error(`Tool not allowed: ${call.function.name}`);
  }

  const args = JSON.parse(call.function.arguments) as {
    city: string;
    unit?: "celsius" | "fahrenheit";
  };

  if (typeof args.city !== "string" || !args.city.trim()) {
    throw new Error("get_weather requires a city");
  }

  // Replace this fixture with your authenticated service call.
  const result = { city: args.city, temperature: 24, unit: args.unit ?? "celsius" };

  messages.push({
    role: "tool",
    tool_call_id: call.id,
    content: JSON.stringify(result),
  });
}

const final = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages,
  tools,
});

console.log(final.choices[0].message.content);

The assistant message containing tool_calls must be included before the corresponding tool messages. Each tool result must use the exact tool_call_id supplied by the model.

Request and response

Request

{
  "model": "gpt-4o-mini",
  "messages": [
    {"role": "user", "content": "What is the weather in Chicago?"}
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {"type": "string"}
          },
          "required": ["city"],
          "additionalProperties": false
        }
      }
    }
  ],
  "tool_choice": "auto"
}

Response requesting a tool

{
  "id": "chatcmpl_123",
  "object": "chat.completion",
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_weather_1",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\":\"Chicago\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}

function.arguments is a JSON string, not an object. Parse it only after the call is complete, then validate the parsed value against your own schema.

Stream tool calls

In a stream, a tool call can arrive across several deltas. Accumulate each call by its index, concatenate its argument fragments in order, and parse the JSON after the stream finishes.

TypeScript

type PendingCall = { id: string; name: string; arguments: string };
const pending = new Map<number, PendingCall>();

const stream = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "What is the weather in Chicago?" }],
  tools,
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta;

  if (delta?.content) process.stdout.write(delta.content);

  for (const call of delta?.tool_calls ?? []) {
    const current = pending.get(call.index) ?? {
      id: "",
      name: "",
      arguments: "",
    };

    if (call.id) current.id = call.id;
    if (call.function?.name) current.name += call.function.name;
    if (call.function?.arguments) current.arguments += call.function.arguments;
    pending.set(call.index, current);
  }
}

for (const call of pending.values()) {
  const args = JSON.parse(call.arguments);
  console.log(call.id, call.name, args);
}

Example argument deltas

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_weather_1","type":"function","function":{"name":"get_weather","arguments":"{\"city\":"}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"Chicago\"}"}}]},"finish_reason":"tool_calls"}]}

Never call JSON.parse on each fragment. A fragment is not required to be valid JSON by itself.

Control tool selection

`tool_choice`	Behavior
`"auto"`	The model may answer normally or request one or more tools.
`"required"`	The model must request a tool.
`"none"`	The model cannot request a tool.
Named function object	The model must request the selected function.

Force one function

{
  "tool_choice": {
    "type": "function",
    "function": {"name": "get_weather"}
  }
}

Support is model- and provider-specific. Check supportsTools, supportsToolChoice, and each endpoint’s supportedParameters in Model capabilities before relying on a feature.

Production checklist

Keep an explicit allowlist of function names.
Validate every argument and enforce length, enum, and range limits.
Apply the current user’s authorization inside each tool implementation.
Set timeouts for network, database, and filesystem operations.
Limit the number of tool rounds to prevent accidental loops.
Return compact structured results; do not send secrets or internal errors back to the model.
Use idempotency keys for tools with side effects such as purchases or messages.
Require confirmation before destructive or high-impact actions.

Streaming

Handle text, usage, errors, and cancellation in a chat stream.

Model capabilities

Find models and provider endpoints that support tools.

Getting Started

AI Gateway

Observability & Analytics

Prompt Management

Async Logging

Custom Logs

Complete tool loop

Request and response

Stream tool calls

Control tool selection

Production checklist

Streaming

Model capabilities

​Complete tool loop

​Request and response

​Stream tool calls

​Control tool selection

​Production checklist

Streaming

Model capabilities

Complete tool loop

Request and response

Stream tool calls

Control tool selection

Production checklist