Streaming chat completions

Streaming lets your application render the beginning of a model response while the model is still generating the rest. Planck uses the OpenAI-compatible Chat Completions stream format, so the OpenAI SDKs can consume the stream directly.

Keep your Planck API key on your server. If you are building a browser or mobile chat UI, stream from your own backend to the client instead of calling Planck directly from public code.

Stream with an SDK

Set stream: true, then iterate over the returned stream. Each chunk contains only the new delta; append each delta.content value to reconstruct the full answer.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inquantum.ai/v1",
  apiKey: process.env.PLANCK_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Explain vector search simply." }],
  stream: true,
});

let answer = "";

for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content ?? "";
  answer += text;
  process.stdout.write(text);

  if (chunk.usage) {
    console.log(`\n${chunk.usage.total_tokens} total tokens`);
  }
}

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inquantum.ai/v1",
    api_key=os.environ["PLANCK_API_KEY"],
)

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain vector search simply."}],
    stream=True,
)

answer = ""

for chunk in stream:
    text = chunk.choices[0].delta.content if chunk.choices else ""
    answer += text or ""
    print(text or "", end="", flush=True)

    if chunk.usage:
        print(f"\n{chunk.usage.total_tokens} total tokens")

curl -N https://api.inquantum.ai/v1/chat/completions \
  -H "Authorization: Bearer $PLANCK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "Explain vector search simply."}
    ],
    "stream": true
  }'

The -N option tells cURL not to buffer the response, so each event appears as soon as it arrives.

Request and stream format

The request is a normal Chat Completions request with stream enabled:

Request

POST /v1/chat/completions HTTP/1.1
Host: api.inquantum.ai
Authorization: Bearer $PLANCK_API_KEY
Content-Type: application/json

{
  "model": "gpt-4o-mini",
  "messages": [
    {"role": "user", "content": "Say hello in one sentence."}
  ],
  "stream": true
}

The response has a text/event-stream content type. Every data: line contains one JSON chunk, and a blank line separates events. The final data: [DONE] event closes the stream.

Response stream

data: {"id":"chatcmpl_123","object":"chat.completion.chunk","created":1788206400,"model":"gpt-4o-mini","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl_123","object":"chat.completion.chunk","created":1788206400,"model":"gpt-4o-mini","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl_123","object":"chat.completion.chunk","created":1788206400,"model":"gpt-4o-mini","choices":[{"index":0,"delta":{"content":"! How can I help you today?"},"finish_reason":null}]}

data: {"id":"chatcmpl_123","object":"chat.completion.chunk","created":1788206400,"model":"gpt-4o-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"req_123","object":"chat.completion.chunk","created":1788206401,"model":"gpt-4o-mini","choices":[],"usage":{"prompt_tokens":13,"completion_tokens":8,"total_tokens":21,"cost_usd":0.000012}}

data: [DONE]

Planck emits one usage-only chunk before [DONE]. Its choices array is empty, so read choices[0] defensively, as the SDK examples above do. The exact chunk boundaries are not stable: never assume that one chunk is one word or one complete JSON value from a tool call.

Read the stream without an SDK

Use an SDK when possible. If you need to consume the SSE response directly, remember that one network read can contain part of an event or several events. Buffer text until you have a complete blank-line-delimited SSE event.

Node.js fetch

async function streamChat() {
  const response = await fetch(
    "https://api.inquantum.ai/v1/chat/completions",
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.PLANCK_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model: "gpt-4o-mini",
        messages: [{ role: "user", content: "Explain vector search simply." }],
        stream: true,
      }),
    },
  );

  if (!response.ok || !response.body) {
    throw new Error(await response.text());
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = "";

  while (true) {
    const { value, done } = await reader.read();
    buffer += decoder.decode(value, { stream: !done });

    const events = buffer.split(/\r?\n\r?\n/);
    buffer = events.pop() ?? "";

    for (const event of events) {
      for (const line of event.split(/\r?\n/)) {
        if (!line.startsWith("data:")) continue;

        const data = line.slice(5).trimStart();
        if (data === "[DONE]") return;

        const chunk = JSON.parse(data);
        if (chunk.error) throw new Error(chunk.error.message);

        const text = chunk.choices?.[0]?.delta?.content;
        if (text) process.stdout.write(text);
      }
    }

    if (done) break;
  }
}

await streamChat();

Errors during a stream

There are two different failure points:

When the failure happens	What your application receives
Before streaming starts	A non-2xx HTTP response with a JSON error body
After streaming starts	An error object inside a `data:` event, followed by `[DONE]`

Once the response headers have been sent, the HTTP status is already committed and can remain 200 even if generation later fails. SDK users should wrap stream creation and iteration in try/catch. Direct SSE consumers must inspect every decoded event for a top-level error object.

Error event

{
  "error": {
    "message": "The provider stream ended unexpectedly",
    "type": "provider_error",
    "code": "stream_error",
    "status": 502,
    "retryable": true
  }
}

Do not automatically retry after displaying partial output unless your product can replace or discard that partial answer. Otherwise, users can see duplicated text.

Cancel a generation

Abort the request when the user presses Stop generating or leaves the page. Planck stops forwarding the stream and records the request as partial.

TypeScript

const controller = new AbortController();

const stream = await client.chat.completions.create(
  {
    model: "gpt-4o-mini",
    messages,
    stream: true,
  },
  { signal: controller.signal },
);

// Connect this to your UI's “Stop generating” action.
controller.abort();

Production checklist

Render delta.content; do not wait for the final assembled message.
Treat chunks as arbitrary fragments and accumulate tool-call arguments before parsing them.
Handle the usage-only chunk with an empty choices array.
Handle errors from both the initial HTTP response and the open stream.
Support cancellation and expect cancelled streams to have partial usage.
Avoid retrying a stream after partial text is visible unless you reset the displayed answer.
Proxy browser requests through your backend so your API key is never exposed.

Chat & Responses

Compare Chat Completions, Responses, and Anthropic-compatible routes.

Chat Completions reference

See the complete request schema and supported parameters.

Error handling

Understand gateway errors, retries, and provider fallback.

Vercel AI SDK

Stream chat in applications built with the Vercel AI SDK.

Getting Started

AI Gateway

Observability & Analytics

Prompt Management

Async Logging

Custom Logs

Streaming chat completions

Stream with an SDK

Request and stream format

Read the stream without an SDK

Errors during a stream

Cancel a generation

Production checklist

Chat & Responses

Chat Completions reference

Error handling

Vercel AI SDK

​Stream with an SDK

​Request and stream format

​Read the stream without an SDK

​Errors during a stream

​Cancel a generation

​Production checklist

​Related guides

Chat & Responses

Chat Completions reference

Error handling

Vercel AI SDK

Stream with an SDK

Request and stream format

Read the stream without an SDK

Errors during a stream

Cancel a generation

Production checklist

Related guides