LLM Caching

Planck caching stores complete eligible LLM responses so repeated requests can return faster and avoid duplicate provider calls.

Looking for provider-level caching? Learn about Prompt Caching to cache prompts directly on provider servers when supported.

How It Works

Planck builds a cache key from the request path, request body, selected cache headers, and optional cache seed. If a matching entry exists, the gateway can return the cached response. If not, the gateway sends the request to the provider and can store the response for future identical requests. Cache behavior is request-scoped. You opt in with Planck-Cache-Enabled.

Quick Start

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inquantum.ai/v1",
  apiKey: process.env.PLANCK_API_KEY,
});

const response = await client.chat.completions.create(
  {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Hello world" }],
  },
  {
    headers: {
      "Planck-Cache-Enabled": "true",
      "Planck-Cache-Control": "max-age=3600",
    },
  }
);

curl https://api.inquantum.ai/v1/chat/completions \
  -H "Authorization: Bearer $PLANCK_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Planck-Cache-Enabled: true" \
  -H "Planck-Cache-Control: max-age=3600" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{ "role": "user", "content": "Hello world" }]
  }'

Headers

string

required

Enables response caching for the request. Use "true" to enable.

string

Sets cache duration. Use max-age=<seconds>, for example max-age=3600.

string

Number of different responses to store for the same request key. This is useful for non-deterministic prompts.

string

Creates a separate cache namespace, such as a user id, organization id, environment, or test run id.

string

Comma-separated JSON keys to exclude from cache key generation.

Header values must be strings. For example, use "Planck-Cache-Bucket-Max-Size": "3".

Cache Seeds

Use a seed when the same request body should cache separately for different users, tenants, or environments:

await client.chat.completions.create(
  {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Summarize my account." }],
  },
  {
    headers: {
      "Planck-Cache-Enabled": "true",
      "Planck-Cache-Seed": "user-123",
    },
  }
);

Changing the seed creates a separate cache namespace.

Ignore Keys

Use Planck-Cache-Ignore-Keys for request fields that should not affect the cached response:

await client.chat.completions.create(
  {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Explain caching." }],
    request_id: "req_123",
  },
  {
    headers: {
      "Planck-Cache-Enabled": "true",
      "Planck-Cache-Ignore-Keys": "request_id",
    },
  }
);

This is useful for timestamps, tracing ids, or other metadata fields that do not change the LLM output.

Response Headers

When caching is enabled, inspect response headers to understand cache behavior:

Header	Description
`Planck-Cache`	Cache status, such as `HIT` or `MISS`.
`Planck-Cache-Bucket-Idx`	Cache bucket index used for the response.

Limitations

Caching is intended for eligible JSON gateway requests.
Streaming cache behavior may differ from non-streaming responses.
Any non-ignored request body change creates a new cache key.
Do not cache requests that include sensitive data unless your retention and privacy settings allow it.

Prompt Caching

Cache prompts on provider servers for reduced token costs and faster processing.

Custom Properties

Add metadata to requests for filtering and analysis.

Rate Limiting

Control request frequency and combine with caching for cost optimization.

User Metrics

Track usage and savings by user or application.

Need more help?

Additional questions or feedback? Reach out to help@inquantum.ai or schedule a call with us.

Getting Started

AI Gateway

Observability & Analytics

Prompt Management

Async Logging

Custom Logs

How It Works

Quick Start

Headers

Cache Seeds

Ignore Keys

Response Headers

Limitations

Prompt Caching

Custom Properties

Rate Limiting

User Metrics

​How It Works

​Quick Start

​Headers

​Cache Seeds

​Ignore Keys

​Response Headers

​Limitations

​Related Features

Prompt Caching

Custom Properties

Rate Limiting

User Metrics

How It Works

Quick Start

Headers

Cache Seeds

Ignore Keys

Response Headers

Limitations

Related Features