> ## Documentation Index
> Fetch the complete documentation index at: https://docs.inquantum.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Manual Logger - TypeScript

> Integrate any custom LLM with Planck using the TypeScript Manual Logger. Step-by-step guide for NodeJS implementation to connect your proprietary or open-source models.

# TypeScript Manual Logger

Logging calls to custom models is supported via the Planck NodeJS SDK.

<Tabs>
  <Tab title="Basic Usage">
    <Steps>
      <Step title="To get started, install the `@inquantum/planck-helpers` package">
        ```bash theme={null}
        npm install @inquantum/planck-helpers
        ```
      </Step>

      <Step title="Set `PLANCK_API_KEY` as an environment variable">
        ```bash theme={null}
        export PLANCK_API_KEY=sk-<your-api-key>
        ```

        <Info>You can also set the Planck API Key in your code (See below)</Info>
      </Step>

      <Step title="Create a new PlanckManualLogger instance">
        ```typescript theme={null}
        import { PlanckManualLogger } from "@inquantum/planck-helpers";

        const planckLogger = new PlanckManualLogger({
        apiKey: process.env.PLANCK_API_KEY, // Can be set as env variable
        headers: {} // Additional headers to be sent with the request
        });

        ```
      </Step>

      <Step title="Log your request">
        ```typescript theme={null}
        const reqBody = {
          model: "text-embedding-ada-002",
          input: "The food was delicious and the waiter was very friendly.",
          encoding_format: "float"
        }
        const res = await planckLogger.logRequest(
          reqBody,
          async (resultRecorder) => {
            const r = await fetch("https://api.openai.com/v1/embeddings", {
              method: "POST",
              headers: {
                "Content-Type": "application/json",
                Authorization: `Bearer ${process.env.OPENAI_API_KEY}`
              },
              body: JSON.stringify(reqBody)
            })
            const resBody = await r.json();
            resultRecorder.appendResults(resBody);
            return resBody; // this will be returned by the logRequest function
          },
          {
           // Additional headers to be sent with the request
          }
        );
        ```
      </Step>
    </Steps>
  </Tab>

  <Tab title="Logging Chat Completion">
    <Steps>
      <Step title="Install dependencies">
        ```bash theme={null}
        npm install @inquantum/planck-helpers openai
        ```
      </Step>

      <Step title="Set up the logger and OpenAI client">
        ```typescript theme={null}
        import { PlanckManualLogger } from "@inquantum/planck-helpers";
        import OpenAI from "openai";

        // Initialize the Planck logger
        const planck = new PlanckManualLogger({
          apiKey: process.env.PLANCK_API_KEY!,
        });

        // Initialize the OpenAI client
        const openai = new OpenAI({
          apiKey: process.env.OPENAI_API_KEY!,
        });
        ```
      </Step>

      <Step title="Using logSingleRequest for non-streaming responses">
        ```typescript theme={null}
        // Define your request
        const requestBody = {
          model: "gpt-4o-mini",
          messages: [
            { role: "user", content: "Explain quantum computing in simple terms" },
          ],
        };

        // Make the API call
        const response = await openai.chat.completions.create(requestBody);

        // Log the request and response to Planck
        await planck.logSingleRequest(requestBody, JSON.stringify(response), {
          additionalHeaders: { "Planck-User-Id": "user-123" }, // Optional additional headers
        });

        console.log(response.choices[0].message.content);
        ```
      </Step>

      <Step title="Using logSingleStream for streaming responses">
        ```typescript theme={null}
        const streamingRequestBody = {
          model: "gpt-4o-mini",
          messages: [{ role: "user", content: "Write a short story about AI" }],
          stream: true,
        };

        const streamingResponse = await openai.chat.completions.create(
          streamingRequestBody
        );

        const [streamForUser, streamForLogging] = stream.tee();

        planck.logSingleStream(streamingRequestBody, streamForLogging, {
          "Planck-User-Id": "user-123",
        });
        ```
      </Step>
    </Steps>
  </Tab>

  <Tab title="On Vercel">
    <Steps>
      <Step title="Install dependencies">
        ```bash theme={null}
        npm install @inquantum/planck-helpers together-ai next
        ```
      </Step>

      <Step title="Create an API route handler with Together AI">
        ```typescript theme={null}
        // app/api/chat/route.ts
        import { PlanckManualLogger } from "@inquantum/planck-helpers";
        import { after } from "next/server";
        import Together from "together-ai";

        export async function POST(request: Request) {
        const { question } = await request.json();

          const together = new Together();
          const planck = new PlanckManualLogger({
            apiKey: process.env.PLANCK_API_KEY!,

          });

          const nonStreamingBody = {
            model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
            messages: [{ role: "user", content: question }],
            stream: false,
          } as Together.Chat.CompletionCreateParamsNonStreaming & { stream: false };

          const completion = await together.chat.completions.create(nonStreamingBody);

          after(
            planck.logSingleRequest(nonStreamingBody, JSON.stringify(completion), {
              additionalHeaders: { "Planck-User-Id": "123" },
            }),
          );

          const body = {
            model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
            messages: [{ role: "user", content: question }],
            stream: true,
          } as Together.Chat.CompletionCreateParamsStreaming & { stream: true };

          const response = await together.chat.completions.create(body);
          const [stream1, stream2] = response.tee();
          after(
            planck.logSingleStream(body, stream2.toReadableStream(), {
              "Planck-User-Id": "123",
            }),
          );

          return new Response(stream1.toReadableStream());

        }

        ```
      </Step>

      <Step title="Important note about the `after` function">
        <Warning>
          The `after` function allows you to perform operations after the response has been sent to the client. This is crucial for logging operations as it ensures they don't delay the response to the user.

          When using this approach:

          * Logging happens asynchronously after the response is sent
          * The user experience isn't affected by logging latency
          * You still capture all the necessary data for observability

          This is especially important for streaming responses where any delay would be noticeable to the user.
        </Warning>
      </Step>
    </Steps>
  </Tab>
</Tabs>

## API Reference

### PlanckManualLogger

```typescript theme={null}
class PlanckManualLogger {
constructor(opts: IPlanckManualLogger);
}

type IPlanckManualLogger = {
apiKey: string;
headers?: Record<string, string>;
loggingEndpoint?: string; // defaults to https://api.hconeai.com/custom/v1/log
};
```

### PlanckLogBuilder

```typescript theme={null}
class PlanckLogBuilder {
  constructor(
    logger: PlanckManualLogger,
    request: PlanckLogRequest,
    additionalHeaders?: Record<string, string>
  );

  setError(error: any): void;
  toReadableStream<T>(stream: Stream<T>): ReadableStream;
  setResponse(body: string): void;
  sendLog(): Promise<void>;
}
```

The `PlanckLogBuilder` provides a simplified way to handle streaming LLM responses with better error handling and async support. It's created using the `logBuilder` method of `PlanckManualLogger`.

#### Methods

* `setError(error: any)`: Sets an error that occurred during the request
* `toReadableStream<T>(stream: Stream<T>)`: Collects streaming responses and converts them to a readable stream while capturing the response for logging
* `setResponse(body: string)`: Sets the response body for non-streaming responses
* `sendLog()`: Sends the log to Planck

### logRequest

```typescript theme={null}
logRequest<T>(
    request: PlanckLogRequest,
    operation: (resultRecorder: PlanckResultRecorder) => Promise<T>,
    additionalHeaders?: Record<string, string>
  ): Promise<T>
```

#### Parameters

1. `request`: `PlanckLogRequest` - The request object to log

```typescript theme={null}
type PlanckLogRequest = ILogRequest | PlanckCustomEventRequest; // ILogRequest is the type for the request object for custom model logging

// The name and structure of the prompt field depends on the model you are using.
// Eg: for chat models it is named "messages", for embeddings models it is named "input".
// Hence, the only enforced type is `model`, you need still add the respective prompt property for your model.
// You may also add more properties (eg: temperature, stop reason etc)
type ILogRequest = {
  model: string;
  [key: string]: any;
};
```

2. `operation`: `(resultRecorder: PlanckResultRecorder) => Promise<T>` - The operation to be executed and logged

```typescript theme={null}
class PlanckResultRecorder {
  private results: Record<string, any> = {};

  appendResults(data: Record<string, any>): void {
    this.results = { ...this.results, ...data };
  }

  getResults(): Record<string, any> {
    return this.results;
  }
}
```

3. `additionalHeaders`: `Record<string, string>`
   * Additional headers to be sent with the request
   * This can be used to use features like [session management](/features/sessions), [custom properties](/features/advanced-usage/custom-properties), etc.

## Available Methods

The `PlanckManualLogger` class provides several methods for logging different types of requests and responses. Here's a comprehensive overview of each method:

### logRequest

Used for logging non-streaming requests and responses with full control over the operation.

```typescript theme={null}
logRequest<T>(
  request: PlanckLogRequest,
  operation: (resultRecorder: PlanckResultRecorder) => Promise<T>,
  additionalHeaders?: Record<string, string>
): Promise<T>
```

**Parameters:**

* `request`: The request object to log
* `operation`: A function that performs the actual API call and records the results
* `additionalHeaders`: Optional additional headers to include with the log request

**Example:**

```typescript theme={null}
const result = await planck.logRequest(
  requestBody,
  async (resultRecorder) => {
    const response = await llmProvider.createCompletion(requestBody);
    resultRecorder.appendResults(response);
    return response;
  },
  { "Planck-User-Id": userId }
);
```

### logStream

Used for logging streaming operations with full control over stream handling.

```typescript theme={null}
logStream<T>(
  request: PlanckLogRequest,
  operation: (resultRecorder: PlanckStreamResultRecorder) => Promise<T>,
  additionalHeaders?: Record<string, string>
): Promise<T>
```

**Parameters:**

* `request`: The request object to log
* `operation`: A function that performs the streaming API call and attaches the stream to the recorder
* `additionalHeaders`: Optional additional headers to include with the log request

**Example:**

```typescript theme={null}
const stream = await planck.logStream(
  requestBody,
  async (resultRecorder) => {
    const response = await llmProvider.createChatCompletion({
      stream: true,
      ...requestBody,
    });
    const [stream1, stream2] = response.tee();
    resultRecorder.attachStream(stream2.toReadableStream());
    return stream1;
  },
  { "Planck-User-Id": userId }
);
```

### logSingleStream

A simplified method for logging a single ReadableStream without needing to manage the operation.

```typescript theme={null}
logSingleStream(
  request: PlanckLogRequest,
  stream: ReadableStream,
  additionalHeaders?: Record<string, string>
): Promise<void>
```

**Parameters:**

* `request`: The request object to log
* `stream`: The ReadableStream to consume and log
* `additionalHeaders`: Optional additional headers to include with the log request

**Example:**

```typescript theme={null}
const response = await llmProvider.createChatCompletion({
  stream: true,
  ...requestBody,
});
const stream = response.toReadableStream();
const [streamForUser, streamForLogging] = stream.tee();

planck.logSingleStream(requestBody, streamForLogging, {
  "Planck-User-Id": userId,
});

return streamForUser;
```

### logSingleRequest

Used for logging a single request with a response body without needing to manage the operation.

```typescript theme={null}
logSingleRequest(
  request: PlanckLogRequest,
  body: string,
  options: {
    additionalHeaders?: Record<string, string>;
    latencyMs?: number;
  }
): Promise<void>
```

**Parameters:**

* `request`: The request object to log
* `body`: The response body as a string
* `additionalHeaders`: Optional additional headers to include with the log request

**Example:**

```typescript theme={null}
const response = await llmProvider.createCompletion(requestBody);
await planck.logSingleRequest(requestBody, JSON.stringify(response), {
  additionalHeaders: { "Planck-User-Id": userId },
});
```

### logBuilder

The recommended method for handling streaming responses with better error handling and simplified workflow.

```typescript theme={null}
logBuilder(
  request: PlanckLogRequest,
  additionalHeaders?: Record<string, string>
): PlanckLogBuilder
```

**Parameters:**

* `request`: The request object to log
* `additionalHeaders`: Optional additional headers to include with the log request

**Example:**

```typescript theme={null}
// Create a log builder
const planckLogBuilder = planck.logBuilder(requestBody, {
  "Planck-User-Id": userId,
});

try {
  // Make the LLM API call
  const response = await llmProvider.createChatCompletion({
    stream: true,
    ...requestBody,
  });

  // Convert the API response to a readable stream and return it
  return new Response(planckLogBuilder.toReadableStream(response));
} catch (error) {
  // Record any errors that occur
  planckLogBuilder.setError(error);
  throw error;
} finally {
  // Send the log (can be used with Vercel's "after" function)
  await planckLogBuilder.sendLog();
}
```

## Streaming Examples

### Using the Async Stream Parser

Planck provides an asynchronous stream parser for efficient handling of streamed responses. This is particularly useful when working with custom integrations that support streaming.

Here's an example of how to use the async stream parser with a custom integration:

```typescript theme={null}
import { PlanckManualLogger } from "@inquantum/planck-helpers";

// Initialize the Planck logger
const planckLogger = new PlanckManualLogger({
  apiKey: process.env.PLANCK_API_KEY!,
  headers: {}, // You can add custom headers here
});

// Your custom model API call that returns a stream
const response = await customModelAPI.generateStream(prompt);

// If your API supports splitting the stream
const [stream1, stream2] = response.tee();

// Log the stream to Planck using the async stream parser
planckLogger.logStream(requestBody, async (resultRecorder) => {
  resultRecorder.attachStream(stream1);
});

// Process the stream for your application
for await (const chunk of stream2) {
  console.log(chunk);
}
```

The async stream parser offers several benefits:

* Processes stream chunks asynchronously for better performance
* Reduces latency when handling large streamed responses
* Provides more reliable token counting for streamed content

### Using Vercel's `after` Function with Streaming

When building applications with Next.js App Router on Vercel, you can use the `after` function to log streaming responses without blocking the client response:

```typescript theme={null}
import { PlanckManualLogger } from "@inquantum/planck-helpers";
import { after } from "next/server";
import Together from "together-ai";

export async function POST(request: Request) {
  const { prompt } = await request.json();

  const together = new Together({ apiKey: process.env.TOGETHER_API_KEY });
  const planck = new PlanckManualLogger({
    apiKey: process.env.PLANCK_API_KEY!,
  });

  // Example with non-streaming response
  const nonStreamingBody = {
    model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: prompt }],
    stream: false,
  };

  const completion = await together.chat.completions.create(nonStreamingBody);

  // Log non-streaming response after sending the response to the client
  after(
    planck.logSingleRequest(nonStreamingBody, JSON.stringify(completion))
  );

  // Example with streaming response
  const streamingBody = {
    model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: prompt }],
    stream: true,
  };

  const response = await together.chat.completions.create(streamingBody);
  const [stream1, stream2] = response.tee();

  // Log streaming response after sending the response to the client
  after(planck.logSingleStream(streamingBody, stream2.toReadableStream()));

  return new Response(stream1.toReadableStream());
}
```

```
```