> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pandaprobe.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM

> Auto-trace LiteLLM completion calls across every provider

[LiteLLM](https://docs.litellm.ai) gives you a single, OpenAI-compatible interface for calling 100+ LLM providers. `wrap_litellm` instruments the module-level `litellm.completion` / `litellm.acompletion` functions so every call — to OpenAI, Anthropic, Gemini, or any other supported provider — is traced automatically as an `LLM` span.

### Installation

<Tabs>
  <Tab title="pip">
    ```bash theme={null}
    pip install "pandaprobe[litellm]"
    ```
  </Tab>

  <Tab title="uv">
    ```bash theme={null}
    uv add "pandaprobe[litellm]"
    ```
  </Tab>
</Tabs>

### Setup

Unlike the other wrappers, LiteLLM's entry points are **module-level functions**, not a client instance — so `wrap_litellm` patches the `litellm` module rather than an object. It is idempotent (a second call is a no-op) and returns the module.

<Tabs>
  <Tab title="Patch the module">
    ```python theme={null}
    import litellm
    from pandaprobe.wrappers import wrap_litellm

    wrap_litellm(litellm)

    response = litellm.completion(
        model="gpt-5.4",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    ```
  </Tab>

  <Tab title="Import and patch">
    ```python theme={null}
    from pandaprobe.wrappers import wrap_litellm

    # Omit the argument to import litellm and patch it globally.
    litellm = wrap_litellm()

    response = litellm.completion(
        model="gpt-5.4",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    ```
  </Tab>
</Tabs>

Both the synchronous `completion` and asynchronous `acompletion` functions are patched, covering blocking and streaming calls.

## Completion API

Span name: `"litellm-chat"`, SpanKind: `LLM`

```python theme={null}
response = litellm.completion(
    model="gpt-5.4",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing."},
    ],
    temperature=0.7,
)
```

**What gets traced**

* Input: `messages` array, captured directly (LiteLLM already speaks the universal OpenAI schema)
* Output: assistant message from `choices[0].message`
* Model name (from the response body — the resolved provider model)
* Token usage: `prompt_tokens`, `completion_tokens`, `total_tokens`, plus detail fields (for example `reasoning_tokens` from `completion_tokens_details`)
* Model parameters: `temperature`, `top_p`, `max_tokens`, `max_completion_tokens`, `reasoning_effort`, `thinking`, `response_format`, `stop`, and other safe parameters only
* Reasoning: LiteLLM's normalized `reasoning_content` is stored in span metadata as `reasoning_summary`

## Any provider, one call

LiteLLM routes to the target provider based on the `model` string — the wrapper and trace shape are identical regardless of which provider serves the request. Set the matching provider API key in your environment.

<CodeGroup>
  ```python title="OpenAI" theme={null}
  litellm.completion(
      model="gpt-5.4",
      messages=[{"role": "user", "content": "Hi"}],
  )  # OPENAI_API_KEY
  ```

  ```python title="Anthropic Claude" theme={null}
  litellm.completion(
      model="anthropic/claude-sonnet-4-6",
      messages=[{"role": "user", "content": "Hi"}],
  )  # ANTHROPIC_API_KEY
  ```

  ```python title="Google Gemini" theme={null}
  litellm.completion(
      model="gemini/gemini-3.1-flash-lite",
      messages=[{"role": "user", "content": "Hi"}],
  )  # GEMINI_API_KEY
  ```
</CodeGroup>

## Streaming

<CodeGroup>
  ```python title="completion — sync" theme={null}
  stream = litellm.completion(
      model="gpt-5.4",
      messages=[{"role": "user", "content": "Hello!"}],
      stream=True,
      stream_options={"include_usage": True},
  )
  for chunk in stream:
      delta = chunk.choices[0].delta.content
      if delta:
          print(delta, end="")
  ```

  ```python title="acompletion — async" theme={null}
  stream = await litellm.acompletion(
      model="gpt-5.4",
      messages=[{"role": "user", "content": "Hello!"}],
      stream=True,
      stream_options={"include_usage": True},
  )
  async for chunk in stream:
      delta = chunk.choices[0].delta.content
      if delta:
          print(delta, end="")
  ```
</CodeGroup>

Streaming is fully supported. The wrapper records `completion_start_time` on the first chunk for time-to-first-token tracking, and reduces chunks to a single response for the span output. Pass `stream_options={"include_usage": True}` so LiteLLM emits token usage on the terminal chunk — otherwise streamed spans have no usage totals.

## Async

The async entry point is the module-level `litellm.acompletion`, instrumented by the same `wrap_litellm` call — no separate client. It supports both blocking and streaming, mirroring `completion`.

```python theme={null}
response = await litellm.acompletion(
    model="anthropic/claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Hello!"}],
)
```

## Token usage mapping

LiteLLM normalizes every provider's usage into the OpenAI shape, so the mapping is the identity:

| LiteLLM Field                                | PandaProbe Field    |
| -------------------------------------------- | ------------------- |
| `prompt_tokens`                              | `prompt_tokens`     |
| `completion_tokens`                          | `completion_tokens` |
| `total_tokens`                               | `total_tokens`      |
| `completion_tokens_details.reasoning_tokens` | `reasoning_tokens`  |

## Using the LiteLLM proxy

If you run LiteLLM as a **proxy server** instead of the embedded SDK, it exposes an OpenAI-compatible endpoint. Point the `openai` client at the proxy base URL and trace it with [`wrap_openai`](/tracing/wrappers/openai) — no LiteLLM-specific wrapping is needed.

```python theme={null}
from openai import OpenAI
from pandaprobe.wrappers import wrap_openai

client = wrap_openai(OpenAI(base_url="http://0.0.0.0:4000", api_key="..."))
```

<Note>
  Wrappers work seamlessly with manual instrumentation. If a `litellm.completion` call happens inside a `pandaprobe.start_trace()` or `@pandaprobe.trace` context, the LLM span is automatically nested as a child span.
</Note>