LiteLLM

LiteLLM gives you a single, OpenAI-compatible interface for calling 100+ LLM providers. wrap_litellm instruments the module-level litellm.completion / litellm.acompletion functions so every call — to OpenAI, Anthropic, Gemini, or any other supported provider — is traced automatically as an LLM span.

Installation

pip install "pandaprobe[litellm]"

uv add "pandaprobe[litellm]"

Setup

Unlike the other wrappers, LiteLLM’s entry points are module-level functions, not a client instance — so wrap_litellm patches the litellm module rather than an object. It is idempotent (a second call is a no-op) and returns the module.

Patch the module
Import and patch

import litellm
from pandaprobe.wrappers import wrap_litellm

wrap_litellm(litellm)

response = litellm.completion(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}],
)

from pandaprobe.wrappers import wrap_litellm

# Omit the argument to import litellm and patch it globally.
litellm = wrap_litellm()

response = litellm.completion(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}],
)

Both the synchronous completion and asynchronous acompletion functions are patched, covering blocking and streaming calls.

Completion API

Span name: "litellm-chat", SpanKind: LLM

response = litellm.completion(
    model="gpt-5.4",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing."},
    ],
    temperature=0.7,
)

What gets traced

Input: messages array, captured directly (LiteLLM already speaks the universal OpenAI schema)
Output: assistant message from choices[0].message
Model name (from the response body — the resolved provider model)
Token usage: prompt_tokens, completion_tokens, total_tokens, plus detail fields (for example reasoning_tokens from completion_tokens_details)
Model parameters: temperature, top_p, max_tokens, max_completion_tokens, reasoning_effort, thinking, response_format, stop, and other safe parameters only
Reasoning: LiteLLM’s normalized reasoning_content is stored in span metadata as reasoning_summary

Any provider, one call

LiteLLM routes to the target provider based on the model string — the wrapper and trace shape are identical regardless of which provider serves the request. Set the matching provider API key in your environment.

litellm.completion(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Hi"}],
)  # OPENAI_API_KEY

litellm.completion(
    model="anthropic/claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Hi"}],
)  # ANTHROPIC_API_KEY

litellm.completion(
    model="gemini/gemini-3.1-flash-lite",
    messages=[{"role": "user", "content": "Hi"}],
)  # GEMINI_API_KEY

Streaming

stream = litellm.completion(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

stream = await litellm.acompletion(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
    stream_options={"include_usage": True},
)
async for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

Streaming is fully supported. The wrapper records completion_start_time on the first chunk for time-to-first-token tracking, and reduces chunks to a single response for the span output. Pass stream_options={"include_usage": True} so LiteLLM emits token usage on the terminal chunk — otherwise streamed spans have no usage totals.

Async

The async entry point is the module-level litellm.acompletion, instrumented by the same wrap_litellm call — no separate client. It supports both blocking and streaming, mirroring completion.

response = await litellm.acompletion(
    model="anthropic/claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Hello!"}],
)

Token usage mapping

LiteLLM normalizes every provider’s usage into the OpenAI shape, so the mapping is the identity:

LiteLLM Field	PandaProbe Field
`prompt_tokens`	`prompt_tokens`
`completion_tokens`	`completion_tokens`
`total_tokens`	`total_tokens`
`completion_tokens_details.reasoning_tokens`	`reasoning_tokens`

Using the LiteLLM proxy

If you run LiteLLM as a proxy server instead of the embedded SDK, it exposes an OpenAI-compatible endpoint. Point the openai client at the proxy base URL and trace it with wrap_openai — no LiteLLM-specific wrapping is needed.

from openai import OpenAI
from pandaprobe.wrappers import wrap_openai

client = wrap_openai(OpenAI(base_url="http://0.0.0.0:4000", api_key="..."))

Wrappers work seamlessly with manual instrumentation. If a litellm.completion call happens inside a pandaprobe.start_trace() or @pandaprobe.trace context, the LLM span is automatically nested as a child span.

Get Started

Tracing

Installation

Setup

Completion API

Any provider, one call

Streaming

Async

Token usage mapping

Using the LiteLLM proxy

​Installation

​Setup

​Completion API

​Any provider, one call

​Streaming

​Async

​Token usage mapping

​Using the LiteLLM proxy

Installation

Setup

Completion API

Any provider, one call

Streaming

Async

Token usage mapping

Using the LiteLLM proxy