Prerequisites
- Captured traces in the project you want to evaluate. Start with the Tracing Quickstart if needed.
- A shared
session_idfor related traces when running session evaluation. - Project access through the dashboard, or an API key and project name for API and CLI requests.
PandaProbe Cloud manages the evaluation LLM infrastructure. Self-hosted deployments need provider credentials configured on the server for judge and embedding calls.
Configure a run
- Choose traces or sessions and select the matching metrics.
- Select specific IDs, or use filters to define the target set. Filtered runs can sample a fraction of matches.
- Optionally choose a judge model. Session runs also support custom signal weights.
- Submit the run and review its progress and scores when processing finishes.
failed_count and individual score statuses when reviewing results.

