Skip to main content
Run evaluations against traces or sessions already captured in your project. Use the dashboard for interactive review, the API or CLI for automation, and monitors for recurring checks.

Prerequisites

  • Captured traces in the project you want to evaluate. Start with the Tracing Quickstart if needed.
  • A shared session_id for related traces when running session evaluation.
  • Project access through the dashboard, or an API key and project name for API and CLI requests.
PandaProbe Cloud manages the evaluation LLM infrastructure. Self-hosted deployments need provider credentials configured on the server for judge and embedding calls.

Configure a run

  1. Choose traces or sessions and select the matching metrics.
  2. Select specific IDs, or use filters to define the target set. Filtered runs can sample a fraction of matches.
  3. Optionally choose a judge model. Session runs also support custom signal weights.
  4. Submit the run and review its progress and scores when processing finishes.
Each score includes a value, computation status, reason, and metadata. A completed run can contain failed computations; check failed_count and individual score statuses when reviewing results.

Choose a setup method