Prerequisites
Install the CLI, then authenticate and select your project:PANDAPROBE_API_KEY and PANDAPROBE_PROJECT_NAME; set PANDAPROBE_ENDPOINT for a custom API endpoint. See CLI authentication.
Evaluation commands default to
--target trace. Use --target session when creating, inspecting, or reading scores from a session run. Add --project your-project-name to override the configured project for a command.Discover metrics and targets
loop_detection is computed during session evaluation and is excluded from the standalone trace metric list.
Evaluate selected IDs
Replace the example IDs below with IDs from your project.- Traces
- Sessions
id and status; evaluation continues in the background.
Evaluate filtered data
Useruns create to select targets with filters. For example, evaluate 25% of completed traces tagged production:
Use ISO 8601 timestamps for dates.
--date-to is exclusive. --filter-name matches part of a trace name; --name labels the evaluation run itself. With no filters, the run selects all targets in the project; the default sampling rate is 1.0.
Both create and batch accept --model to override the judge model. --sampling-rate applies only to filtered runs. Use pandaprobe evals runs create --help for flag details.
Review progress and scores
Use the run ID returned when you submitted the evaluation:- Traces
- Sessions
status, evaluated_count, and failed_count to track processing. Runs progress from PENDING to RUNNING, then COMPLETED or FAILED. COMPLETED can include individual computation failures. A score’s SUCCESS status means it was computed; inspect its value and reason to assess quality.
To find previous runs or scores for one target:
Use results in scripts
JSON goes to stdout and errors go to stderr. Withjq installed, select the fields your workflow needs:
A zero CLI exit code means the command succeeded, not that evaluation has finished or met your quality bar. A CI check should wait for a terminal run status, check computation failures, and then compare score values with its own criteria.
--format table. See the CLI reference for output formats and exit codes, or Scheduling to create recurring monitors from the CLI.
