Skip to main content
Use the PandaProbe CLI for terminal workflows and shell-based automation. It calls the same evaluation API as the dashboard and works with data already captured in your project.

Prerequisites

Install the CLI, then authenticate and select your project:
Cloud login opens a browser and saves the API key and project name locally. For self-hosted deployments or CI, use an API key with PANDAPROBE_API_KEY and PANDAPROBE_PROJECT_NAME; set PANDAPROBE_ENDPOINT for a custom API endpoint. See CLI authentication.
Evaluation commands default to --target trace. Use --target session when creating, inspecting, or reading scores from a session run. Add --project your-project-name to override the configured project for a command.

Discover metrics and targets

Choose metrics from the matching list. loop_detection is computed during session evaluation and is excluded from the standalone trace metric list.

Evaluate selected IDs

Replace the example IDs below with IDs from your project.
Both ID flags accept comma-separated lists. The command returns the created run as JSON, including its id and status; evaluation continues in the background.

Evaluate filtered data

Use runs create to select targets with filters. For example, evaluate 25% of completed traces tagged production:
For sessions, select sessions with at least three traces and optionally customize signal weights:
Omitted signal weights retain their defaults. The available filter flags depend on the target: Use ISO 8601 timestamps for dates. --date-to is exclusive. --filter-name matches part of a trace name; --name labels the evaluation run itself. With no filters, the run selects all targets in the project; the default sampling rate is 1.0. Both create and batch accept --model to override the judge model. --sampling-rate applies only to filtered runs. Use pandaprobe evals runs create --help for flag details.

Review progress and scores

Use the run ID returned when you submitted the evaluation:
Read status, evaluated_count, and failed_count to track processing. Runs progress from PENDING to RUNNING, then COMPLETED or FAILED. COMPLETED can include individual computation failures. A score’s SUCCESS status means it was computed; inspect its value and reason to assess quality. To find previous runs or scores for one target:

Use results in scripts

JSON goes to stdout and errors go to stderr. With jq installed, select the fields your workflow needs:
A zero CLI exit code means the command succeeded, not that evaluation has finished or met your quality bar. A CI check should wait for a terminal run status, check computation failures, and then compare score values with its own criteria.
For readable terminal output, add --format table. See the CLI reference for output formats and exit codes, or Scheduling to create recurring monitors from the CLI.