Dashboard setup
Create monitors from the Evaluations tab in the PandaProbe dashboard.1
Open Evaluations
Open the Evaluations tab from the dashboard navigation.
2
Open Monitors
Select the Monitors card from the Evaluations landing page.
3
Click Create monitor
Click Create monitor to open the monitor sidebar.
4
Configure the monitor
Add a name, choose the target type (
TRACE or SESSION), select metrics, and add filters that define the traces or sessions the monitor should evaluate.5
Set the cadence
Choose how often the monitor should create a new eval run. Cadence controls the recurring schedule.
6
Submit
Click Create monitor. The monitor starts in the background and creates eval runs on its configured schedule.
Monitor fields
When creating a monitor from the dashboard, configure:- Name: a human-readable label for the monitor.
- Target type:
TRACEfor trace evaluation orSESSIONfor session evaluation. - Metrics: the trace-level or session-level metrics to run.
- Filters: the matching traces or sessions to evaluate.
- Sampling rate: the portion of matching data to evaluate on each run.
- Cadence: how often PandaProbe creates a new eval run.
- Model: optional model selection for LLM-as-judge metrics.
- Customize signal weights: optional for session monitors.
- Skip run if no new data since last run: enables
only_if_changed(on by default).
Filters
- Trace monitors
- Session monitors
Trace monitors support Started after, Started before, Trace status, Trace name contains, Session ID, User ID, and Tags.
Sampling rate
Sampling rate controls what portion of matching data is evaluated each time the monitor runs. For example:1.0evaluates all matching traces or sessions.0.5evaluates 50% of matching traces or sessions.0.1evaluates 10% of matching traces or sessions.
API setup
Use the API authentication setup. Paths below are relative tohttps://api.pandaprobe.com or your self-hosted API endpoint.
Create a monitor
Request fields
Session monitor example
CLI setup
After installing and authenticating the CLI, create a monitor with the same metrics and filters:--target session with session metrics such as agent_reliability. The CLI also supports --signal-weights for session monitors and --only-if-changed=false to disable the new-data check.
Cadence options
Monitors support predefined intervals and custom cron expressions.
Predefined intervals schedule the next run relative to creation or the previous scheduling event. Use a cron expression when you need a fixed time of day. Cron schedules use UTC.
The only_if_changed flag
When only_if_changed is true (the default), PandaProbe checks for traces whose started_at is later than the monitor’s previous run time. For session monitors, those traces must have a session ID. The check also applies the configured date, user, and tag filters.
This decides whether to run; it does not restrict evaluation to new data. Once triggered, the run evaluates the monitor’s full filtered and sampled target set. Changes to older traces alone do not satisfy the new-data check.
Set it to false when you want the monitor to run on every cadence tick, even if the underlying data has not changed.
Manage monitors
Monitors have two states:
Common API operations:
trigger to create an immediate eval run from a monitor without waiting for the next scheduled cadence.
Next steps
Evaluation via UI
Create one-off trace and session eval runs from the dashboard.
Evaluation via API
Create and manage eval runs programmatically.

