/evaluations. The paths below are relative to https://api.pandaprobe.com for PandaProbe Cloud, or your self-hosted API endpoint.
For server-side API access, send your API key in
X-API-Key and project name in X-Project-Name. The dashboard uses bearer authentication with the active project context.Content-Type: application/json when posting the request bodies shown below. See the HTTP API reference for generated endpoint schemas.
Discover available metrics
Before creating eval runs, check which metrics are available.List trace metrics
loop_detection is excluded because it requires session context. Example entries:
List session metrics
Check LLM provider availability
Trace evaluation runs
Create a filtered trace eval run
202 Accepted immediately.
Request body:
Response (202):
Create a batch trace eval run
Rate limit: 50/min
Poll eval run status
status is COMPLETED or FAILED, then inspect failed_count and the individual scores. COMPLETED can include failed computations; a score’s SUCCESS status describes computation, not whether its value meets your quality bar.
List eval runs
Get scores for a run
Retry failed metrics
422 if the original run has no failures.
Rate limit: 50/min
Delete an eval run
eval_run_id set to null. Pass ?delete_scores=true to also delete all scores from this run.
Trace scores
Create a manual score
NUMERIC scores must be in the range [0.0, 1.0]. BOOLEAN scores must be
"true" or "false".List trace scores
Get latest scores for a trace
created_at. Use this to display a score overview panel for a specific trace.
Update a score
value, reason, and metadata can be changed. status is automatically set to SUCCESS and source to ANNOTATION.
Delete a score
Session evaluation runs
Session eval runs follow the same pattern as trace eval runs but target sessions instead of traces.Create a filtered session eval run
Rate limit: 50/min
Create a batch session eval run
Other session run endpoints
Session scores
Analytics
PandaProbe provides analytics endpoints for both trace and session scores.Trace score analytics
Summary — aggregated stats per metric:
Distribution — histogram of score values:
Session score analytics
Session score analytics mirror the trace analytics:
Session score history — track how a session’s score evolves over re-evaluations:
sort_order=asc to surface the worst sessions first.
Get an eval run template
To help build eval run requests, PandaProbe can generate a pre-filled template for a metric:Error handling
Next steps
Scheduling
Set up automated recurring evaluations with monitors.
Metrics
Calculations and metadata for trace and session metrics.

