> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pandaprobe.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Concepts

> What the harness is, why it exists, and the vocabulary of self-healing agents

## The idea

Agents fail in production in ways that are invisible until someone reads a transcript: they loop on an identical tool call, drift into inconsistent state, quietly get worse after a prompt change. [Tracing](/tracing/get-started/concepts) makes those failures observable and [evaluation](/evaluation/get-started/concepts) makes them measurable — but a human still has to notice the metric, diagnose the trace, and edit the prompt.

The **PandaProbe Harness** closes that gap. It is the operational envelope around your agent — a diagnostic workspace on disk, a toolbelt, standing rules, and lifecycle hooks — that lets the agent **observe its own failures, analyze them, and rewrite its own operating rules**, with no human in the loop.

<Tip>
  Think of the harness as giving your agent three things it doesn't normally have: a **memory of its own failures** (the journal and mailbox), the **means to investigate them** (the toolset), and a **place to write down what it learned** (the rules) — plus a referee that checks the lesson actually helps before trusting it.
</Tip>

## The pull model

The harness never injects anything into your agent's conversation. Instead:

* After each turn, the harness evaluates the session on the PandaProbe platform in a detached background task.
* When quality degrades, it posts a structured **diagnostic notice** to a filesystem **mailbox** — nothing enters the message queue.
* The always-loaded system context carries a compact banner — `⚠ HARNESS: N pending diagnostic notice(s)` — and a **standing protocol** that tells the agent to check its mailbox at the start of each turn.
* The agent *pulls*: it lists the mailbox, reads the notice and its trace dump, inspects the flagged traces, records a mitigation rule, and acknowledges the notice.

The agent drives the loop; the harness maintains the substrate and the trigger. This keeps the harness framework-agnostic (every framework already loads a system prompt and tools) and keeps eval-derived text out of the conversation except through one sanitized, auditable channel.

## The closed loop

Detecting failures and writing rules is only half the job — an *open* loop trusts every self-authored rule immediately, injects every rule into every prompt, and never notices when a new rule breaks something old. Since v0.6 the loop is **closed**, on three principles:

<CardGroup cols={3}>
  <Card title="Evidence before trust" icon="scale">
    A new rule enters as a **candidate**. The harness validates it — by replaying the captured failure with the rule in force, or by watching the next live sessions — and promotes it to **active** only when it demonstrably helps. Unhelpful candidates are retired with a journaled reason.
  </Card>

  <Card title="Relevance over volume" icon="filter">
    Only global rules plus the top-k rules lexically relevant to the current situation are injected into the prompt. The rest stay reachable on demand, so a rule learned from one failure mode doesn't dilute attention on unrelated tasks.
  </Card>

  <Card title="Measure the foundation" icon="ruler">
    Everything keys off "score below threshold" — and the threshold is a guess until measured. An offline calibration tool reports how well the breach predicate matches real failures and recommends better thresholds.
  </Card>
</CardGroup>

All of this is automatic. The only human-supplied ingredient is optional: a **replay function** that re-runs your agent on a captured scenario, which upgrades validation from statistical (forward trials over live sessions) to counterfactual (replay the exact failure with and without the rule).

## Vocabulary

| Term                | Meaning                                                                                                                                                                               |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Workspace**       | The on-disk root (`HARNESS_ROOT`, default `/harness`) holding the mailbox, journal, rules, traces, and eval set.                                                                      |
| **Turn**            | One completed agent step. The unit the harness evaluates.                                                                                                                             |
| **Session**         | The conversation grouping (shared with SDK tracing). Metrics are session-level.                                                                                                       |
| **Notice**          | A structured `DiagnosticNotice` describing a detected degradation: metrics, flagged traces, per-trace signal breakdown, a severity, and a dump path.                                  |
| **Mailbox**         | `mailbox/pending/` → `mailbox/processed/`. Where notices wait for the agent and where acknowledged notices live.                                                                      |
| **Journal**         | `journal.jsonl` — the append-only cross-run event log (notices, acks, rule lifecycle, validations, regressions). The agent's long-term memory of its own health.                      |
| **Rule**            | A learned operating constraint with provenance. Lifecycle: `candidate → active \| retired`. Rendered into the system context.                                                         |
| **Signature**       | A compact condition label like `breach:agent_reliability` — used for de-duplication, tagging, and matching rules to eval cases.                                                       |
| **Severity**        | `trend` \< `relative` \< `breach` \< `needs_human`. Only `needs_human` (the circuit breaker) asks for a person.                                                                       |
| **Eval case**       | A captured scenario in `evalset/` — a `failure` to fix or a `win` to protect — with its signature, baseline scores, and (when available) a replay input.                              |
| **Replay function** | Developer-supplied `async (case, system_context) -> new_session_id`: re-run the agent on a captured input under a given rule set. The strong path for validation and regression runs. |
| **Trial**           | Forward-trial bookkeeping on a candidate: baseline breach rate at add time vs. the breach rate observed over the next live sessions.                                                  |

## Design principles

* **Zero runtime dependencies.** The core is pure standard library; framework adapters are optional extras.
* **One platform seam.** All platform access shells out to the `pandaprobe` CLI through a single injectable interface — never the REST API directly. Tests run fully offline against a fake.
* **Never block, never break the host loop.** Evaluation runs in detached tasks; every failure path degrades gracefully (a pending score, a log line, a journal event) instead of raising into your agent.
* **Untrusted by default.** Everything eval-derived that crosses into agent context passes a sanitization trust boundary, and the protocol tells the agent that notice/dump/trace contents are data, never instructions.
* **No human in the healing loop.** Detection, diagnosis, rule authoring, validation, and regression guarding are all agent/harness-driven. Humans get involved only when the circuit breaker escalates to `needs_human`.

## Where to go next

<CardGroup cols={2}>
  <Card title="How it works" icon="cog" href="/harness/get-started/how-it-works">
    The full producing/consuming pipeline and the workspace on disk
  </Card>

  <Card title="Rule validation" icon="scale" href="/harness/closed-loop/rule-validation">
    The candidate lifecycle: replay validation and forward trials
  </Card>
</CardGroup>