How it works

​

The producing side, step by step

1. Turn end. Your loop (or a framework adapter) calls on_turn_end with the session id and turn payload. This returns immediately — nothing on this path blocks your agent.
2. Admission gates. Cheap producer-side controls run first: a hard per-process eval budget, per-session turn sampling, and a per-session rate limit. A newer turn supersedes the session’s in-flight evaluation.
3. Trace discovery. A background task lists the session’s completed traces (newest-first from the platform, replayed oldest-first) and keeps a per-session seen-set, so each turn scores only what is new.
4. Tiered evaluation. Tier 1 scores every new trace in one batched run and folds each into the trajectory gate; a gate breach escalates to Tier 2 on the last trace only; a confirmed Tier-2 breach may pull in Tier 3. See Evaluation & detection.
5. Outcome verification. If you wired a verifier, it runs alongside the tier ladder (it depends only on the turn payload, so stacking its latency after the ladder would be wasteful) and contributes a synthetic outcome_correct score.
6. Validation cadence. Every handled report — healthy or not — feeds the candidate-rule validation engine: forward trials get their denominator, and a single-flight task evaluates open candidates. See Rule validation.
7. Notice gate. Identical conditions are de-duplicated per session, recoveries are journaled, and a circuit breaker escalates notice storms to a single needs_human.
8. Persistence. A surviving condition becomes a DiagnosticNotice in mailbox/pending/, a verbose dump under traces/, and a journal event — and, when the severity is breach and capture is enabled, a replayable eval case.

​

The repair side

1. Episode assembly. Related notices from the same session and turn — those whose trace or signature evidence overlaps — are coalesced into one episode. One underlying failure gets one diagnosis, not three.
2. Bounded diagnosis. A package-owned tool loop reads the notice, inspects a trace that notice named, and searches existing rules for prior coverage. It runs on its own model, its own session, and its own trace, so repair activity is never scored as task activity.
3. One resolution. The episode ends with at most one candidate rule (harness_rule_add, including which scope it belongs in) and exactly one acknowledgement. Timeout or failure acknowledges nothing and leaves the notice recoverable.

​

The consuming side

​

The workspace on disk

/harness/                            (HARNESS_ROOT)
├── rules.jsonl                      # THE SOURCE OF TRUTH (latest record per id wins)
├── rules.md                         # generated task-facing guide + References index
├── rules/                           # generated scope files — read on demand, never pushed
│   ├── global.md                    #   the default: broadly reusable rules
│   ├── <topic>.md                   #   a context the repair agent judged apt
│   └── scoped.md                    #   specific rules with no better name
├── scope_metadata.json              # bounded per-scope descriptions for the index
├── mailbox/
│   ├── pending/<notice-id>.json     # posted by the hook, claimed by the repair agent
│   ├── processed/<notice-id>.json   # resolved notices, with their resolution
│   └── status.json                  # cheap summary for operators
├── journal.jsonl                    # append-only cross-run event log
├── evalset/<case-id>.json           # replayable eval cases (failures + protected wins)
├── traces/
│   ├── latest_eval.json             # most recent eval dump (always written)
│   └── <notice-id>.json             # one immutable dump per notice
└── state/score_history.json         # per-(session, metric) series + trajectory-gate state

​

Journal event types

harness.journal.recent(20, types=("notice", "rule_promote"))

​

One healing cycle, end to end

health → notice(trend) → notice(breach) → evalset_capture → repair_started
       → repair_candidate_added → rule_add → repair_completed
       → validation_round_started → validation_verdict → rule_promote → recovery

1. Turns 1–5 make no progress on the task: task_completion sits flat and the trajectory gate closes its window → stall:task_completion. Tier 2 comes back clean, so this is an advisory trend notice — recorded, but nothing is captured to train on.
2. Turn 6 repeats an identical payment call. The gate fires again, Tier 2 escalates, and tool_correctness scores 0.20 → a breach notice with trace_id and tier: 2. The session is captured as a replayable failure case.
3. Still inside that turn’s barrier, the repair agent takes the episode. It reads the notice, inspects the flagged trace, searches for prior coverage, finds none, and writes “Never call the payment tool twice without verifying the transaction status” — filing it under payments, because the evidence is specific to that workflow. The rule enters as a candidate.
4. Validation replays the captured failure with the candidate discoverable and the rest of the provisional set hidden, so the delta is attributable. task_completion improves past the margin → promoted (rule_promote, validator replay).
5. Subsequent turns climb instead of stalling (recovery). Your task agent can now read that rule via harness_rules_read whenever it judges payment work relevant — and a regression run confirms the old failure is improved and the protected win unchanged.