The repair agent needs a model.
Harness.create() raises ValueError unless you set repair_model (or HARNESS_REPAIR_MODEL), because no billable default is ever chosen for you. observe_only=True is the one non-mutating exception.Why a second agent
The obvious design is to let the task agent heal itself: it already has the context, so hand it the mailbox and some tools. A measured AppWorld run showed what that actually costs. Given ten administrative tools and a standing instruction to check its mailbox, the agent spent its turns operating the harness instead of doing the task — and nine of the thirteen rules it wrote were about gaming its own diagnostic protocol rather than about the work. The failure is structural, not a prompting mistake. Diagnosis and execution are different jobs that want different context, different tools, and different success criteria. Splitting them means the task agent’s whole surface is the task, and the repair agent’s whole surface is one failure.The ownership line
One episode, end to end
1
Notices are grouped
Pending notices from the same session and turn are coalesced into one episode when their trace or signature evidence overlaps — one underlying failure, one diagnosis. The episode keeps every notice id, and one resolution acknowledges the whole group atomically.
2
The assignment is built
The episode becomes a
RepairAssignment: the notices with their alerting metrics, thresholds and judge reason strings, the flagged trace ids, the dump path, any host RuleScopeHint metadata and task_summary, and your domain_policy. All of it is sanitized and bounded, and the prompt declares it untrusted data rather than instructions.3
The repair agent runs
A bounded tool loop over PandaProbe’s official LiteLLM wrapper, in a fresh async context under its own session id. It reads the notice, inspects a trace the notice named, searches existing rules for prior coverage, and then either writes one candidate or resolves without one.
4
It resolves exactly once
An episode ends in one
RepairStatus: candidate_added, duplicate (an active rule already covers it), already_covered (a candidate is already testing it), no_proposal, unactionable, timed_out, failed, or cancelled. At most one candidate per episode, always.5
Settlement returns the outcome
settle() waits for evaluation, notice persistence, and one bounded repair attempt, then hands you a RepairResult. Timeout, cancellation, or failure acknowledges nothing — the notice stays pending and recoverable, and the developer task never fails.The repair capability set
The repair agent’s tools are package-internal and scoped to its own episode. It cannot read a notice it was not assigned, or inspect a trace that notice does not name.Novelty comes first
Before a proposal is accepted, the store checks it against live rules in the target scope — normalized exact text, failure signatures, bounded tags, and deterministic lexical overlap. A covered proposal resolves asduplicate or already_covered instead of adding near-identical guidance. The prompt reinforces it: minor wording changes, a narrower example, or another occurrence of the same workflow do not justify a new rule.
A rule’s metric is also validated against the evaluator’s metric registry at this boundary. That field is not a label — validation matches it against breach:<metric>-style signatures to decide which sessions and replay cases count as evidence. A name that matches nothing reads as “never breached”, which can invert the rule’s own verdict, so an unknown or composite value is rejected for the model to correct.
Scope selection
The repair agent also decides where the rule is filed, as part of the same call — no extra model round for a filename.globalis the default: broadly reusable rules, not tied to one task, workflow, application, tool, or domain.- A concise contextual name — an application, workflow, or domain drawn from the evidence — is preferred whenever the rule genuinely belongs to that context. The catalog is open; a new name simply creates
rules/<scope>.md. scopedis the fallback: the rule is specific, but no meaningful stable name could be determined.
RuleScopeHint metadata and a short task_summary on the turn payload. Both inform the decision; neither dictates it.
task_summary exists because a task id is opaque. Told only 3ab5b8b_2, a model has nothing to name a scope after; told what the task asked for, it can tell a Spotify library task from a Venmo payment. It is sanitized, length-bounded, and framed as untrusted data like every other externally-authored string.../../etc/passwd into etc-passwd would file a rule under a name nobody chose); a generic host or integration label is refused, because it names where the agent ran rather than what failed; and with no expressed choice the default applies — silence never resolves to scoped.
Trace isolation
Repair activity must never be scored as task activity, or the harness would be grading itself.- Repair completions run under a distinct SDK session,
repair-<task-session-id>-<episode-id>, with repair-role metadata. Exact-session task-trace discovery excludes them. - With
trace_repair_agent=true, each run exports one trace namedpandaprobe: aharnessCHAIN span containing repeatedrepair-agentandtoolsAGENT spans, eachrepair-agentholding the wrapper’slitellm-chatLLM span and eachtoolsspan holding one TOOL child per workspace call. - With it disabled (the default), the SDK context is non-exporting, so the wrapper cannot create an accidental standalone trace. Task tracing is identical either way.
Cost and bounds
Repair is the loop’s second model consumer, and it is bounded on four axes:repair_timeout_s (60 s), repair_max_turns (6), repair_max_tokens (4096), and one episode per settled turn at most. Notice coalescing bounds it further — three related notices on one turn produce one episode, not three.
The six-turn default accommodates providers that emit a single tool call per round, letting one episode complete read → inspect → search → add → acknowledge without any provider-specific orchestration. repair_reasoning_effort defaults to "none" because current OpenAI reasoning models require it to use function tools on the wrapped chat-completions path; it is forwarded only when LiteLLM reports support.
Observing repair
repair_started, repair_model_turn, repair_tool_call, repair_candidate_added, repair_notice_resolved, and one terminal event per episode (repair_completed, repair_duplicate, repair_already_covered, repair_no_proposal, repair_unactionable, repair_timed_out, repair_failed). Each carries the episode id, grouped notice ids, recommended and selected scope, considered rule ids, and suppression reason — never prompts, credentials, provider responses, or unbounded diagnostic payloads.