Rules workspace

Rules are the harness’s output: operating constraints learned from real failures, written by the repair agent and read by your task agent on demand. They live in a workspace PandaProbe owns — a generated guide that indexes scope files, and scope files holding the rules themselves.

Nothing here is injected into your prompt. rules.md is the task-facing directory: the read-only workflow plus a generated References index — no rule text. Your agent pulls the bodies it wants with harness_rules_read, or none at all.

Why nothing is injected

The obvious design is to inline every rule into every prompt. It does not hold up. In a measured AppWorld run of a build that did exactly that — the whole corpus in context, ten administrative tools, and a check-your-mailbox-every-turn mandate — the agent spent its turns operating the harness instead of doing the task, and nine of the thirteen rules it wrote were about gaming its own diagnostic protocol rather than about the work. So the harness uses the shape that already works for agent skills: a small, stable index that says what exists and how to reach it, with content behind a read. The corpus can grow without crowding out the task, and the agent decides what is relevant instead of a lexical scorer guessing on its behalf.

The layout

<HARNESS_ROOT>/
├── rules.jsonl               # THE SOURCE OF TRUTH (latest record per id wins)
├── rules.md                  # generated task-facing guide + References index.
│                             #   Contains NO rule text.
├── rules/
│   ├── global.md             # the default: broadly reusable rules
│   ├── payments.md           # any <topic>.md the repair agent judged apt
│   └── scoped.md             # specific rules with no better name
└── scope_metadata.json       # bounded per-scope descriptions for the index

rules.jsonl is the source of truth; every .md file is a projection, regenerated from the store on every change. Read the markdown freely; change rules through the store.

Scopes

scope is a free-form string on each rule, and it is the file the rule lands in. The repair agent chooses it from the failure evidence, as part of the call that writes the rule.

Scope	Meaning
`global`	The default. Broadly reusable — not tied to one task, workflow, application, tool, or domain.
`<topic>`	A concise contextual name drawn from the evidence: an application, a workflow, a domain. Preferred whenever the rule genuinely belongs to that context. The catalog is open; a new name simply creates its file.
`scoped`	The fallback. The rule is specific, but no meaningful stable name could be determined.

Silence means global, not scoped. scoped is a considered verdict — “specific, but unnameable” — so a rule with no expressed scope takes the broadly-applicable default rather than landing in a catch-all. That inverts the v0.8 behavior, where an unspecified scope became scoped and, in practice, everything ended up in one file.

Hosts may pass bounded RuleScopeHint metadata and a short task_summary on the turn payload to inform the choice; neither dictates it. PandaProbe enforces only filename safety and refuses a generic host or integration label, since that names where the agent ran rather than what failed.

The generated guide

rules.md sits beside the rules.jsonl store and the rules/ scope files it indexes. It has SKILL-style frontmatter declaring the four allowed read-only tools, a stable body explaining the pull workflow, and a generated References section — global first, then every other live scope alphabetically:

---
name: pandaprobe-learned-rules
description: Learned rules maintained by PandaProbe. Use when prior validated or
  provisional rules may help the current task; inspect only the relevant scopes
  through the allowed read-only tools.
allowed-tools:
  - harness_rules_list
  - harness_rules_read
  - harness_rules_search
  - harness_rule_status
---

# PandaProbe Learned Rules

PandaProbe keeps learned rules in scoped reference files. This guide is the
task-facing directory: it explains the read-only tools and points to live scopes,
but never contains rule bodies. …

## References
<!-- REFERENCES — generated by the harness; do not edit below -->

- [`global`](rules/global.md)
  Broadly reusable execution and verification rules.
  2 active, 0 provisional

- [`payments`](rules/payments.md)
  Payments workflows.
  3 active, 1 provisional

Each entry carries the scope key, its path, one bounded description, and active/provisional counts — never rule text. With no rules yet, References says so rather than being empty or misleading.

The guide is a derived view, not a document you maintain. Everything above the References marker comes from the packaged template and is versioned with the package; everything below is regenerated from rules.jsonl on every rule mutation and at startup. Hand-edit either half and the next rule write overwrites it. Delete the file and it comes back.

harness.system_context(session_id) does not return this guide. The task preamble is one stable sentence saying learned rules exist and naming the four tools. An agent reads the guide only by calling harness_rules_list — nothing is read on its behalf, and nothing is read before a turn.

A rule file

# Rules — payments

_Rules learned in the context of `payments`._

- **r-9772c134b8**: Never call the payment tool twice without first verifying
  the transaction status identifier.
  - rationale: Repeated identical payment call flagged by tool_correctness.
    (added 2026-07-30T05:35:11+00:00, from notice n-20260730T053511-f2019e57)

### Provisional rules (under evaluation)

_The following candidate rules are available but not yet validated. Treat
them as tentative and prefer validated rules when they conflict._

- **r-23b0460306** (candidate): Verify the transaction status before retrying.
  - rationale: … (added …; trial: 2/5 sessions observed)

Candidates always render in full — a rule must be in force to be measurable — under a heading that makes their unproven status unmistakable. Retrieval may narrow the actives in a file, never the candidates: filtering a trial would starve it.

The rule record

Field	Purpose
`id`, `created_at`	Identity (`r-` + 10 hex chars) and provenance timestamp.
`rule`, `rationale`	The constraint and why it exists — both sanitized on write.
`scope`	The rule file it lives in.
`applicability`	How widely it applies — `global`, `topical`, or `task` — as a separate axis from where it is filed.
`source_notice_id`, `metric`	Provenance: which notice motivated it, which metric it targets.
`status`	`candidate` → `active` \| `retired`. See Rule validation.
`tags`	Retrieval hooks, auto-derived from the source notice (signatures, metric and signal names, severity) plus any explicit tags. Used for ranking only, never for placement.
`trial`	Validation bookkeeping: baseline vs. observed breach rates, sessions watched, replay attempts, verdict.

Guard rails on add:

Dedup — a rule whose normalized text matches a live rule returns the existing rule instead of creating a duplicate.
Novelty — a proposal already covered by a live rule in the same scope resolves as duplicate or already_covered instead of adding near-identical guidance. This, not a global count, is what controls proliferation.
Cap — max_active_rules defaults to 0, meaning unlimited. Set a positive value as an operator safety bound; nothing is ever silently evicted.
Metric validation — metric must name a real evaluator metric. Validation matches it against breach:<metric> signatures to decide what counts as evidence, so a name matching nothing reads as “never breached” and can invert the rule’s own verdict.
Sanitization — rule text and rationale cross the trust boundary before they can ever reach a prompt.

Older stores

A record with no scope reads as the default, global — like any other unspecified field. Existing global, scoped, and custom scope files keep working untouched: nothing is migrated, moved, or duplicated, and there is no migration command. The first render regenerates rules.md and the rules/ subtree from whatever the store holds. See Upgrading to v0.9. A scope whose rules all retire keeps its file, rewritten empty, rather than being deleted — a prior context may reference the path, and a file saying “no rules here” is a better answer than a missing one.

Retrieval

rule_retrieval (default true) and rules_context_topk (default 8) narrow the active rules inside a rendered scope file when a caller passes a query. They do not shape the system prompt, which carries no rule bodies to shape, and they do not affect the task-facing read/search/list tools — those never preselect on the agent’s behalf. The scorer is deliberately simple and dependency-free: casefolded word tokens, tag hits weighted double over rule/rationale text, normalized by query size, ties broken by recency. breach:tool_correctness tokenizes to breach + tool_correctness, so signatures, metric names, and task words all match naturally. Behavior at the edges:

No query → everything renders; retrieval never hides rules without a reason.
No overlap → ranking degrades to recency under the same top-k.
Candidates never consume top-k slots.
A trimmed file carries a one-line note pointing at harness_rules_search / harness_rules_list, so every rule stays reachable.

Working with rules programmatically

harness.rules.active()                      # validated rules
harness.rules.candidates()                  # rules under trial
harness.rules.scopes()                      # ["global", "scoped", "payments"]
harness.rules.by_scope("payments")
harness.rules.read_scope("global")          # the rendered file, as the agent sees it
harness.rules.search("payment breach")      # ranked (rule, score) pairs
harness.rules.retire("r-9772c134b8", reason="superseded by a broader rule")
harness.rules.effectiveness()               # per-rule notice counts before/after

The task-agent equivalents are exactly four reads: harness_rules_read, harness_rules_list, harness_rules_search, and harness_rule_status — see the task toolset. Writing (harness_rule_add) belongs to the repair agent; promotion and retirement belong to validation alone.

render_markdown() renders the guide plus every scope file, candidates included. It is an explicit operator call — not the system prompt, and no longer the replay context either: a replay now receives a capability-only context and discovers rules through the same read-only tools, with only the candidate under test visible.

Get Started

The Self-Heal Loop

Closing the Loop

Integrations

Reference

Why nothing is injected

The layout

Scopes

The generated guide

A rule file

The rule record

Older stores

Retrieval

Working with rules programmatically

​Why nothing is injected

​The layout

​Scopes

​The generated guide

​A rule file

​The rule record

​Older stores

​Retrieval

​Working with rules programmatically

Why nothing is injected

The layout

Scopes

The generated guide

A rule file

The rule record

Older stores

Retrieval

Working with rules programmatically