open/config-audit
1
0
Fork 0
Code Issues Releases Activity

docs: /doctor-overlap decision merged into v5.14 plan + positioning

Browse source 
Oppgave A measured (fasit-first, CC 2.1.220): /doctor overlaps our
judgment lenses and alarms, not the deterministic validators. Binding
outcome: 0 whole scanners removed, 2 measured function-duplicates
scheduled for removal (GAP autoMode dimension, CA-SKL-002 alarm role),
5 surfaces repositioned. Plan rewritten with D-chunks + B2-B4 merged
against the existing C-chunks and open dogfood posts; old rejection #5
(prose contradiction detection) superseded by B3 with evidence.

README gains the division-of-labor section vs /doctor; CLAUDE.md gains
the never-duplicate-a-doctor-check invariant.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NeaMRXVGzh9oSwigJDjE9
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 11:41:07 +02:00
commit d68b2152e3
3 changed files with 200 additions and 145 deletions
Download patch file Download diff file Expand all files Collapse all files

2
CLAUDE.md
View file

@ -4,6 +4,8 @@ Claude Code Configuration Intelligence — know if your config is correct, find
Per-command flags, patterns, and feature lists live in `README.md` and `/config-audit help`. This file carries what's invariant for working on the plugin.
**Positioning vs. built-in `/doctor` (measured 2026-08-03, binding):** we are the deterministic/reproducible/all-scope/zero-quota side; `/doctor` is usage-weighted one-shot judgment. Never build a feature whose whole value is duplicating a `/doctor` check — see README «config-audit vs. the built-in /doctor» and `docs/v5.13-model-routing-effort-deadref-plan.md` §A.
## Commands
### Core (just run `/config-audit` to get started)

28
README.md
View file

@ -40,6 +40,7 @@ A Claude Code plugin that checks configuration health, suggests context-aware im
- [Gotchas](#gotchas)
- [Data Storage & Safety Guarantees](#data-storage--safety-guarantees)
- [What This Plugin Does Not Cover](#what-this-plugin-does-not-cover)
- [config-audit vs. the built-in /doctor](#config-audit-vs-the-built-in-doctor)
- [Version History](#version-history)
- [License](#license)
@ -685,6 +686,33 @@ This plugin is cautious by design — configuration files are important, and a b
---
## config-audit vs. the built-in /doctor
Claude Code ships `/doctor` (alias `/checkup`, v2.1.205+): an agent-driven setup checkup that
diagnoses and — with your confirmation — fixes issues. The overlap with config-audit was
measured (2026-08-03, CC 2.1.220, prediction-before-measurement protocol), and the two tools
divide cleanly:
| | Built-in `/doctor` | config-audit |
|---|---|---|
| Engine | Model-driven judgment, one machine, one run | Deterministic scanners, byte-stable, reproducible |
| Cost | A full agent session per run (quota) | Free, seconds, scriptable/CI-able |
| Scope | Current setup; trims **checked-in** CLAUDE.md only | All scopes incl. **local/private** files, plus cross-repo `campaign` |
| Evidence | **Usage telemetry** (transcripts, lifetime counters) — its unique edge | Static analysis with provenance-stamped best-practices register |
| Memory | None between runs | Baselines (`drift`), suppressions with audit trail, backup/rollback |
Division of labor per area: `/doctor` parses settings — the SET scanner validates the schema
exhaustively (unknown/deprecated keys, types, whole cascade). `/doctor` measures hook
*latency* — the HKV scanner validates hook *correctness*. `/doctor` judges conflicts per run —
the CNF scanner detects them deterministically. `/doctor` trims derivable content from
checked-in CLAUDE.md — `optimize --subtract` covers **all** scopes with a coded load-bearing
floor (`/doctor` itself refers local-file trimming to `optimize --subtract`). `/doctor`
estimates context weight once — `tokens`/`manifest` measure it deterministically and
cache-aware. Run `/doctor` for a usage-weighted one-shot cleanup; run config-audit for
reproducible, all-scope, zero-quota auditing.
---
## Version History
| Version | Date | Highlights |

323
docs/v5.13-model-routing-effort-deadref-plan.md
View file

@ -1,161 +1,186 @@
# v5.13 Plan — Model Routing, Effort Awareness, Dead References
# v5.14 Plan — Doctor Overlap, Model Routing, Effort Awareness, Dead References
> **Target version is now v5.14.** The `5.13.0` slot was consumed by the pipeline-hardening batch
> (`optimize --subtract` is a feature, so that release could not be a patch). The filename is kept so
> existing references resolve; only the target moved.
> **Filename note:** kept as `v5.13-…` so existing references resolve; the target has been
> v5.14 since the `5.13.0` slot was consumed by the pipeline-hardening batch. Rewritten
> 2026-08-03 (session #53) to merge the `/doctor`-overlap decision (Oppgave A) and the
> context-engineering follow-ups (B1–B4) from `docs/v5.14-doctor-overlap-brief.md` into the
> pre-existing chunks. One plan, top-to-bottom, no relitigation.
Derived from an external video analysis ("The Model Isn't the Moat", 2026-07) cross-checked
against primary sources and against what config-audit already encodes. Every claim acted on
here was verified against Anthropic's own docs; video-only claims are explicitly rejected below.
## Inputs and their status
## Source verification (done 2026-07-14)
| Input | Status |
|---|---|
| `/doctor`-overlap measurement (session #53, fasit-first) | **DONE** — decisions binding, recorded in `docs/doctor-overlap-results.local.md` |
| B1 register freshness defect (evidence-age / supersededBy / sources[]) | **DONE** — landed as `d66035e` (TDD, 1477/1477 green) |
| Video-derived model/effort chunks (verified 2026-07-14) | Open — carried below unchanged |
| Open posts from dogfood sessions #45–#51 | Open — detail lives in `STATE.md`; referenced here by ID only |
## A. The `/doctor` verdict (measured 2026-08-03, CC 2.1.220 — binding)
`/doctor` (alias `/checkup`, v2.1.205+) is an **agent-driven, usage-data-backed, quota-priced,
non-reproducible** checkup: 10 checks incl. unused skills/plugins/MCP vs. context cost,
CLAUDE.md dedup/contradiction judgment, derivable-content trim (checked-in files only),
lazy-loading migration, hook latency, and a chars÷4 context manifest. It overlaps our
**judgment lenses and alarms — not the deterministic validators.** Full per-scanner table in
`docs/doctor-overlap-results.local.md`.
Binding outcomes (0 whole scanners removed; 2 measured function-duplicates removed;
5 surfaces repositioned):
| ID | Chunk | What |
|---|---|---|
| **D1** | GAP autoMode removal | Delete the «No autoMode classifier» gap dimension (`feature-gap-scanner.mjs:485`) — `/doctor` Sjekk 8 checks AND fixes it. Byte-stability: existing-scanner finding-removal variant of [[adding-scanner-byte-stability]]; humanizer entries for the removed title go too. |
| **D2** | SKL alarm re-scope | Remove CA-SKL-002's aggregate-over-budget **alarm** role (native startup warning v2.1.105/2.1.181 + `/doctor` Sjekk 6 both do it better-placed); re-position CA-SKL-001 as per-description **attribution** (author lens, repo scope); CA-SKL-003 untouched. Exact form (delete vs. re-scope 002) decided inside the chunk against the frozen-baseline cost. |
| **D3** | Positioning rewrite | **DONE in #53** — README section «config-audit vs. the built-in /doctor» (division-of-labor table + per-area split for SET/HKV/CNF/OPT/tokens/manifest, citing `/doctor`'s own referral to `optimize --subtract`) + CLAUDE.md invariant line. Remaining refinement (per-command copy in `commands/*.md`, if wanted) rides along with D1/D2. |
**Strategic line (constrains all copy):** our defensible identity = determinism/byte-stability,
all scopes incl. LOCAL, cross-repo campaign breadth, zero quota. `/doctor`'s unmatchable edge =
usage telemetry. Long-term (v5.15+ candidate, NOT this release): transcript/usage telemetry as a
scanner *input*, so the axes compose instead of competing.
## B. Context-engineering follow-ups (article 2026-07-24, brief §2)
- **B1 — DONE** (`d66035e`): `sources[]` + `published` + `supersededBy` + evidence-age rule
(green-but-outdated is now expressible and flagged; re-verifying the old source no longer
clears it). BP-SUB-001 carries the article as verified corroborating source.
**Deviation from brief, verified 2026-08-03:** the article contains NO mechanism-choice or
size-limit content → it was NOT added to BP-MECH-*/BP-SIZE-001 (that would be false
provenance). If a future read finds real coverage, add it then.
- **B2 — new lens axis (own chunk):** `BP-JUDG-001` + `CA-OPT-002` for instructions that are
local and specific but **over-specify and cage judgment** (article rule 1). NOT an extension
of `--subtract` — the floor (`floor-exclusion.mjs`) rightly protects these blocks from
deletion; this axis says *keep the content, loosen the phrasing*. Same precision gate as
`optimization-lens-agent` (cite rule + source, stay silent when unsure). Mixing the axes is
the ÅS#5 defect class.
- **B3 — two-layer duplication/contradiction (CNF extension):** article rule 4 + `/doctor`'s
measured Sjekk 2 catch (global CLAUDE.md model-policy ↔ agent frontmatter) prove the class
exists and is catchable. Extend `conflict-detector` with cross-layer checks where the pair is
structurable (CLAUDE.md/rule, rule/skill-description, CLAUDE.md-keyword ↔ frontmatter field).
This **supersedes the old "Explicitly rejected #5"** below — it now has evidence.
- **B4 — thinner gaps (backlog, after everything above):** rule 2 (skills/agents leaning on
examples where a parameter enum would do) and rule 6 (reference *form*: prefer in-code files;
`import-resolver` is the natural owner). Park until D/B2/B3 land.
- **Rules 3 and 5 need nothing** (verified in brief §B4): progressive disclosure = BP-LOAD-001..006
+ BP-MECH-003 + `token-hotspots`/`manifest`; router pattern + auto-memory covered.
## C. Carried chunks (verified 2026-07-14 — unchanged specs)
Source-verification table, rejected-claims list, and full chunk specs below are carried
verbatim from the 2026-07-14 revision; only numbering context changed.
### C1 — Register entries: model routing + effort (dogfoods `knowledge-refresh`)
- **BP-MODEL-001** (`model-fit`): mechanical/read-only subagents can pin a cheaper model via
`model:` frontmatter; orchestrator keeps the strong model. Source: code.claude.com/docs/en/sub-agents
→ `confirmed`.
- **BP-MODEL-002** (`model-fit`): reasoning effort tunable at five levels in five places;
default `high`; higher is not universally better. Source: code.claude.com/docs/en/model-config
→ `confirmed`.
- Schema per `scanners/lib/best-practices-register.mjs:42-102`. New sources SHOULD carry
`published` so the B1 evidence-age rule has teeth.
### C2 — fix-engine effort hygiene (tiny, TDD)
`fix-engine.mjs:26` `VALID_EFFORT_LEVELS` missing `xhigh` → nearest-match "fix" for `xhig`
corrects to `high`. Red test first: `findNearestEffortLevel('xhig') === 'xhigh'`. Align with
`settings-validator.mjs:75`.
### C3 — CA-CML dead prose references (new deterministic check)
Flag backtick-quoted relative file paths in CLAUDE.md prose that do not exist on disk (today
only `@import` targets are checked). Conservative v1: skip URLs, globs, placeholders,
absolute/`~/` paths. Severity low. New CA-CML-NNN (verify next free NNN at implementation).
Byte-stability per [[adding-scanner-byte-stability]] incl. humanizer step 7.
### C4 — feature-gap + inventory: model/effort awareness
New T3 opportunity check: authored agents where NO agent sets `model:`/`effort:` → routing
opportunity citing BP-MODEL-001/002. Fires only when authored agents exist; opportunity
framing, suppressable. `whats-active`/`manifest` surface `model`/`effort` per agent.
Humanizer step 7; verify via direct `scan()` ([[agent-commands-need-scanner-scoping]]).
Known tension with the operator's own Opus-for-everything policy stands as written 2026-07-14:
the check serves general users; on this machine it gets suppressed.
### C5 — planner-agent adversarial gate (AFTER DEL B 3.2 dogfood)
Required "Failure modes" section in `agents/planner-agent.md`'s action-plan contract.
Sequencing: only after the DEL B fasit pass that judges planner-agent, or the fasit target
moves mid-evaluation.
## D. Open posts from dogfooding (detail in STATE.md — not restated here)
C-SKL1 (#37) · M-BUG-26 · **M-BUG-28** (suppression-ID positional instability — ID-semantics
change touching all scanners + frozen snapshots, own chunk) · **M-BUG-41** (no scope-gate from
scan to write, two arms — design change, own chunk) · **arg-sluk CLI arm**
(`optimize-lens-cli` + `token-hotspots-cli`, `KNOWN_OPEN` in
`tests/scanners/cli-unknown-flag-rejection.test.mjs`) · **P6/M-BUG-44** (scanner-side stdout
with `--output-file`) · knowledge-refresh write-CLI · cleanup-invisible session files ·
web-poll candidates (nothing written; primary sources unread).
## Priority order for v5.14
Cheap-and-loud first, judgment-heavy later; D-chunks early because they DELETE code the rest
must not build on:
1. **C2** (minutes, pure TDD)
2. **Arg-sluk CLI arm** (closes M-BUG-46's class; makes later CLI work loud)
3. **D1 + D2** (doctor-duplicate removals — before anything extends GAP/SKL)
4. **D3** (positioning docs — cheap, high external value)
5. **M-BUG-28** (suppression ID semantics — big but foundational; suppressions guard everything after)
6. **C1** (register entries; exercises B1's `published` field)
7. **C4**, then **C3** (feature/scanner additions on the now-clean base)
8. **B2** (new lens axis)
9. **B3** (CNF two-layer extension)
10. **M-BUG-41** (scope-gate design)
11. **P6/M-BUG-44**, knowledge-refresh write-CLI, cleanup glob (small batch)
12. **C5** (after DEL B 3.2), **B4** backlog last
13. Release-cut v5.14.0 via `release-plugin.mjs` when the batch is coherent (version level from
`git log v5.13.0..HEAD` — D1/D2 removals + C3/C4 additions ⇒ minor)
## Source verification (done 2026-07-14 — carried)
| Claim from video | Verdict | Source |
|---|---|---|
| Orchestrator + cheaper worker models is a supported, recommended pattern | VERIFIED | code.claude.com/docs/en/sub-agents ("Control costs by routing tasks to faster, cheaper models like Haiku"), code.claude.com/docs/en/workflows |
| Reasoning effort is tunable per settings / session / launch / **per-agent frontmatter** / SDK; levels `low, medium, high, xhigh, max` | VERIFIED | code.claude.com/docs/en/model-config#adjust-effort-level, sub-agents doc |
| Leaked Fable 5 system prompt principles ("partial recognition ≠ current knowledge"; "a prompt implying a file is present doesn't mean one is"; answer-first-then-one-question; tool-call scaling 1 / 3–5 / 5–10) | VERIFIED near-verbatim, **provenance unconfirmed** (third-party leak repo, not Anthropic-confirmed) | github.com/asgeirtj/system_prompts_leaks `Anthropic/claude-fable-5.md` |
| "Fable 5 on low ≈ Opus 4.8 on high, slightly higher cost/quality" score-vs-cost chart | **CONTRADICTED** — no such chart/statement on Anthropic's pages; GPT-5.5 appears only in a testimonial | anthropic.com/news/claude-fable-5-mythos-5 |
| Orchestrator + cheaper worker models is supported/recommended | VERIFIED | code.claude.com/docs/en/sub-agents, /workflows |
| Effort tunable per settings/session/launch/agent-frontmatter/SDK; `low..max` | VERIFIED | code.claude.com/docs/en/model-config#adjust-effort-level |
| Leaked Fable 5 system-prompt principles | VERIFIED near-verbatim, provenance unconfirmed | github.com/asgeirtj/system_prompts_leaks |
| "Fable low ≈ Opus high" chart | **CONTRADICTED** | anthropic.com/news/claude-fable-5-mythos-5 |
## Already covered — no action
## Explicitly rejected (unchanged unless noted)
| Video idea | Existing coverage |
|---|---|
| "Process is the moat" (config/harness > raw model) | The plugin's entire thesis |
| Extract repeated procedure into a skill | BP-MECH-003 + CA-OPT-001 (`optimization-lens-scanner.mjs:121`) |
| CLAUDE.md size/ownership discipline | BP-SIZE-001 + CA-CML line/size checks (`claude-md-linter.mjs:109/:120/:140`) |
| Check that referenced imports exist | CA-IMP broken `@import` (`lib/import-resolver.mjs:88`) — but **only** `@import`, see Chunk 3 |
| Plugin's own agents are model-routed | Agents table already pins sonnet for mechanical, opus for judgment |
1. "Fable low ≈ Opus high" framing — never encode.
2. Tool-call-count effort scaling as register entry — unconfirmed leak, not carried.
3. Cost/intelligence/"taste" routing-table generator — subjective, doesn't fit provenance-gated design.
4. "Fable mode" skill — out of plugin scope.
5. ~~CLAUDE.md prose contradiction detection~~ — **superseded by B3** (2026-08-03: article
rule 4 + `/doctor` Sjekk 2 measurement supplied the evidence the 2026-07-14 rejection lacked).
## Gaps → chunks
## Verification (per chunk, unchanged discipline)
Verified gap summary (register-mapper sweep, 2026-07-14): no scanner audits per-agent
`model:`/`effort:` frontmatter; effort has validity-check only (`settings-validator.mjs:195`),
no recommendation; no dead-reference check for prose file mentions in CLAUDE.md; no
adversarial/failure-mode requirement in planner-agent; `fix-engine.mjs:26` effort list
omits `xhigh` (settings-validator has all five).
- Full suite green (`node --test 'tests/**/*.test.mjs'`; baseline 2026-08-03: 1477/0), frozen
`tests/snapshots/v5.0.0/` untouched (`git status --porcelain` empty), red test before every
production change.
- **D1:** GAP fixture with autoMode absent → no finding; humanizer has no orphaned entries
(M-16/M-17 checks reversed for removal); frozen baselines untouched or consciously re-seeded
per [[adding-scanner-byte-stability]].
- **D2:** over-budget fixture → no 002-alarm (or re-scoped payload per in-chunk decision);
001 fires per oversized description with attribution copy; 003 unchanged.
- **D3:** README/CLAUDE.md name `/doctor` explicitly; `self-audit --check-readme` PASS.
- **C1–C5:** criteria as specified 2026-07-14 (C2 red-first nearest-match; C3 fixture
missing-path fires / URL-glob-placeholder silent; C4 authored-agent matrix; C5 failure-modes
section present).
- **B2:** fixture with a precise-but-caging instruction → CA-OPT-002 with rule+source citation;
floor-protected block WITHOUT caging phrasing → silent (axis separation proven).
- **B3:** fixture with same instruction in CLAUDE.md + rule → CNF finding; single-layer only →
silent ([[guard-can-be-green-on-its-own-defect]]: assert the blanket invariant).
### Chunk 1 — Register entries: model routing + effort (dogfoods `knowledge-refresh`)
## Key assumptions (test at implementation)
Add to `knowledge/best-practices.json` via the knowledge-refresh flow (human-approved write):
- **BP-MODEL-001** (`category: model-fit`): subagents doing mechanical/read-only work can pin a
cheaper model via `model:` frontmatter; orchestrator keeps the strong model. Source:
code.claude.com/docs/en/sub-agents → `confidence: confirmed`.
- **BP-MODEL-002** (`category: model-fit`): reasoning effort is tunable at five levels in five
places (settings `effortLevel`, `/effort`, `--effort`, per-agent `effort` frontmatter, SDK);
default `high`; higher effort is not universally better for simple tasks. Source:
code.claude.com/docs/en/model-config → `confidence: confirmed`.
Schema per `scanners/lib/best-practices-register.mjs:42-102` (id/claim/confidence/source.url/
source.verified required). This chunk doubles as the DEL B dogfood of `/config-audit
knowledge-refresh` (each chunk is also a plugin test).
### Chunk 2 — fix-engine effort hygiene (tiny, TDD)
`fix-engine.mjs:26` `VALID_EFFORT_LEVELS = ['low','medium','high','max']` — missing `xhigh`.
Consequence: nearest-match "fix" for a typo like `xhig` corrects to `high`, not `xhigh`.
Red test first: `findNearestEffortLevel('xhig') === 'xhigh'`. Align list with
`settings-validator.mjs:75`.
### Chunk 3 — CA-CML dead prose references (new deterministic check)
The strongest video-derived principle ("a prompt implying a file is present doesn't mean one
is") applied to CLAUDE.md quality: flag file paths mentioned in CLAUDE.md **prose** that do not
exist on disk. Today only `@import` targets are existence-checked; stale pointers like
`docs/foo.md` or `scripts/bar.sh` rot silently and burn always-loaded tokens on misdirection.
Conservative v1 to control false positives:
- Only backtick-quoted tokens that look like relative file paths (contain `/` or a known
extension), resolved against the CLAUDE.md's own directory.
- Skip URLs, globs (`*`), placeholders (`{...}`, `<...>`, `$VAR`, `${...}`), absolute and
`~/` paths (machine-specific), and paths under `.gitignore`d dirs if cheap to determine.
- Severity: low. New CA-CML-NNN (verify next free NNN at implementation — IDs are dynamic).
Byte-stability: follow [[adding-scanner-byte-stability]] steps for a new finding type in an
EXISTING scanner — frozen `tests/snapshots/v5.0.0/` must stay untouched; default-output
snapshots regenerate (`UPDATE_SNAPSHOT=1`) only if a fixture actually carries the new type;
humanizer step 7 (M-16/M-17 lessons): `TRANSLATIONS`-static entry for the new RAW title
(CML category mapping already exists).
### Chunk 4 — feature-gap + inventory: model/effort awareness
- New T3 opportunity check in `feature-gap-scanner.mjs`: authored agents
(`isAuthoredConfig`, M-BUG-13 lesson) where **no** agent sets `model:` or `effort:` →
"all agents inherit the session model/effort — mechanical agents can be routed cheaper /
effort-calibrated" citing BP-MODEL-001/002. Fires only when authored agents exist
(M-BUG-15 lesson: no enhancement-check on empty collections). Opportunity framing, never
failure — deliberate max-model setups are a valid choice; finding is suppressable
(`.config-audit-ignore`).
- `whats-active` / `manifest`: surface `model`/`effort` per agent in the inventory tables.
- Humanizer wiring step 7 for the new GAP finding; verify via direct `scan()` output, not the
self-suppressed default output ([[agent-commands-need-scanner-scoping]]).
- feat commit → docs-gate: README + CLAUDE.md diffs required.
### Chunk 5 — planner-agent adversarial gate (do AFTER DEL B pipeline dogfood)
Add a required "Failure modes" section to `agents/planner-agent.md`'s action-plan contract:
before an action plan is emitted, list what could go wrong per change + rollback trigger.
Mirrors the video's scoping-vs-devil's-advocate distinction; currently absent (zero
adversarial requirements in agents/). **Sequencing constraint:** DEL B step 3.2 judges
planner-agent against a fasit — change the agent only after that dogfood pass, or the
fasit target moves mid-evaluation.
## Explicitly rejected (do not revisit without new evidence)
1. **"Fable low ≈ Opus high" cost/score framing** — contradicted by Anthropic's own pages.
Never encode in register, copy, or recommendations.
2. **Tool-call-count effort scaling (1 / 3–5 / 5–10) as a register entry** — source is an
unconfirmed third-party leak → would be `confidence: inferred`, never surfaced. Not worth
carrying.
3. **Cost/intelligence/"taste" model-routing table generator** — subjective scores don't fit
the deterministic, provenance-gated design. BP-MODEL-001 covers the actionable core.
4. **"Fable mode" skill** — a user-level skill, not configuration auditing. Out of plugin scope.
5. **CLAUDE.md prose contradiction detection** — real gap (CA-CNF only covers
settings/permissions/hooks) but not video-driven; keep this plan surgical.
## Known tension (named, not resolved here)
The operator's own global policy is Opus/max-effort for ALL subagents, never Haiku — the
opposite of Chunk 4's recommendation. Both are legitimate: the docs-backed routing advice
optimizes cost at equal quality for the general user; the operator deliberately buys maximum
quality. Chunk 4's copy must respect that (opportunity framing + suppressability), and on this
machine the finding will simply be suppressed or ignored. The plugin serves general users;
the operator's setup is not the target of the check.
## Verification
Global, after every chunk:
- `node --test 'tests/**/*.test.mjs'` → green (baseline 1359/0; count grows with new tests)
- `git status --porcelain tests/snapshots/v5.0.0/` → empty (frozen untouched)
- TDD: red test exists and fails BEFORE each production change
Per chunk:
- **C1:** `node --test tests/lib/best-practices-register.test.mjs` green;
`node scanners/knowledge-refresh-cli.mjs` classifies BP-MODEL-001/002 as fresh
- **C2:** `findNearestEffortLevel('xhig')` → `xhigh` (red first); `grep xhigh scanners/fix-engine.mjs` non-empty
- **C3:** fixture CLAUDE.md referencing `docs/missing.md` → finding fires; existing file /
URL / glob / placeholder / `~/` path → silent; humanized output has non-contradictory copy
- **C4:** authored-agent fixture without model/effort → opportunity fires; with either set →
silent; zero authored agents → silent; `userImpactCategory` ≠ `Other` end-to-end via direct scan()
- **C5:** dogfood plan run produces a Failure-modes section; DEL B 3.2 fasit judged BEFORE the change
## Key assumptions (test before/at implementation)
1. **Per-agent `effort` frontmatter is official** — verified 2026-07-14 against
code.claude.com/docs/en/sub-agents + /model-config; re-fetch both pages at implementation
(docs move).
2. **New finding type in existing scanner leaves frozen snapshots untouched** — M-17 precedent
says yes when no v5.0.0 fixture carries the type; verify by running the suite and inspecting
which snapshots differ before committing.
3. **Next free CA-CML/CA-GAP finding numbers** — IDs are built dynamically; grep tests +
snapshots for the highest used NNN before assigning.
## Sequencing vs DEL B (one plan, no relitigation)
This plan does NOT preempt the active DEL B sequence. Recommended order:
1. DEL B step 3 pipeline dogfood (`analyze → plan → implement → rollback`) — unchanged, next.
2. Batch patch release M-11→M-17 — unchanged.
3. v5.13 chunks 1→5 (chunk 1 doubles as the `knowledge-refresh` dogfood already queued in
DEL B "Resten"; chunk 5 explicitly waits for step 3.2). Release as minor v5.13.0 via
`release-plugin.mjs` when all chunks land.
1. Per-agent `effort` frontmatter still official — re-fetch sub-agents + model-config pages.
2. Finding-type REMOVAL in an existing scanner leaves frozen v5.0.0 snapshots untouched only if
no frozen fixture carries the type — verify per D1/D2 before committing; re-seed consciously
if not.
3. Next free CA-CML/CA-GAP/CA-OPT NNN — grep tests + snapshots before assigning.
4. `/doctor`'s check set is version-fluid (2.1.205→220 changed it materially) — re-run the
overlap measurement cheaply (CLI + one in-session run) before executing D1–D3 if CC has
moved significantly past 2.1.220.