config-audit/docs/v5.13-model-routing-effort-deadref-plan.md

# v5.13 Plan — Model Routing, Effort Awareness, Dead References

Derived from an external video analysis ("The Model Isn't the Moat", 2026-07) cross-checked
against primary sources and against what config-audit already encodes. Every claim acted on
here was verified against Anthropic's own docs; video-only claims are explicitly rejected below.

## Source verification (done 2026-07-14)

| Claim from video | Verdict | Source |
|---|---|---|
| Orchestrator + cheaper worker models is a supported, recommended pattern | VERIFIED | code.claude.com/docs/en/sub-agents ("Control costs by routing tasks to faster, cheaper models like Haiku"), code.claude.com/docs/en/workflows |
| Reasoning effort is tunable per settings / session / launch / **per-agent frontmatter** / SDK; levels `low, medium, high, xhigh, max` | VERIFIED | code.claude.com/docs/en/model-config#adjust-effort-level, sub-agents doc |
| Leaked Fable 5 system prompt principles ("partial recognition ≠ current knowledge"; "a prompt implying a file is present doesn't mean one is"; answer-first-then-one-question; tool-call scaling 1 / 3–5 / 5–10) | VERIFIED near-verbatim, **provenance unconfirmed** (third-party leak repo, not Anthropic-confirmed) | github.com/asgeirtj/system_prompts_leaks `Anthropic/claude-fable-5.md` |
| "Fable 5 on low ≈ Opus 4.8 on high, slightly higher cost/quality" score-vs-cost chart | **CONTRADICTED** — no such chart/statement on Anthropic's pages; GPT-5.5 appears only in a testimonial | anthropic.com/news/claude-fable-5-mythos-5 |

## Already covered — no action

| Video idea | Existing coverage |
|---|---|
| "Process is the moat" (config/harness > raw model) | The plugin's entire thesis |
| Extract repeated procedure into a skill | BP-MECH-003 + CA-OPT-001 (`optimization-lens-scanner.mjs:121`) |
| CLAUDE.md size/ownership discipline | BP-SIZE-001 + CA-CML line/size checks (`claude-md-linter.mjs:109/:120/:140`) |
| Check that referenced imports exist | CA-IMP broken `@import` (`lib/import-resolver.mjs:88`) — but **only** `@import`, see Chunk 3 |
| Plugin's own agents are model-routed | Agents table already pins sonnet for mechanical, opus for judgment |

## Gaps → chunks

Verified gap summary (register-mapper sweep, 2026-07-14): no scanner audits per-agent
`model:`/`effort:` frontmatter; effort has validity-check only (`settings-validator.mjs:195`),
no recommendation; no dead-reference check for prose file mentions in CLAUDE.md; no
adversarial/failure-mode requirement in planner-agent; `fix-engine.mjs:26` effort list
omits `xhigh` (settings-validator has all five).

### Chunk 1 — Register entries: model routing + effort (dogfoods `knowledge-refresh`)

Add to `knowledge/best-practices.json` via the knowledge-refresh flow (human-approved write):

- **BP-MODEL-001** (`category: model-fit`): subagents doing mechanical/read-only work can pin a
  cheaper model via `model:` frontmatter; orchestrator keeps the strong model. Source:
  code.claude.com/docs/en/sub-agents → `confidence: confirmed`.
- **BP-MODEL-002** (`category: model-fit`): reasoning effort is tunable at five levels in five
  places (settings `effortLevel`, `/effort`, `--effort`, per-agent `effort` frontmatter, SDK);
  default `high`; higher effort is not universally better for simple tasks. Source:
  code.claude.com/docs/en/model-config → `confidence: confirmed`.

Schema per `scanners/lib/best-practices-register.mjs:42-102` (id/claim/confidence/source.url/
source.verified required). This chunk doubles as the DEL B dogfood of `/config-audit
knowledge-refresh` (each chunk is also a plugin test).

### Chunk 2 — fix-engine effort hygiene (tiny, TDD)

`fix-engine.mjs:26` `VALID_EFFORT_LEVELS = ['low','medium','high','max']` — missing `xhigh`.
Consequence: nearest-match "fix" for a typo like `xhig` corrects to `high`, not `xhigh`.
Red test first: `findNearestEffortLevel('xhig') === 'xhigh'`. Align list with
`settings-validator.mjs:75`.

### Chunk 3 — CA-CML dead prose references (new deterministic check)

The strongest video-derived principle ("a prompt implying a file is present doesn't mean one
is") applied to CLAUDE.md quality: flag file paths mentioned in CLAUDE.md **prose** that do not
exist on disk. Today only `@import` targets are existence-checked; stale pointers like
`docs/foo.md` or `scripts/bar.sh` rot silently and burn always-loaded tokens on misdirection.

Conservative v1 to control false positives:
- Only backtick-quoted tokens that look like relative file paths (contain `/` or a known
  extension), resolved against the CLAUDE.md's own directory.
- Skip URLs, globs (`*`), placeholders (`{...}`, `<...>`, `$VAR`, `${...}`), absolute and
  `~/` paths (machine-specific), and paths under `.gitignore`d dirs if cheap to determine.
- Severity: low. New CA-CML-NNN (verify next free NNN at implementation — IDs are dynamic).

Byte-stability: follow [[adding-scanner-byte-stability]] steps for a new finding type in an
EXISTING scanner — frozen `tests/snapshots/v5.0.0/` must stay untouched; default-output
snapshots regenerate (`UPDATE_SNAPSHOT=1`) only if a fixture actually carries the new type;
humanizer step 7 (M-16/M-17 lessons): `TRANSLATIONS`-static entry for the new RAW title
(CML category mapping already exists).

### Chunk 4 — feature-gap + inventory: model/effort awareness

- New T3 opportunity check in `feature-gap-scanner.mjs`: authored agents
  (`isAuthoredConfig`, M-BUG-13 lesson) where **no** agent sets `model:` or `effort:` →
  "all agents inherit the session model/effort — mechanical agents can be routed cheaper /
  effort-calibrated" citing BP-MODEL-001/002. Fires only when authored agents exist
  (M-BUG-15 lesson: no enhancement-check on empty collections). Opportunity framing, never
  failure — deliberate max-model setups are a valid choice; finding is suppressable
  (`.config-audit-ignore`).
- `whats-active` / `manifest`: surface `model`/`effort` per agent in the inventory tables.
- Humanizer wiring step 7 for the new GAP finding; verify via direct `scan()` output, not the
  self-suppressed default output ([[agent-commands-need-scanner-scoping]]).
- feat commit → docs-gate: README + CLAUDE.md diffs required.

### Chunk 5 — planner-agent adversarial gate (do AFTER DEL B pipeline dogfood)

Add a required "Failure modes" section to `agents/planner-agent.md`'s action-plan contract:
before an action plan is emitted, list what could go wrong per change + rollback trigger.
Mirrors the video's scoping-vs-devil's-advocate distinction; currently absent (zero
adversarial requirements in agents/). **Sequencing constraint:** DEL B step 3.2 judges
planner-agent against a fasit — change the agent only after that dogfood pass, or the
fasit target moves mid-evaluation.

## Explicitly rejected (do not revisit without new evidence)

1. **"Fable low ≈ Opus high" cost/score framing** — contradicted by Anthropic's own pages.
   Never encode in register, copy, or recommendations.
2. **Tool-call-count effort scaling (1 / 3–5 / 5–10) as a register entry** — source is an
   unconfirmed third-party leak → would be `confidence: inferred`, never surfaced. Not worth
   carrying.
3. **Cost/intelligence/"taste" model-routing table generator** — subjective scores don't fit
   the deterministic, provenance-gated design. BP-MODEL-001 covers the actionable core.
4. **"Fable mode" skill** — a user-level skill, not configuration auditing. Out of plugin scope.
5. **CLAUDE.md prose contradiction detection** — real gap (CA-CNF only covers
   settings/permissions/hooks) but not video-driven; keep this plan surgical.

## Known tension (named, not resolved here)

The operator's own global policy is Opus/max-effort for ALL subagents, never Haiku — the
opposite of Chunk 4's recommendation. Both are legitimate: the docs-backed routing advice
optimizes cost at equal quality for the general user; the operator deliberately buys maximum
quality. Chunk 4's copy must respect that (opportunity framing + suppressability), and on this
machine the finding will simply be suppressed or ignored. The plugin serves general users;
the operator's setup is not the target of the check.

## Verification

Global, after every chunk:
- `node --test 'tests/**/*.test.mjs'` → green (baseline 1359/0; count grows with new tests)
- `git status --porcelain tests/snapshots/v5.0.0/` → empty (frozen untouched)
- TDD: red test exists and fails BEFORE each production change

Per chunk:
- **C1:** `node --test tests/lib/best-practices-register.test.mjs` green;
  `node scanners/knowledge-refresh-cli.mjs` classifies BP-MODEL-001/002 as fresh
- **C2:** `findNearestEffortLevel('xhig')` → `xhigh` (red first); `grep xhigh scanners/fix-engine.mjs` non-empty
- **C3:** fixture CLAUDE.md referencing `docs/missing.md` → finding fires; existing file /
  URL / glob / placeholder / `~/` path → silent; humanized output has non-contradictory copy
- **C4:** authored-agent fixture without model/effort → opportunity fires; with either set →
  silent; zero authored agents → silent; `userImpactCategory` ≠ `Other` end-to-end via direct scan()
- **C5:** dogfood plan run produces a Failure-modes section; DEL B 3.2 fasit judged BEFORE the change

## Key assumptions (test before/at implementation)

1. **Per-agent `effort` frontmatter is official** — verified 2026-07-14 against
   code.claude.com/docs/en/sub-agents + /model-config; re-fetch both pages at implementation
   (docs move).
2. **New finding type in existing scanner leaves frozen snapshots untouched** — M-17 precedent
   says yes when no v5.0.0 fixture carries the type; verify by running the suite and inspecting
   which snapshots differ before committing.
3. **Next free CA-CML/CA-GAP finding numbers** — IDs are built dynamically; grep tests +
   snapshots for the highest used NNN before assigning.

## Sequencing vs DEL B (one plan, no relitigation)

This plan does NOT preempt the active DEL B sequence. Recommended order:
1. DEL B step 3 pipeline dogfood (`analyze → plan → implement → rollback`) — unchanged, next.
2. Batch patch release M-11→M-17 — unchanged.
3. v5.13 chunks 1→5 (chunk 1 doubles as the `knowledge-refresh` dogfood already queued in
   DEL B "Resten"; chunk 5 explicitly waits for step 3.2). Release as minor v5.13.0 via
   `release-plugin.mjs` when all chunks land.