config-audit/docs/v5.13-model-routing-effort-deadref-plan.md

v5.13 Plan — Model Routing, Effort Awareness, Dead References

Derived from an external video analysis ("The Model Isn't the Moat", 2026-07) cross-checked against primary sources and against what config-audit already encodes. Every claim acted on here was verified against Anthropic's own docs; video-only claims are explicitly rejected below.

Source verification (done 2026-07-14)

Claim from video	Verdict	Source
Orchestrator + cheaper worker models is a supported, recommended pattern	VERIFIED	code.claude.com/docs/en/sub-agents ("Control costs by routing tasks to faster, cheaper models like Haiku"), code.claude.com/docs/en/workflows
Reasoning effort is tunable per settings / session / launch / per-agent frontmatter / SDK; levels low, medium, high, xhigh, max	VERIFIED	code.claude.com/docs/en/model-config#adjust-effort-level, sub-agents doc
Leaked Fable 5 system prompt principles ("partial recognition ≠ current knowledge"; "a prompt implying a file is present doesn't mean one is"; answer-first-then-one-question; tool-call scaling 1 / 3–5 / 5–10)	VERIFIED near-verbatim, provenance unconfirmed (third-party leak repo, not Anthropic-confirmed)	github.com/asgeirtj/system_prompts_leaks Anthropic/claude-fable-5.md
"Fable 5 on low ≈ Opus 4.8 on high, slightly higher cost/quality" score-vs-cost chart	CONTRADICTED — no such chart/statement on Anthropic's pages; GPT-5.5 appears only in a testimonial	anthropic.com/news/claude-fable-5-mythos-5

Already covered — no action

Video idea	Existing coverage
"Process is the moat" (config/harness > raw model)	The plugin's entire thesis
Extract repeated procedure into a skill	BP-MECH-003 + CA-OPT-001 (optimization-lens-scanner.mjs:121)
CLAUDE.md size/ownership discipline	BP-SIZE-001 + CA-CML line/size checks (claude-md-linter.mjs:109/:120/:140)
Check that referenced imports exist	CA-IMP broken @import (lib/import-resolver.mjs:88) — but only @import, see Chunk 3
Plugin's own agents are model-routed	Agents table already pins sonnet for mechanical, opus for judgment

Gaps → chunks

Verified gap summary (register-mapper sweep, 2026-07-14): no scanner audits per-agent model:/effort: frontmatter; effort has validity-check only (settings-validator.mjs:195), no recommendation; no dead-reference check for prose file mentions in CLAUDE.md; no adversarial/failure-mode requirement in planner-agent; fix-engine.mjs:26 effort list omits xhigh (settings-validator has all five).

Chunk 1 — Register entries: model routing + effort (dogfoods knowledge-refresh)

Add to knowledge/best-practices.json via the knowledge-refresh flow (human-approved write):

• BP-MODEL-001 (category: model-fit): subagents doing mechanical/read-only work can pin a cheaper model via model: frontmatter; orchestrator keeps the strong model. Source: code.claude.com/docs/en/sub-agents → confidence: confirmed.
• BP-MODEL-002 (category: model-fit): reasoning effort is tunable at five levels in five places (settings effortLevel, /effort, --effort, per-agent effort frontmatter, SDK); default high; higher effort is not universally better for simple tasks. Source: code.claude.com/docs/en/model-config → confidence: confirmed.

Schema per scanners/lib/best-practices-register.mjs:42-102 (id/claim/confidence/source.url/ source.verified required). This chunk doubles as the DEL B dogfood of /config-audit knowledge-refresh (each chunk is also a plugin test).

Chunk 2 — fix-engine effort hygiene (tiny, TDD)

fix-engine.mjs:26 VALID_EFFORT_LEVELS = ['low','medium','high','max'] — missing xhigh. Consequence: nearest-match "fix" for a typo like xhig corrects to high, not xhigh. Red test first: findNearestEffortLevel('xhig') === 'xhigh'. Align list with settings-validator.mjs:75.

Chunk 3 — CA-CML dead prose references (new deterministic check)

The strongest video-derived principle ("a prompt implying a file is present doesn't mean one is") applied to CLAUDE.md quality: flag file paths mentioned in CLAUDE.md prose that do not exist on disk. Today only @import targets are existence-checked; stale pointers like docs/foo.md or scripts/bar.sh rot silently and burn always-loaded tokens on misdirection.

Conservative v1 to control false positives:

• Only backtick-quoted tokens that look like relative file paths (contain / or a known extension), resolved against the CLAUDE.md's own directory.
• Skip URLs, globs (*), placeholders ({...}, <...>, $VAR, ${...}), absolute and ~/ paths (machine-specific), and paths under .gitignored dirs if cheap to determine.
• Severity: low. New CA-CML-NNN (verify next free NNN at implementation — IDs are dynamic).

Byte-stability: follow adding-scanner-byte-stability steps for a new finding type in an EXISTING scanner — frozen tests/snapshots/v5.0.0/ must stay untouched; default-output snapshots regenerate (UPDATE_SNAPSHOT=1) only if a fixture actually carries the new type; humanizer step 7 (M-16/M-17 lessons): TRANSLATIONS-static entry for the new RAW title (CML category mapping already exists).

Chunk 4 — feature-gap + inventory: model/effort awareness

• New T3 opportunity check in feature-gap-scanner.mjs: authored agents (isAuthoredConfig, M-BUG-13 lesson) where no agent sets model: or effort: → "all agents inherit the session model/effort — mechanical agents can be routed cheaper / effort-calibrated" citing BP-MODEL-001/002. Fires only when authored agents exist (M-BUG-15 lesson: no enhancement-check on empty collections). Opportunity framing, never failure — deliberate max-model setups are a valid choice; finding is suppressable (.config-audit-ignore).
• whats-active / manifest: surface model/effort per agent in the inventory tables.
• Humanizer wiring step 7 for the new GAP finding; verify via direct scan() output, not the self-suppressed default output (agent-commands-need-scanner-scoping).
• feat commit → docs-gate: README + CLAUDE.md diffs required.

Chunk 5 — planner-agent adversarial gate (do AFTER DEL B pipeline dogfood)

Add a required "Failure modes" section to agents/planner-agent.md's action-plan contract: before an action plan is emitted, list what could go wrong per change + rollback trigger. Mirrors the video's scoping-vs-devil's-advocate distinction; currently absent (zero adversarial requirements in agents/). Sequencing constraint: DEL B step 3.2 judges planner-agent against a fasit — change the agent only after that dogfood pass, or the fasit target moves mid-evaluation.

Explicitly rejected (do not revisit without new evidence)

1. "Fable low ≈ Opus high" cost/score framing — contradicted by Anthropic's own pages. Never encode in register, copy, or recommendations.
2. Tool-call-count effort scaling (1 / 3–5 / 5–10) as a register entry — source is an unconfirmed third-party leak → would be confidence: inferred, never surfaced. Not worth carrying.
3. Cost/intelligence/"taste" model-routing table generator — subjective scores don't fit the deterministic, provenance-gated design. BP-MODEL-001 covers the actionable core.
4. "Fable mode" skill — a user-level skill, not configuration auditing. Out of plugin scope.
5. CLAUDE.md prose contradiction detection — real gap (CA-CNF only covers settings/permissions/hooks) but not video-driven; keep this plan surgical.

Known tension (named, not resolved here)

The operator's own global policy is Opus/max-effort for ALL subagents, never Haiku — the opposite of Chunk 4's recommendation. Both are legitimate: the docs-backed routing advice optimizes cost at equal quality for the general user; the operator deliberately buys maximum quality. Chunk 4's copy must respect that (opportunity framing + suppressability), and on this machine the finding will simply be suppressed or ignored. The plugin serves general users; the operator's setup is not the target of the check.

Verification

Global, after every chunk:

• node --test 'tests/**/*.test.mjs' → green (baseline 1359/0; count grows with new tests)
• git status --porcelain tests/snapshots/v5.0.0/ → empty (frozen untouched)
• TDD: red test exists and fails BEFORE each production change

Per chunk:

• C1: node --test tests/lib/best-practices-register.test.mjs green; node scanners/knowledge-refresh-cli.mjs classifies BP-MODEL-001/002 as fresh
• C2: findNearestEffortLevel('xhig') → xhigh (red first); grep xhigh scanners/fix-engine.mjs non-empty
• C3: fixture CLAUDE.md referencing docs/missing.md → finding fires; existing file / URL / glob / placeholder / ~/ path → silent; humanized output has non-contradictory copy
• C4: authored-agent fixture without model/effort → opportunity fires; with either set → silent; zero authored agents → silent; userImpactCategory ≠ Other end-to-end via direct scan()
• C5: dogfood plan run produces a Failure-modes section; DEL B 3.2 fasit judged BEFORE the change

Key assumptions (test before/at implementation)

1. Per-agent effort frontmatter is official — verified 2026-07-14 against code.claude.com/docs/en/sub-agents + /model-config; re-fetch both pages at implementation (docs move).
2. New finding type in existing scanner leaves frozen snapshots untouched — M-17 precedent says yes when no v5.0.0 fixture carries the type; verify by running the suite and inspecting which snapshots differ before committing.
3. Next free CA-CML/CA-GAP finding numbers — IDs are built dynamically; grep tests + snapshots for the highest used NNN before assigning.

Sequencing vs DEL B (one plan, no relitigation)

This plan does NOT preempt the active DEL B sequence. Recommended order:

1. DEL B step 3 pipeline dogfood (analyze → plan → implement → rollback) — unchanged, next.
2. Batch patch release M-11→M-17 — unchanged.
3. v5.13 chunks 1→5 (chunk 1 doubles as the knowledge-refresh dogfood already queued in DEL B "Resten"; chunk 5 explicitly waits for step 3.2). Release as minor v5.13.0 via release-plugin.mjs when all chunks land.