docs(plan): v5.5+ steering-model coverage plan (A–E)
Doc-grounded plan for the five new-functionality items greenlit after checking the "seven steering mechanisms" framing against live CC docs (2026-06-19): A durability/compaction findings, B load-pattern token accounting, C output-style scanner, D mechanism-fit heuristic, E agent-listing cost + plugin-agent dead-config. Includes a full verification log (CONFIRMED/REFUTED/UNVERIFIED per claim + source), dependency graph, phased rollout (v5.5/v5.6/v5.7), testable acceptance criteria, and open decisions. PLAN only — each feature awaits its own GO. No code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
This commit is contained in:
parent
b6a62d7699
commit
e71e60f14e
1 changed files with 240 additions and 0 deletions
240
docs/v5.5-steering-model-plan.md
Normal file
240
docs/v5.5-steering-model-plan.md
Normal file
|
|
@ -0,0 +1,240 @@
|
|||
# v5.5+ Plan — Steering-Model Coverage
|
||||
|
||||
_Status: PLAN (awaiting GO per feature). Created 2026-06-20. Plans live next to the work (continuity rule)._
|
||||
|
||||
## Why this plan exists
|
||||
|
||||
A "seven ways to steer Claude Code" framing (CLAUDE.md / rules / skills / sub-agents /
|
||||
hooks / output styles / mechanism-fit) was used as a **check-against** reference — not a
|
||||
spec. Checking it against the live docs (`code.claude.com/docs`, map updated 2026-06-19)
|
||||
both corrected the video **and** surfaced gaps in config-audit's own coverage.
|
||||
|
||||
The unifying insight: every steering mechanism has a different **loading model** and
|
||||
**compaction-survival** profile, and the docs now publish both explicitly. config-audit
|
||||
already audits *structure* well; it does **not** yet model *when a thing is loaded* or
|
||||
*whether it survives compaction* — which is exactly the "wrong mechanism = wasted tokens /
|
||||
unloaded instruction" axis. This plan closes that.
|
||||
|
||||
The small correctness items the check surfaced (HKV hook-events, RUL globs wording) are
|
||||
**already landed** on `main` (`b6a62d7`) and bound for the pending **v5.4.1** patch. This
|
||||
document covers only the **new functionality** (v5.5+), greenlit by the operator: A, B, C,
|
||||
D, E.
|
||||
|
||||
---
|
||||
|
||||
## Verification log (Verifiseringsplikt)
|
||||
|
||||
Every claim this plan builds on, with source and status. `CONFIRMED` = stated in primary
|
||||
docs; `REFUTED` = docs contradict; `UNVERIFIED` = not found in primary docs (do not assert).
|
||||
|
||||
| # | Claim | Status | Source |
|
||||
|---|-------|--------|--------|
|
||||
| V1 | Project-root CLAUDE.md + unscoped rules are **re-injected from disk after compaction** | CONFIRMED | `context-window.md#what-survives-compaction` |
|
||||
| V2 | Rules with `paths:` frontmatter are **lost after compaction** until a matching file is read again | CONFIRMED | `context-window.md#what-survives-compaction` |
|
||||
| V3 | Nested (subdir) CLAUDE.md is **lost after compaction** until a file in that dir is read again | CONFIRMED | `context-window.md`, `memory.md` |
|
||||
| V4 | Path-scoped rules trigger on **Read** of a matching file (not every tool use) | CONFIRMED | `memory.md#path-specific-rules` |
|
||||
| V5 | The only documented rule-scoping frontmatter field is **`paths`** (not `globs`) | CONFIRMED | `memory.md#path-specific-rules` |
|
||||
| V6 | `.claude/rules/` is an **official** feature; all `.md` auto-discovered recursively; unscoped = always-on | CONFIRMED | `memory.md#organize-rules-with-claude/rules/` |
|
||||
| V7 | Skill **name+description load every turn**; body loads on invoke | CONFIRMED | `skills.md`, `features-overview.md` |
|
||||
| V8 | Skill listing description cap = **1,536 chars** default of `maxSkillDescriptionChars` (configurable, v2.1.105+) | CONFIRMED | `skills.md`, `settings.md` |
|
||||
| V9 | Output styles **still exist** (not deprecated); only the standalone `/output-style` command was removed (v2.1.91) → use `/config` | CONFIRMED | `output-styles.md`, changelog |
|
||||
| V10 | Output styles **modify the system prompt** (add to end); a **custom** style without `keep-coding-instructions: true` **removes built-in software-engineering instructions** | CONFIRMED | `output-styles.md` |
|
||||
| V11 | `force-for-plugin: true` auto-applies a plugin's style, **overriding the user's `outputStyle`** | CONFIRMED | `output-styles.md` |
|
||||
| V12 | CLAUDE.md is "**a user message after the system prompt**", whereas output styles are a system-prompt mechanism | CONFIRMED | `output-styles.md` (comparison table) |
|
||||
| V13 | Output styles are "the most expensive" steering mechanism | **UNVERIFIED** — docs say cost rises but prompt-cache mitigates; no ranking | `output-styles.md` |
|
||||
| V14 | Sub-agent runs in isolated context; only a summary returns | CONFIRMED | `sub-agents.md` |
|
||||
| V15 | For **plugin** subagents, `hooks` / `mcpServers` / `permissionMode` frontmatter are **silently ignored** | CONFIRMED | `sub-agents.md` |
|
||||
| V16 | Agent `skills:` frontmatter **preloads full skill content at startup** | CONFIRMED | `sub-agents.md` |
|
||||
| V17 | Required agent frontmatter = **only `name` + `description`** | CONFIRMED | `sub-agents.md` |
|
||||
| V18 | Hooks: **~30 events** (not "five"); hook scripts run outside context, but injected `additionalContext` IS saved to transcript (subject to compaction) | CONFIRMED | `hooks.md` |
|
||||
| V19 | An official **mechanism-fit comparison table** exists (output style vs CLAUDE.md vs `--append-system-prompt` vs agents vs skills) | CONFIRMED | `output-styles.md` |
|
||||
| U1 | hook event `post-session` (kebab) | **UNVERIFIED** — config-audit's `VALID_EVENTS` lists it (2.1.169 comment) but not found in current `hooks.md`; a test depends on it. Follow-up check before v5.5. | — |
|
||||
| U2 | A plugin.json `outputStyles` path-override key (PLH `SHADOWING_PATH_FIELDS`) | **CONFIRM** — pinned to path-behavior docs in PLH; one research aside disputed it. Re-confirm against `plugin-reference` before touching. See `[[plugin-json-path-behavior]]`. | — |
|
||||
|
||||
---
|
||||
|
||||
## The five features
|
||||
|
||||
Design constraints throughout: deterministic where possible; additive findings that do
|
||||
**not** fire on the SC-5 snapshot fixture stay byte-stable (the v5.3/v5.4 pattern); every
|
||||
new finding gets a stable title (fix-engine + humanizer couple on title — see `b6a62d7`).
|
||||
|
||||
### Foundation (prerequisite for B/A/C/E): enumeration
|
||||
|
||||
`active-config-reader.mjs` today enumerates CLAUDE.md cascade, hooks, MCP, skills, plugins —
|
||||
but **not** rules, agents, or output styles (confirmed by the coverage scan). B, and parts
|
||||
of A/C/E, need these enumerated with a `loadPattern` tag. This is the first build step.
|
||||
|
||||
- Add enumeration for: `.claude/rules/*.md` (+ `~/.claude/rules/`), agents (`.claude/agents/`,
|
||||
`~/.claude/agents/`, plugin agents), output styles (`.claude/output-styles/`,
|
||||
`~/.claude/output-styles/`).
|
||||
- Tag each source kind with `loadPattern ∈ { always, on-demand, external }` and
|
||||
`survivesCompaction ∈ { yes, no, n/a }` derived from V1–V3, V6, V7, V18.
|
||||
- **Test:** fixture with root+subdir CLAUDE.md, scoped+unscoped rule, project agent, output
|
||||
style → assert each is enumerated with the correct `loadPattern`/`survivesCompaction`.
|
||||
|
||||
### A — Durability / compaction-survival findings
|
||||
|
||||
**Problem.** A must-always-hold instruction placed where it does **not** survive compaction
|
||||
(nested CLAUDE.md per V3, path-scoped rule per V2) silently disappears mid-session after a
|
||||
`/compact`. The docs publish exactly which mechanisms survive (V1–V3) — so this is
|
||||
doc-grounded, not a guess.
|
||||
|
||||
**Shape (additive, no new scanner — count stays 13).**
|
||||
- **RUL** new finding `CA-RUL-NNN` — a **large** path-scoped rule (e.g. > 50 lines, reuse the
|
||||
existing unscoped-size heuristic) carries an informational note: its content is **lost after
|
||||
compaction** until a matching file is re-read (V2/V4). Severity **low** (awareness, not a bug).
|
||||
- **CML** new finding `CA-CML-NNN` — a **nested** (non-root) CLAUDE.md of meaningful size
|
||||
notes it is **not re-injected after compaction** (V3). Severity **low**.
|
||||
- Keep it strictly **structural** (size + location), never semantic ("is this instruction
|
||||
critical?") — that stays deterministic.
|
||||
|
||||
**Key assumption + test.** Discovery distinguishes root vs nested CLAUDE.md and scoped vs
|
||||
unscoped rules. → Test with a fixture asserting the finding fires for nested/scoped+large and
|
||||
**not** for root/unscoped.
|
||||
|
||||
**Byte-stability.** Additive; snapshot fixture has no nested CLAUDE.md / large scoped rule →
|
||||
SC-5 byte-stable (re-verify).
|
||||
|
||||
### B — Load-pattern accounting in `tokens` + `manifest`
|
||||
|
||||
**Problem.** `manifest.mjs` and TOK rank all sources uniformly by `estimated_tokens`. A skill
|
||||
listing entry (paid **every turn**, V7) ranks identically to a skill body (paid **on invoke**).
|
||||
The docs give a precise per-mechanism loading + survival model (V1–V3, V7, V18) that we can
|
||||
surface. This is the **core** of the "wrong mechanism = wasted tokens" thesis.
|
||||
|
||||
**Shape.**
|
||||
- Add a `loadPattern` (and `survivesCompaction`) column to `manifest` output and to TOK
|
||||
source records, derived from the foundation enumeration.
|
||||
- Add an **always-loaded subtotal** to the manifest summary: "≈X tokens enter context every
|
||||
turn before you type" (project-root CLAUDE.md + unscoped rules + skill/agent listing +
|
||||
output style + MCP schemas), vs an on-demand subtotal.
|
||||
- Optional follow-up: cost-benefit hint (a large `always` source dwarfing the on-demand pool).
|
||||
|
||||
**Key assumption + test.** Source-kind → `loadPattern` is a deterministic mapping. → Test:
|
||||
manifest on a mixed fixture yields the correct always-subtotal; `--json` includes the new field.
|
||||
|
||||
**Byte-stability — RISK.** B changes the **manifest/tokens output format** → **not**
|
||||
byte-stable. Requires snapshot regen (SC-5) and a decision on `--json` back-compat (add field
|
||||
vs version the schema). This is the highest-format-risk item; sequence it deliberately.
|
||||
|
||||
### C — Output-style scanner (new `CA-OST`, count 13 → 14)
|
||||
|
||||
**Problem.** Output styles are live (V9) and the **most surprising** surface: a custom style
|
||||
silently strips built-in software-engineering instructions (V10), and `force-for-plugin`
|
||||
overrides the user's choice (V11). config-audit does not scan them at all today (GAP/SET/PLH
|
||||
only touch adoption / a settings key / plugin folder-shadow).
|
||||
|
||||
**Shape (new scanner — output styles are a genuinely new file surface, so a new scanner is
|
||||
warranted, unlike the additive v5.3/v5.4 work).**
|
||||
- `CA-OST-001` — **custom** output style missing `keep-coding-instructions: true` → flags that
|
||||
built-in SWE instructions are removed (V10). Severity **medium** (silently changes coding
|
||||
behavior). The headline footgun.
|
||||
- `CA-OST-002` — `force-for-plugin: true` in a project/user style → it overrides the user's
|
||||
`outputStyle` (V11). Severity **low**.
|
||||
- `CA-OST-003` — settings `outputStyle` value resolving to a non-existent style → dead config.
|
||||
Severity **low/medium**. (Could live in SET instead; decide during build.)
|
||||
- Active output-style token cost → feed B's `always` accounting (V10: system-prompt, every turn).
|
||||
- **Not** asserting V13 ("most expensive") anywhere — unverified.
|
||||
|
||||
**Key assumption + test.** Discovery can find `.claude/output-styles/*.md` and
|
||||
`~/.claude/output-styles/*.md` (new discovery type). → Test: fixture with a custom style
|
||||
missing the flag fires `CA-OST-001`; a built-in style reference does not.
|
||||
|
||||
**Byte-stability.** New scanner; fixture-gated → SC-5 byte-stable if it does not fire on the
|
||||
snapshot project (re-verify). Self-audit scanner-count badge moves 13 → 14 (update README +
|
||||
CLAUDE.md inventory + the "count stays 13" lore).
|
||||
|
||||
### D — Mechanism-fit detector (heuristic — lowest precision, sequence last)
|
||||
|
||||
**Problem.** The docs publish a mechanism-fit table (V19) and the architectural distinction
|
||||
that CLAUDE.md is a per-turn user message while output styles are system-prompt (V12). Common
|
||||
mismatch: an "every time / before each" instruction in CLAUDE.md that should be a **hook**; a
|
||||
path-specific instruction in root CLAUDE.md that should be a **path-scoped rule** (V4/V6).
|
||||
|
||||
**Shape (heuristic; additive to CML or a small new check).**
|
||||
- `CA-???-001` — imperative lifecycle phrasing in CLAUDE.md (`every time`, `before each`,
|
||||
`always run`, `after you …`, `whenever you …`) describing a tool/lifecycle action → suggest
|
||||
a hook. Framed as **Missed opportunity**, severity **low**, never "Fix this now".
|
||||
- `CA-???-002` — path-specific phrasing in **root** CLAUDE.md (`in src/**`, `for *.ts files`)
|
||||
→ suggest a path-scoped rule.
|
||||
|
||||
**Key assumption + test — THE RISK.** Phrase heuristics must have low false-positive rate. →
|
||||
Test with a positive corpus (clear mismatches → flagged) **and** a negative corpus (normal
|
||||
project conventions → silent). Must be suppressible (`.config-audit-ignore`). If precision is
|
||||
poor in testing, **ship behind a flag or defer** — do not ship a noisy heuristic.
|
||||
|
||||
**Byte-stability.** Additive; risk it fires on the snapshot's CLAUDE.md → check carefully,
|
||||
may need snapshot regen.
|
||||
|
||||
### E — Agent-listing cost + plugin-agent dead-config (additive to PLH; feeds B)
|
||||
|
||||
**Problem.** Agents' name+description load for delegation (every turn), like skills — but
|
||||
unlike skills they're unmetered. And `hooks`/`mcpServers`/`permissionMode` in a **plugin**
|
||||
agent's frontmatter are silently ignored (V15) — dead config, and a false sense of
|
||||
isolation/security if `permissionMode` is among them.
|
||||
|
||||
**Shape (additive to PLH).**
|
||||
- `CA-PLH-NNN` — plugin agent declaring `hooks` / `mcpServers` / `permissionMode` → dead config
|
||||
(V15). Severity **low** generally; **medium** for `permissionMode` (false security).
|
||||
- Agent description cost → feed B's `always` accounting. **No hard cap claimed** — research
|
||||
found no documented agent-description limit (unlike skills' 1,536, V8). Treat as cost signal,
|
||||
not a limit.
|
||||
- `skills:` preload in agent frontmatter → informational note: full skill content injected at
|
||||
startup (V16).
|
||||
|
||||
**Key assumption + test.** PLH can tell a **plugin** agent from a user/project agent (V15
|
||||
only applies to plugin subagents). → Test: plugin-agent fixture with `permissionMode` fires
|
||||
medium dead-config; identical user-level agent does **not**.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & phased rollout
|
||||
|
||||
```
|
||||
Foundation (enumeration: rules, agents, output styles + loadPattern/survivesCompaction)
|
||||
├── A (durability) additive RUL+CML doc-grounded, low risk
|
||||
├── E (plugin-agent dead) additive PLH doc-grounded, low risk
|
||||
├── C (output-style) new CA-OST (13→14) doc-grounded, new surface
|
||||
└── B (load-pattern) manifest/tokens format doc-grounded, FORMAT risk
|
||||
D (mechanism-fit) heuristic independent, precision risk
|
||||
```
|
||||
|
||||
**Recommended phasing** (chunk-work rule — session-sized, checkpoint STATE between):
|
||||
|
||||
- **v5.5.0 "steering-model I"** — Foundation + A + E. Additive, doc-grounded, byte-stable,
|
||||
count stays 13. Lowest risk, ships the compaction-survival insight.
|
||||
- **v5.6.0 "steering-model II"** — C (new `CA-OST`, count → 14) + B (load-pattern accounting,
|
||||
manifest format change + snapshot regen + `--json` back-compat decision).
|
||||
- **v5.7.0 (optional)** — D, only if the negative-corpus test shows acceptable precision;
|
||||
otherwise behind a flag or dropped.
|
||||
|
||||
(Operator may prefer a single larger v5.5 — but B's format change and D's precision risk argue
|
||||
for separating them from the low-risk additive batch.)
|
||||
|
||||
---
|
||||
|
||||
## Acceptance criteria (whole programme)
|
||||
|
||||
- [ ] Every new finding's primary claim traces to a `CONFIRMED` row above (no `UNVERIFIED`
|
||||
assertions in user-facing text — V13 and U1/U2 must be resolved or omitted).
|
||||
- [ ] `node --test 'tests/**/*.test.mjs'` green; README test badge == suite count
|
||||
(`self-audit --check-readme` `passed: true`).
|
||||
- [ ] `self-audit` stays **A / A**, no critical/high on the plugin itself.
|
||||
- [ ] Each new finding has: a positive fixture (fires) **and** a negative fixture (silent).
|
||||
- [ ] fix-engine + humanizer-data entries added/updated for any new finding title (title
|
||||
coupling — see `b6a62d7`).
|
||||
- [ ] SC-5 snapshot: byte-stable for additive findings (A/E); regenerated + reviewed for B
|
||||
(and D if it touches the snapshot project).
|
||||
- [ ] Scanner-count lore updated everywhere if C lands (README badge, CLAUDE.md inventory,
|
||||
`docs/scanner-internals.md`, the "count stays 13" notes).
|
||||
- [ ] U1 (`post-session`) and U2 (plugin `outputStyles` key) resolved before the features that
|
||||
depend on them.
|
||||
|
||||
## Open decisions for the operator
|
||||
|
||||
1. **Phasing**: three releases (recommended) vs one v5.5 with everything?
|
||||
2. **C placement**: new `CA-OST` scanner (count → 14) vs folding output-style checks into
|
||||
SET+a file check (keeps 13)? (Recommend new scanner — distinct surface.)
|
||||
3. **D**: build now (heuristic, precision-gated) vs defer until A/B/C/E prove the theme?
|
||||
4. **B `--json`**: add `loadPattern` field in place (mild back-compat risk) vs schema version bump?
|
||||
Loading…
Add table
Add a link
Reference in a new issue