open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

feat(scanners): a redundancy claim that belongs to one model is scoped to it

Browse source 
Anthropic documents that Claude Opus 5 verifies its own work, and that telling
it to double-check or to delegate verification to a subagent causes
over-verification -- token cost with no quality gain. The general subtraction
detector (BP-SUB-001) already surfaces those blocks for every user, with no
model-awareness at all.

`optimize --subtract --for-model <name>` adds the missing half. It ANNOTATES a
subset of the candidates --subtract already produced; it is not a second
detector and can never widen the candidate set. A second SUBTRACT_DETECTORS
entry would have collided with BP-SUB-001 on de-dup, and a prose-only signal in
the agent prompt would have been untestable.

There is no auto-detection, by measurement rather than omission: a CLAUDE.md has
no frontmatter and no resolvable target model, and this operator's own `route`
skill deliberately runs a different model per session -- the same file is read
by whichever model comes next. So the model is named, and the citation is
reported as conditional everywhere a human sees it (agent report copy, and the
Step 7a listing that is the last surface before an approval file).

Precision comes from the TARGET, not the verb list. Measured across the
409-file corpus: 392 BP-SUB-001 candidates, 31 (7.9%) carry a verify verb, and
0 also carry a reflexive or delegated target. Two independent raw-text greps
found 0 as well, so the zero is the corpus rather than an over-narrow regex.
Those 31 verb-only blocks -- "sjekk relevante config-filer", "Type-sjekk:
pyright", "To verify plugin functionality" -- are exactly the false positives a
verb-only version would have produced, which is BP-JUDG-001's 7/7 failure
arriving one lens over. The numbers live in the register entry's note and are
pinned by a test, because a session that cannot see the measurement reads the
zero as a broken detector and loosens it.

`recognized` is reported separately from `matchedCount`: a typo'd model name and
a genuinely clean config both yield zero, and without the distinction the CLI
would report a silent no-op as good news. Dogfooded on the real machine --
`opus-5` gives recognized:true/matchedCount:0, `oppus5` gives recognized:false.

source.published is absent because the guide carries no visible publish date;
its absence is asserted so a later session does not invent one to match the
other entries' shape. Both quoted sentences were verified verbatim 2026-08-12.

The payload stays additive -- forModel and per-candidate modelScope appear only
under the flag, so a plain --subtract run is byte-identical to before (asserted
on the serialized bytes, since a key set to undefined passes a shallow check).

Suite 1724 -> 1752 (+28). The one remaining failure is the pre-existing
drift-cli --output-file crash, untouched by this work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRuXt6tZyowi8QYNKLSHQm
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 23:16:23 +02:00
commit 7df8e0d65b
11 changed files with 650 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

34
CLAUDE.md
View file

@ -17,7 +17,7 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
| `/config-audit tokens` | Prompt-cache-aware token hotspots, each tagged with its load pattern; cache-aware |
| `/config-audit manifest` | Ranked table of every token source + always-loaded subtotal |
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact |
| `/config-audit optimize` | Mechanism-fit lens (procedure→skill, lifecycle→hook, path→rule, never→permission). Agent-driven, **not byte-stable**. `--subtract` adds the subtraction axis (what no longer earns its always-loaded rent, `BP-SUB-001`) — opt-in, proposes only; `--subtract --apply` executes the removals the operator picks |
| `/config-audit optimize` | Mechanism-fit lens (procedure→skill, lifecycle→hook, path→rule, never→permission). Agent-driven, **not byte-stable**. `--subtract` adds the subtraction axis (what no longer earns its always-loaded rent, `BP-SUB-001`) — opt-in, proposes only; `--subtract --apply` executes the removals the operator picks; `--subtract --for-model <name>` annotates the candidates a named model documents as redundant (`BP-PROMPT-001`) |
| `/config-audit fix` | Auto-fix deterministic issues with backup + verification |
| `/config-audit rollback` | Restore configuration from backup |
| `/config-audit plan` | Create action plan from findings |
@ -165,6 +165,38 @@ a trailing `:54-56` locator is a recorded v1 miss, not a bug to fix by loosening
**Subtraction floor (invariant).** `optimize --subtract` is the only lens that proposes removing config, so `scanners/lib/floor-exclusion.mjs` runs as a deterministic pre-step *before* the judge — a load-bearing block is never a candidate, and that guarantee must not be moved into the agent prompt. Two rules follow from it: (1) **staleness is not a deletion signal** — an outdated version pin inside a floor block is a `drift`/`CA-CML` dead-reference concern; (2) **tier 2 ≠ tier 3** — a compensatory block that keeps earning its place returns, and reporting it as dead weight is wrong even when the label matches. Norwegian keywords need the Unicode boundaries in `subtraction-prefilter.mjs`; JS `\b` is ASCII-only, so `/\bunngå\b/` silently never matches.
**Model-scoped annotation (invariant).** `--for-model <name>` (`BP-PROMPT-001`,
`scanners/lib/prompting-model-scope.mjs`) is an **annotation on existing
`BP-SUB-001` candidates, never a detector** — it tags a subset of what
`--subtract` already found and can never widen the candidate set. Four
properties are load-bearing. (1) **The model must be named.** There is no
auto-detection and there cannot be: a CLAUDE.md has no frontmatter and no
resolvable target model, and the operator's own `route` skill deliberately runs
a different model per session — so the same file is read by whichever model
comes next. (2) **Precision is the TARGET, not the verb list.** The verb set is
as broad as `subtraction-prefilter`'s `IMPERATIVE_RE`; what narrows it is
requiring a co-occurring reflexive ("your own work", "før du svarer") or
delegated ("subagent") target. Measured across the 409-file corpus: 392
`BP-SUB-001` candidates, **31 carry a verify verb, 0 also carry a target**, and
two independent raw-text greps also found 0 — so the zero is the corpus, not an
over-narrow regex. Those 31 (`sjekk relevante config-filer`, `Type-sjekk:
pyright`) are exactly the false positives a verb-only version would have
produced, which is `BP-JUDG-001`'s 7/7 failure arriving one lens over. The
numbers live in the register entry's `note` and are pinned by
`best-practices-register.test.mjs`, because a later session that cannot see the
measurement reads the zero as a broken detector and loosens it. (3) **`recognized`
is not `matchedCount`.** A typo'd model name and a genuinely clean config both
yield zero matches; without the separate flag the CLI would report a silent
no-op as good news. Dogfooded on the real machine: `opus-5` → `recognized:true,
matchedCount:0`; `oppus5` → `recognized:false`. (4) **The citation is
conditional wherever a human sees it** — agent report copy and the Step 7a
approval listing both say so. `--for-model` and `--apply` are separate CLIs that
never see each other's flags, so a CLI-level refusal is impossible; the
safeguard has to live in the copy. The payload stays additive: `forModel` and
per-candidate `modelScope` appear **only** when the flag is passed, so a plain
`--subtract` run is byte-identical to the pre-flag payload (asserted on the
serialized bytes, since a key set to `undefined` passes a shallow key check).
**Subtraction write path (invariant).** `--apply` routes through `scanners/lib/subtraction-write.mjs`, never through `fix-engine` or the `plan`/`implement` pipeline, and both exclusions are **measured**: the subtraction axis is absent from the orchestrated envelope, so `verifyFixes`' re-scan would mark every removal `verified` whether or not it happened (a success-shaped no-op), and the findings pipeline needs a finding code — which names a deterministic check, not a prose judgement. Three properties are load-bearing and each has a guard seen red against its own defect: removals are validated against the ORIGINAL content and applied in **descending** line order (an ascending pass shifts later spans out from under themselves); the **range** check is not redundant with the text check (`line: 0` makes `slice(-1, 0)` empty, so an empty `text` matches and `splice(-1, 1)` deletes the file's LAST line); and `createBackup` skips a nonexistent path while still returning an id, so coverage of every file about to be written is **asserted from the manifest** before a byte changes. The floor is *repeated* here, not moved: `floor-exclusion` still vetoes before anything is proposed, and the engine refuses a load-bearing block again so a hand-built approval cannot route around it. The archive rule (`mv` to `_archive/`) is file-level and does not apply to a block excision — the timestamped backup is the recovery artifact, and inventing a second copy with no restorer behind it would be worse than none.
## Testing