feat(scanners): a redundancy claim that belongs to one model is scoped to it
Anthropic documents that Claude Opus 5 verifies its own work, and that telling it to double-check or to delegate verification to a subagent causes over-verification -- token cost with no quality gain. The general subtraction detector (BP-SUB-001) already surfaces those blocks for every user, with no model-awareness at all. `optimize --subtract --for-model <name>` adds the missing half. It ANNOTATES a subset of the candidates --subtract already produced; it is not a second detector and can never widen the candidate set. A second SUBTRACT_DETECTORS entry would have collided with BP-SUB-001 on de-dup, and a prose-only signal in the agent prompt would have been untestable. There is no auto-detection, by measurement rather than omission: a CLAUDE.md has no frontmatter and no resolvable target model, and this operator's own `route` skill deliberately runs a different model per session -- the same file is read by whichever model comes next. So the model is named, and the citation is reported as conditional everywhere a human sees it (agent report copy, and the Step 7a listing that is the last surface before an approval file). Precision comes from the TARGET, not the verb list. Measured across the 409-file corpus: 392 BP-SUB-001 candidates, 31 (7.9%) carry a verify verb, and 0 also carry a reflexive or delegated target. Two independent raw-text greps found 0 as well, so the zero is the corpus rather than an over-narrow regex. Those 31 verb-only blocks -- "sjekk relevante config-filer", "Type-sjekk: pyright", "To verify plugin functionality" -- are exactly the false positives a verb-only version would have produced, which is BP-JUDG-001's 7/7 failure arriving one lens over. The numbers live in the register entry's note and are pinned by a test, because a session that cannot see the measurement reads the zero as a broken detector and loosens it. `recognized` is reported separately from `matchedCount`: a typo'd model name and a genuinely clean config both yield zero, and without the distinction the CLI would report a silent no-op as good news. Dogfooded on the real machine -- `opus-5` gives recognized:true/matchedCount:0, `oppus5` gives recognized:false. source.published is absent because the guide carries no visible publish date; its absence is asserted so a later session does not invent one to match the other entries' shape. Both quoted sentences were verified verbatim 2026-08-12. The payload stays additive -- forModel and per-candidate modelScope appear only under the flag, so a plain --subtract run is byte-identical to before (asserted on the serialized bytes, since a key set to undefined passes a shallow check). Suite 1724 -> 1752 (+28). The one remaining failure is the pre-existing drift-cli --output-file crash, untouched by this work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BRuXt6tZyowi8QYNKLSHQm
This commit is contained in:
parent
05b4e9d797
commit
7df8e0d65b
11 changed files with 650 additions and 10 deletions
|
|
@ -139,6 +139,73 @@ describe('bundled register integrity (Verifiseringsplikt)', () => {
|
|||
);
|
||||
});
|
||||
|
||||
// The model-scoped prompting entry is an ANNOTATION on an existing BP-SUB-001
|
||||
// candidate, not a detector of its own — same discipline as BP-JUDG-001 above,
|
||||
// and asserted for the same reason: without this, a later session "completes"
|
||||
// the entry by wiring a second SUBTRACT_DETECTORS entry, which would both
|
||||
// widen the candidate set and collide with BP-SUB-001 on de-dup.
|
||||
//
|
||||
// `modelScope` is pinned because it is a DATA CONTRACT, not a label:
|
||||
// `matchModelScope` looks the requested `--for-model` name up in exactly this
|
||||
// array. Renaming or dropping it makes every lookup silently return null —
|
||||
// the CLI would report `recognized: false` for a model the register still
|
||||
// claims to cover, which is a quiet wrong answer, not a loud failure.
|
||||
it('carries the model-scoped prompting entry (BP-PROMPT-001) as an annotation, not a detector', () => {
|
||||
const e = getEntry(reg, 'BP-PROMPT-001');
|
||||
assert.ok(e, 'BP-PROMPT-001 missing from the bundled register');
|
||||
assert.equal(e.confidence, 'confirmed', 'BP-PROMPT-001 must be confirmed');
|
||||
assert.equal(e.category, 'prompting-fit', 'BP-PROMPT-001 wrong category');
|
||||
assert.equal(
|
||||
e.mechanism,
|
||||
'deletion',
|
||||
'BP-PROMPT-001 rides the subtraction axis — an addition-shaped claim must not reuse this entry'
|
||||
);
|
||||
assert.equal(
|
||||
e.lensCheck ?? null,
|
||||
null,
|
||||
'BP-PROMPT-001 must NOT name a lensCheck — it annotates BP-SUB-001 candidates, it does not detect'
|
||||
);
|
||||
assert.deepStrictEqual(
|
||||
e.modelScope,
|
||||
['opus-5'],
|
||||
'BP-PROMPT-001 modelScope is the lookup key matchModelScope resolves --for-model against'
|
||||
);
|
||||
assert.equal(
|
||||
e.source.url,
|
||||
'https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5',
|
||||
'BP-PROMPT-001 primary source must be the official Opus 5 prompting guide'
|
||||
);
|
||||
// The guide carries no visible publish/last-updated date (re-checked
|
||||
// 2026-08-12). Asserting its ABSENCE keeps a later session from inventing
|
||||
// one to satisfy a pattern the other entries happen to have.
|
||||
assert.equal(
|
||||
e.source.published,
|
||||
undefined,
|
||||
'BP-PROMPT-001 source has no visible published date — do not fabricate one'
|
||||
);
|
||||
assert.match(
|
||||
String(e.note),
|
||||
/--for-model/,
|
||||
'BP-PROMPT-001 must record that it is gated behind an explicit flag'
|
||||
);
|
||||
// The corpus numbers live in the entry, not only in a gitignored fasit,
|
||||
// for the same reason BP-JUDG-001's 409 does: a later session that cannot
|
||||
// see the measurement re-derives it, and the cheapest re-derivation is to
|
||||
// read the zero as a broken detector and loosen it. 31 is the load-bearing
|
||||
// half — how many verb-only blocks the TARGET requirement rejected, i.e.
|
||||
// the false positives a naive version of this check would have produced.
|
||||
assert.match(
|
||||
String(e.note),
|
||||
/409/,
|
||||
'BP-PROMPT-001 must carry the corpus size it was measured against'
|
||||
);
|
||||
assert.match(
|
||||
String(e.note),
|
||||
/(^|\s)31(\s|,)/,
|
||||
'BP-PROMPT-001 must carry the count the target requirement rejected — the reason it is narrow'
|
||||
);
|
||||
});
|
||||
|
||||
// The register's own consumers key on lensCheck; an entry that names one it
|
||||
// does not have would reach a payload with no detector behind it.
|
||||
it('every lensCheck in the register is backed by a detector', () => {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue