Kjell Tore Guttormsen
7df8e0d65b
Anthropic documents that Claude Opus 5 verifies its own work, and that telling it to double-check or to delegate verification to a subagent causes over-verification -- token cost with no quality gain. The general subtraction detector (BP-SUB-001) already surfaces those blocks for every user, with no model-awareness at all. `optimize --subtract --for-model <name>` adds the missing half. It ANNOTATES a subset of the candidates --subtract already produced; it is not a second detector and can never widen the candidate set. A second SUBTRACT_DETECTORS entry would have collided with BP-SUB-001 on de-dup, and a prose-only signal in the agent prompt would have been untestable. There is no auto-detection, by measurement rather than omission: a CLAUDE.md has no frontmatter and no resolvable target model, and this operator's own `route` skill deliberately runs a different model per session -- the same file is read by whichever model comes next. So the model is named, and the citation is reported as conditional everywhere a human sees it (agent report copy, and the Step 7a listing that is the last surface before an approval file). Precision comes from the TARGET, not the verb list. Measured across the 409-file corpus: 392 BP-SUB-001 candidates, 31 (7.9%) carry a verify verb, and 0 also carry a reflexive or delegated target. Two independent raw-text greps found 0 as well, so the zero is the corpus rather than an over-narrow regex. Those 31 verb-only blocks -- "sjekk relevante config-filer", "Type-sjekk: pyright", "To verify plugin functionality" -- are exactly the false positives a verb-only version would have produced, which is BP-JUDG-001's 7/7 failure arriving one lens over. The numbers live in the register entry's note and are pinned by a test, because a session that cannot see the measurement reads the zero as a broken detector and loosens it. `recognized` is reported separately from `matchedCount`: a typo'd model name and a genuinely clean config both yield zero, and without the distinction the CLI would report a silent no-op as good news. Dogfooded on the real machine -- `opus-5` gives recognized:true/matchedCount:0, `oppus5` gives recognized:false. source.published is absent because the guide carries no visible publish date; its absence is asserted so a later session does not invent one to match the other entries' shape. Both quoted sentences were verified verbatim 2026-08-12. The payload stays additive -- forModel and per-candidate modelScope appear only under the flag, so a plain --subtract run is byte-identical to before (asserted on the serialized bytes, since a key set to undefined passes a shallow check). Suite 1724 -> 1752 (+28). The one remaining failure is the pre-existing drift-cli --output-file crash, untouched by this work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BRuXt6tZyowi8QYNKLSHQm
109 lines
3.9 KiB
JavaScript
109 lines
3.9 KiB
JavaScript
/**
|
|
* Model-scoped ANNOTATION on top of the subtraction lens.
|
|
*
|
|
* This module generates no candidates of its own. `optimize --subtract` already
|
|
* surfaces compensatory instructions through the single `compensatory-instruction`
|
|
* detector (`BP-SUB-001`); this answers a narrower question over text that
|
|
* already passed it: is this specifically the class of instruction a named model
|
|
* documents as redundant — self-verification, or verification delegated to a
|
|
* subagent?
|
|
*
|
|
* A second competing detector is deliberately NOT what this is. The register
|
|
* entry it cites carries `lensCheck: null`, the same discipline `BP-JUDG-001`
|
|
* shipped under: a plausible-looking detector for "instruction a model no longer
|
|
* needs" measured 7/7 false positives across 409 real CLAUDE.md files, so this
|
|
* claim class rides an existing measured detector rather than widening the
|
|
* candidate set.
|
|
*
|
|
* Precision comes from requiring a reflexive/delegate TARGET alongside the
|
|
* verify verb, never from narrowing the verb list. A bare "check"/"verify" also
|
|
* matches EXTERNAL verification ("check the CI status") — which stays a true
|
|
* negative here even though it is, correctly, still a `BP-SUB-001` candidate
|
|
* upstream.
|
|
*
|
|
* Pure: text → annotation or null. Zero external dependencies.
|
|
*/
|
|
import { LB, RB } from './subtraction-prefilter.mjs';
|
|
|
|
/**
|
|
* The reflexive self-verification target. This is what narrows a bare
|
|
* verify/check imperative down to the specific claim the model's documented
|
|
* self-correction contradicts.
|
|
*/
|
|
const SELF_TARGET_RE = new RegExp(
|
|
LB +
|
|
'(?:your (?:own )?(?:work|output|answer|changes)|yourself|' +
|
|
'before (?:responding|submitting|finalizing)|dine egne?|deg selv|før du svarer)' +
|
|
RB,
|
|
'i',
|
|
);
|
|
|
|
/**
|
|
* Verify verbs — deliberately as broad as `subtraction-prefilter.mjs`'s
|
|
* `IMPERATIVE_RE`. Breadth here is safe because a co-occurring target is
|
|
* required; narrowing the list would only lose true positives.
|
|
*/
|
|
const VERIFY_VERB_RE = new RegExp(
|
|
LB +
|
|
'(?:double-check|re-verify|re-check|confirm|verify|review|check|' +
|
|
'dobbeltsjekk|verifiser|sjekk)' +
|
|
RB,
|
|
'i',
|
|
);
|
|
|
|
/**
|
|
* Delegated verification. Order-free on purpose: it must match both
|
|
* "verify X with a subagent" and "use a subagent to verify X".
|
|
*/
|
|
const DELEGATE_VERIFY_RE = new RegExp(
|
|
LB + '(?:subagent|sub-agent|another (?:agent|instance)|task tool)' + RB,
|
|
'i',
|
|
);
|
|
|
|
/**
|
|
* Does this text carry a self- or delegate-targeted verification instruction?
|
|
*
|
|
* @param {string} text
|
|
* @returns {boolean}
|
|
*/
|
|
export function isModelContradictedVerification(text) {
|
|
return (
|
|
VERIFY_VERB_RE.test(text) &&
|
|
(SELF_TARGET_RE.test(text) || DELEGATE_VERIFY_RE.test(text))
|
|
);
|
|
}
|
|
|
|
/**
|
|
* Model names are compared on their alphanumeric skeleton, so "opus-5",
|
|
* "Opus 5" and "OPUS5" are one model. A typo'd name simply fails to match —
|
|
* the CLI reports that it did not recognize the name rather than reporting a
|
|
* silent zero.
|
|
*
|
|
* @param {unknown} s
|
|
* @returns {string}
|
|
*/
|
|
const normalizeModel = (s) =>
|
|
String(s || '')
|
|
.toLowerCase()
|
|
.replace(/[^a-z0-9]/g, '');
|
|
|
|
/**
|
|
* Annotate a subtraction candidate with a model-scoped register citation.
|
|
*
|
|
* @param {string} text candidate block text (already a `BP-SUB-001` candidate)
|
|
* @param {string|null|undefined} targetModel the model named by `--for-model`
|
|
* @param {Array<{id: string, claim: string, modelScope?: string[]}>} entries
|
|
* confirmed register entries with `category: 'prompting-fit'`
|
|
* @returns {{registerId: string, claim: string, requestedModel: string}|null}
|
|
*/
|
|
export function matchModelScope(text, targetModel, entries) {
|
|
if (!targetModel || !isModelContradictedVerification(text)) return null;
|
|
const wanted = normalizeModel(targetModel);
|
|
const entry = (entries || []).find((e) =>
|
|
(e.modelScope || []).some((m) => normalizeModel(m) === wanted),
|
|
);
|
|
if (!entry) return null;
|
|
return { registerId: entry.id, claim: entry.claim, requestedModel: targetModel };
|
|
}
|
|
|
|
export { normalizeModel };
|