docs(knowledge): the judgment axis is knowledge, not a detector (B2)

Article rule 1 ("give Claude judgement instead of rules") gets its register
entry, and deliberately no detector. `BP-JUDG-001` carries `lensCheck: null`.

The cut between deterministic prefilter and prose judge was the open design
decision. It was settled by measurement, and the measurement declined both
halves:

- 409 real CLAUDE.md files (38488 lines, 8689 prose blocks): the caging class
  fires 7 times, and all 7 are false positives ("rendered prose-side",
  "naming is a flag on the class"). Verified along an independent grep path
  that bypasses block-splitting and sentence-splitting entirely, in both word
  orders: 5 lines and 1 line, none an instruction.
- The narrow variant (absolute + form noun + numeric cap) fired 8 times —
  one duplicated block seen seven times across plugin caches, precision 0 %.
  The pre-committed rule required 90 % over 20 distinct fires.
- Where the shape does occur — 45 lines across 4755 skill/agent/command files
  — it is the author's editorial policy (emoji, sentence length, slide
  titles). Nothing in the text separates that from a vendor's over-tight
  guardrail, and the article's reasoning does not transfer: the model is not
  the author of a user's config.

So no CA-OPT-002; finding-codes keeps OPT next-free = 2. The numbers live in
the entry's own `note`, so the next session does not re-derive the question.

Two premises the chunk falsified. The brief justified a separate axis by
saying these blocks sit inside `floor-exclusion`'s floor — but the article's
own canonical line carries no floor marker at all, so "inside the floor"
cannot define the axis (the corpus tendency is 76 %, which is a tendency, not
a mechanism). And the fasit's own form-noun vocabulary was wrong: `name` and
`format` alone drove 97 % of fires.

Not folded into `--subtract`: a third "loosen instead of delete" verdict in
the subtraction judge is the AS#5 mixing STATE forbids, and with the corrected
vocabulary there are 0 collisions to arbitrate anyway.

Guards, both seen red against their own defect first: the entry must exist,
be confirmed, date its source and name NO lensCheck; and every lensCheck in
the register must be backed by a real detector.

No behaviour changed — no new finding, no output change, nothing consumes the
entry yet — hence `docs`, not `feat`. Suite 1701 -> 1703/0; frozen v5.0.0 and
default-output baselines 0 changed files. Fasit:
docs/b2-judgment-lens-fasit.local.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017mCkx9wGywqNQzsXkBMzJ1
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 20:31:22 +02:00
commit 6bb100f2e0
3 changed files with 86 additions and 8 deletions

View file

@ -46,12 +46,19 @@ scanner *input*, so the axes compose instead of competing.
**Deviation from brief, verified 2026-08-03:** the article contains NO mechanism-choice or **Deviation from brief, verified 2026-08-03:** the article contains NO mechanism-choice or
size-limit content → it was NOT added to BP-MECH-*/BP-SIZE-001 (that would be false size-limit content → it was NOT added to BP-MECH-*/BP-SIZE-001 (that would be false
provenance). If a future read finds real coverage, add it then. provenance). If a future read finds real coverage, add it then.
- **B2 — new lens axis (own chunk):** `BP-JUDG-001` + `CA-OPT-002` for instructions that are - **B2 — DONE, and the detector was DECLINED by measurement.** `BP-JUDG-001` ships as
local and specific but **over-specify and cage judgment** (article rule 1). NOT an extension register knowledge with `lensCheck: null`; **no `CA-OPT-002`** (OPT next-free stays 2).
of `--subtract` — the floor (`floor-exclusion.mjs`) rightly protects these blocks from Measured over **409 real CLAUDE.md files** (38 488 lines, 8 689 prose blocks): the caging
deletion; this axis says *keep the content, loosen the phrasing*. Same precision gate as class fires **7 times, all 7 false positives**, confirmed along an independent grep path in
`optimization-lens-agent` (cite rule + source, stay silent when unsure). Mixing the axes is both word orders (5 lines / 1 line, none an instruction). Where the shape *does* occur — 45
the ÅS#5 defect class. lines across 4 755 skill/agent/command files — it is the author's **editorial policy** (emoji,
sentence length, slide titles), and nothing in the text separates that from a vendor's
over-tight guardrail: the article's reasoning does not transfer, because the model is not the
author of a user's config. Precision-first ⇒ silence. Two premises the chunk falsified: the
brief's «these blocks are inside the floor» (the article's own example carries **no** floor
marker; the corpus tendency is 76 %, which is not a mechanism), and the §4 form-noun
vocabulary (`name`/`format` alone were 97 % of fires). Full record:
`docs/b2-judgment-lens-fasit.local.md` §9.
- **B3 — two-layer duplication/contradiction (CNF extension):** article rule 4 + `/doctor`'s - **B3 — two-layer duplication/contradiction (CNF extension):** article rule 4 + `/doctor`'s
measured Sjekk 2 catch (global CLAUDE.md model-policy ↔ agent frontmatter) prove the class measured Sjekk 2 catch (global CLAUDE.md model-policy ↔ agent frontmatter) prove the class
exists and is catchable. Extend `conflict-detector` with cross-layer checks where the pair is exists and is catchable. Extend `conflict-detector` with cross-layer checks where the pair is
@ -201,8 +208,11 @@ Check 3) and fails the binding `/doctor` positioning; and regenerating destroys
- **C1C5:** criteria as specified 2026-07-14 (C2 red-first nearest-match; C3 fixture - **C1C5:** criteria as specified 2026-07-14 (C2 red-first nearest-match; C3 fixture
missing-path fires / URL-glob-placeholder silent; C4 authored-agent matrix; C5 failure-modes missing-path fires / URL-glob-placeholder silent; C4 authored-agent matrix; C5 failure-modes
section present). section present).
- **B2:** fixture with a precise-but-caging instruction → CA-OPT-002 with rule+source citation; - **B2 (superseded by the measurement above):** the criterion was written for a detector that
floor-protected block WITHOUT caging phrasing → silent (axis separation proven). measurement declined. What was verified instead: `BP-JUDG-001` present, `confirmed`, primary
source dated `2026-07-24`, `lensCheck` **absent** — plus a guard that every `lensCheck` in the
register is backed by a real detector, so no later session can "complete" the entry by wiring
one. Both assertions seen RED against their own defect before landing.
- **B3:** fixture with same instruction in CLAUDE.md + rule → CNF finding; single-layer only → - **B3:** fixture with same instruction in CLAUDE.md + rule → CNF finding; single-layer only →
silent ([[guard-can-be-green-on-its-own-defect]]: assert the blanket invariant). silent ([[guard-can-be-green-on-its-own-defect]]: assert the blanket invariant).

View file

@ -263,6 +263,22 @@
"note": "Verbatim: 'Claude will be more predisposed to double-checking additional hypotheses or verifying correctness at higher effort levels, but it generally won't artificially inflate usage for simple tasks at higher effort levels.'; 'In fact, our team pays close attention to \"overthinking\" during model training as it degrades effectiveness.'" "note": "Verbatim: 'Claude will be more predisposed to double-checking additional hypotheses or verifying correctness at higher effort levels, but it generally won't artificially inflate usage for simple tasks at higher effort levels.'; 'In fact, our team pays close attention to \"overthinking\" during model training as it degrades effectiveness.'"
} }
] ]
},
{
"id": "BP-JUDG-001",
"claim": "An instruction that fixes a form decision — how code or prose should look (comment density, docstring length, naming shape, sentence or paragraph count) — as an absolute rule buys a guardrail current models no longer need, and is wrong for the cases the rule did not anticipate. Anthropic removed its own example from the Claude Code system prompt: \"Never write multi-paragraph docstrings or multi-line comment blocks — one short line max\" was replaced by \"Write code that reads like the surrounding code: match its comment density, naming, and idiom.\"",
"appliesTo": "claude-md",
"recommendation": "Where an absolute governs a form decision rather than a local fact, state the outcome you want and let the model judge the instance. This does not apply to safety rules, tool or version facts, or a house style you hold deliberately — those are the reason the claim is not machine-checkable.",
"confidence": "confirmed",
"category": "judgment-fit",
"lensCheck": null,
"note": "KNOWLEDGE ONLY — no detector, by measurement (docs/b2-judgment-lens-fasit.local.md §9). Across 409 real CLAUDE.md files (38488 lines, 8689 prose blocks) the class fired 7 times and all 7 were false positives; an independent grep in both word orders found 5 lines and 1 line respectively, none an instruction. Where the shape does occur — 45 matching lines across 4755 skill/agent/command files — it is the author's editorial policy (emoji, sentence length, slide titles), which nothing in the text separates from a vendor's over-tight guardrail. Precision-first: no CA-OPT code was allocated and OPT next-free stays 2.",
"source": {
"url": "https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models",
"title": "The new rules of context engineering for Claude 5 generation models",
"published": "2026-07-24",
"verified": "2026-08-12"
}
} }
] ]
} }

View file

@ -7,6 +7,8 @@ import {
CONFIDENCE_LEVELS, CONFIDENCE_LEVELS,
REGISTER_PATH, REGISTER_PATH,
} from '../../scanners/lib/best-practices-register.mjs'; } from '../../scanners/lib/best-practices-register.mjs';
import { LENS_DETECTORS } from '../../scanners/lib/lens-prefilter.mjs';
import { SUBTRACT_DETECTORS } from '../../scanners/lib/subtraction-prefilter.mjs';
// A minimal well-formed entry; negative tests clone + mutate this. // A minimal well-formed entry; negative tests clone + mutate this.
const validEntry = () => ({ const validEntry = () => ({
@ -107,6 +109,56 @@ describe('bundled register integrity (Verifiseringsplikt)', () => {
); );
} }
}); });
// B2. The judgment entry is knowledge WITHOUT a detector, and that is the
// measured outcome, not an omission: the caging class the article names fired
// 7 times across 409 real CLAUDE.md files and all 7 were false positives
// (docs/b2-judgment-lens-fasit.local.md §9.4). Asserting the absent lensCheck
// is what stops a later session from "completing" the entry by wiring a
// detector the corpus refused.
it('carries the judgment entry (B2) as detector-less knowledge', () => {
const e = getEntry(reg, 'BP-JUDG-001');
assert.ok(e, 'BP-JUDG-001 missing from the bundled register');
assert.equal(e.confidence, 'confirmed', 'BP-JUDG-001 must be confirmed');
assert.equal(e.category, 'judgment-fit', 'BP-JUDG-001 wrong category');
assert.equal(
e.lensCheck ?? null,
null,
'BP-JUDG-001 must NOT name a lensCheck — no detector survived measurement'
);
assert.equal(
e.source.url,
'https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models',
'BP-JUDG-001 primary source must be the article that states rule 1'
);
assert.equal(e.source.published, '2026-07-24', 'BP-JUDG-001 must date its primary source');
assert.match(
String(e.note),
/409/,
'BP-JUDG-001 must carry the measured negative result, so it is not re-derived'
);
});
// The register's own consumers key on lensCheck; an entry that names one it
// does not have would reach a payload with no detector behind it.
it('every lensCheck in the register is backed by a detector', () => {
const detectors = new Set([
...LENS_DETECTORS.map((d) => d.lensCheck),
...SUBTRACT_DETECTORS.map((d) => d.lensCheck),
// Scanner-side checks that are their own detector.
'procedure-in-claude-md',
'CA-OST-001',
'CA-CML-001',
'CA-SKL-002',
]);
for (const e of reg.entries) {
if (e.lensCheck == null) continue;
assert.ok(
detectors.has(e.lensCheck),
`${e.id} names lensCheck "${e.lensCheck}" with no detector behind it`
);
}
});
}); });
describe('validateRegister (negative cases)', () => { describe('validateRegister (negative cases)', () => {