Kjell Tore Guttormsen
6bb100f2e0
Article rule 1 ("give Claude judgement instead of rules") gets its register
entry, and deliberately no detector. `BP-JUDG-001` carries `lensCheck: null`.
The cut between deterministic prefilter and prose judge was the open design
decision. It was settled by measurement, and the measurement declined both
halves:
- 409 real CLAUDE.md files (38488 lines, 8689 prose blocks): the caging class
fires 7 times, and all 7 are false positives ("rendered prose-side",
"naming is a flag on the class"). Verified along an independent grep path
that bypasses block-splitting and sentence-splitting entirely, in both word
orders: 5 lines and 1 line, none an instruction.
- The narrow variant (absolute + form noun + numeric cap) fired 8 times —
one duplicated block seen seven times across plugin caches, precision 0 %.
The pre-committed rule required 90 % over 20 distinct fires.
- Where the shape does occur — 45 lines across 4755 skill/agent/command files
— it is the author's editorial policy (emoji, sentence length, slide
titles). Nothing in the text separates that from a vendor's over-tight
guardrail, and the article's reasoning does not transfer: the model is not
the author of a user's config.
So no CA-OPT-002; finding-codes keeps OPT next-free = 2. The numbers live in
the entry's own `note`, so the next session does not re-derive the question.
Two premises the chunk falsified. The brief justified a separate axis by
saying these blocks sit inside `floor-exclusion`'s floor — but the article's
own canonical line carries no floor marker at all, so "inside the floor"
cannot define the axis (the corpus tendency is 76 %, which is a tendency, not
a mechanism). And the fasit's own form-noun vocabulary was wrong: `name` and
`format` alone drove 97 % of fires.
Not folded into `--subtract`: a third "loosen instead of delete" verdict in
the subtraction judge is the AS#5 mixing STATE forbids, and with the corrected
vocabulary there are 0 collisions to arbitrate anyway.
Guards, both seen red against their own defect first: the entry must exist,
be confirmed, date its source and name NO lensCheck; and every lensCheck in
the register must be backed by a real detector.
No behaviour changed — no new finding, no output change, nothing consumes the
entry yet — hence `docs`, not `feat`. Suite 1701 -> 1703/0; frozen v5.0.0 and
default-output baselines 0 changed files. Fasit:
docs/b2-judgment-lens-fasit.local.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017mCkx9wGywqNQzsXkBMzJ1
284 lines
16 KiB
JSON
284 lines
16 KiB
JSON
{
|
|
"version": 1,
|
|
"note": "Machine-readable best-practices register. SOURCE OF TRUTH for the optimization lens (v5.7 CA-OPT). Human-readable mirror lives in knowledge/*.md. Every entry is provenance-stamped (source.url + source.verified; optional corroborating sources[] with published dates feed the evidence-age freshness rule; source.supersededBy marks a source replaced by a newer one) and carries a confidence; only CONFIRMED claims are consumed user-facing (Verifiseringsplikt). Curated manually + by /config-audit knowledge-refresh (human-approved). Seeded from docs/v5.5-steering-model-plan.md V-rows + the Anthropic 'Steering Claude Code' blog.",
|
|
"entries": [
|
|
{
|
|
"id": "BP-MECH-001",
|
|
"claim": "Lifecycle automation phrased as an instruction in CLAUDE.md (\"every time\", \"before each\", \"always run X after Y\") should be a hook — a behavior the model chooses to follow is not deterministic.",
|
|
"mechanism": "hook",
|
|
"appliesTo": "claude-md",
|
|
"recommendation": "Move the behavior to a PreToolUse/PostToolUse/Stop hook so it runs deterministically, outside the model's discretion.",
|
|
"confidence": "confirmed",
|
|
"severity": "low",
|
|
"category": "mechanism-fit",
|
|
"lensCheck": "claude-md-lifecycle-phrasing",
|
|
"source": {
|
|
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
|
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-MECH-002",
|
|
"claim": "A file- or path-specific constraint placed in root CLAUDE.md or an unscoped rule should be a path-scoped rule (paths: frontmatter), so it loads only when a matching file is touched.",
|
|
"mechanism": "rule",
|
|
"appliesTo": "claude-md",
|
|
"recommendation": "Move it to .claude/rules/ with a paths: frontmatter; unscoped instructions cost tokens every turn whether relevant or not.",
|
|
"confidence": "confirmed",
|
|
"severity": "low",
|
|
"category": "mechanism-fit",
|
|
"lensCheck": "unscoped-path-specific-instruction",
|
|
"source": {
|
|
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
|
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-MECH-003",
|
|
"claim": "A multi-step procedure (deploy/release checklist) in CLAUDE.md should be a skill — CLAUDE.md is for facts Claude should hold all the time; procedures belong in skills.",
|
|
"mechanism": "skill",
|
|
"appliesTo": "claude-md",
|
|
"recommendation": "Extract the procedure into .claude/skills/; its body then loads only on invoke instead of every turn.",
|
|
"confidence": "confirmed",
|
|
"severity": "low",
|
|
"category": "mechanism-fit",
|
|
"lensCheck": "procedure-in-claude-md",
|
|
"source": {
|
|
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
|
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-MECH-004",
|
|
"claim": "An absolute prohibition phrased as a \"never do X\" instruction is the wrong tool; for something that absolutely must not happen, use permissions or a PreToolUse hook.",
|
|
"mechanism": "permission",
|
|
"appliesTo": "claude-md",
|
|
"recommendation": "Enforce hard prohibitions via permission deny rules or a PreToolUse hook (exit code 2 denies the call), not prose instructions.",
|
|
"confidence": "confirmed",
|
|
"severity": "low",
|
|
"category": "mechanism-fit",
|
|
"lensCheck": "never-instruction",
|
|
"source": {
|
|
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
|
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-MECH-005",
|
|
"claim": "A custom output style without keep-coding-instructions: true removes Claude Code's built-in software-engineering instructions when active.",
|
|
"mechanism": "output-style",
|
|
"appliesTo": "output-style",
|
|
"recommendation": "Set keep-coding-instructions: true, or prefer a built-in style (Explanatory / Learning / Proactive) before writing a custom one.",
|
|
"confidence": "confirmed",
|
|
"severity": "medium",
|
|
"category": "mechanism-fit",
|
|
"lensCheck": "CA-OST-001",
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/output-styles",
|
|
"title": "Output styles",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-LOAD-001",
|
|
"claim": "Project-root CLAUDE.md and unscoped rules are re-injected from disk after compaction (they survive a /compact).",
|
|
"appliesTo": "claude-md",
|
|
"confidence": "confirmed",
|
|
"category": "loading-model",
|
|
"lensCheck": null,
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/context-window",
|
|
"title": "Context window — what survives compaction",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-LOAD-002",
|
|
"claim": "Path-scoped rules are lost after compaction until a matching file is read again, and they trigger on Read of a matching file (not on every tool use).",
|
|
"appliesTo": "rule",
|
|
"confidence": "confirmed",
|
|
"category": "loading-model",
|
|
"lensCheck": null,
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/memory",
|
|
"title": "Memory — path-specific rules",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-LOAD-003",
|
|
"claim": "A nested (non-root) CLAUDE.md is lost after compaction until a file in its directory is read again.",
|
|
"appliesTo": "claude-md",
|
|
"confidence": "confirmed",
|
|
"category": "loading-model",
|
|
"lensCheck": null,
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/context-window",
|
|
"title": "Context window — what survives compaction",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-LOAD-004",
|
|
"claim": "A skill's name + description load every turn; its body loads only on invoke.",
|
|
"appliesTo": "skill",
|
|
"confidence": "confirmed",
|
|
"category": "loading-model",
|
|
"lensCheck": null,
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/skills",
|
|
"title": "Skills",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-LOAD-005",
|
|
"claim": "Hook scripts run outside the model context, but any additionalContext they inject is saved to the transcript and is therefore subject to compaction.",
|
|
"appliesTo": "hook",
|
|
"confidence": "confirmed",
|
|
"category": "loading-model",
|
|
"lensCheck": null,
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/hooks",
|
|
"title": "Hooks",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-LOAD-006",
|
|
"claim": "A subagent runs in an isolated, fresh context window; only its final summary returns to the main session (parent instructions are not auto-injected).",
|
|
"appliesTo": "agent",
|
|
"confidence": "confirmed",
|
|
"category": "loading-model",
|
|
"lensCheck": null,
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/sub-agents",
|
|
"title": "Subagents",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-SIZE-001",
|
|
"claim": "Keep CLAUDE.md under 200 lines; give it an owner and review changes to it like code. Every line costs tokens whether relevant or not.",
|
|
"appliesTo": "claude-md",
|
|
"recommendation": "Trim CLAUDE.md to facts; move procedures to skills and path-specific rules to .claude/rules/.",
|
|
"confidence": "confirmed",
|
|
"severity": "medium",
|
|
"category": "size-budget",
|
|
"lensCheck": "CA-CML-001",
|
|
"source": {
|
|
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
|
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-SIZE-002",
|
|
"claim": "The skill-listing description cap is 1,536 characters (maxSkillDescriptionChars, configurable, v2.1.105+); the name + description load every turn.",
|
|
"appliesTo": "skill",
|
|
"confidence": "confirmed",
|
|
"severity": "low",
|
|
"category": "size-budget",
|
|
"lensCheck": "CA-SKL-002",
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/skills",
|
|
"title": "Skills",
|
|
"verified": "2026-06-20"
|
|
}
|
|
},
|
|
{
|
|
"id": "BP-SUB-001",
|
|
"claim": "Every line of CLAUDE.md loads into every session whether or not it is relevant, which consumes tokens and dilutes adherence. A line that states a local fact the model cannot derive (build commands, directory layout, conventions, team norms) earns that cost; a line that only restates general engineering behaviour pays it without being the kind of content CLAUDE.md is for, and is a candidate for removal.",
|
|
"mechanism": "deletion",
|
|
"appliesTo": "claude-md",
|
|
"recommendation": "Review the block for removal, then re-add it only if the model actually stumbles on it repeatedly. Local facts (remotes, versions, paths, conventions) and policy invariants are the floor and are never removal candidates.",
|
|
"confidence": "confirmed",
|
|
"severity": "low",
|
|
"category": "subtraction",
|
|
"lensCheck": "compensatory-instruction",
|
|
"source": {
|
|
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
|
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
|
"verified": "2026-07-31"
|
|
},
|
|
"sources": [
|
|
{
|
|
"url": "https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models",
|
|
"title": "The new rules of context engineering for Claude 5 generation models",
|
|
"published": "2026-07-24",
|
|
"verified": "2026-08-03",
|
|
"note": "Near-verbatim coverage: 'briefly describe what your repo is for, but spend most of the tokens on gotchas inside of the codebase'; 'Avoid stating the obvious things Claude should know by looking at your file system or your repo.'"
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"id": "BP-MODEL-001",
|
|
"claim": "A subagent's `model` frontmatter field defaults to `inherit`, so a subagent that names no model runs on the main conversation's model. Routing mechanical or read-only subagents to a cheaper alias (`haiku`, `sonnet`) while the orchestrating session keeps the stronger model is the documented way to control cost. The pin is not absolute: Claude Code resolves the model as CLAUDE_CODE_SUBAGENT_MODEL, then a per-invocation `model` parameter, then the frontmatter, then the main conversation's model.",
|
|
"mechanism": "model",
|
|
"appliesTo": "agent",
|
|
"recommendation": "Set `model:` explicitly on subagents whose work is mechanical or read-only (search, extraction, summarisation) and leave the orchestrator on the stronger model. Omitting the field is not a neutral default — it inherits, so every subagent costs what the session costs.",
|
|
"confidence": "confirmed",
|
|
"severity": "low",
|
|
"category": "model-fit",
|
|
"lensCheck": null,
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/sub-agents",
|
|
"title": "Create custom subagents — supported frontmatter fields / choose a model",
|
|
"verified": "2026-08-10"
|
|
},
|
|
"sources": [
|
|
{
|
|
"url": "https://claude.com/blog/claude-model-and-effort-level-in-claude-code",
|
|
"title": "Choosing a Claude model and effort level in Claude Code",
|
|
"published": "2026-07-07",
|
|
"verified": "2026-08-10",
|
|
"note": "Verbatim: 'Pick a smaller model when the work is routine. For example, edits you can describe precisely, mechanical changes, or questions about code that's already in context.'"
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"id": "BP-MODEL-002",
|
|
"claim": "Reasoning effort is an axis separate from model choice: five levels (`low`, `medium`, `high`, `xhigh`, `max`) on current models, four on Opus 4.6 and Sonnet 4.6, which omit `xhigh`; the default is `high` on every model that supports effort except Opus 4.7, which defaults to `xhigh`. Higher is not universally better — `max` \"can improve performance on demanding tasks but may show diminishing returns and is prone to overthinking\". Effort is settable in six places: `/effort`, the slider in `/model`, the `--effort` flag, CLAUDE_CODE_EFFORT_LEVEL, `effortLevel` in settings, and `effort:` in skill or subagent frontmatter; the environment variable takes precedence over all of them.",
|
|
"mechanism": "effort",
|
|
"appliesTo": "agent",
|
|
"recommendation": "Treat effort as a per-task dial rather than a global maximum: pin a lower `effort:` in the frontmatter of mechanical skills and subagents, and reserve `xhigh`/`max` for work whose product is judgment. The scale is calibrated per model, so the same level name is not the same amount of thinking across models — and CLAUDE_CODE_EFFORT_LEVEL silently overrides every other source, so verify which level is actually in force.",
|
|
"confidence": "confirmed",
|
|
"severity": "low",
|
|
"category": "model-fit",
|
|
"lensCheck": null,
|
|
"source": {
|
|
"url": "https://code.claude.com/docs/en/model-config",
|
|
"title": "Model configuration — adjust effort level / set the effort level",
|
|
"verified": "2026-08-10"
|
|
},
|
|
"sources": [
|
|
{
|
|
"url": "https://claude.com/blog/claude-model-and-effort-level-in-claude-code",
|
|
"title": "Choosing a Claude model and effort level in Claude Code",
|
|
"published": "2026-07-07",
|
|
"verified": "2026-08-10",
|
|
"note": "Verbatim: 'Claude will be more predisposed to double-checking additional hypotheses or verifying correctness at higher effort levels, but it generally won't artificially inflate usage for simple tasks at higher effort levels.'; 'In fact, our team pays close attention to \"overthinking\" during model training as it degrades effectiveness.'"
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"id": "BP-JUDG-001",
|
|
"claim": "An instruction that fixes a form decision — how code or prose should look (comment density, docstring length, naming shape, sentence or paragraph count) — as an absolute rule buys a guardrail current models no longer need, and is wrong for the cases the rule did not anticipate. Anthropic removed its own example from the Claude Code system prompt: \"Never write multi-paragraph docstrings or multi-line comment blocks — one short line max\" was replaced by \"Write code that reads like the surrounding code: match its comment density, naming, and idiom.\"",
|
|
"appliesTo": "claude-md",
|
|
"recommendation": "Where an absolute governs a form decision rather than a local fact, state the outcome you want and let the model judge the instance. This does not apply to safety rules, tool or version facts, or a house style you hold deliberately — those are the reason the claim is not machine-checkable.",
|
|
"confidence": "confirmed",
|
|
"category": "judgment-fit",
|
|
"lensCheck": null,
|
|
"note": "KNOWLEDGE ONLY — no detector, by measurement (docs/b2-judgment-lens-fasit.local.md §9). Across 409 real CLAUDE.md files (38488 lines, 8689 prose blocks) the class fired 7 times and all 7 were false positives; an independent grep in both word orders found 5 lines and 1 line respectively, none an instruction. Where the shape does occur — 45 matching lines across 4755 skill/agent/command files — it is the author's editorial policy (emoji, sentence length, slide titles), which nothing in the text separates from a vendor's over-tight guardrail. Precision-first: no CA-OPT code was allocated and OPT next-free stays 2.",
|
|
"source": {
|
|
"url": "https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models",
|
|
"title": "The new rules of context engineering for Claude 5 generation models",
|
|
"published": "2026-07-24",
|
|
"verified": "2026-08-12"
|
|
}
|
|
}
|
|
]
|
|
}
|