feat(knowledge): model + effort routing enter the register, dated [skip-docs]

C1: two entries the optimization lens can cite for the model/effort axis,
both read out of the primary sources in this session rather than from the
plan's 2026-07-14 summary of them.

Verifying corrected the plan's own numbers: effort is settable in SIX
places, not five (/effort, the /model slider, --effort,
CLAUDE_CODE_EFFORT_LEVEL, settings effortLevel, skill/subagent
frontmatter), the default is high on every supporting model EXCEPT Opus
4.7 (xhigh), and the level count is model-dependent (Opus 4.6 and Sonnet
4.6 have no xhigh).

- BP-MODEL-001: subagent `model` defaults to `inherit`, so a subagent
  that names no model costs what the session costs; the documented pin
  is overridable by CLAUDE_CODE_SUBAGENT_MODEL and per-invocation model
- BP-MODEL-002: effort is an axis separate from model choice, and higher
  is not universally better (`max` "may show diminishing returns and is
  prone to overthinking")
- both carry the 2026-07-07 model/effort blog as a corroborating source
  with a real `published` date, so B1's evidence-age rule has teeth:
  measured stale with reasons ['evidence-age'] at a reference date 378
  days past publication while their verified stamps are pristine. The
  docs pages themselves get NO published date — they carry none, and
  guessing one in the field whose whole job is dating evidence is the
  lie the rule exists to catch
- new blanket guard: every corroborating source must carry a parseable
  published date. Without it newestEvidenceMs() returns null and the
  entry stays green on evidence of any age — a silent hole. Seen red
  against its own defect before it was trusted
- knowledge-refresh-cli's stale branch no longer expires on every new
  entry: its reference date has to sit after every `verified` stamp, was
  bumped once for BP-SUB-001 and would have needed a third bump now, so
  it moves to a date no stamp can reach

Register 14 -> 16 entries. Frozen v5.0.0 snapshots untouched (no scanner
output changes); suite 1579/0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NKjojcdYYiCQP5AudUyQ5e
This commit is contained in:
Kjell Tore Guttormsen 2026-08-10 04:36:09 +02:00
commit e861e63a7b
3 changed files with 103 additions and 11 deletions

View file

@ -213,6 +213,56 @@
"note": "Near-verbatim coverage: 'briefly describe what your repo is for, but spend most of the tokens on gotchas inside of the codebase'; 'Avoid stating the obvious things Claude should know by looking at your file system or your repo.'" "note": "Near-verbatim coverage: 'briefly describe what your repo is for, but spend most of the tokens on gotchas inside of the codebase'; 'Avoid stating the obvious things Claude should know by looking at your file system or your repo.'"
} }
] ]
},
{
"id": "BP-MODEL-001",
"claim": "A subagent's `model` frontmatter field defaults to `inherit`, so a subagent that names no model runs on the main conversation's model. Routing mechanical or read-only subagents to a cheaper alias (`haiku`, `sonnet`) while the orchestrating session keeps the stronger model is the documented way to control cost. The pin is not absolute: Claude Code resolves the model as CLAUDE_CODE_SUBAGENT_MODEL, then a per-invocation `model` parameter, then the frontmatter, then the main conversation's model.",
"mechanism": "model",
"appliesTo": "agent",
"recommendation": "Set `model:` explicitly on subagents whose work is mechanical or read-only (search, extraction, summarisation) and leave the orchestrator on the stronger model. Omitting the field is not a neutral default — it inherits, so every subagent costs what the session costs.",
"confidence": "confirmed",
"severity": "low",
"category": "model-fit",
"lensCheck": null,
"source": {
"url": "https://code.claude.com/docs/en/sub-agents",
"title": "Create custom subagents — supported frontmatter fields / choose a model",
"verified": "2026-08-10"
},
"sources": [
{
"url": "https://claude.com/blog/claude-model-and-effort-level-in-claude-code",
"title": "Choosing a Claude model and effort level in Claude Code",
"published": "2026-07-07",
"verified": "2026-08-10",
"note": "Verbatim: 'Pick a smaller model when the work is routine. For example, edits you can describe precisely, mechanical changes, or questions about code that's already in context.'"
}
]
},
{
"id": "BP-MODEL-002",
"claim": "Reasoning effort is an axis separate from model choice: five levels (`low`, `medium`, `high`, `xhigh`, `max`) on current models, four on Opus 4.6 and Sonnet 4.6, which omit `xhigh`; the default is `high` on every model that supports effort except Opus 4.7, which defaults to `xhigh`. Higher is not universally better — `max` \"can improve performance on demanding tasks but may show diminishing returns and is prone to overthinking\". Effort is settable in six places: `/effort`, the slider in `/model`, the `--effort` flag, CLAUDE_CODE_EFFORT_LEVEL, `effortLevel` in settings, and `effort:` in skill or subagent frontmatter; the environment variable takes precedence over all of them.",
"mechanism": "effort",
"appliesTo": "agent",
"recommendation": "Treat effort as a per-task dial rather than a global maximum: pin a lower `effort:` in the frontmatter of mechanical skills and subagents, and reserve `xhigh`/`max` for work whose product is judgment. The scale is calibrated per model, so the same level name is not the same amount of thinking across models — and CLAUDE_CODE_EFFORT_LEVEL silently overrides every other source, so verify which level is actually in force.",
"confidence": "confirmed",
"severity": "low",
"category": "model-fit",
"lensCheck": null,
"source": {
"url": "https://code.claude.com/docs/en/model-config",
"title": "Model configuration — adjust effort level / set the effort level",
"verified": "2026-08-10"
},
"sources": [
{
"url": "https://claude.com/blog/claude-model-and-effort-level-in-claude-code",
"title": "Choosing a Claude model and effort level in Claude Code",
"published": "2026-07-07",
"verified": "2026-08-10",
"note": "Verbatim: 'Claude will be more predisposed to double-checking additional hypotheses or verifying correctness at higher effort levels, but it generally won't artificially inflate usage for simple tasks at higher effort levels.'; 'In fact, our team pays close attention to \"overthinking\" during model training as it degrades effectiveness.'"
}
]
} }
] ]
} }

View file

@ -73,6 +73,40 @@ describe('bundled register integrity (Verifiseringsplikt)', () => {
assert.match(e.source.verified, /^\d{4}-\d{2}-\d{2}$/, `${e.id} bad verified date`); assert.match(e.source.verified, /^\d{4}-\d{2}-\d{2}$/, `${e.id} bad verified date`);
} }
}); });
// A corroborating source without a published date is a SILENT hole: newestEvidenceMs()
// returns null for the entry and the evidence-age rule cannot judge evidence it cannot
// date, so the entry stays green on evidence of any age. Assert the blanket invariant,
// not the entries that happen to have one today.
it('every corroborating source carries a published date (evidence-age rule has teeth)', () => {
for (const e of reg.entries) {
if (!Array.isArray(e.sources)) continue;
e.sources.forEach((s, i) => {
assert.match(
String(s && s.published),
/^\d{4}-\d{2}-\d{2}$/,
`${e.id} sources[${i}] (${s && s.url}) missing a parseable published date`
);
});
}
});
it('carries the model-routing entries (C1), each dated by a published source', () => {
for (const id of ['BP-MODEL-001', 'BP-MODEL-002']) {
const e = getEntry(reg, id);
assert.ok(e, `${id} missing from the bundled register`);
assert.equal(e.confidence, 'confirmed', `${id} must be confirmed`);
assert.equal(e.category, 'model-fit', `${id} wrong category`);
assert.ok(
e.source.url.startsWith('https://code.claude.com/docs/'),
`${id} primary source must be the official docs, got ${e.source.url}`
);
assert.ok(
Array.isArray(e.sources) && e.sources.some((s) => s.published),
`${id} must carry a corroborating source with a published date (B1 evidence-age rule)`
);
}
});
}); });
describe('validateRegister (negative cases)', () => { describe('validateRegister (negative cases)', () => {

View file

@ -9,19 +9,24 @@ import { tmpdir } from 'node:os';
const __dirname = fileURLToPath(new URL('.', import.meta.url)); const __dirname = fileURLToPath(new URL('.', import.meta.url));
const CLI = resolve(__dirname, '../../scanners/knowledge-refresh-cli.mjs'); const CLI = resolve(__dirname, '../../scanners/knowledge-refresh-cli.mjs');
// The reference date must sit AFTER the newest entry's `verified` date, so every // A fixed reference date keeps both branches reachable regardless of the real clock.
// entry is at least one day old and both the fresh and stale branches are
// reachable deterministically, regardless of the real clock.
// //
// It used to be 2026-06-21, on the premise that every seed entry was verified // REF is used by the fresh branch only, and needs no maintenance: an entry verified
// 2026-06-20. That premise expired when BP-SUB-001 was added with a genuine // after it has a negative age and still counts as fresh.
// 2026-07-31 verification date — an entry verified *after* the reference date has
// a negative age and counts as fresh, so `--stale-after 0` no longer emptied the
// fresh bucket. Backdating the entry to fit the test would have been a lie about
// when its source was checked; moving the reference date is the honest fix.
// Bump this again when a newer entry lands.
const REF = '2026-08-01'; const REF = '2026-08-01';
// The stale branch needs a date strictly after EVERY entry's `verified` stamp — with
// `--stale-after 0` an entry verified later than the reference date has a negative age,
// lands in the fresh bucket, and empties nothing.
//
// This was REF itself, bumped 2026-06-21 → 2026-08-01 when BP-SUB-001 landed with a
// genuine 2026-07-31 stamp, and it expired a second time when BP-MODEL-001/002 landed
// verified 2026-08-10. Backdating an entry to fit the test would lie about when its
// source was read, and re-deriving the date from the register would just assert the
// register against itself. A date no verification stamp can reach satisfies the
// requirement by construction, so no future entry expires it.
const REF_AFTER_EVERY_STAMP = '2999-01-01';
function runCli(extraArgs) { function runCli(extraArgs) {
try { try {
const stdout = execFileSync('node', [CLI, ...extraArgs], { encoding: 'utf-8', timeout: 15000 }); const stdout = execFileSync('node', [CLI, ...extraArgs], { encoding: 'utf-8', timeout: 15000 });
@ -41,7 +46,10 @@ describe('knowledge-refresh-cli — exit codes', () => {
it('exits 1 (advisory) when one or more entries are stale', () => { it('exits 1 (advisory) when one or more entries are stale', () => {
// stale-after 0 → anything verified before the reference date is stale. // stale-after 0 → anything verified before the reference date is stale.
const { status, stdout } = runCli(['--reference-date', REF, '--stale-after', '0']); const { status, stdout } = runCli([
'--reference-date', REF_AFTER_EVERY_STAMP,
'--stale-after', '0',
]);
assert.equal(status, 1); assert.equal(status, 1);
const out = JSON.parse(stdout); const out = JSON.parse(stdout);
assert.ok(out.counts.stale > 0); assert.ok(out.counts.stale > 0);