config-audit/scanners/lib/finding-codes.mjs
Kjell Tore Guttormsen 9ae4be26d2 feat(scanners): model/effort routing becomes a lever, not a 25th dimension (C4)
New GAP finding CA-GAP-028: authored subagents exist and not one of them names
`model:` or `effort:`, so every delegated task runs on the main conversation's
model (`model` defaults to `inherit`). Cites BP-MODEL-001/002, landed in C1.
`whats-active` and `manifest` now carry `model`/`effort` per agent.

Shipped as a conditional LEVER rather than a 25th dimension, and the choice was
made by measurement: as a t3 dimension the agent-less marketplace-medium fixture
would count it vacuously-present, moving the denominators 41->42 and utilization
44->45 — which flips `segment` "Developing"->"Competent" in the frozen v5.0.0
posture baseline, a field strip-retired-gap.mjs does not mask. A lever never
enters those denominators. The general rule is now an invariant in CLAUDE.md.

One check across both axes, not one per axis: it fires only when neither is used
anywhere, so a deliberate everything-on-one-model policy stays silent. Cost is
recall, chosen for precision.

Found by dogfooding, fixed red-first: `model: inherit` is the documented default
spelled out, so it must not count as routing — otherwise a config opts out of the
opportunity without changing anything real.

Two pre-existing defects surfaced and closed on the way:
- The humanizer guard asserted TRANSLATIONS.GAP.static EQUALS the dimension
  titles, which forbade humanizing any lever — all three existing levers fell
  through to the generic "feature opportunity" default, wrong for a budget lever.
  Guard now requires coverage of every emittable title, seen red against those
  three before the entries were written.
- Two hand-written copies of the lever list (finding-codes guard, humanizer
  guard) merged into one exported LEVERS registry carrying code AND title.
- suppression-validation pinned CA-GAP-028 as an unoccupied number; C4 claimed
  it. Fixed structurally with a derived first-free id, not by picking a new
  literal — same class as #60's "bump this again".

Suite 1596/0. Frozen v5.0.0 snapshots untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq3nye21RVYk4pZLeT8pGz
2026-08-10 05:07:23 +02:00

320 lines
11 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/**
* Finding-code registry — the authority for the {NNN} in `CA-{SCANNER}-{NNN}`.
*
* A finding ID names the CHECK, not the finding's position in a run (M-BUG-28).
* Before this registry, `{NNN}` came from an emission counter, so the same check
* carried different IDs on different configurations: fixing an unrelated earlier
* gap silently renumbered every later one, and a `.config-audit-ignore` entry
* retargeted to a neighbouring finding without the user changing anything.
*
* Rules for editing this file:
*
* 1. A number, once published, belongs to its check forever. Adding a check
* takes the next free number for that scanner — never the next source-order
* position, and never a number listed in RETIRED.
* 2. Removing a check moves its key to RETIRED. The number is never reissued;
* a user's suppression must go dead rather than quietly point at a
* different finding. (D1 retired GAP `t3_8` under the old scheme, which is
* the incident that motivated the registry.)
* 3. Several call sites may share one code when they are arms of one check —
* e.g. the forward/reverse arms of a permission conflict. Duplicate
* emission is legal; a finding is identified by (id, file, line).
* 4. Numbers below are NOT all source order: the ones marked "documented"
* are pinned by README / command copy that shipped before the registry.
*
* GAP keys are the `GAP_CHECKS[].id` values from `feature-gap-scanner.mjs`,
* which were already stable. They are declared here rather than derived, so
* numbers live in exactly one place; `tests/lib/finding-codes.test.mjs` binds
* the two together instead of a second copy of the table drifting.
*/
/**
* @type {Record<string, Record<string, number>>}
* scanner prefix → check key → number
*/
export const FINDING_CODES = {
// ── CML: claude-md-linter (source order) ────────────────────────────────
// `over-char-budget` has two call sites: the conservative 200k anchor and the
// `--context-window` calibrated variant. One check, one code.
CML: {
'no-claude-md': 1,
'nested-not-reinjected': 2,
'over-500-lines': 3,
'over-200-lines': 4,
'over-char-budget': 5,
'nearly-empty': 6,
'missing-sections': 7,
'no-headings': 8,
'deep-relative-import': 9,
'html-comments': 10,
'repeated-content': 11,
'todo-markers': 12,
},
// ── SET: settings-validator (source order) ──────────────────────────────
SET: {
'invalid-json': 1,
'key-typo': 2,
'deprecated-key': 3,
'type-mismatch': 4,
'invalid-effort-level': 5,
'missing-schema': 6,
'no-deny-rules': 7,
'no-allow-rules': 8,
'many-additional-dirs': 9,
'automode-not-object': 10,
'automode-unknown-subkey': 11,
'automode-subkey-not-string-array': 12,
'automode-in-shared-settings': 13,
'hooks-as-array': 14,
},
// ── HKV: hook-validator (source order) ──────────────────────────────────
HKV: {
'invalid-json': 1,
'hooks-not-object': 2,
'unknown-event': 3,
'handlers-not-array': 4,
'matcher-not-string': 5,
'missing-hooks-array': 6,
'invalid-handler-type': 7,
'script-not-found': 8,
'verbose-output': 9,
'unfiltered-additional-context': 10,
'timeout-not-number': 11,
'timeout-out-of-range': 12,
},
// ── RUL: rules-validator (source order) ─────────────────────────────────
RUL: {
'no-frontmatter': 1,
'globs-instead-of-paths': 2,
'pattern-matches-nothing': 3,
'nearly-empty': 4,
'large-unscoped': 5,
'large-scoped-lost-after-compaction': 6,
'not-markdown': 7,
},
// ── MCP: mcp-config-validator (source order) ────────────────────────────
MCP: {
'invalid-json': 1,
'unknown-server-type': 2,
'sse-transport': 3,
'unreferenced-env-var': 4,
'unknown-server-field': 5,
},
// ── IMP: import-resolver (source order) ─────────────────────────────────
IMP: {
'tilde-path': 1,
'broken-link': 2,
'circular-reference': 3,
'deep-chain': 4,
},
// ── CNF: conflict-detector ──────────────────────────────────────────────
// `permission-allow-deny` covers both arms (allow-in-A/deny-in-B and reverse).
CNF: {
'settings-key-conflict': 1,
'permission-allow-deny': 2,
'duplicate-hook': 3,
},
// ── DIS: disabled-in-schema-scanner (source order) ──────────────────────
DIS: {
'deny-and-allow': 1,
'ineffective-allow-wildcard': 2,
'forbidden-param-deny': 3,
'forbidden-param-allow': 4,
},
// ── CPS: cache-prefix-scanner (CPS-001 documented) ──────────────────────
CPS: {
'volatile-in-prefix': 1,
'volatile-in-import': 2,
},
// ── COL: collision-scanner (source order) ───────────────────────────────
COL: {
'skill-user-vs-plugin': 1,
'skill-multi-plugin': 2,
},
// ── AGT: agent-listing-scanner (both documented; source order matches) ──
AGT: {
'description-bloat': 1,
'aggregate-listing-budget': 2,
},
// ── OST: output-style-scanner (all three documented) ────────────────────
OST: {
'strips-coding-instructions': 1,
'plugin-forces-style': 2,
'style-not-found': 3,
},
// ── OPT: optimization-lens-scanner (documented) ─────────────────────────
OPT: {
'procedure-should-be-skill': 1,
},
// ── SKL: skill-listing-scanner (all three documented) ───────────────────
// `aggregate-listing-budget` has two call sites: the conservative 200k anchor
// and the calibrated `--context-window` variant. One check, one code.
SKL: {
'description-over-cap': 1,
'aggregate-listing-budget': 2,
'oversized-body': 3,
},
// ── TOK: token-hotspots ─────────────────────────────────────────────────
// 1/2/3/5/6 are documented (README + commands/tokens.md). `mcp-schema-deferral`
// is documented as 006 although it is the 8th call site in source order, so
// `cascade-over-budget` and `stale-plugin-cache` take the free 7 and 8.
TOK: {
'volatile-top': 1,
'redundant-permissions': 2,
'deep-import-chain': 3,
'bloated-skill-description': 4,
'mcp-schema-budget': 5,
'mcp-schema-deferral': 6,
'cascade-over-budget': 7,
'stale-plugin-cache': 8,
},
// ── PLH: plugin-health-scanner ──────────────────────────────────────────
// 15 and 16 are documented (README v5.4.0 entry) but sit at source positions
// 3 and 4; the remaining checks take {1…14, 17, 18, 19} in source order.
PLH: {
'invalid-plugin-json': 1,
'missing-required-field': 2,
'missing-plugin-json': 3,
'claude-md-missing-section': 4,
'missing-claude-md': 5,
'command-missing-frontmatter': 6,
'command-missing-field': 7,
'agent-missing-frontmatter': 8,
'agent-missing-field': 9,
'agent-ignored-key': 10,
'hooks-json-invalid-structure': 11,
'hooks-json-array': 12,
'hooks-json-invalid': 13,
'unknown-plugin-file': 14,
'plugin-json-shadows-default': 15,
'skills-array-entry': 16,
'no-plugins-found': 17,
'command-name-collision': 18,
'namespace-collision': 19,
},
// ── GAP: feature-gap-scanner ────────────────────────────────────────────
// Keys are GAP_CHECKS[].id for dimensions, and the lever code for the
// conditional levers the scanner emits after the loop. Numbers 124 happen to
// follow the current table order because that is how the dimensions were first
// published — NOT because position determines the number. A new check takes the
// next free number wherever it sits in the file (M-BUG-28).
GAP: {
t1_1: 1,
t1_2: 2,
t1_3: 3,
t1_4: 4,
t1_5: 5,
t2_1: 6,
t2_2: 7,
t2_3: 8,
t2_4: 9,
t2_5: 10,
t2_6: 11,
t2_7: 12,
t3_1: 13,
t3_2: 14,
t3_3: 15,
t3_4: 16,
t3_5: 17,
t3_6: 18,
t3_7: 19,
t4_1: 20,
t4_2: 21,
t4_3: 22,
t4_4: 23,
t4_5: 24,
'bundled-skills-lever': 25,
'cli-over-mcp-lever': 26,
'filter-hook-output-lever': 27,
'agent-model-routing-lever': 28,
},
};
/**
* Keys withdrawn from a scanner. Their numbers are never reissued, so a stale
* suppression goes dead instead of silently naming a different check.
* @type {Record<string, string[]>}
*/
export const RETIRED_CODES = {
// D1 (4027cdc, 2026-08-09): "No autoMode classifier" — /doctor Check 8 covers
// auto mode with usage-weighted judgement, so the nudge went. The number it
// occupied under the old counter scheme is not reused.
GAP: ['t3_8'],
};
/**
* Resolve a check key to its published number.
* Throws rather than falling back: a fallback would let a half-converted scanner
* ship IDs that look valid, which is the silent-degradation class this registry
* exists to remove.
* @param {string} scanner - scanner prefix, e.g. 'GAP'
* @param {string} code - check key, e.g. 't3_7'
* @returns {number}
*/
export function codeNumber(scanner, code) {
const table = FINDING_CODES[scanner];
if (!table) {
throw new Error(`finding(): unknown scanner "${scanner}" — add it to FINDING_CODES`);
}
if (code === undefined || code === null || code === '') {
throw new Error(`finding(): missing "code" for scanner ${scanner} — every finding must name its check`);
}
if (!Object.prototype.hasOwnProperty.call(table, code)) {
const retired = (RETIRED_CODES[scanner] || []).includes(code);
throw new Error(
retired
? `finding(): check "${code}" is RETIRED for ${scanner} — retired numbers are never reissued`
: `finding(): undeclared check "${code}" for ${scanner} — add it to FINDING_CODES with the next free number`
);
}
return table[code];
}
/**
* Render a finding ID from a check key.
* @param {string} scanner
* @param {string} code
* @returns {string} e.g. 'CA-GAP-019'
*/
export function findingId(scanner, code) {
return `CA-${scanner}-${String(codeNumber(scanner, code)).padStart(3, '0')}`;
}
/**
* Every declared ID, as a flat set — used to validate suppression patterns so a
* stale pin is reported instead of silently matching nothing.
* @returns {Set<string>}
*/
export function allFindingIds() {
const ids = new Set();
for (const [scanner, table] of Object.entries(FINDING_CODES)) {
for (const n of Object.values(table)) {
ids.add(`CA-${scanner}-${String(n).padStart(3, '0')}`);
}
}
return ids;
}
/**
* Scanner prefixes the registry knows about.
* @returns {string[]}
*/
export function knownScanners() {
return Object.keys(FINDING_CODES);
}