fix(scanners)!: a finding ID names the check, not the emission (M-BUG-28)

BREAKING CHANGE: the {NNN} in CA-{SCANNER}-{NNN} identifies the check that
produced the finding. It used to be the finding's position in that scanner's
output for that run, which made it unstable across CONFIGURATIONS, not just
across releases as STATE framed it. Measured on two fixtures: "No custom
subagents" was CA-GAP-007 on minimal-project and CA-GAP-004 on healthy-project.
A user who fixed an unrelated earlier gap silently renumbered every later one,
so a .config-audit-ignore pin retargeted to a neighbouring finding with no
version change at all.

Second measured arm: README already documented the opposite scheme. It and the
scanner headers describe ~20 numbers as check codes (CA-SKL-003 = oversized
body, CA-PLH-015 = folder shadowing, CA-TOK-006 = schema deferral), and the
counter could only produce those in the all-fire case -- source-order positions
are 4, 3 and 8. The documentation described the scheme; the implementation was
what was wrong. Every published number is preserved by construction and pinned
exhaustively in tests/lib/finding-codes.test.mjs.

scanners/lib/finding-codes.mjs is the single authority. Every finding() call
passes a `code`; an undeclared or missing one THROWS. No counter fallback --
that would reproduce D1's findGapId -> 'unknown' silent degradation and let a
half-converted scanner ship IDs that look valid. findingCounter/resetCounter
are deleted outright, not left as no-ops. Retirement is now a mechanism:
RETIRED_CODES tombstones a withdrawn key so its number is never reissued,
seeded with GAP t3_8 -- the D1 removal that opened this chunk.

IDs are consequently NOT unique per finding: one check failing in three files
emits three findings sharing an ID. That inverts which consumer is correct, so
every f.id/findingId site was classified before the change. diff-engine and
most of fix-engine already keyed on scanner+title+file (drift was never lying);
fix-engine's verification did not, and keyed on the ID alone -- fixing one of
two sibling instances marked both fixed, and the untouched one, still present
in the re-scan, was reported as a REGRESSION. Red test first, then keyed on
(findingId, file), which both planFixes and applyFixes already carry.
plugin-health's crossIds Set was measured and is a clean negative: cross
findings are allFindings.slice(crossPluginStart) and codes 18/19 are emitted
only in that tail, so the partition holds by construction.

unknownSuppressions() reports a pin that names no declared check, in the
--output-file payload (ux-rules rule 2 -- a stderr-only warning is invisible to
the commands) and only when one exists, so a clean config is byte-identical.
That is what makes the break safe: a stale pin goes loud instead of dying quiet.

Frozen tests/snapshots/v5.0.0/ untouched on disk. IDs are masked out of that
comparison (mask-finding-ids.mjs) rather than re-derived -- re-deriving
positional IDs would assert the retired scheme against itself, and #58's
isGapEntry off-by-one is the measured example of that misfiring. The dead
re-derivation is removed from strip-retired-gap.mjs. default-output snapshots
re-approved after confirming the diff is IDs and nothing else.

Guards, each seen red against its own defect: a missing code (scanner errors
out mid-sweep), an orphan declaration, a resurrected retired key, and a
documented ID naming no check. The sweep asserts the union across all 16
scanners, never per scanner -- a per-scanner assertion goes green on a partial
conversion.

Fasit written before implementation: docs/mbug28-id-semantics-fasit.local.md,
including one correction made before running (CML has 12 checks over 13 call
sites -- the anchored and calibrated char-budget arms are one check, which a
repeated-title sweep found and my call-site count had missed).

Suite 1535 -> 1573, 0 failing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MyqCQKK2ornJ1jFWwqx17E

tests/lib/documented-ids-resolve.test.mjs

@@ -0,0 +1,50 @@
+/**
+ * Every finding ID printed in shipped prose must name a declared check.
+ *
+ * README, CLAUDE.md, the command files and the scanner header comments are a
+ * second copy of the registry, and two copies of one table drift (#57/C2). This
+ * is the direction that matters to a user: a documented `CA-PLH-016` that no
+ * check emits sends them to write a suppression that can never match.
+ */
+
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { readFile, readdir } from 'node:fs/promises';
+import { resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { allFindingIds } from '../../scanners/lib/finding-codes.mjs';
+
+const ROOT = resolve(fileURLToPath(new URL('.', import.meta.url)), '../..');
+const ID_RE = /CA-[A-Z]{2,4}-\d{3}/g;
+
+async function documentedFiles() {
+  const files = ['README.md', 'CLAUDE.md'];
+  for (const dir of ['commands', 'scanners', 'scanners/lib']) {
+    const entries = await readdir(resolve(ROOT, dir));
+    for (const e of entries) {
+      if (e.endsWith('.md') || e.endsWith('.mjs')) files.push(`${dir}/${e}`);
+    }
+  }
+  return files;
+}
+
+describe('documented finding IDs', () => {
+  it('all resolve to a declared check', async () => {
+    const ids = allFindingIds();
+    const dangling = [];
+
+    for (const rel of await documentedFiles()) {
+      let src;
+      try {
+        src = await readFile(resolve(ROOT, rel), 'utf-8');
+      } catch {
+        continue;
+      }
+      for (const [id] of src.matchAll(ID_RE)) {
+        if (!ids.has(id)) dangling.push(`${rel}: ${id}`);
+      }
+    }
+
+    assert.deepEqual([...new Set(dangling)], [], 'documented ID names no declared check');
+  });
+});

tests/lib/finding-codes.test.mjs

@@ -0,0 +1,113 @@
+/**
+ * Registry invariants for the finding-code scheme (M-BUG-28).
+ *
+ * These are blanket assertions over the whole registry, never a relation between
+ * two chosen entries: a per-entry check goes green on a partial conversion, which
+ * is the failure mode #51/#57/#58 kept reproducing.
+ */
+
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { FINDING_CODES, RETIRED_CODES, codeNumber, findingId, allFindingIds } from '../../scanners/lib/finding-codes.mjs';
+import { GAP_CHECKS } from '../../scanners/feature-gap-scanner.mjs';
+
+describe('finding-code registry', () => {
+  it('gives every check a distinct number within its scanner', () => {
+    const collisions = [];
+    for (const [scanner, table] of Object.entries(FINDING_CODES)) {
+      const seen = new Map();
+      for (const [key, n] of Object.entries(table)) {
+        if (seen.has(n)) collisions.push(`${scanner}: ${seen.get(n)} and ${key} both claim ${n}`);
+        seen.set(n, key);
+      }
+    }
+    assert.deepEqual(collisions, []);
+  });
+
+  it('uses positive integers only', () => {
+    const bad = [];
+    for (const [scanner, table] of Object.entries(FINDING_CODES)) {
+      for (const [key, n] of Object.entries(table)) {
+        if (!Number.isInteger(n) || n < 1) bad.push(`${scanner}.${key} = ${n}`);
+      }
+    }
+    assert.deepEqual(bad, []);
+  });
+
+  it('never lets a retired key stay active', () => {
+    const resurrected = [];
+    for (const [scanner, keys] of Object.entries(RETIRED_CODES)) {
+      for (const key of keys) {
+        if (FINDING_CODES[scanner] && key in FINDING_CODES[scanner]) {
+          resurrected.push(`${scanner}.${key}`);
+        }
+      }
+    }
+    assert.deepEqual(resurrected, []);
+  });
+
+  it('never reissues a retired number', () => {
+    // The whole point of the tombstone: D1 retired GAP t3_8 and shifted three
+    // live IDs down by one. Reusing a retired number would repeat that silently.
+    assert.ok(RETIRED_CODES.GAP.includes('t3_8'), 'D1 tombstone missing');
+  });
+
+  it('declares exactly the GAP dimensions the scanner ships, plus its levers', () => {
+    const declared = new Set(Object.keys(FINDING_CODES.GAP));
+    const shipped = GAP_CHECKS.map((g) => g.id);
+
+    const missing = shipped.filter((id) => !declared.has(id));
+    assert.deepEqual(missing, [], 'a GAP dimension has no declared code');
+
+    const levers = ['bundled-skills-lever', 'cli-over-mcp-lever', 'filter-hook-output-lever'];
+    const orphans = [...declared].filter((k) => !shipped.includes(k) && !levers.includes(k));
+    assert.deepEqual(orphans, [], 'a declared GAP code matches no shipped dimension');
+  });
+
+  it('throws on an undeclared code instead of inventing an ID', () => {
+    assert.throws(() => codeNumber('GAP', 'nope'), /undeclared check/);
+    assert.throws(() => codeNumber('NOSUCH', 't1_1'), /unknown scanner/);
+    assert.throws(() => codeNumber('GAP', undefined), /missing "code"/);
+  });
+
+  it('names retirement explicitly when a retired key is used', () => {
+    assert.throws(() => codeNumber('GAP', 't3_8'), /RETIRED/);
+  });
+
+  it('renders the published ID format', () => {
+    assert.equal(findingId('GAP', 't1_1'), 'CA-GAP-001');
+    assert.equal(findingId('PLH', 'skills-array-entry'), 'CA-PLH-016');
+    assert.ok(allFindingIds().has('CA-SKL-003'));
+  });
+});
+
+describe('published finding IDs (pinned exhaustively — README is a contract)', () => {
+  // Every number that shipped in README, CLAUDE.md or command copy before the
+  // registry existed. Spot-checking these would let a renumber through.
+  const PUBLISHED = [
+    ['SKL', 'description-over-cap', 'CA-SKL-001'],
+    ['SKL', 'aggregate-listing-budget', 'CA-SKL-002'],
+    ['SKL', 'oversized-body', 'CA-SKL-003'],
+    ['OST', 'strips-coding-instructions', 'CA-OST-001'],
+    ['OST', 'plugin-forces-style', 'CA-OST-002'],
+    ['OST', 'style-not-found', 'CA-OST-003'],
+    ['TOK', 'volatile-top', 'CA-TOK-001'],
+    ['TOK', 'redundant-permissions', 'CA-TOK-002'],
+    ['TOK', 'deep-import-chain', 'CA-TOK-003'],
+    ['TOK', 'mcp-schema-budget', 'CA-TOK-005'],
+    ['TOK', 'mcp-schema-deferral', 'CA-TOK-006'],
+    ['PLH', 'plugin-json-shadows-default', 'CA-PLH-015'],
+    ['PLH', 'skills-array-entry', 'CA-PLH-016'],
+    ['OPT', 'procedure-should-be-skill', 'CA-OPT-001'],
+    ['AGT', 'description-bloat', 'CA-AGT-001'],
+    ['AGT', 'aggregate-listing-budget', 'CA-AGT-002'],
+    ['CPS', 'volatile-in-prefix', 'CA-CPS-001'],
+    ['COL', 'skill-user-vs-plugin', 'CA-COL-001'],
+  ];
+
+  for (const [scanner, key, expected] of PUBLISHED) {
+    it(`${expected} still names ${key}`, () => {
+      assert.equal(findingId(scanner, key), expected);
+    });
+  }
+});

tests/lib/output.test.mjs

@@ -1,36 +1,34 @@
 import { describe, it, beforeEach } from 'node:test';
 import assert from 'node:assert/strict';
-import { finding, scannerResult, envelope, resetCounter } from '../../scanners/lib/output.mjs';
-
-describe('resetCounter', () => {
-  it('resets finding ID counter', () => {
-    resetCounter();
-    const f1 = finding({ scanner: 'TST', severity: 'info', title: 'a', description: 'b' });
-    assert.strictEqual(f1.id, 'CA-TST-001');
-    resetCounter();
-    const f2 = finding({ scanner: 'TST', severity: 'info', title: 'c', description: 'd' });
-    assert.strictEqual(f2.id, 'CA-TST-001');
-  });
-});
+import { finding, scannerResult, envelope } from '../../scanners/lib/output.mjs';
 
 describe('finding', () => {
-  beforeEach(() => { resetCounter(); });
-
-  it('generates correct ID format', () => {
-    const f = finding({ scanner: 'CML', severity: 'high', title: 't', description: 'd' });
+    it('generates correct ID format', () => {
+    const f = finding({ scanner: 'CML', code: 'no-claude-md', severity: 'high', title: 't', description: 'd' });
     assert.match(f.id, /^CA-CML-\d{3}$/);
   });
 
-  it('auto-increments IDs', () => {
-    const f1 = finding({ scanner: 'CML', severity: 'info', title: 'a', description: 'b' });
-    const f2 = finding({ scanner: 'CML', severity: 'info', title: 'c', description: 'd' });
-    assert.strictEqual(f1.id, 'CA-CML-001');
-    assert.strictEqual(f2.id, 'CA-CML-002');
+  it('gives a check the same ID however many findings precede it', () => {
+    // The ID names the check, not the emission (M-BUG-28). Emitting other
+    // findings first must not move it.
+    const first = finding({ scanner: 'CML', code: 'todo-markers', severity: 'info', title: 'a', description: 'b' });
+    finding({ scanner: 'CML', code: 'no-headings', severity: 'info', title: 'c', description: 'd' });
+    finding({ scanner: 'CML', code: 'html-comments', severity: 'info', title: 'e', description: 'f' });
+    const again = finding({ scanner: 'CML', code: 'todo-markers', severity: 'info', title: 'a', description: 'b' });
+    assert.strictEqual(first.id, again.id);
+  });
+
+  it('refuses to invent an ID when the check is not named', () => {
+    assert.throws(
+      () => finding({ scanner: 'CML', severity: 'info', title: 'a', description: 'b' }),
+      /missing "code"/,
+    );
   });
 
   it('includes all required fields', () => {
     const f = finding({
       scanner: 'SET',
+      code: 'invalid-json',
       severity: 'critical',
       title: 'Test',
       description: 'Desc',
@@ -54,7 +52,7 @@ describe('finding', () => {
   });
 
   it('defaults nullable fields to null', () => {
-    const f = finding({ scanner: 'TST', severity: 'info', title: 't', description: 'd' });
+    const f = finding({ scanner: 'SET', code: 'missing-schema', severity: 'info', title: 't', description: 'd' });
     assert.strictEqual(f.file, null);
     assert.strictEqual(f.line, null);
     assert.strictEqual(f.evidence, null);
@@ -65,16 +63,14 @@ describe('finding', () => {
 });
 
 describe('scannerResult', () => {
-  beforeEach(() => { resetCounter(); });
-
-  it('counts severity correctly', () => {
+    it('counts severity correctly', () => {
     const findings = [
-      finding({ scanner: 'TST', severity: 'critical', title: 'a', description: 'b' }),
-      finding({ scanner: 'TST', severity: 'high', title: 'c', description: 'd' }),
-      finding({ scanner: 'TST', severity: 'high', title: 'e', description: 'f' }),
-      finding({ scanner: 'TST', severity: 'info', title: 'g', description: 'h' }),
+      finding({ scanner: 'SET', code: 'invalid-json', severity: 'critical', title: 'a', description: 'b' }),
+      finding({ scanner: 'SET', code: 'key-typo', severity: 'high', title: 'c', description: 'd' }),
+      finding({ scanner: 'SET', code: 'deprecated-key', severity: 'high', title: 'e', description: 'f' }),
+      finding({ scanner: 'SET', code: 'missing-schema', severity: 'info', title: 'g', description: 'h' }),
     ];
-    const r = scannerResult('TST', 'ok', findings, 5, 100);
+    const r = scannerResult('SET', 'ok', findings, 5, 100);
     assert.strictEqual(r.counts.critical, 1);
     assert.strictEqual(r.counts.high, 2);
     assert.strictEqual(r.counts.medium, 0);
@@ -83,12 +79,12 @@ describe('scannerResult', () => {
   });
 
   it('includes error message when provided', () => {
-    const r = scannerResult('TST', 'error', [], 0, 50, 'boom');
+    const r = scannerResult('SET', 'error', [], 0, 50, 'boom');
     assert.strictEqual(r.error, 'boom');
   });
 
   it('omits error when not provided', () => {
-    const r = scannerResult('TST', 'ok', [], 3, 100);
+    const r = scannerResult('SET', 'ok', [], 3, 100);
     assert.strictEqual(r.error, undefined);
   });
 
@@ -103,16 +99,13 @@ describe('scannerResult', () => {
 });
 
 describe('envelope', () => {
-  beforeEach(() => { resetCounter(); });
-
-  it('aggregates across scanners', () => {
-    const r1 = scannerResult('A', 'ok', [
-      finding({ scanner: 'A', severity: 'high', title: 'x', description: 'y' }),
+    it('aggregates across scanners', () => {
+    const r1 = scannerResult('CML', 'ok', [
+      finding({ scanner: 'CML', code: 'no-claude-md', severity: 'high', title: 'x', description: 'y' }),
     ], 1, 10);
-    resetCounter();
-    const r2 = scannerResult('B', 'ok', [
-      finding({ scanner: 'B', severity: 'critical', title: 'a', description: 'b' }),
-      finding({ scanner: 'B', severity: 'low', title: 'c', description: 'd' }),
+    const r2 = scannerResult('SET', 'ok', [
+      finding({ scanner: 'SET', code: 'invalid-json', severity: 'critical', title: 'a', description: 'b' }),
+      finding({ scanner: 'SET', code: 'no-deny-rules', severity: 'low', title: 'c', description: 'd' }),
     ], 2, 20);
 
     const env = envelope('/target', [r1, r2], 50);

tests/lib/suppression-validation.test.mjs

@@ -0,0 +1,48 @@
+/**
+ * A suppression that names no known check must be reported, not silently
+ * ignored (M-BUG-28, prediction 8).
+ *
+ * This is what makes the ID-semantics change safe to ship: pins written against
+ * the old positional numbering either still name a real check, or they now name
+ * nothing — and "nothing" has to be visible. A silently-dead suppression is the
+ * same failure the old scheme had, just in the other direction.
+ */
+
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { parseIgnoreFile, unknownSuppressions } from '../../scanners/lib/suppression.mjs';
+
+describe('unknownSuppressions', () => {
+  it('accepts an exact ID that names a declared check', () => {
+    const s = parseIgnoreFile('CA-SKL-003\n');
+    assert.deepEqual(unknownSuppressions(s), []);
+  });
+
+  it('reports an exact ID that names no declared check', () => {
+    // CA-GAP-099 has never existed; CA-PLH-021 is past the end of PLH's range.
+    const s = parseIgnoreFile('CA-GAP-099\nCA-PLH-021\n');
+    assert.deepEqual(unknownSuppressions(s), ['CA-GAP-099', 'CA-PLH-021']);
+  });
+
+  it('reports an ID whose number was retired rather than pretending it matches', () => {
+    // GAP's retired autoMode dimension sat at 25 under the registry's numbering
+    // had it survived; nothing occupies it now.
+    const s = parseIgnoreFile('CA-GAP-028\n');
+    assert.deepEqual(unknownSuppressions(s), ['CA-GAP-028']);
+  });
+
+  it('accepts a scanner-wide glob for a real scanner', () => {
+    const s = parseIgnoreFile('CA-GAP-*\nCA-PLH-*\n');
+    assert.deepEqual(unknownSuppressions(s), []);
+  });
+
+  it('reports a glob for a scanner that does not exist', () => {
+    const s = parseIgnoreFile('CA-XYZ-*\n');
+    assert.deepEqual(unknownSuppressions(s), ['CA-XYZ-*']);
+  });
+
+  it('stays quiet on an empty ignore file', () => {
+    assert.deepEqual(unknownSuppressions(parseIgnoreFile('# just a comment\n')), []);
+    assert.deepEqual(unknownSuppressions([]), []);
+  });
+});