Kjell Tore Guttormsen 7a794b47eb fix(scanners)!: a finding ID names the check, not the emission (M-BUG-28)

BREAKING CHANGE: the {NNN} in CA-{SCANNER}-{NNN} identifies the check that
produced the finding. It used to be the finding's position in that scanner's
output for that run, which made it unstable across CONFIGURATIONS, not just
across releases as STATE framed it. Measured on two fixtures: "No custom
subagents" was CA-GAP-007 on minimal-project and CA-GAP-004 on healthy-project.
A user who fixed an unrelated earlier gap silently renumbered every later one,
so a .config-audit-ignore pin retargeted to a neighbouring finding with no
version change at all.

Second measured arm: README already documented the opposite scheme. It and the
scanner headers describe ~20 numbers as check codes (CA-SKL-003 = oversized
body, CA-PLH-015 = folder shadowing, CA-TOK-006 = schema deferral), and the
counter could only produce those in the all-fire case -- source-order positions
are 4, 3 and 8. The documentation described the scheme; the implementation was
what was wrong. Every published number is preserved by construction and pinned
exhaustively in tests/lib/finding-codes.test.mjs.

scanners/lib/finding-codes.mjs is the single authority. Every finding() call
passes a `code`; an undeclared or missing one THROWS. No counter fallback --
that would reproduce D1's findGapId -> 'unknown' silent degradation and let a
half-converted scanner ship IDs that look valid. findingCounter/resetCounter
are deleted outright, not left as no-ops. Retirement is now a mechanism:
RETIRED_CODES tombstones a withdrawn key so its number is never reissued,
seeded with GAP t3_8 -- the D1 removal that opened this chunk.

IDs are consequently NOT unique per finding: one check failing in three files
emits three findings sharing an ID. That inverts which consumer is correct, so
every f.id/findingId site was classified before the change. diff-engine and
most of fix-engine already keyed on scanner+title+file (drift was never lying);
fix-engine's verification did not, and keyed on the ID alone -- fixing one of
two sibling instances marked both fixed, and the untouched one, still present
in the re-scan, was reported as a REGRESSION. Red test first, then keyed on
(findingId, file), which both planFixes and applyFixes already carry.
plugin-health's crossIds Set was measured and is a clean negative: cross
findings are allFindings.slice(crossPluginStart) and codes 18/19 are emitted
only in that tail, so the partition holds by construction.

unknownSuppressions() reports a pin that names no declared check, in the
--output-file payload (ux-rules rule 2 -- a stderr-only warning is invisible to
the commands) and only when one exists, so a clean config is byte-identical.
That is what makes the break safe: a stale pin goes loud instead of dying quiet.

Frozen tests/snapshots/v5.0.0/ untouched on disk. IDs are masked out of that
comparison (mask-finding-ids.mjs) rather than re-derived -- re-deriving
positional IDs would assert the retired scheme against itself, and #58's
isGapEntry off-by-one is the measured example of that misfiring. The dead
re-derivation is removed from strip-retired-gap.mjs. default-output snapshots
re-approved after confirming the diff is IDs and nothing else.

Guards, each seen red against its own defect: a missing code (scanner errors
out mid-sweep), an orphan declaration, a resurrected retired key, and a
documented ID naming no check. The sweep asserts the union across all 16
scanners, never per scanner -- a per-scanner assertion goes green on a partial
conversion.

Fasit written before implementation: docs/mbug28-id-semantics-fasit.local.md,
including one correction made before running (CML has 12 checks over 13 call
sites -- the anchored and calibrated char-budget arms are one check, which a
repeated-title sweep found and my call-site count had missed).

Suite 1535 -> 1573, 0 failing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MyqCQKK2ornJ1jFWwqx17E

2026-08-09 23:26:36 +02:00

8.1 KiB

Raw Blame History

Config-Audit Plugin

Claude Code Configuration Intelligence — know if your config is correct, find what could improve it, fix it automatically. Three pillars: Health (deterministic scanners), Opportunities (context-aware recommendations), Action (auto-fix with backup/rollback).

Per-command flags, patterns, and feature lists live in README.md and /config-audit help. This file carries what's invariant for working on the plugin.

Positioning vs. built-in /doctor (measured 2026-08-03, binding): we are the deterministic/reproducible/all-scope/zero-quota side; /doctor is usage-weighted one-shot judgment. Never build a feature whose whole value is duplicating a /doctor check — see README «config-audit vs. the built-in /doctor» and docs/v5.13-model-routing-effort-deadref-plan.md §A.

Commands

Core (just run `/config-audit` to get started)

Command	Description
`/config-audit`	Full audit with auto-scope detection
`/config-audit posture`	A-F health scorecard (10 quality areas)
`/config-audit tokens`	Prompt-cache-aware token hotspots, each tagged with its load pattern; cache-aware
`/config-audit manifest`	Ranked table of every token source + always-loaded subtotal
`/config-audit feature-gap`	Context-aware feature recommendations grouped by impact
`/config-audit optimize`	Mechanism-fit lens (procedure→skill, lifecycle→hook, path→rule, never→permission). Agent-driven, not byte-stable. `--subtract` adds the subtraction axis (what no longer earns its always-loaded rent, `BP-SUB-001`) — opt-in, proposes only
`/config-audit fix`	Auto-fix deterministic issues with backup + verification
`/config-audit rollback`	Restore configuration from backup
`/config-audit plan`	Create action plan from findings
`/config-audit implement`	Execute plan with backups + auto-verify
`/config-audit help`	Show all commands

Additional

Command	Description
`/config-audit drift`	Compare current config against saved baseline
`/config-audit plugin-health`	Audit plugin structure, frontmatter, cross-plugin coherence
`/config-audit whats-active`	Read-only inventory of active plugins/skills/MCP/hooks/CLAUDE.md (with token estimates)
`/config-audit knowledge-refresh`	Refresh the best-practices register (stale check + web poll). Human-approved writes; not byte-stable
`/config-audit campaign`	Machine-wide audit ledger + token bill across repos. Human-approved writes; not byte-stable
`/config-audit discover`	Run discovery phase only
`/config-audit analyze`	Run analysis phase only
`/config-audit interview`	Gather user preferences (opt-in)
`/config-audit status`	Show current session state
`/config-audit cleanup`	Clean up old sessions

Agents

Agent	Role	Model	Color	Tools
scanner-agent	Find config files	sonnet	cyan	Read, Glob, Grep, Write
analyzer-agent	Generate report	sonnet	blue	Read, Glob, Grep, Write
planner-agent	Create action plan	opus	yellow	Read, Glob, Write
implementer-agent	Execute changes	sonnet	magenta	Read, Write, Edit, Bash, Glob
verifier-agent	Verify results	sonnet	purple	Read, Glob, Grep
feature-gap-agent	Feature recommendations	opus	green	Read, Glob, Grep, Write
optimization-lens-agent	Mechanism-fit precision gate	opus	orange	Read, Glob, Grep, Write

Hooks

Event	Script	Purpose
PreToolUse	`auto-backup-config.mjs`	Backup config files before Edit/Write
PostToolUse	`post-edit-verify.mjs`	Verify after Edit/Write, block on new critical/high
SessionStart	`session-start.mjs`	Check for active (unfinished) sessions
Stop	`stop-session-reminder.mjs`	Remind about current session phase

Reference docs (read on demand)

docs/scanner-internals.md — scanner inventory, lib modules, action engines, knowledge base, per-scanner/per-block implementation notes (design rationale, primary-source verification, byte-stability lessons)
docs/humanizer.md — plain-language output (v5.1.0), humanizer vocabularies, output modes

Plain-Language Output (v5.1.0)

Default output of all commands routes through humanizeEnvelope (lib/humanizer.mjs), decorating each finding with userImpactCategory, userActionLanguage, and relevanceContext. --raw and --json bypass the humanizer for byte-stable v5.0.0 output. Full detail: docs/humanizer.md.

Suppressions

Create .config-audit-ignore at project root — one exact ID or glob per line (CA-SET-003, CA-GAP-*). Suppressed findings are tracked in the envelope's suppressed_findings for audit trail. Disable with --no-suppress.

Architecture

Workflow: /config-audit → discover + analyze (auto) → plan → implement → verify. Auto-detects scope from git context; override with full|repo|home|current; --delta for incremental. Session state lives under ~/.claude/config-audit/sessions/{id}/ (scope.yaml, discovery.json, state.yaml, findings/, analysis-report.md, action-plan.md, backups/, implementation-log.md).

Finding ID format: CA-{SCANNER}-{NNN} — e.g. CA-CML-001, CA-SET-003, CA-HKV-002, CA-RUL-005, CA-TOK-005, CA-CPS-001, CA-SKL-001, CA-OST-001, CA-OPT-001, CA-AGT-001.

{NNN} names the CHECK, never the emission position (invariant). scanners/lib/finding-codes.mjs is the single authority: every finding() call passes a code, and an undeclared or missing one throws — there is no counter fallback, because a fallback lets a half-converted scanner ship IDs that look valid. Adding a check takes the next free number for that scanner, never the next source-order position; removing one moves its key to RETIRED_CODES and its number is never reissued. IDs are therefore not unique per finding — one check failing in three files emits three findings sharing an ID, and (id, file, line) is the instance key that fix-engine verification uses. Frozen v5.0.0 baselines mask IDs (tests/helpers/mask-finding-ids.mjs) instead of re-deriving them; the check→number pairs are pinned exhaustively in tests/lib/finding-codes.test.mjs.

Conventions

Enforced conventions live in .claude/rules/ (auto-loaded as project instructions):

ux-rules.md — output/narration/formatting for all commands (never dump raw JSON, narrate before each step, space-separated command suggestions)
command-development.md — required command frontmatter + plugin:action naming
agent-development.md — agent frontmatter + "when to use" conventions
state-management.md — update state.yaml after every workflow phase

Coding style: scanners are zero-dependency Node ESM; new findings use the CA-{SCANNER}-{NNN} ID format; byte-stable CLIs are verified against frozen tests/snapshots/v5.0.0/ baselines.

Subtraction floor (invariant). optimize --subtract is the only lens that proposes removing config, so scanners/lib/floor-exclusion.mjs runs as a deterministic pre-step before the judge — a load-bearing block is never a candidate, and that guarantee must not be moved into the agent prompt. Two rules follow from it: (1) staleness is not a deletion signal — an outdated version pin inside a floor block is a drift/CA-CML dead-reference concern; (2) tier 2 ≠ tier 3 — a compensatory block that keeps earning its place returns, and reporting it as dead weight is wrong even when the label matches. Norwegian keywords need the Unicode boundaries in subtraction-prefilter.mjs; JS \b is ASCII-only, so /\bunngå\b/ silently never matches.

Testing

node --test 'tests/**/*.test.mjs'

Test fixtures in tests/fixtures/. Per-scanner and per-build-block implementation notes (design rationale, primary-source verification, byte-stability lessons) live in docs/scanner-internals.md → Implementation notes.

Gotchas

Session directories accumulate — use /config-audit cleanup to manage
Scanners run on Node.js ≥ 18 (uses node:test, node:fs/promises)
Plugin CLAUDE.md files in node_modules should be excluded via scope

8.1 KiB Raw Blame History