`write-scope.mjs` has existed since M-BUG-41, but only one writer ever called it. Measured 2026-08-12: 9 files under `scanners/` write to disk, 1 imported the gate; 21 command templates, 17 mention a write, 5 call `write-scope-cli`. Five templates paraphrasing one policy is the shape that put the lever table in five copies (#61) — one level up. The defect was never "8 ungated writers = 8 bugs". Four of them write the plugin's own bookkeeping and must STAY ungated: a gate that fires on every run gets switched off, and then it guards nothing. The defect is that nothing declared WHICH, so the question was answered by reading, and answered differently each time it was asked. `tests/lib/write-gate-coverage.test.mjs` makes the answer structural: every writer either imports the gate or holds an EXEMPT entry naming where the bytes land. Seen RED against today's tree before the fix (4 ungated writers), and each of its four assertions was separately seen red against its own defect. Two premises in the plan text were falsified by measuring them first: - `scan-orchestrator` was carried as "plugin-managed, legitimately exempt". `--save-baseline` derives its path from the SCAN TARGET, so `--global` lands `~/.claude/.config-audit-baseline.json` — user-scope, require-ok. It is gated. `lib/baseline.mjs` is the genuinely exempt one. - the first sweep scored 9 writers with a regex that could not match `writeFileSync(`, so `lib/backup.mjs` — a real writer — read as clean. The guard covers sync and async forms, strips comments before matching, and asserts non-emptiness so a regex that stops matching cannot make every other assertion vacuously green (#63, #64). Gated: fix-engine, rollback-engine, campaign-export-cli, scan-orchestrator. All five call sites share ONE reduction, `evaluateWriteTargets` — four copies of classify/strongestGate/dedup is the drift this exists to prevent. `campaign export` still DISCLOSES rather than refuses: cross-repo is by design there, and tightening it into a refusal would break the feature. A dry run is still not a write, so it is never gated (#63). A refusal is a verdict about a config that WAS examined, so it rides in the payload and keeps the 0/1/2 exit contract (#62) — and the verdict now reaches the success payload too, since stderr is discarded by `2>/dev/null` (F3's class). commands/fix.md carries `--approve-scope` from the answer the user gives, with the rule stated where it can be read: classifying is not approving. Dogfooded end to end: a target outside the session root refuses with zero bytes written, then applies under `--approve-scope`. Suite 1703 -> 1707/0. Frozen v5.0.0 + default-output snapshots: 0 changed files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pkn22uGCgk6QZA738zNmHL
155 lines
15 KiB
Markdown
155 lines
15 KiB
Markdown
# Config-Audit Plugin
|
|
|
|
Claude Code Configuration Intelligence — know if your config is correct, find what could improve it, fix it automatically. Three pillars: **Health** (deterministic scanners), **Opportunities** (context-aware recommendations), **Action** (auto-fix with backup/rollback).
|
|
|
|
Per-command flags, patterns, and feature lists live in `README.md` and `/config-audit help`. This file carries what's invariant for working on the plugin.
|
|
|
|
**Positioning vs. built-in `/doctor` (measured 2026-08-03, binding):** we are the deterministic/reproducible/all-scope/zero-quota side; `/doctor` is usage-weighted one-shot judgment. Never build a feature whose whole value is duplicating a `/doctor` check — see README «config-audit vs. the built-in /doctor» and `docs/v5.13-model-routing-effort-deadref-plan.md` §A.
|
|
|
|
## Commands
|
|
|
|
### Core (just run `/config-audit` to get started)
|
|
|
|
| Command | Description |
|
|
|---------|-------------|
|
|
| `/config-audit` | Full audit with auto-scope detection |
|
|
| `/config-audit posture` | A-F health scorecard (10 quality areas) |
|
|
| `/config-audit tokens` | Prompt-cache-aware token hotspots, each tagged with its load pattern; cache-aware |
|
|
| `/config-audit manifest` | Ranked table of every token source + always-loaded subtotal |
|
|
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact |
|
|
| `/config-audit optimize` | Mechanism-fit lens (procedure→skill, lifecycle→hook, path→rule, never→permission). Agent-driven, **not byte-stable**. `--subtract` adds the subtraction axis (what no longer earns its always-loaded rent, `BP-SUB-001`) — opt-in, proposes only; `--subtract --apply` executes the removals the operator picks |
|
|
| `/config-audit fix` | Auto-fix deterministic issues with backup + verification |
|
|
| `/config-audit rollback` | Restore configuration from backup |
|
|
| `/config-audit plan` | Create action plan from findings |
|
|
| `/config-audit implement` | Execute plan with backups + auto-verify |
|
|
| `/config-audit help` | Show all commands |
|
|
|
|
### Additional
|
|
|
|
| Command | Description |
|
|
|---------|-------------|
|
|
| `/config-audit drift` | Compare current config against saved baseline |
|
|
| `/config-audit plugin-health` | Audit plugin structure, frontmatter, cross-plugin coherence |
|
|
| `/config-audit whats-active` | Read-only inventory of active plugins/skills/agents/MCP/hooks/CLAUDE.md (with token estimates, and `model`/`effort` per agent) |
|
|
| `/config-audit knowledge-refresh` | Refresh the best-practices register (stale check + web poll). Human-approved writes; **not byte-stable** |
|
|
| `/config-audit campaign` | Machine-wide audit ledger + token bill across repos. Human-approved writes; **not byte-stable** |
|
|
| `/config-audit discover` | Run discovery phase only |
|
|
| `/config-audit analyze` | Run analysis phase only |
|
|
| `/config-audit interview` | Gather user preferences (opt-in) |
|
|
| `/config-audit status` | Show current session state |
|
|
| `/config-audit cleanup` | Clean up old sessions |
|
|
|
|
## Agents
|
|
|
|
| Agent | Role | Model | Color | Tools |
|
|
|-------|------|-------|-------|-------|
|
|
| scanner-agent | Find config files | sonnet | cyan | Read, Glob, Grep, Write |
|
|
| analyzer-agent | Generate report | sonnet | blue | Read, Glob, Grep, Write |
|
|
| planner-agent | Create action plan | opus | yellow | Read, Glob, Write |
|
|
| implementer-agent | Execute changes | sonnet | magenta | Read, Write, Edit, Bash, Glob |
|
|
| verifier-agent | Verify results | sonnet | purple | Read, Glob, Grep |
|
|
| feature-gap-agent | Feature recommendations | opus | green | Read, Glob, Grep, Write |
|
|
| optimization-lens-agent | Mechanism-fit precision gate | opus | orange | Read, Glob, Grep, Write |
|
|
|
|
## Hooks
|
|
|
|
| Event | Script | Purpose |
|
|
|-------|--------|---------|
|
|
| PreToolUse | `auto-backup-config.mjs` | Backup config files before Edit/Write |
|
|
| PostToolUse | `post-edit-verify.mjs` | Verify after Edit/Write, block on new critical/high |
|
|
| SessionStart | `session-start.mjs` | Check for active (unfinished) sessions |
|
|
| Stop | `stop-session-reminder.mjs` | Remind about current session phase |
|
|
|
|
## Reference docs (read on demand)
|
|
|
|
- `docs/scanner-internals.md` — scanner inventory, lib modules, action engines, knowledge base, per-scanner/per-block implementation notes (design rationale, primary-source verification, byte-stability lessons)
|
|
- `docs/humanizer.md` — plain-language output (v5.1.0), humanizer vocabularies, output modes
|
|
|
|
## Plain-Language Output (v5.1.0)
|
|
|
|
Default output of all commands routes through `humanizeEnvelope` (`scanners/lib/humanizer.mjs`), decorating each finding with `userImpactCategory`, `userActionLanguage`, and `relevanceContext`. `--raw` and `--json` bypass the humanizer for byte-stable v5.0.0 output. Full detail: `docs/humanizer.md`.
|
|
|
|
## Suppressions
|
|
|
|
Create `.config-audit-ignore` at project root — one exact ID or glob per line (`CA-SET-003`, `CA-GAP-*`). Suppressed findings are tracked in the envelope's `suppressed_findings` for audit trail. Disable with `--no-suppress`.
|
|
|
|
## Architecture
|
|
|
|
Workflow: `/config-audit → discover + analyze (auto) → plan → implement → verify`. Auto-detects scope from git context; override with `full|repo|home|current`; `--delta` for incremental. Session state lives under `~/.claude/config-audit/sessions/{id}/` (scope.yaml, discovery.json, state.yaml, findings/, analysis-report.md, action-plan.md, backups/, implementation-log.md).
|
|
|
|
Finding ID format: `CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-SKL-001`, `CA-OST-001`, `CA-OPT-001`, `CA-AGT-001`.
|
|
|
|
**GAP dimensions vs. levers (invariant).** `GAP_CHECKS` holds the 24 *dimensions* — always evaluated, always counted in the utilization denominators (`TIER_COUNTS` / `TOTAL_DIMENSIONS` in `scoring.mjs`, and `TITLE_TO_ID` there). A *lever* is a finding the scanner emits after the loop and only under a measured condition; it carries no tier, never enters those denominators, and is registered in the exported `LEVERS` object (code + title in one place, because the finding-code guard needs the code and the humanizer-coverage guard needs the title). Adding a dimension moves every user's utilization score and can flip the reported `segment` in a frozen baseline — adding a lever cannot. When a check is only meaningful for configs that already have some feature, it is a lever.
|
|
|
|
**`{NNN}` names the CHECK, never the emission position (invariant).** `scanners/lib/finding-codes.mjs` is the single authority: every `finding()` call passes a `code`, and an undeclared or missing one **throws** — there is no counter fallback, because a fallback lets a half-converted scanner ship IDs that look valid. Adding a check takes the next free number for that scanner, never the next source-order position; removing one moves its key to `RETIRED_CODES` and its number is never reissued. IDs are therefore **not unique per finding** — one check failing in three files emits three findings sharing an ID, and `(id, file, line)` is the instance key that `fix-engine` verification uses. Frozen `v5.0.0` baselines mask IDs (`tests/helpers/mask-finding-ids.mjs`) instead of re-deriving them; the check→number pairs are pinned exhaustively in `tests/lib/finding-codes.test.mjs`.
|
|
|
|
## Conventions
|
|
|
|
Enforced conventions live in `.claude/rules/` (auto-loaded as project instructions):
|
|
- `ux-rules.md` — output/narration/formatting for all commands (never dump raw JSON, narrate before each step, space-separated command suggestions)
|
|
- `command-development.md` — required command frontmatter + `plugin:action` naming
|
|
- `agent-development.md` — agent frontmatter + "when to use" conventions
|
|
- `state-management.md` — update `state.yaml` after every workflow phase
|
|
|
|
Coding style: scanners are zero-dependency Node ESM; new findings use the `CA-{SCANNER}-{NNN}` ID format; byte-stable CLIs are verified against frozen `tests/snapshots/v5.0.0/` baselines.
|
|
|
|
**Write-scope gate (invariant).** Every write target is classified by `scanners/lib/write-scope.mjs` before it reaches an approval surface, and the **scope class decides the gate's strength — never the command asking**. Five command-owned policies would drift apart the way five copies of the lever table did. `SCOPE_CLASSES` is the single source for class, gate (`silent`/`disclose`/`require-ok`), wording and predicate; templates render `disclosures[]` from `write-scope-cli.mjs` rather than restating what a class means. Two orderings in that object are load-bearing and were measured, not reasoned about: `plugin-managed` before `user-scope` (both `~/.claude/config-audit/` and legacy `~/.config-audit/` are live, so the other order fires the gate on every session write and gets it switched off), and `user-scope` before `cross-repo` (`~/.claude/.git` exists, so a plain `.git`-upward walk calls `~/.claude/CLAUDE.md` merely "another repo" and silently downgrades the strongest gate). `disclose` ≠ `require-ok`: `campaign export` is cross-repo *by design*, so tightening it into a refusal breaks the feature. Distinct from the `require-target-dir.mjs` guard, which asks whether a scan **root** is readable (exit 3) — a different invariant, not to be merged.
|
|
|
|
**Write-gate coverage (invariant).** The gate above only counts where it is *called*, and for four
|
|
releases it was called from prose: `write-scope.mjs` existed, but exactly one writer imported it
|
|
(`lib/subtraction-write.mjs`) while five command templates paraphrased the policy. Measured
|
|
2026-08-12: 9 files under `scanners/` write to disk, 1 imported the gate. The defect was never
|
|
"8 ungated writers = 8 bugs" — four of them write the plugin's own bookkeeping and MUST stay
|
|
ungated, because a gate that fires on every run gets switched off. The defect is that **nothing
|
|
declared which**, so the question was answered by reading, and answered differently each time.
|
|
`tests/lib/write-gate-coverage.test.mjs` is now the authority: every writer must either import
|
|
the gate or hold an `EXEMPT` entry naming **where the bytes land**. Three properties are
|
|
load-bearing. (1) **A rationale is a claim, not a label** — `scan-orchestrator` was carried in
|
|
the plan text as exempt while `--save-baseline` derived its path from the *scan target*, so
|
|
`--global` landed `~/.claude/.config-audit-baseline.json` (`user-scope`/`require-ok`); it is
|
|
gated, and `lib/baseline.mjs` — which writes only under `~/.config-audit/baselines` — is the
|
|
genuinely exempt one. (2) **Sync variants count**: `writeFile(` does not match `writeFileSync(`,
|
|
and `lib/backup.mjs` uses only the sync forms, so the first sweep scored a real writer as clean
|
|
and was green on its own subject. (3) **The sweep asserts non-emptiness** — a regex that stops
|
|
matching makes every other assertion here vacuously green. The exemption table is stale-checked
|
|
in both directions: an entry naming a file that no longer writes, or one that has since been
|
|
gated, fails. `evaluateWriteTargets` in `write-scope.mjs` is the one reduction (classify →
|
|
`strongestGate` → dedup disclosures) that all five call sites share; four copies of those four
|
|
lines is the drift shape `SCOPE_CLASSES` exists to prevent one level down. Approval is carried by
|
|
`--approve-scope`, and **classifying is not approving**: a template that sets the flag because it
|
|
already ran `write-scope-cli` has rebuilt the prose contract this guard replaced.
|
|
|
|
**Dead-prose-reference silence list (invariant).** `CA-CML-013` is a precision-first check, so its
|
|
design lives in what it *declines* to flag, and that list is measured (407 real CLAUDE.md files),
|
|
never argued. Three rules are load-bearing and each has a guard seen red against its own defect.
|
|
(1) **Containment is checked against the scan root, not the file's own directory** — a `../` chain
|
|
that leaves the tree is silenced (`outside-scan-tree`) rather than resolved, because a base a `..`
|
|
chain can escape is not a base: measured, `../../../../etc/passwd` resolved to the real file and
|
|
silenced its own finding. A legitimate `../docs/x.md` inside the same repo still resolves.
|
|
(2) **A bare token is a concept, not a reference** — `README.md` (no separator) and `docs/`
|
|
(single segment) are excluded on the same reasoning one level apart; admitting bare filenames
|
|
tripled the output with name-drops of tools living elsewhere, and single-segment folders are 26 %
|
|
of the remainder, led by a remote namespace prefix. (3) **Rule ORDER is the reported reason** —
|
|
first match wins, so `npm test` is silenced as `whitespace` (a command), not as `no-separator`,
|
|
and two silences with different causes keep their own fixtures. The check emits **one finding per
|
|
file** (the `todo-markers` / `repeated-content` idiom), because per-token emission measured 699
|
|
findings where per-file measured 128. Silence is the safe failure direction here: a path carrying
|
|
a trailing `:54-56` locator is a recorded v1 miss, not a bug to fix by loosening a rule.
|
|
|
|
**Subtraction floor (invariant).** `optimize --subtract` is the only lens that proposes removing config, so `scanners/lib/floor-exclusion.mjs` runs as a deterministic pre-step *before* the judge — a load-bearing block is never a candidate, and that guarantee must not be moved into the agent prompt. Two rules follow from it: (1) **staleness is not a deletion signal** — an outdated version pin inside a floor block is a `drift`/`CA-CML` dead-reference concern; (2) **tier 2 ≠ tier 3** — a compensatory block that keeps earning its place returns, and reporting it as dead weight is wrong even when the label matches. Norwegian keywords need the Unicode boundaries in `subtraction-prefilter.mjs`; JS `\b` is ASCII-only, so `/\bunngå\b/` silently never matches.
|
|
|
|
**Subtraction write path (invariant).** `--apply` routes through `scanners/lib/subtraction-write.mjs`, never through `fix-engine` or the `plan`/`implement` pipeline, and both exclusions are **measured**: the subtraction axis is absent from the orchestrated envelope, so `verifyFixes`' re-scan would mark every removal `verified` whether or not it happened (a success-shaped no-op), and the findings pipeline needs a finding code — which names a deterministic check, not a prose judgement. Three properties are load-bearing and each has a guard seen red against its own defect: removals are validated against the ORIGINAL content and applied in **descending** line order (an ascending pass shifts later spans out from under themselves); the **range** check is not redundant with the text check (`line: 0` makes `slice(-1, 0)` empty, so an empty `text` matches and `splice(-1, 1)` deletes the file's LAST line); and `createBackup` skips a nonexistent path while still returning an id, so coverage of every file about to be written is **asserted from the manifest** before a byte changes. The floor is *repeated* here, not moved: `floor-exclusion` still vetoes before anything is proposed, and the engine refuses a load-bearing block again so a hand-built approval cannot route around it. The archive rule (`mv` to `_archive/`) is file-level and does not apply to a block excision — the timestamped backup is the recovery artifact, and inventing a second copy with no restorer behind it would be worse than none.
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
node --test 'tests/**/*.test.mjs'
|
|
```
|
|
|
|
Test fixtures in `tests/fixtures/`. Per-scanner and per-build-block implementation notes (design rationale, primary-source verification, byte-stability lessons) live in `docs/scanner-internals.md` → **Implementation notes**.
|
|
|
|
## Gotchas
|
|
|
|
- Session directories accumulate — use `/config-audit cleanup` to manage
|
|
- Scanners run on Node.js ≥ 18 (uses node:test, node:fs/promises)
|
|
- Plugin CLAUDE.md files in node_modules should be excluded via scope
|