`import-resolver` follows @import targets; a path written in ordinary prose was checked by nothing. CA-CML-013 resolves those too — one finding per file, severity low, against both the CLAUDE.md's own directory and the scan root, because a nested file may legitimately write repo-root-relative paths. The design work here is the SILENCE list, and every entry on it was measured against 407 real CLAUDE.md files rather than argued for: - Bare filenames excluded: admitting them tripled the output (2350 vs 810), led by name-drops of tools that exist elsewhere on the machine. - Org/repo slugs, npm packages, pytest node ids and prose enumerations excluded: 111 fires, inspected, all false positives. - Bare folder names excluded on the same reasoning one level up: 183 of the remaining 699 fires (26%), led by `open/` — a Forgejo remote namespace prefix, not a directory. This one overturned a premise the fasit had asserted without measuring; the deviation is recorded rather than the prediction quietly edited. - Containment is checked against the scan root, not the file's own dir: a base a `..` chain can escape is not a base. Measured — without it, `../../../../etc/passwd` resolved to the real file and silenced its own finding, while a legitimate `../docs/x.md` still resolves. Rule ORDER is the reported reason (first match wins), so `npm test` is silenced as a command rather than as a bare token, and two silences with different causes keep their own fixtures. Twelve classes, pinned by name. Both load-bearing rules were seen RED against their own defect: deleting containment fails 1 test, deleting the slug rule fails 6. Dogfooded through the argv the command template itself constructs, which found a true positive in our own CLAUDE.md — `lib/humanizer.mjs` where the file is `scanners/lib/humanizer.mjs`. Fixed here. Suite 1662 -> 1701, 0 failing. Frozen v5.0.0 and default-output baselines: 0 changed files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HJbfM3N8zWQ1wA2voTrZxz
131 lines
13 KiB
Markdown
131 lines
13 KiB
Markdown
# Config-Audit Plugin
|
|
|
|
Claude Code Configuration Intelligence — know if your config is correct, find what could improve it, fix it automatically. Three pillars: **Health** (deterministic scanners), **Opportunities** (context-aware recommendations), **Action** (auto-fix with backup/rollback).
|
|
|
|
Per-command flags, patterns, and feature lists live in `README.md` and `/config-audit help`. This file carries what's invariant for working on the plugin.
|
|
|
|
**Positioning vs. built-in `/doctor` (measured 2026-08-03, binding):** we are the deterministic/reproducible/all-scope/zero-quota side; `/doctor` is usage-weighted one-shot judgment. Never build a feature whose whole value is duplicating a `/doctor` check — see README «config-audit vs. the built-in /doctor» and `docs/v5.13-model-routing-effort-deadref-plan.md` §A.
|
|
|
|
## Commands
|
|
|
|
### Core (just run `/config-audit` to get started)
|
|
|
|
| Command | Description |
|
|
|---------|-------------|
|
|
| `/config-audit` | Full audit with auto-scope detection |
|
|
| `/config-audit posture` | A-F health scorecard (10 quality areas) |
|
|
| `/config-audit tokens` | Prompt-cache-aware token hotspots, each tagged with its load pattern; cache-aware |
|
|
| `/config-audit manifest` | Ranked table of every token source + always-loaded subtotal |
|
|
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact |
|
|
| `/config-audit optimize` | Mechanism-fit lens (procedure→skill, lifecycle→hook, path→rule, never→permission). Agent-driven, **not byte-stable**. `--subtract` adds the subtraction axis (what no longer earns its always-loaded rent, `BP-SUB-001`) — opt-in, proposes only; `--subtract --apply` executes the removals the operator picks |
|
|
| `/config-audit fix` | Auto-fix deterministic issues with backup + verification |
|
|
| `/config-audit rollback` | Restore configuration from backup |
|
|
| `/config-audit plan` | Create action plan from findings |
|
|
| `/config-audit implement` | Execute plan with backups + auto-verify |
|
|
| `/config-audit help` | Show all commands |
|
|
|
|
### Additional
|
|
|
|
| Command | Description |
|
|
|---------|-------------|
|
|
| `/config-audit drift` | Compare current config against saved baseline |
|
|
| `/config-audit plugin-health` | Audit plugin structure, frontmatter, cross-plugin coherence |
|
|
| `/config-audit whats-active` | Read-only inventory of active plugins/skills/agents/MCP/hooks/CLAUDE.md (with token estimates, and `model`/`effort` per agent) |
|
|
| `/config-audit knowledge-refresh` | Refresh the best-practices register (stale check + web poll). Human-approved writes; **not byte-stable** |
|
|
| `/config-audit campaign` | Machine-wide audit ledger + token bill across repos. Human-approved writes; **not byte-stable** |
|
|
| `/config-audit discover` | Run discovery phase only |
|
|
| `/config-audit analyze` | Run analysis phase only |
|
|
| `/config-audit interview` | Gather user preferences (opt-in) |
|
|
| `/config-audit status` | Show current session state |
|
|
| `/config-audit cleanup` | Clean up old sessions |
|
|
|
|
## Agents
|
|
|
|
| Agent | Role | Model | Color | Tools |
|
|
|-------|------|-------|-------|-------|
|
|
| scanner-agent | Find config files | sonnet | cyan | Read, Glob, Grep, Write |
|
|
| analyzer-agent | Generate report | sonnet | blue | Read, Glob, Grep, Write |
|
|
| planner-agent | Create action plan | opus | yellow | Read, Glob, Write |
|
|
| implementer-agent | Execute changes | sonnet | magenta | Read, Write, Edit, Bash, Glob |
|
|
| verifier-agent | Verify results | sonnet | purple | Read, Glob, Grep |
|
|
| feature-gap-agent | Feature recommendations | opus | green | Read, Glob, Grep, Write |
|
|
| optimization-lens-agent | Mechanism-fit precision gate | opus | orange | Read, Glob, Grep, Write |
|
|
|
|
## Hooks
|
|
|
|
| Event | Script | Purpose |
|
|
|-------|--------|---------|
|
|
| PreToolUse | `auto-backup-config.mjs` | Backup config files before Edit/Write |
|
|
| PostToolUse | `post-edit-verify.mjs` | Verify after Edit/Write, block on new critical/high |
|
|
| SessionStart | `session-start.mjs` | Check for active (unfinished) sessions |
|
|
| Stop | `stop-session-reminder.mjs` | Remind about current session phase |
|
|
|
|
## Reference docs (read on demand)
|
|
|
|
- `docs/scanner-internals.md` — scanner inventory, lib modules, action engines, knowledge base, per-scanner/per-block implementation notes (design rationale, primary-source verification, byte-stability lessons)
|
|
- `docs/humanizer.md` — plain-language output (v5.1.0), humanizer vocabularies, output modes
|
|
|
|
## Plain-Language Output (v5.1.0)
|
|
|
|
Default output of all commands routes through `humanizeEnvelope` (`scanners/lib/humanizer.mjs`), decorating each finding with `userImpactCategory`, `userActionLanguage`, and `relevanceContext`. `--raw` and `--json` bypass the humanizer for byte-stable v5.0.0 output. Full detail: `docs/humanizer.md`.
|
|
|
|
## Suppressions
|
|
|
|
Create `.config-audit-ignore` at project root — one exact ID or glob per line (`CA-SET-003`, `CA-GAP-*`). Suppressed findings are tracked in the envelope's `suppressed_findings` for audit trail. Disable with `--no-suppress`.
|
|
|
|
## Architecture
|
|
|
|
Workflow: `/config-audit → discover + analyze (auto) → plan → implement → verify`. Auto-detects scope from git context; override with `full|repo|home|current`; `--delta` for incremental. Session state lives under `~/.claude/config-audit/sessions/{id}/` (scope.yaml, discovery.json, state.yaml, findings/, analysis-report.md, action-plan.md, backups/, implementation-log.md).
|
|
|
|
Finding ID format: `CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-SKL-001`, `CA-OST-001`, `CA-OPT-001`, `CA-AGT-001`.
|
|
|
|
**GAP dimensions vs. levers (invariant).** `GAP_CHECKS` holds the 24 *dimensions* — always evaluated, always counted in the utilization denominators (`TIER_COUNTS` / `TOTAL_DIMENSIONS` in `scoring.mjs`, and `TITLE_TO_ID` there). A *lever* is a finding the scanner emits after the loop and only under a measured condition; it carries no tier, never enters those denominators, and is registered in the exported `LEVERS` object (code + title in one place, because the finding-code guard needs the code and the humanizer-coverage guard needs the title). Adding a dimension moves every user's utilization score and can flip the reported `segment` in a frozen baseline — adding a lever cannot. When a check is only meaningful for configs that already have some feature, it is a lever.
|
|
|
|
**`{NNN}` names the CHECK, never the emission position (invariant).** `scanners/lib/finding-codes.mjs` is the single authority: every `finding()` call passes a `code`, and an undeclared or missing one **throws** — there is no counter fallback, because a fallback lets a half-converted scanner ship IDs that look valid. Adding a check takes the next free number for that scanner, never the next source-order position; removing one moves its key to `RETIRED_CODES` and its number is never reissued. IDs are therefore **not unique per finding** — one check failing in three files emits three findings sharing an ID, and `(id, file, line)` is the instance key that `fix-engine` verification uses. Frozen `v5.0.0` baselines mask IDs (`tests/helpers/mask-finding-ids.mjs`) instead of re-deriving them; the check→number pairs are pinned exhaustively in `tests/lib/finding-codes.test.mjs`.
|
|
|
|
## Conventions
|
|
|
|
Enforced conventions live in `.claude/rules/` (auto-loaded as project instructions):
|
|
- `ux-rules.md` — output/narration/formatting for all commands (never dump raw JSON, narrate before each step, space-separated command suggestions)
|
|
- `command-development.md` — required command frontmatter + `plugin:action` naming
|
|
- `agent-development.md` — agent frontmatter + "when to use" conventions
|
|
- `state-management.md` — update `state.yaml` after every workflow phase
|
|
|
|
Coding style: scanners are zero-dependency Node ESM; new findings use the `CA-{SCANNER}-{NNN}` ID format; byte-stable CLIs are verified against frozen `tests/snapshots/v5.0.0/` baselines.
|
|
|
|
**Write-scope gate (invariant).** Every write target is classified by `scanners/lib/write-scope.mjs` before it reaches an approval surface, and the **scope class decides the gate's strength — never the command asking**. Five command-owned policies would drift apart the way five copies of the lever table did. `SCOPE_CLASSES` is the single source for class, gate (`silent`/`disclose`/`require-ok`), wording and predicate; templates render `disclosures[]` from `write-scope-cli.mjs` rather than restating what a class means. Two orderings in that object are load-bearing and were measured, not reasoned about: `plugin-managed` before `user-scope` (both `~/.claude/config-audit/` and legacy `~/.config-audit/` are live, so the other order fires the gate on every session write and gets it switched off), and `user-scope` before `cross-repo` (`~/.claude/.git` exists, so a plain `.git`-upward walk calls `~/.claude/CLAUDE.md` merely "another repo" and silently downgrades the strongest gate). `disclose` ≠ `require-ok`: `campaign export` is cross-repo *by design*, so tightening it into a refusal breaks the feature. Distinct from the `require-target-dir.mjs` guard, which asks whether a scan **root** is readable (exit 3) — a different invariant, not to be merged.
|
|
|
|
**Dead-prose-reference silence list (invariant).** `CA-CML-013` is a precision-first check, so its
|
|
design lives in what it *declines* to flag, and that list is measured (407 real CLAUDE.md files),
|
|
never argued. Three rules are load-bearing and each has a guard seen red against its own defect.
|
|
(1) **Containment is checked against the scan root, not the file's own directory** — a `../` chain
|
|
that leaves the tree is silenced (`outside-scan-tree`) rather than resolved, because a base a `..`
|
|
chain can escape is not a base: measured, `../../../../etc/passwd` resolved to the real file and
|
|
silenced its own finding. A legitimate `../docs/x.md` inside the same repo still resolves.
|
|
(2) **A bare token is a concept, not a reference** — `README.md` (no separator) and `docs/`
|
|
(single segment) are excluded on the same reasoning one level apart; admitting bare filenames
|
|
tripled the output with name-drops of tools living elsewhere, and single-segment folders are 26 %
|
|
of the remainder, led by a remote namespace prefix. (3) **Rule ORDER is the reported reason** —
|
|
first match wins, so `npm test` is silenced as `whitespace` (a command), not as `no-separator`,
|
|
and two silences with different causes keep their own fixtures. The check emits **one finding per
|
|
file** (the `todo-markers` / `repeated-content` idiom), because per-token emission measured 699
|
|
findings where per-file measured 128. Silence is the safe failure direction here: a path carrying
|
|
a trailing `:54-56` locator is a recorded v1 miss, not a bug to fix by loosening a rule.
|
|
|
|
**Subtraction floor (invariant).** `optimize --subtract` is the only lens that proposes removing config, so `scanners/lib/floor-exclusion.mjs` runs as a deterministic pre-step *before* the judge — a load-bearing block is never a candidate, and that guarantee must not be moved into the agent prompt. Two rules follow from it: (1) **staleness is not a deletion signal** — an outdated version pin inside a floor block is a `drift`/`CA-CML` dead-reference concern; (2) **tier 2 ≠ tier 3** — a compensatory block that keeps earning its place returns, and reporting it as dead weight is wrong even when the label matches. Norwegian keywords need the Unicode boundaries in `subtraction-prefilter.mjs`; JS `\b` is ASCII-only, so `/\bunngå\b/` silently never matches.
|
|
|
|
**Subtraction write path (invariant).** `--apply` routes through `scanners/lib/subtraction-write.mjs`, never through `fix-engine` or the `plan`/`implement` pipeline, and both exclusions are **measured**: the subtraction axis is absent from the orchestrated envelope, so `verifyFixes`' re-scan would mark every removal `verified` whether or not it happened (a success-shaped no-op), and the findings pipeline needs a finding code — which names a deterministic check, not a prose judgement. Three properties are load-bearing and each has a guard seen red against its own defect: removals are validated against the ORIGINAL content and applied in **descending** line order (an ascending pass shifts later spans out from under themselves); the **range** check is not redundant with the text check (`line: 0` makes `slice(-1, 0)` empty, so an empty `text` matches and `splice(-1, 1)` deletes the file's LAST line); and `createBackup` skips a nonexistent path while still returning an id, so coverage of every file about to be written is **asserted from the manifest** before a byte changes. The floor is *repeated* here, not moved: `floor-exclusion` still vetoes before anything is proposed, and the engine refuses a load-bearing block again so a hand-built approval cannot route around it. The archive rule (`mv` to `_archive/`) is file-level and does not apply to a block excision — the timestamped backup is the recovery artifact, and inventing a second copy with no restorer behind it would be worse than none.
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
node --test 'tests/**/*.test.mjs'
|
|
```
|
|
|
|
Test fixtures in `tests/fixtures/`. Per-scanner and per-build-block implementation notes (design rationale, primary-source verification, byte-stability lessons) live in `docs/scanner-internals.md` → **Implementation notes**.
|
|
|
|
## Gotchas
|
|
|
|
- Session directories accumulate — use `/config-audit cleanup` to manage
|
|
- Scanners run on Node.js ≥ 18 (uses node:test, node:fs/promises)
|
|
- Plugin CLAUDE.md files in node_modules should be excluded via scope
|