Compare commits
40 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| d5714261d1 | |||
| 44b222859e | |||
| b35ff449e8 | |||
| c3af74dfbc | |||
| adcf44fe4e | |||
| 7df8e0d65b | |||
| 05b4e9d797 | |||
| dc800560b7 | |||
| 30c78aeda0 | |||
| 749b710de7 | |||
| e60b80978b | |||
| 6bb100f2e0 | |||
| dbb6a6a3cf | |||
| 33bfd5ff5b | |||
| 000e47f9d2 | |||
| 1543830c52 | |||
| 49bae2657f | |||
| 9ae4be26d2 | |||
| e861e63a7b | |||
| 542f983178 | |||
| 7a794b47eb | |||
| 4027cdcf54 | |||
| 182a37c1af | |||
| 3d6ddb273c | |||
| c76dc537ce | |||
| 0b763f25c1 | |||
| ca199cc5f6 | |||
| b3f6866644 | |||
| 1b49bcc766 | |||
| d68b2152e3 | |||
| c8d4dc9421 | |||
| d66035ed86 | |||
| caea8aca23 | |||
| acd1cf1248 | |||
| 09f817977c | |||
| de8a7b5d51 | |||
| b85919f2ec | |||
| 001090261e | |||
| 05f1e954d0 | |||
| 1182f85767 |
170 changed files with 12451 additions and 1138 deletions
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"name": "config-audit",
|
||||
"description": "Multi-agent workflow for analyzing, reporting, and optimizing Claude Code configuration across your entire machine",
|
||||
"version": "5.13.0",
|
||||
"version": "6.0.0",
|
||||
"author": {
|
||||
"name": "Kjell Tore Guttormsen"
|
||||
},
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ All command files MUST include:
|
|||
---
|
||||
name: plugin:command
|
||||
description: Short description of what this command does
|
||||
allowed-tools: Read, Write, Bash, Task
|
||||
allowed-tools: Read, Write, Bash, Agent
|
||||
model: sonnet
|
||||
---
|
||||
```
|
||||
|
|
|
|||
318
CHANGELOG.md
318
CHANGELOG.md
|
|
@ -5,6 +5,324 @@ All notable changes to this project will be documented in this file.
|
|||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [6.0.0] - 2026-08-18
|
||||
|
||||
### Summary
|
||||
"Prose is not a contract" — a MAJOR release whose theme is a *class* of defect rather than a feature
|
||||
area. Across three sweeps (Q1, Q2, Q_AUDIT) the same shape kept surfacing: a command template stated
|
||||
an invariant in prose, code on the other side depended on it, and nothing checked that the two still
|
||||
agreed. The write-scope **gate** was policy paraphrased in five templates while exactly one writer
|
||||
imported it. The **argv** a template built was never checked against the CLI receiving it —
|
||||
`--stale-after 30` arrived as a single argument, matched no flag, and the command reported success
|
||||
about a threshold the user had just overridden. And the **data contracts** — backup manifests,
|
||||
`state.yaml`, `scope.yaml` — are hand-built by the model and parsed by engines that know one frozen
|
||||
example. The first two are now enforced in code and tests; the third is measured and ranked in
|
||||
`docs/q-audit-prose-invariants.md`, with the recovery path (`rollback`) at the top as the surface
|
||||
that runs precisely when the user is already in trouble.
|
||||
|
||||
**Breaking — a finding ID's `{NNN}` names the check, not its emission position (`M-BUG-28`).**
|
||||
IDs are therefore not unique per finding: one check failing in three files emits three findings
|
||||
sharing an ID, and `(id, file, line)` is the instance key. Any consumer keying on `id` alone must
|
||||
move to the triple. `scanners/lib/finding-codes.mjs` is now the single authority — an undeclared or
|
||||
missing code **throws**, with no counter fallback, because a fallback lets a half-converted scanner
|
||||
ship IDs that look valid. Retired numbers are never reissued. Frozen `v5.0.0` baselines mask IDs
|
||||
rather than re-deriving them.
|
||||
|
||||
**37** commits since 5.13.0. **1752** tests, 0 failing. GAP dimensions **25 → 24** (one `/doctor`
|
||||
duplicate retired). No component-count change: scanners **16**, agents **7**, commands **21**,
|
||||
hooks **4**, knowledge entries **8**.
|
||||
|
||||
### Fixed
|
||||
- **`M-BUG-45` — `/config-audit knowledge-refresh --stale-after N` was silently dead under zsh.** The
|
||||
command built `STALE_AFTER="--stale-after 30"` and expanded it unquoted, relying on the shell to
|
||||
split it into two argv entries. bash does; **zsh — the macOS default since Catalina — does not**.
|
||||
The CLI received one argv entry reading `--stale-after 30`, matched no flag, and fell back to the
|
||||
90-day default while reporting success: "✓ All 14 register entries were re-verified within the last
|
||||
90 days" — a true-sounding sentence about a threshold the user had just overridden. Measured:
|
||||
`set -- $STALE_AFTER; echo $#` prints 1 under zsh, 2 under bash. The threshold is now passed as its
|
||||
own quoted argument, and a guard rejects any command template that packs a flag and its value into
|
||||
one variable.
|
||||
- **`M-BUG-46` — four CLIs accepted unknown flags in silence.** No `else` branch at all in the parse
|
||||
loop, so an unrecognised flag vanished without a trace: a typo'd `--ledger-file` made
|
||||
`campaign-cli` report confidently on the *default* ledger instead of the one the caller named, and
|
||||
a mistyped `--stale-after` reverted to 90 days. This is what made `M-BUG-45` silent rather than
|
||||
loud. `campaign-cli` and `knowledge-refresh-cli` now fail with exit 3 and name the offending flag;
|
||||
`optimize-lens-cli` and `token-hotspots-cli` share the defect and are closed together with their
|
||||
positional-swallow arm in the v5.14 argument-handling work (tracked in the guard's `KNOWN_OPEN`).
|
||||
- **`M-BUG-47` — the machine-wide token bill counted repos it could not read.** `refresh-tokens`
|
||||
routed a repo to `skipped[]` only when `readActiveConfig` *threw*, but that function resolves any
|
||||
path and its sub-readers all tolerate ENOENT, so a repo that does not exist yields an empty config
|
||||
instead of an error. Measured: a phantom path landed in `swept[]` with a 0-token delta,
|
||||
`skipped[]` was empty, and the roll-up claimed `reposWithTokens: 3` for a machine with two real
|
||||
repos — so the command's own honesty clause ("name those repos plainly so the user knows the bill
|
||||
omits them") could never fire. Readability is now checked before the sweep.
|
||||
- **`M-BUG-48` — `campaign add` vouched for paths that do not exist.** `add /finnes/ikke` returned
|
||||
`added: [...]` and exit 0, and the phantom row then sat in the backlog permanently. Paths are still
|
||||
tracked (an unmounted volume is a legitimate reason for a repo to be absent today) but are now
|
||||
reported separately as `addedUnverified`, and `campaign.md` names them instead of glossing over them.
|
||||
- **`M-BUG-49` — `posture` reported a crash as a passing grade.** Its top-level catch set
|
||||
`process.exitCode = 1`, while every command in this plugin is told that "codes 0, 1, 2 are normal
|
||||
(PASS/WARNING/FAIL). Only 3 is a real error". A fatal error was therefore indistinguishable from a
|
||||
WARNING, and the command went on to Read a payload file that was never written. Measured as the
|
||||
single outlier: 1 of 14 scanners. Now exits 3.
|
||||
- **`M-BUG-50` — `knowledge-refresh` read one register and wrote another, and the gate saw neither.**
|
||||
Step 6 said `Edit knowledge/best-practices.json` — an unanchored relative path — while the CLI reads
|
||||
`${CLAUDE_PLUGIN_ROOT}/knowledge/best-practices.json`, which for a marketplace install is the plugin
|
||||
cache. So a normal user has no such file in their repo at all; in the plugin's own checkout the
|
||||
command read the cache and wrote the working tree; and step 6.3's validation gate ran the cached
|
||||
test against the cached register — **validating the copy that was not edited, and passing no matter
|
||||
what was written.** Every write step is now anchored, and the command states where the register
|
||||
actually lives (a marketplace copy is discarded on the next plugin upgrade).
|
||||
- **No scanner created its `--output-file` parent directory.** `saveLedger` always did; the payload
|
||||
write never did — an accidental asymmetry across all 13 writers. `commands/campaign.md` writes its
|
||||
report under `~/.claude/config-audit/sessions/`, so on a fresh machine — precisely the first run
|
||||
that `campaign-cli` otherwise handles gracefully with `initialized: false` — the write threw ENOENT
|
||||
and the command's exit-code table reported it as a possibly-corrupt ledger, steering the user away
|
||||
from the one action that would have helped. All payload writes now go through
|
||||
`scanners/lib/write-output.mjs`.
|
||||
- **`M-BUG-40`, fifth arm — `posture` wrote four temp files it could never read back.** #49 closed the
|
||||
`$$`/cross-block class in four commands, but `posture.md` survived it, and so did the guard written
|
||||
to prevent exactly this. The guard compared each `$$` path against the block that created it, so a
|
||||
path written **once** and then read via prose ("Read the JSON output file using the Read tool") had
|
||||
no second occurrence to flag. Measured live: the scanner wrote `/tmp/config-audit-posture-21614.json`
|
||||
from PID 21614 while the next Bash call ran as PID 23772, and the read step had no path to hand the
|
||||
Read tool at all. The invariant is now blanket — **no `$$` in any temp path in any command file** —
|
||||
which also caught `fix.md` and `feature-gap.md`. All five sites now use fixed literal paths, repeated
|
||||
literally in every step that needs them.
|
||||
- **`M-BUG-43` — commands leaked whole JSON payloads into the transcript.** Every scanner except
|
||||
`scan-orchestrator` writes its payload to stdout when `--raw`/`--json` is set, **even when
|
||||
`--output-file` was given** — and the command templates redirected only stderr. Measured on a real
|
||||
repo: `posture` 255 182 B, `whats-active` 35 922 B, `drift` 28 316 B, `manifest` 23 825 B, `tokens`
|
||||
8 768 B. `fix` and `feature-gap` were the worst case: both ran posture with `--json`, never read the
|
||||
file they wrote, and in practice recovered a single letter grade from a quarter-megabyte dump — in
|
||||
the plugin that exists to cut token cost. 13 invocations across 10 command files now redirect stdout,
|
||||
and the two commands that needed the data read it from their output file instead.
|
||||
- **`tokens` swallowed two documented flags.** `--json` and `--with-telemetry-recipe` were listed as
|
||||
recognized flags but never threaded into the CLI call, so `--json` returned the *humanized* payload
|
||||
where the docs promised byte-stable v5.0.0 output (measured: 4/4 findings carried humanizer fields,
|
||||
and the title read "Your file starts with content that changes between turns" instead of
|
||||
"Cache-breaking volatile content at top of CLAUDE.md"), while `--with-telemetry-recipe` silently
|
||||
produced no `telemetry_recipe_path` — the very flag the command's own closing tip recommends.
|
||||
- **`M-BUG-42` — `manifest` asked for a field the scanner never emits.** The render contract used
|
||||
`{load}`; the payload carries `loadPattern`. The Load column — which the command's own prose calls
|
||||
the whole point of the view — would render blank for all 96 rows. `posture`'s headline had the same
|
||||
shape (`{qualityAreaCount}`, never emitted) and now takes its count from the humanized scorecard
|
||||
rather than `areas.length`, which counts a Feature Coverage row the table below deliberately excludes.
|
||||
|
||||
### Removed
|
||||
- **GAP dimension `No autoMode classifier` (D1) — retired as a `/doctor` duplicate.** CC 2.1.226's
|
||||
`/doctor` Check 8 covers auto mode with usage-weighted judgement, and the binding positioning
|
||||
(README «config-audit vs. the built-in /doctor») forbids carrying a feature whose whole value is
|
||||
duplicating a `/doctor` check. What is retired is only the *"adopt this feature"* nudge; the
|
||||
deterministic side stays untouched — SET still validates `autoMode` structure and still flags it
|
||||
as dead config in shared project settings. GAP dimensions: **25 → 24**.
|
||||
|
||||
The title lived in **four** tables, not the two the removal was scoped against: the dimension
|
||||
list, the scoring `TITLE_TO_ID` map, the humanizer's static translations, and — the one that
|
||||
moves a user-visible number — the scoring denominators (`TIER_COUNTS` t3 8→7,
|
||||
`TOTAL_DIMENSIONS` 25→24, `MAX_WEIGHTED` 42→41). `findGapId` falls back to `'unknown'` silently,
|
||||
so a partial removal would have degraded without failing. A blanket sync invariant now asserts
|
||||
all four against `GAP_CHECKS` rather than checking occurrences pairwise, and each arm was
|
||||
verified red against its own defect.
|
||||
|
||||
Utilization shifts accordingly (fixture: 43 → 44). `risk_score`, `risk_band`, `verdict`,
|
||||
`overallGrade`, `maturity` and `segment` are byte-identical across the change — the dimension was
|
||||
severity `info` (zero risk weight) and GAP is excluded from the overall grade.
|
||||
|
||||
**Frozen `tests/snapshots/v5.0.0/` stays untouched.** The removal-twin normalizer
|
||||
(`tests/helpers/strip-retired-gap.mjs`, mirroring `strip-added-scanner.mjs`) strips the retired
|
||||
dimension from whichever side still carries it and re-derives GAP IDs — retiring a dimension from
|
||||
mid-list shifts every later ID by one. The derived utilization figures are dropped from
|
||||
comparison rather than recomputed, since recomputing them in a test helper would assert the new
|
||||
arithmetic against itself; they are covered exactly in `tests/lib/scoring.test.mjs`. Re-seeding
|
||||
the baselines was rejected: it would silently bake in any other drift accumulated across every
|
||||
scanner those four files cover.
|
||||
|
||||
### Added
|
||||
- Four command-template shape tests (1449 → 1453), each verified to fail before the fix: a blanket
|
||||
`$$` ban, stdout-redirect discipline for any scanner invoked with `--output-file` in raw/json mode,
|
||||
flag threading from prose to shell, and a render-contract test that checks every `{field}` against a
|
||||
**live payload generated from a fixture** rather than a hardcoded key list, which would drift.
|
||||
|
||||
- **`M-BUG-40` — command templates assumed shell state survives between fenced blocks.** It does not:
|
||||
every ```` ```bash ```` fence is executed as its own Bash call, in its own process. A variable
|
||||
assigned in one block is empty in the next, and `$$` (the PID) differs between calls, so a
|
||||
`/tmp/foo-$$.json` path created in one block can never be reconstructed in a later one. The defect
|
||||
was surfaced by dogfooding `plan` + `implement`, and **confirmed at runtime by the planner agent
|
||||
itself**, which reported that `Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was
|
||||
documented in both files while being functionally dead. A machine sweep found the same root in
|
||||
**20 places across 9 files**, far past the two predicted: `$RAW_FLAG` referenced from non-shell
|
||||
agent prompts (`analyze`, `plan`, `implement`); `$TMPFILE` referenced across blocks in `tokens`,
|
||||
`manifest`, `whats-active` and `plugin-health`, so each command could not read the file it had just
|
||||
written; `$GLOBAL_FLAG` in `fix`; `$TODAY` in `campaign`, which was **never assigned in any block**
|
||||
and passed `--reference-date ""` to a write CLI; and three `$$` temp paths handed to the Read tool
|
||||
in `fix`, which expands neither `$$` nor variables. All now follow the hardened `drift.md` pattern:
|
||||
a fixed literal path, or a re-derivation inside each block that needs it.
|
||||
- **`implement` handed out a rollback ID it never captured.** The backup directory was created with
|
||||
`mkdir -p .../$(date +%Y%m%d_%H%M%S)/`, so the timestamp existed only inside a command
|
||||
substitution, while step 6 promised `/config-audit rollback {timestamp}` — the one message a user
|
||||
reads after a bad run. The step now prints `BACKUP_ID` and substitutes it literally.
|
||||
- **`plan` reported "No analysis results found" for valid sessions.** Step 1 pointed the Read tool at
|
||||
`~/.claude/config-audit/sessions/*/state.yaml`; Read takes one literal path and does not expand
|
||||
`*`, so the lookup failed and the command reported the session as missing. It now uses Glob, and
|
||||
additionally verifies `analysis-report.md` exists before spawning the planner agent — a session can
|
||||
carry a valid `state.yaml` and still be missing its report.
|
||||
- **Phase commands wrote `state.yaml` with two of the four required fields.** `.claude/rules/state-management.md`
|
||||
mandates `current_phase`, `completed_phases`, `next_phase` and `updated_at`; `analyze`, `discover`,
|
||||
`implement`, `interview` and `plan` named only a subset. Because the planner agent writes all four,
|
||||
a follow-up full-file Write naming two **deletes** the other two — the fields that make an
|
||||
interrupted run resumable.
|
||||
- **`implement` documented a rollback semantics that does not exist.** Its "## Rollback" section
|
||||
promised to "delete newly created files", while `rollback.md` deliberately leaves them in place and
|
||||
lists them under "Left in place" (deletion is unimplemented; `M-BUG-26` remains open). The doc now
|
||||
mirrors actual behaviour rather than describing a half-restore as clean.
|
||||
- **`implement` claimed a score delta with no source**, since nothing captured the pre-change grade
|
||||
before the edits ran, and its implied posture call omitted both `--output-file` and `2>/dev/null`
|
||||
required by the output rules. It now reports a delta only when a pre-change grade was actually
|
||||
measured.
|
||||
- **`verifier-agent` was instructed to write a report it has no tool to write** (`tools: Read, Glob,
|
||||
Grep`, and "Read-only validation" by design). It now returns findings as its final message and the
|
||||
command appends them with Bash `>>`, preserving both the read-only design and the shared-log
|
||||
append discipline.
|
||||
- **Dead tool name in always-loaded context:** `.claude/rules/command-development.md` taught
|
||||
`allowed-tools: ... Task` while every command uses `Agent`, and `interview.md` carried two more
|
||||
`Task` references. `planner-agent.md` also contradicted itself — line 110 forbids inline file
|
||||
content while its own output template demonstrated exactly that, pushing plans past the 200-line
|
||||
budget the same file sets.
|
||||
|
||||
### Fixed (previously released work)
|
||||
- **`M-BUG-39` — every scanner CLI could truncate its own output when piped.** `process.exit()`
|
||||
terminates immediately, but Node writes stdout **asynchronously** when stdout is a pipe, so whatever
|
||||
is still buffered is discarded. `scan-orchestrator.mjs` measured **246 854 bytes to a file vs
|
||||
65 536 to a pipe** (and 131 072 on another run — the cut point is a nondeterministic flush race),
|
||||
handing any machine consumer truncated, unparseable JSON that reads like a corrupt file rather than
|
||||
a cut-off. Reported by `org-ops`, whose census pipes our envelope. The whole class is closed, not
|
||||
just the CLI where it was observable: `campaign-cli`, `campaign-export-cli`, `campaign-write-cli`,
|
||||
`knowledge-refresh-cli`, `drift-cli` and `fix-cli` all exited the same way on their success paths
|
||||
and were green only because their payloads happen to fit the pipe buffer today. All 38 sites across
|
||||
14 files now set `process.exitCode` and return, letting Node exit once stdout drains — the pattern
|
||||
`self-audit.mjs` and llm-security's orchestrator already used. `fail()` throws instead of exiting so
|
||||
it keeps its never-returns contract; exit codes and `Error:`/`Fatal:` stderr text are unchanged.
|
||||
Guarded by `tests/scanners/cli-pipe-integrity.test.mjs`: one behavioural test that pipes a >128 KB
|
||||
envelope, one class sweep over `scanners/*.mjs`.
|
||||
- **`M-BUG-36` — `/config-audit drift --list` showed nothing.** `drift-cli.mjs` accepted
|
||||
`--output-file` but list mode ignored it, and the listing itself goes to **stderr**, which
|
||||
`commands/drift.md` discards with `2>/dev/null` (ux-rules rule 2). The command received **0 bytes**
|
||||
and could render no baselines at all. List mode now honours `--output-file`; `--raw`/`--json` stdout
|
||||
is unchanged and byte-stable. Fourth instance of the stderr-only class after `M-BUG-33`.
|
||||
- **`M-BUG-37` — `/config-audit feature-gap` promised a backup it never made.** Step 6's
|
||||
"Create backup" ran `fix-cli.mjs <path> --json`, but fix-cli is **dry-run by default**: no backup was
|
||||
written and `backupId` came back `null`, after which the command edited the user's configuration
|
||||
believing it could be restored. Passing `--apply` would have been worse — it executes unrelated
|
||||
auto-fixes the user never selected. The step now copies the files itself and states plainly that
|
||||
plain copies are restored by copying them back, not by `/config-audit rollback` (`M-BUG-31` class).
|
||||
- **`M-BUG-38` — `fix-cli.mjs` sent users to a script that does not exist.** After applying fixes it
|
||||
printed `Rollback: node scanners/rollback-cli.mjs <id>`; there is no `rollback-cli.mjs` — only
|
||||
`rollback-engine.mjs`, driven by `/config-audit rollback`. A dead reference in the one message a
|
||||
user reaches for after a bad fix. Now points at the command.
|
||||
- **`M-BUG-21` (fourth arm) — command templates fed bracketed placeholder flags to the arg loops.**
|
||||
Five templates (`config-audit.md`, `discover.md`, `fix.md`, `tokens.md`, `whats-active.md`) carried
|
||||
literal `[--global]` / `[--full-machine]` / `[--verbose]` inside executable bash blocks. A bracketed
|
||||
placeholder does not start with `-`, so every scanner's `else if (!args[i].startsWith('-'))` branch
|
||||
takes it as the **scan target** — silently scanning a path that does not exist. Replaced with empty
|
||||
shell variables that expand to nothing when the flag does not apply.
|
||||
- **`/config-audit interview` and `analyze` never said which session they act on.** Both referenced
|
||||
`{session-id}` with no resolution rule, while every other session-aware command globs
|
||||
`sessions/*/state.yaml` and takes the most recent. Two runs could write to two different sessions.
|
||||
Both now resolve the session explicitly and exit when none exists.
|
||||
- **`/config-audit interview` could rewind a finished session.** The mandated state write had no bound,
|
||||
so running the optional interview against a session that had already reached `implement` reset
|
||||
`current_phase` and re-added phases. It now appends `interview` only if absent and leaves the
|
||||
furthest phase reached intact.
|
||||
- **`/config-audit cleanup` interpolated an unvalidated id into `rm -rf`.** An empty or malformed
|
||||
`{session-id}` expands the path to `sessions//`, deleting **every** session. The id must now match
|
||||
`^[0-9]{8}_[0-9]{6}$` or come verbatim from the directory listing; anything else is refused and
|
||||
reported.
|
||||
- **`/config-audit status` advertised a command that does not exist.** It documented
|
||||
`/config-audit resume {session-id}`; there is no `resume` command. Replaced with how session
|
||||
selection actually works. A test now fails on any `/config-audit <word>` reference in `commands/`
|
||||
without a matching file.
|
||||
- **`/config-audit status all` was documented but never parsed.** The flag-parse step knew only
|
||||
`--raw`. It now parses `all` and routes to the all-sessions table.
|
||||
|
||||
### Fixed (previously)
|
||||
- **`M-BUG-21` (third arm) — `plugin-health-scanner.mjs` swallowed unknown flags, and the wrong
|
||||
target looked *green*.** The same `else if (!args[i].startsWith('-')) targetPath = args[i]` loop:
|
||||
`--output-file /tmp/x.json` was dropped and `/tmp/x.json` became the scan target. Where `drift`
|
||||
produced phantom drift, this produced a **reassuring** answer — a non-existent path discovers no
|
||||
plugins, so the scanner reported `No plugins found` (info) and exit `0`. Unknown options and a
|
||||
value-less `--output-file` now exit `3`.
|
||||
- **`M-BUG-33` — `/config-audit plugin-health` read zero bytes.** The scanner had no `--output-file`
|
||||
(ux-rules rule 2) and its default-mode report goes to **stderr**, which `commands/plugin-health.md`
|
||||
discards with `2>/dev/null` before telling the agent to "read stdout output (JSON)". The command's
|
||||
default path could not produce the report it documents. `--output-file` now writes a humanized
|
||||
payload; `--raw`/`--json` stdout is unchanged and byte-stable.
|
||||
- **`M-BUG-34` — the report's per-plugin table and Cross-Plugin section were unbuildable.** Per-plugin
|
||||
data (`commandCount`, `agentCount`) and the grade formula never left `scan()` — the only grade code,
|
||||
`formatPluginHealthReport`, had no caller — and cross-plugin findings were flattened into `findings`
|
||||
behind a `category: 'plugin-hygiene'` they share with per-plugin findings. The command mandated both,
|
||||
so it had to fabricate them. The payload now carries `plugins[]` (name, declaredName, counts, score,
|
||||
grade via the shared `pluginGrade`) and `cross_plugin_findings[]` (also marked `crossPlugin: true`),
|
||||
via a new `scanDetailed()`; `scan()`'s frozen v5.0.0 envelope is untouched.
|
||||
- **`M-BUG-35` — `.claude-plugin/marketplace.json` was reported as an unknown file.** It is the
|
||||
documented, required location for a marketplace catalog, and a marketplace entry with
|
||||
`"source": "./"` makes the repo root its own plugin — such a repo legitimately carries both files.
|
||||
Genuinely unexpected files in `.claude-plugin/` are still flagged.
|
||||
- **`commands/posture.md` discarded both optional scanners' output.** Its `--drift` and
|
||||
`--plugin-health` sections ran `drift-cli.mjs` / `plugin-health-scanner.mjs` in default mode under
|
||||
`2>/dev/null` and read stdout, which is empty in that mode. Both calls now use `--output-file`.
|
||||
- **`M-BUG-21` — `drift-cli.mjs` had no `--output-file`, and its argument loop turned the missing
|
||||
flag into a wrong scan target.** The loop ended in `else if (!arg.startsWith('-')) targetPath = arg`
|
||||
with no unknown-flag branch, so an unrecognised flag was dropped silently and its *value* fell
|
||||
through to the scan target: `drift-cli.mjs . --output-file /tmp/x.json` scanned `/tmp/x.json`, a
|
||||
path that does not exist, and reported the resulting near-empty scan as drift — permanently, and
|
||||
without a warning. The same silence was destructive for `--save --name` with the value omitted:
|
||||
`--name` was ignored, the name stayed `default`, and an existing baseline was **overwritten**.
|
||||
Unknown options and value-less `--name`/`--baseline`/`--output-file` now exit `3`.
|
||||
- **`M-BUG-21` (second arm) — `/config-audit drift` captured nothing at all.** `commands/drift.md`
|
||||
ran the CLI under `2>/dev/null` and told the agent to "read stdout", but the default-mode report,
|
||||
the `--save` confirmation, and the `--list` output all go to **stderr**. All three modes returned
|
||||
empty. `--output-file` now writes the diff (humanized in default mode, raw under `--json`/`--raw`,
|
||||
matching `posture.mjs`), and the command reads that file; `--save` passes `--json` for its
|
||||
confirmation.
|
||||
- **`M-BUG-27` — `drift` compared against baselines anchored to a different directory and called it
|
||||
"improving".** `diff-engine` never checked the baseline's stored `target_path` against the current
|
||||
scan target. Diffing a repo against a baseline saved elsewhere marked every baseline finding
|
||||
"resolved" and every current finding "new" — a 100% phantom diff surfacing as a *reassuring* trend,
|
||||
on the **default** baseline. The CLI now warns on stderr in every mode and carries
|
||||
`_baselineAnchor {matches, baselineTarget, currentTarget}` in the default-mode payload, so a caller
|
||||
running under `2>/dev/null` can still see it. `--json`/`--raw` stdout stays v5.0.0-shaped and the
|
||||
frozen `drift.json` snapshot is untouched.
|
||||
|
||||
- **`M-BUG-21` (third arm) — `fix-cli.mjs` had the same unvalidated argument loop, where it moves the
|
||||
*write* target.** An unrecognised flag was dropped and its value became the scan target, so
|
||||
`fix-cli.mjs <repo> --output-file /tmp/x.json` silently audited `/tmp/x.json`; with `--apply` the
|
||||
same slip relocates what gets written. Unknown options and value-less `--output-file` now exit `3`.
|
||||
`--dry-run` — documented in `commands/fix.md`'s `argument-hint` but never implemented — is now
|
||||
accepted instead of silently dropped, and `--output-file` writes the fix payload to disk so
|
||||
`commands/fix.md` can read a file rather than parse stdout it runs under `2>/dev/null`.
|
||||
- **`M-BUG-31` — `fix` promised a mandatory backup it did not always take.** `fix-cli.mjs` excluded
|
||||
`file-rename` from the backup set, so a rule file whose only defect was its extension was renamed
|
||||
with **no** backup entry — while the command told the user "every fix creates a backup first" and
|
||||
handed back a `backupId` that could not restore it. The source file is now backed up like any other.
|
||||
- **`M-BUG-32` — verification re-scanned a different scope than the fix run.** `verifyFixes` hardcoded
|
||||
`includeGlobal: false`. After a `--global` run every user-scope finding fell out of the re-scan and
|
||||
was therefore counted as *verified*: a clean "fixed" report for files nothing had touched
|
||||
(reproduced against an untouched `~/.claude/CLAUDE.md`). It now inherits the run's scope, and
|
||||
`commands/fix.md` passes `--global` to every step instead of only the display scan.
|
||||
- **`M-BUG-29` — two fixes on one file were applied in an order that guaranteed failure.** A rule file
|
||||
with both `globs:` and a non-`.md` extension had the rename applied first; the frontmatter fix then
|
||||
failed with `ENOENT`. Renames now sort after every other fix.
|
||||
- **`M-BUG-30` — critical fixes sorted last.** `severityOrder[s] || 4` maps `critical` (weight `0`) to
|
||||
`4`, the opposite of the documented "critical first" contract. The old test used the same falsy
|
||||
fallback, so it agreed with the bug. Now `?? 4`.
|
||||
- **A failed fix no longer exits `0`.** `fix-cli.mjs` returns `2` when any planned fix failed, matching
|
||||
the `0/1/2 = PASS/WARNING/FAIL`, `3 = error` convention the other scanners follow.
|
||||
|
||||
**1420** tests (+10). No count change (scanners **16**, agents **7**, commands **21**, hooks **4**).
|
||||
|
||||
## [5.13.0] - 2026-07-31
|
||||
|
||||
### Summary
|
||||
|
|
|
|||
176
CLAUDE.md
176
CLAUDE.md
|
|
@ -4,6 +4,8 @@ Claude Code Configuration Intelligence — know if your config is correct, find
|
|||
|
||||
Per-command flags, patterns, and feature lists live in `README.md` and `/config-audit help`. This file carries what's invariant for working on the plugin.
|
||||
|
||||
**Positioning vs. built-in `/doctor` (measured 2026-08-03, binding):** we are the deterministic/reproducible/all-scope/zero-quota side; `/doctor` is usage-weighted one-shot judgment. Never build a feature whose whole value is duplicating a `/doctor` check — see README «config-audit vs. the built-in /doctor» and `docs/v5.13-model-routing-effort-deadref-plan.md` §A.
|
||||
|
||||
## Commands
|
||||
|
||||
### Core (just run `/config-audit` to get started)
|
||||
|
|
@ -15,7 +17,7 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
|
|||
| `/config-audit tokens` | Prompt-cache-aware token hotspots, each tagged with its load pattern; cache-aware |
|
||||
| `/config-audit manifest` | Ranked table of every token source + always-loaded subtotal |
|
||||
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact |
|
||||
| `/config-audit optimize` | Mechanism-fit lens (procedure→skill, lifecycle→hook, path→rule, never→permission). Agent-driven, **not byte-stable**. `--subtract` adds the subtraction axis (what no longer earns its always-loaded rent, `BP-SUB-001`) — opt-in, proposes only |
|
||||
| `/config-audit optimize` | Mechanism-fit lens (procedure→skill, lifecycle→hook, path→rule, never→permission). Agent-driven, **not byte-stable**. `--subtract` adds the subtraction axis (what no longer earns its always-loaded rent, `BP-SUB-001`) — opt-in, proposes only; `--subtract --apply` executes the removals the operator picks; `--subtract --for-model <name>` annotates the candidates a named model documents as redundant (`BP-PROMPT-001`) |
|
||||
| `/config-audit fix` | Auto-fix deterministic issues with backup + verification |
|
||||
| `/config-audit rollback` | Restore configuration from backup |
|
||||
| `/config-audit plan` | Create action plan from findings |
|
||||
|
|
@ -28,7 +30,7 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
|
|||
|---------|-------------|
|
||||
| `/config-audit drift` | Compare current config against saved baseline |
|
||||
| `/config-audit plugin-health` | Audit plugin structure, frontmatter, cross-plugin coherence |
|
||||
| `/config-audit whats-active` | Read-only inventory of active plugins/skills/MCP/hooks/CLAUDE.md (with token estimates) |
|
||||
| `/config-audit whats-active` | Read-only inventory of active plugins/skills/agents/MCP/hooks/CLAUDE.md (with token estimates, and `model`/`effort` per agent) |
|
||||
| `/config-audit knowledge-refresh` | Refresh the best-practices register (stale check + web poll). Human-approved writes; **not byte-stable** |
|
||||
| `/config-audit campaign` | Machine-wide audit ledger + token bill across repos. Human-approved writes; **not byte-stable** |
|
||||
| `/config-audit discover` | Run discovery phase only |
|
||||
|
|
@ -65,7 +67,7 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
|
|||
|
||||
## Plain-Language Output (v5.1.0)
|
||||
|
||||
Default output of all commands routes through `humanizeEnvelope` (`lib/humanizer.mjs`), decorating each finding with `userImpactCategory`, `userActionLanguage`, and `relevanceContext`. `--raw` and `--json` bypass the humanizer for byte-stable v5.0.0 output. Full detail: `docs/humanizer.md`.
|
||||
Default output of all commands routes through `humanizeEnvelope` (`scanners/lib/humanizer.mjs`), decorating each finding with `userImpactCategory`, `userActionLanguage`, and `relevanceContext`. `--raw` and `--json` bypass the humanizer for byte-stable v5.0.0 output. Full detail: `docs/humanizer.md`.
|
||||
|
||||
## Suppressions
|
||||
|
||||
|
|
@ -77,6 +79,41 @@ Workflow: `/config-audit → discover + analyze (auto) → plan → implement
|
|||
|
||||
Finding ID format: `CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-SKL-001`, `CA-OST-001`, `CA-OPT-001`, `CA-AGT-001`.
|
||||
|
||||
**GAP dimensions vs. levers (invariant).** `GAP_CHECKS` holds the 24 *dimensions* — always evaluated, always counted in the utilization denominators (`TIER_COUNTS` / `TOTAL_DIMENSIONS` in `scoring.mjs`, and `TITLE_TO_ID` there). A *lever* is a finding the scanner emits after the loop and only under a measured condition; it carries no tier, never enters those denominators, and is registered in the exported `LEVERS` object (code + title in one place, because the finding-code guard needs the code and the humanizer-coverage guard needs the title). Adding a dimension moves every user's utilization score and can flip the reported `segment` in a frozen baseline — adding a lever cannot. When a check is only meaningful for configs that already have some feature, it is a lever.
|
||||
|
||||
**`{NNN}` names the CHECK, never the emission position (invariant).** `scanners/lib/finding-codes.mjs` is the single authority: every `finding()` call passes a `code`, and an undeclared or missing one **throws** — there is no counter fallback, because a fallback lets a half-converted scanner ship IDs that look valid. Adding a check takes the next free number for that scanner, never the next source-order position; removing one moves its key to `RETIRED_CODES` and its number is never reissued. IDs are therefore **not unique per finding** — one check failing in three files emits three findings sharing an ID, and `(id, file, line)` is the instance key that `fix-engine` verification uses. Frozen `v5.0.0` baselines mask IDs (`tests/helpers/mask-finding-ids.mjs`) instead of re-deriving them; the check→number pairs are pinned exhaustively in `tests/lib/finding-codes.test.mjs`.
|
||||
|
||||
**A file's contract cannot exceed its tools (invariant).** `agents/verifier-agent.md` said both
|
||||
"Append to: implementation-log.md" (§Output Format) and "never modifies any files" (§Read-Only
|
||||
Guarantee) while granting only `Read, Glob, Grep`. The failure mode is not a blocked write — it is
|
||||
the agent improvising a full-file `Write` on the log it *shares* with the parallel implementer
|
||||
agents, which is the exact defect `implement-log-append.test.mjs` exists to prevent, entering
|
||||
through the one file that test does not read. `tests/agents/agent-write-contract.test.mjs` asserts
|
||||
the blanket invariant over the whole catalogue rather than a fact about one file: an agent whose
|
||||
`tools:` grant no write capability (`Write`/`Edit`/`NotebookEdit`/`Bash`) must instruct no file
|
||||
write **and** state positively that it returns its findings inline. Both sides are read from the
|
||||
file, so stripping `Write` from any agent whose body still instructs a write turns it red, and the
|
||||
sweep asserts a write-tool-less agent exists so the invariant cannot pass vacuously. Measured
|
||||
2026-08-20: **1 of 7** agents carried the defect; the other six all hold `Write`. The orchestrator
|
||||
half was already correct — `implement.md` Step 5 appends with Bash `>>` and tells the agent not to
|
||||
write — so this was a one-way fix in the agent file, not a two-sided one.
|
||||
|
||||
**Frontmatter contracts are derived, never listed (invariant).** `.claude/rules/agent-development.md`
|
||||
and `.claude/rules/command-development.md` state which keys an agent/command MUST carry and that
|
||||
agent colors are unique; nothing enforced any of it, and the only test that read agent frontmatter
|
||||
checked `name:` against a hand-written 3-of-7 list. `tests/agents/frontmatter-contract.test.mjs`
|
||||
takes nothing by hand: required keys are parsed from each rule's own fenced `yaml` block, the files swept
|
||||
are resolved from each rule's own `paths:`, and the plugin name is read from
|
||||
`.claude-plugin/plugin.json` — add a key to a rule and it is enforced on the next run with no test
|
||||
edit, because a hand-kept list of what to sweep is a premise, not a measurement. The repo was
|
||||
already compliant (7/7 agents, 21/21 commands, 0 duplicate colors), so a green first run proves
|
||||
nothing: every arm was seen red against a temporarily introduced defect **one at a time**,
|
||||
including emptying a rule's yaml block to show the derivation is not vacuous. The color **enum** is
|
||||
deliberately *not* guarded — the official subagent docs list `red/blue/green/yellow/purple/orange/
|
||||
pink/cyan` (no `magenta`) while issue #19292 lists `magenta` but neither `purple` nor `orange`, and
|
||||
this plugin ships both `magenta` and `orange`; pinning an unsettled set in a guard would encode an
|
||||
unverified premise rather than measure one.
|
||||
|
||||
## Conventions
|
||||
|
||||
Enforced conventions live in `.claude/rules/` (auto-loaded as project instructions):
|
||||
|
|
@ -87,8 +124,141 @@ Enforced conventions live in `.claude/rules/` (auto-loaded as project instructio
|
|||
|
||||
Coding style: scanners are zero-dependency Node ESM; new findings use the `CA-{SCANNER}-{NNN}` ID format; byte-stable CLIs are verified against frozen `tests/snapshots/v5.0.0/` baselines.
|
||||
|
||||
**Write-scope gate (invariant).** Every write target is classified by `scanners/lib/write-scope.mjs` before it reaches an approval surface, and the **scope class decides the gate's strength — never the command asking**. Five command-owned policies would drift apart the way five copies of the lever table did. `SCOPE_CLASSES` is the single source for class, gate (`silent`/`disclose`/`require-ok`), wording and predicate; templates render `disclosures[]` from `write-scope-cli.mjs` rather than restating what a class means. Two orderings in that object are load-bearing and were measured, not reasoned about: `plugin-managed` before `user-scope` (both `~/.claude/config-audit/` and legacy `~/.config-audit/` are live, so the other order fires the gate on every session write and gets it switched off), and `user-scope` before `cross-repo` (`~/.claude/.git` exists, so a plain `.git`-upward walk calls `~/.claude/CLAUDE.md` merely "another repo" and silently downgrades the strongest gate). `disclose` ≠ `require-ok`: `campaign export` is cross-repo *by design*, so tightening it into a refusal breaks the feature. Distinct from the `require-target-dir.mjs` guard, which asks whether a scan **root** is readable (exit 3) — a different invariant, not to be merged.
|
||||
|
||||
**Write-gate coverage (invariant).** The gate above only counts where it is *called*, and for four
|
||||
releases it was called from prose: `write-scope.mjs` existed, but exactly one writer imported it
|
||||
(`lib/subtraction-write.mjs`) while five command templates paraphrased the policy. Measured
|
||||
2026-08-12: 9 files under `scanners/` write to disk, 1 imported the gate. The defect was never
|
||||
"8 ungated writers = 8 bugs" — four of them write the plugin's own bookkeeping and MUST stay
|
||||
ungated, because a gate that fires on every run gets switched off. The defect is that **nothing
|
||||
declared which**, so the question was answered by reading, and answered differently each time.
|
||||
`tests/lib/write-gate-coverage.test.mjs` is now the authority: every writer must either import
|
||||
the gate or hold an `EXEMPT` entry naming **where the bytes land**. Three properties are
|
||||
load-bearing. (1) **A rationale is a claim, not a label** — `scan-orchestrator` was carried in
|
||||
the plan text as exempt while `--save-baseline` derived its path from the *scan target*, so
|
||||
`--global` landed `~/.claude/.config-audit-baseline.json` (`user-scope`/`require-ok`); it is
|
||||
gated, and `lib/baseline.mjs` — which writes only under `~/.config-audit/baselines` — is the
|
||||
genuinely exempt one. (2) **Sync variants count**: `writeFile(` does not match `writeFileSync(`,
|
||||
and `lib/backup.mjs` uses only the sync forms, so the first sweep scored a real writer as clean
|
||||
and was green on its own subject. (3) **The sweep asserts non-emptiness** — a regex that stops
|
||||
matching makes every other assertion here vacuously green. The exemption table is stale-checked
|
||||
in both directions: an entry naming a file that no longer writes, or one that has since been
|
||||
gated, fails. `evaluateWriteTargets` in `write-scope.mjs` is the one reduction (classify →
|
||||
`strongestGate` → dedup disclosures) that all five call sites share; four copies of those four
|
||||
lines is the drift shape `SCOPE_CLASSES` exists to prevent one level down. Approval is carried by
|
||||
`--approve-scope`, and **classifying is not approving**: a template that sets the flag because it
|
||||
already ran `write-scope-cli` has rebuilt the prose contract this guard replaced.
|
||||
|
||||
**Command→CLI flag contract (invariant).** A command template is a caller with no compiler
|
||||
behind it: it names a scanner and an argv, and nothing used to check that the scanner still
|
||||
accepts them. The measured cost is M-BUG-45 — `--stale-after` reached its CLI malformed, was
|
||||
ignored, and the command reported "✓ all 14 entries re-verified within the last 90 days" about
|
||||
a threshold the user had just overridden. `tests/commands/command-cli-contract.test.mjs` closes
|
||||
that seam, and four properties are load-bearing. (1) **The argv is built from the template's own
|
||||
text** (`tests/helpers/command-invocations.mjs`), never hand-typed — a hand-written call is a
|
||||
path no user takes (#63). Flags appear in *three* forms and all three are read: literal,
|
||||
`if …; then RAW_FLAG="--raw"; fi`, and **comment-only** (`GLOBAL_FLAG="" # --global`); the third
|
||||
is the class that dies unobserved, because the default path leaves the variable empty. (2) **The
|
||||
probe proves itself per CLI** — each must first be seen rejecting a flag that certainly does not
|
||||
exist, or a CLI that exits on a required-arg check before reaching flag parsing passes every pair
|
||||
vacuously. Measured 15/15 report the unknown flag first, so no prefix-argv table is needed, and
|
||||
the second copy of `cli-unknown-flag-rejection`'s `GUARDED` table was therefore never created.
|
||||
(3) **"Unknown" is told from "needs a value" by the CLI's own words**, which is only sound because
|
||||
every CLI classifies the two correctly — measured 14/15, and the fifteenth
|
||||
(`campaign-export-cli`, the last hand-rolled parser, whose `argv[i+1] !== undefined` guards let a
|
||||
trailing `--repo` fall through to the catch-all and be reported as an unknown flag) was moved onto
|
||||
the shared `requireValidArgs` gate rather than special-cased in the test. (4) **Probing a flag
|
||||
runs the CLI, and some flags are writers** — the first run of this guard let `drift-cli --save`
|
||||
default its target to the cwd and overwrite the operator's real
|
||||
`~/.config-audit/baselines/default.json`. Every probe now runs under `hermeticEnv()` with its own
|
||||
empty cwd, and the cwd is *asserted* empty afterwards: isolation that is only a convention is not
|
||||
isolation. Not asserted here: that a template calling a gated writer also calls `write-scope-cli`
|
||||
— measured false-red (`discover`/`config-audit` invoke `scan-orchestrator` without reaching its
|
||||
`--save-baseline` write), so that arm stays in `write-scope-gate-shape.test.mjs`.
|
||||
|
||||
**Dead-prose-reference silence list (invariant).** `CA-CML-013` is a precision-first check, so its
|
||||
design lives in what it *declines* to flag, and that list is measured (407 real CLAUDE.md files),
|
||||
never argued. Three rules are load-bearing and each has a guard seen red against its own defect.
|
||||
(1) **Containment is checked against the scan root, not the file's own directory** — a `../` chain
|
||||
that leaves the tree is silenced (`outside-scan-tree`) rather than resolved, because a base a `..`
|
||||
chain can escape is not a base: measured, `../../../../etc/passwd` resolved to the real file and
|
||||
silenced its own finding. A legitimate `../docs/x.md` inside the same repo still resolves.
|
||||
(2) **A bare token is a concept, not a reference** — `README.md` (no separator) and `docs/`
|
||||
(single segment) are excluded on the same reasoning one level apart; admitting bare filenames
|
||||
tripled the output with name-drops of tools living elsewhere, and single-segment folders are 26 %
|
||||
of the remainder, led by a remote namespace prefix. (3) **Rule ORDER is the reported reason** —
|
||||
first match wins, so `npm test` is silenced as `whitespace` (a command), not as `no-separator`,
|
||||
and two silences with different causes keep their own fixtures. The check emits **one finding per
|
||||
file** (the `todo-markers` / `repeated-content` idiom), because per-token emission measured 699
|
||||
findings where per-file measured 128. Silence is the safe failure direction here: a path carrying
|
||||
a trailing `:54-56` locator is a recorded v1 miss, not a bug to fix by loosening a rule.
|
||||
|
||||
**Subtraction floor (invariant).** `optimize --subtract` is the only lens that proposes removing config, so `scanners/lib/floor-exclusion.mjs` runs as a deterministic pre-step *before* the judge — a load-bearing block is never a candidate, and that guarantee must not be moved into the agent prompt. Two rules follow from it: (1) **staleness is not a deletion signal** — an outdated version pin inside a floor block is a `drift`/`CA-CML` dead-reference concern; (2) **tier 2 ≠ tier 3** — a compensatory block that keeps earning its place returns, and reporting it as dead weight is wrong even when the label matches. Norwegian keywords need the Unicode boundaries in `subtraction-prefilter.mjs`; JS `\b` is ASCII-only, so `/\bunngå\b/` silently never matches.
|
||||
|
||||
**Model-scoped annotation (invariant).** `--for-model <name>` (`BP-PROMPT-001`,
|
||||
`scanners/lib/prompting-model-scope.mjs`) is an **annotation on existing
|
||||
`BP-SUB-001` candidates, never a detector** — it tags a subset of what
|
||||
`--subtract` already found and can never widen the candidate set. Four
|
||||
properties are load-bearing. (1) **The model must be named.** There is no
|
||||
auto-detection and there cannot be: a CLAUDE.md has no frontmatter and no
|
||||
resolvable target model, and the operator's own `route` skill deliberately runs
|
||||
a different model per session — so the same file is read by whichever model
|
||||
comes next. (2) **Precision is the TARGET, not the verb list.** The verb set is
|
||||
as broad as `subtraction-prefilter`'s `IMPERATIVE_RE`; what narrows it is
|
||||
requiring a co-occurring reflexive ("your own work", "før du svarer") or
|
||||
delegated ("subagent") target. Measured across the 409-file corpus: 392
|
||||
`BP-SUB-001` candidates, **31 carry a verify verb, 0 also carry a target**, and
|
||||
two independent raw-text greps also found 0 — so the zero is the corpus, not an
|
||||
over-narrow regex. Those 31 (`sjekk relevante config-filer`, `Type-sjekk:
|
||||
pyright`) are exactly the false positives a verb-only version would have
|
||||
produced, which is `BP-JUDG-001`'s 7/7 failure arriving one lens over. The
|
||||
numbers live in the register entry's `note` and are pinned by
|
||||
`best-practices-register.test.mjs`, because a later session that cannot see the
|
||||
measurement reads the zero as a broken detector and loosens it. (3) **`recognized`
|
||||
is not `matchedCount`.** A typo'd model name and a genuinely clean config both
|
||||
yield zero matches; without the separate flag the CLI would report a silent
|
||||
no-op as good news. Dogfooded on the real machine: `opus-5` → `recognized:true,
|
||||
matchedCount:0`; `oppus5` → `recognized:false`. (4) **The citation is
|
||||
conditional wherever a human sees it** — agent report copy and the Step 7a
|
||||
approval listing both say so. `--for-model` and `--apply` are separate CLIs that
|
||||
never see each other's flags, so a CLI-level refusal is impossible; the
|
||||
safeguard has to live in the copy. The payload stays additive: `forModel` and
|
||||
per-candidate `modelScope` appear **only** when the flag is passed, so a plain
|
||||
`--subtract` run is byte-identical to the pre-flag payload (asserted on the
|
||||
serialized bytes, since a key set to `undefined` passes a shallow key check).
|
||||
|
||||
**Backup/restore is one code path (invariant).** `scanners/rollback-cli.mjs` is the only entry to
|
||||
the backup engine, and BOTH pipelines use it: `fix` backs up through `createBackup` directly,
|
||||
`implement` Step 3 through `--create`. Before R1/R2 the two halves each hid the other's failure.
|
||||
The engine verified every checksum before and after each write, but had no `process.argv` (16 CLIs
|
||||
under `scanners/` had one, it did not), so `commands/rollback.md` restored as model prose — an ESM
|
||||
`import` block a template cannot execute, `cp` offered underneath as the runnable path, and
|
||||
"(checksum verified)" pre-rendered in the success output. `cp` establishes no checksum, so the
|
||||
verification was a property of the template. Meanwhile `implement` hand-built its manifest in the
|
||||
template while `parseManifest` knew one frozen sample of that format, pinned by a HAND-WRITTEN
|
||||
fixture rather than by the template's own text — the #63 shape on the data side, where a renamed
|
||||
key yields zero parsed files and a rollback that reports success having restored nothing. Four
|
||||
properties are load-bearing. (1) **Neither half fixes alone**: a CLI over a prose format still
|
||||
parses prose; a clean format with no runnable entry still cannot restore. (2) **A gated restore is
|
||||
exit 1, not 3** — "this write leaves your project" is a verdict about a write that WAS examined and
|
||||
rides in the payload, where a command running under `2>/dev/null` can act on it (F3's class); 3
|
||||
stays reserved for argv errors and a backup id that resolves in neither root. (3) **`--created`
|
||||
records, it does not copy** — no backup can hold a file that does not exist yet, so those paths go
|
||||
into the manifest for `rollback` to report as left in place; `serializeManifest` emits the bare
|
||||
`created:` key, which is why the pre-R2 `created: <timestamp>` (a VALUE, meaning the backup id)
|
||||
never collides with it. (4) **`parseManifest`'s implement-format branch stays** even though nothing
|
||||
writes that shape now — backups already on disk in it must remain restorable, the same reason
|
||||
`getLegacyBackupDir()` is still read; its fixture changed meaning from "a stand-in for the
|
||||
template's text" to "a golden sample of historical bytes". The output contract is guarded by
|
||||
running the CLI: `tests/commands/backup-restore-contract.test.mjs` checks every field
|
||||
`rollback.md` renders against a real payload, so a renamed key fails there instead of becoming a
|
||||
confident sentence in front of a user who is already in trouble. Distinct from the scope gate,
|
||||
which classifies *where* a restore lands — `rollback.md` still calls `write-scope-cli` before its
|
||||
approval surface, and classifying is still not approving.
|
||||
|
||||
**Subtraction write path (invariant).** `--apply` routes through `scanners/lib/subtraction-write.mjs`, never through `fix-engine` or the `plan`/`implement` pipeline, and both exclusions are **measured**: the subtraction axis is absent from the orchestrated envelope, so `verifyFixes`' re-scan would mark every removal `verified` whether or not it happened (a success-shaped no-op), and the findings pipeline needs a finding code — which names a deterministic check, not a prose judgement. Three properties are load-bearing and each has a guard seen red against its own defect: removals are validated against the ORIGINAL content and applied in **descending** line order (an ascending pass shifts later spans out from under themselves); the **range** check is not redundant with the text check (`line: 0` makes `slice(-1, 0)` empty, so an empty `text` matches and `splice(-1, 1)` deletes the file's LAST line); and `createBackup` skips a nonexistent path while still returning an id, so coverage of every file about to be written is **asserted from the manifest** before a byte changes. The floor is *repeated* here, not moved: `floor-exclusion` still vetoes before anything is proposed, and the engine refuses a load-bearing block again so a hand-built approval cannot route around it. The archive rule (`mv` to `_archive/`) is file-level and does not apply to a block excision — the timestamped backup is the recovery artifact, and inventing a second copy with no restorer behind it would be worse than none.
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
|
|
|
|||
131
GOVERNANCE.md
131
GOVERNANCE.md
|
|
@ -1,131 +0,0 @@
|
|||
# Governance
|
||||
|
||||
How this marketplace is maintained, what you can expect from upstream, and how it's meant to be used.
|
||||
|
||||
## TL;DR
|
||||
|
||||
- Solo-maintained, AI-assisted development, MIT licensed.
|
||||
- **Fork-and-own is the default model.** Upstream is a starting point, not a vendor.
|
||||
- Issues welcome as signals. Pull requests are not accepted — see [Why no PRs](#pull-requests--no).
|
||||
- No SLA. Best-effort bug fixes and security advisories. Breaking changes happen and are noted in each plugin's CHANGELOG.
|
||||
|
||||
---
|
||||
|
||||
## Can I trust this?
|
||||
|
||||
Be honest with yourself about what you're adopting:
|
||||
|
||||
- **One maintainer.** If I get hit by a bus, the bus wins. The repos stay up under MIT, but no one owes you a fix.
|
||||
- **AI-generated code with human review.** Every plugin is built through dialog-driven development with Claude Code. I read, test, and judge the output before it ships, but I'm not auditing every line the way a security firm would. Treat it accordingly.
|
||||
- **No commercial interests.** I'm not selling a SaaS, not steering you toward a paid tier, not collecting telemetry. The plugins run locally in your Claude Code installation.
|
||||
- **MIT licensed.** Fork it, modify it, ship it under your own name.
|
||||
|
||||
If you work somewhere that needs vendor accountability, support contracts, or signed assurances — **this isn't that.** Use it as a reference implementation, fork it into your own organization, and own the result.
|
||||
|
||||
---
|
||||
|
||||
## How this is meant to be used
|
||||
|
||||
### Fork-and-own
|
||||
|
||||
The intended workflow:
|
||||
|
||||
1. **Fork** the marketplace (or a single plugin) into your own organization or namespace.
|
||||
2. **Tailor** it to your context — terminology, integrations, cycle lengths, regulatory framing, whatever doesn't fit out of the box.
|
||||
3. **Maintain it yourself.** Treat your fork as the canonical version for your team.
|
||||
4. **Watch upstream selectively.** Cherry-pick changes that help, ignore changes that don't. There's no obligation to stay in sync.
|
||||
|
||||
This isn't a workaround for not accepting PRs. It's the actual recommended adoption pattern, especially for plugins like `okr` and `ms-ai-architect` where every Norwegian public sector organization will need its own tildelingsbrev mappings, terminology, and integrations. A central "one true plugin" would be wrong for everyone.
|
||||
|
||||
### What to change first when you fork
|
||||
|
||||
Each plugin differs, but the common edits are:
|
||||
|
||||
- **Identity** — rename the plugin, replace authorship, update README.
|
||||
- **External integrations** — issue trackers, knowledge bases, dashboards, observability backends. The plugins ship as starting points, not pre-wired. Every organization must configure its own integrations.
|
||||
- **Norwegian-specific framing** — relevant for `okr` and `ms-ai-architect`. Other plugins are jurisdiction-neutral. Rewrite for your jurisdiction if you're outside Norway.
|
||||
- **Reference docs** — the knowledge base in each plugin reflects my reading. Replace with your organization's authoritative sources.
|
||||
- **Hooks and policies** — security thresholds, blocked commands, and audit gates are tuned to my taste. Tune them to yours.
|
||||
|
||||
### Staying current with upstream
|
||||
|
||||
If you want to pull in upstream changes later:
|
||||
|
||||
- **Cherry-pick, don't merge.** Each plugin moves independently and breaking changes land without ceremony.
|
||||
- **Read the CHANGELOG first.** Every plugin has one.
|
||||
- **Keep your customizations in clearly-named files.** The harder upstream is to merge cleanly, the more painful staying current becomes. A `local/` directory or `*.local.md` convention helps.
|
||||
|
||||
---
|
||||
|
||||
## What upstream provides
|
||||
|
||||
| | What I do | What I don't |
|
||||
|---|---|---|
|
||||
| **Bug fixes** | Best-effort when I notice or get a clear report | No SLA, no triage commitment |
|
||||
| **Security issues** | Investigate within reasonable time, document in CHANGELOG | No CVE process, no embargo coordination |
|
||||
| **New features** | When they fit my own usage | Not on request |
|
||||
| **Norwegian public sector context** | Kept current as long as the project lives | If I lose interest or change jobs, the framing freezes |
|
||||
| **Breaking changes** | Documented in CHANGELOG | They happen — version pin if you need stability |
|
||||
| **Compatibility** | Tracked against current Claude Code releases | No long-term support branches |
|
||||
|
||||
If any of this is a dealbreaker — fork now, version-pin, and stop reading upstream.
|
||||
|
||||
---
|
||||
|
||||
## How to contribute
|
||||
|
||||
### Issues — yes, please
|
||||
|
||||
Issues are the most valuable thing you can send me:
|
||||
|
||||
- **Bug reports** with reproduction steps. Even a screenshot helps.
|
||||
- **Use-case feedback.** "I tried to use this in my organization and X didn't fit" is genuinely useful, even if I can't fix it for you.
|
||||
- **Pointers to better sources.** If you know a DFØ veileder, an NSM guideline, or an academic paper that contradicts what's in a knowledge base, tell me.
|
||||
- **Security findings.** See each plugin's `SECURITY.md` for disclosure preference where one exists; otherwise email rather than open a public issue.
|
||||
|
||||
### Pull requests — no
|
||||
|
||||
This is deliberate, not laziness:
|
||||
|
||||
- **Solo review is a bottleneck.** Honest PR review takes me longer than rewriting from scratch. The math doesn't work.
|
||||
- **Forks are where the value is.** The fork-and-own model means upstream consolidation isn't the point. Your organization's adaptations belong in your fork, not mine.
|
||||
- **AI-generated code complicates provenance.** Every line here is produced through dialog with Claude Code, with me as the judge. Mixing in PRs from contributors with different processes and licensing assumptions creates a mess I'd rather not untangle.
|
||||
|
||||
If you've built something useful on top of a fork, **publish it under your own name and link back.** I'll happily list notable forks here once they exist.
|
||||
|
||||
### Notable forks
|
||||
|
||||
*(To be populated as forks emerge. If you've forked one of these plugins for production use, open an issue and I'll add a link.)*
|
||||
|
||||
---
|
||||
|
||||
## Relationship between plugins
|
||||
|
||||
These plugins are **independent**. Install one without the others, fork one without the others. They share conventions (slash command naming, hook patterns, AI-generated disclosure) but no runtime dependencies.
|
||||
|
||||
The marketplace is a **catalog**, not a suite. Don't fork the whole repo unless you actually want to maintain everything.
|
||||
|
||||
---
|
||||
|
||||
## Versioning and stability
|
||||
|
||||
- **Semantic versioning per plugin.** Each plugin has its own `CHANGELOG.md` and version number.
|
||||
- **Breaking changes happen.** I bump the major version when they do, but I don't run an LTS branch.
|
||||
- **Pin your version.** If stability matters more than features, install a specific version and stay there until you choose to upgrade.
|
||||
|
||||
---
|
||||
|
||||
## Public sector adoption notes
|
||||
|
||||
For Norwegian etater specifically:
|
||||
|
||||
- **DPIA-relevant data flows are documented in the relevant plugin README where applicable.** Read them before installation.
|
||||
- **No data leaves your machine** beyond what Claude Code itself sends to Anthropic. The plugins themselves do not call external services unless you configure an integration.
|
||||
- **Drøftingsplikt and ledelsesansvar** are not replaced by these tools. The `okr` plugin coaches; it does not decide. The `ms-ai-architect` plugin advises; it does not approve.
|
||||
- **Choose your Claude deployment carefully.** claude.ai vs. API direct vs. Bedrock in EU region have different data residency profiles. The plugins don't choose for you.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT for all plugins in this marketplace. See each plugin's `LICENSE` file.
|
||||
266
README.md
266
README.md
|
|
@ -1,27 +1,46 @@
|
|||
# Config-Audit Plugin for Claude Code
|
||||
# config-audit
|
||||
|
||||
> Know if your configuration is correct. Find what could improve it. Fix it automatically.
|
||||
Multi-agent workflow for analyzing, reporting, and optimizing Claude Code configuration across your entire machine
|
||||
|
||||
> **Solo-maintained, fork-and-own.** This plugin is a starting point, not a vendor product. Issues are welcome as signals; pull requests are not accepted. See [GOVERNANCE.md](GOVERNANCE.md) for the full model and what upstream provides.
|
||||
Know if your configuration is correct. Find what could improve it. Fix it automatically.
|
||||
|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development. [Full disclosure →](../../README.md#ai-generated-code-disclosure)*
|
||||
> **Solo-maintained, fork-and-own.** This plugin is a starting point, not a vendor product. Issues are welcome as signals; pull requests are not accepted. See [GOVERNANCE.md](https://git.fromaitochitta.com/open/repo-standard/src/branch/main/GOVERNANCE.md) for the full model and what upstream provides.
|
||||
|
||||

|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development. Every change is human-directed, reviewed, and validated before commit. Per Anthropic Consumer Terms §4, ownership of outputs is assigned to the user; this plugin is licensed MIT.*
|
||||
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
||||
A Claude Code plugin that checks configuration health, suggests context-aware improvements, and auto-fixes issues — `CLAUDE.md`, `settings.json`, hooks, rules, MCP servers, `@imports`, and plugins. 16 deterministic scanners across 10 quality areas, context-aware feature recommendations, auto-fix with backup/rollback, a prompt-cache-aware Token Hotspots scanner with optional API-calibrated `--accurate-tokens` mode, plus cache-prefix stability, dead-tool, cross-plugin collision, output-style, and always-loaded agent-listing-budget detection. Zero external dependencies.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
|
||||
claude plugin install config-audit@ktg-plugin-marketplace
|
||||
```
|
||||
|
||||
Or enable directly in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"config-audit@ktg-plugin-marketplace": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) installed
|
||||
- Node.js 18+ — the scanners also run standalone from a clone, with no other dependencies
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [What's New in v5.4.0](#whats-new-in-v540)
|
||||
- [What Is This?](#what-is-this)
|
||||
- [The Configuration Problem](#the-configuration-problem)
|
||||
- [Quick Start](#quick-start)
|
||||
|
|
@ -39,37 +58,13 @@ A Claude Code plugin that checks configuration health, suggests context-aware im
|
|||
- [Testing](#testing)
|
||||
- [Gotchas](#gotchas)
|
||||
- [Data Storage & Safety Guarantees](#data-storage--safety-guarantees)
|
||||
- [What This Plugin Does Not Cover](#what-this-plugin-does-not-cover)
|
||||
- [Version History](#version-history)
|
||||
- [Non-goals](#non-goals)
|
||||
- [config-audit vs. the built-in /doctor](#config-audit-vs-the-built-in-doctor)
|
||||
- [Changelog](#changelog)
|
||||
- [License](#license)
|
||||
|
||||
---
|
||||
|
||||
## What's New in v5.4.0
|
||||
|
||||
**Plugin-hygiene & settings-validation hardening.** Three additive findings extend the plugin and
|
||||
settings surfaces — no new scanner, so the count stays **13**:
|
||||
|
||||
- **PLH plugin-folder shadowing** (`CA-PLH-015`) — flags a `plugin.json` component-path key in the
|
||||
*replaces* set (`commands`/`agents`/`outputStyles`) that points at a custom path while the
|
||||
default folder of that name still exists, so the folder is silently ignored (dead config).
|
||||
Mirrors Claude Code's own warning in `/doctor`, `claude plugin list`, and the `/plugin` detail
|
||||
view. `skills` is excluded (it *adds to* the default scan, never shadows), as are
|
||||
`hooks`/`mcpServers`/`lspServers` (own merge rules); a custom path resolving *into* the default
|
||||
folder is not flagged.
|
||||
- **PLH `skills:`-array validation** (`CA-PLH-016`) — validates each `plugin.json` `skills` entry
|
||||
(string or array) resolves to an existing directory inside the plugin root; flags `non-string`,
|
||||
`escapes-root`, `not-found`, and `not-a-directory` entries. Mirrors `claude plugin validate`.
|
||||
- **SET `autoMode` structure + dead-config** — checks that `autoMode` is an object whose only keys
|
||||
are `environment`/`allow`/`soft_deny`/`hard_deny`, each a string array (the literal `"$defaults"`
|
||||
is valid); unknown sub-keys and wrong types are flagged (medium). Separately, `autoMode` placed
|
||||
in **shared** project settings (`.claude/settings.json`) is flagged as dead config (low) —
|
||||
Claude Code's classifier does not read it there.
|
||||
|
||||
All three extend existing PLH and SET scanners. `--json` and `--raw` output remain byte-stable.
|
||||
|
||||
---
|
||||
|
||||
## What Is This?
|
||||
|
||||
Claude Code reads instructions from at least 7 different file types across multiple scopes: `CLAUDE.md`, `settings.json`, `.claude/rules/`, `hooks.json`, `.mcp.json`, `.claudeignore`, and `settings.local.json`. Each can exist at project level, user level, or both. Plugins add more. The system is powerful — but nobody tells you what you're using wrong, what you're missing, or what's silently conflicting.
|
||||
|
|
@ -154,28 +149,7 @@ Also **Grade A** — with only 3 opportunities remaining. This project has CLAUD
|
|||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) installed
|
||||
- Node.js 18+ (for standalone CLI tools)
|
||||
|
||||
### Installation
|
||||
|
||||
Add the marketplace and browse plugins with `/plugin`:
|
||||
|
||||
```bash
|
||||
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
|
||||
```
|
||||
|
||||
Or enable directly in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"config-audit@ktg-plugin-marketplace": true
|
||||
}
|
||||
}
|
||||
```
|
||||
Install first — see [Install](#install) above.
|
||||
|
||||
### First Scan
|
||||
|
||||
|
|
@ -198,7 +172,7 @@ The CLI tools work standalone — no Claude Code session needed, just Node.js 18
|
|||
|
||||
Most configuration tools stop at "is it valid?" Config-audit goes further: **what could improve your setup, and is it relevant to your project?**
|
||||
|
||||
The feature opportunity scanner checks 25 dimensions and groups recommendations by impact:
|
||||
The feature opportunity scanner checks 24 dimensions and groups recommendations by impact:
|
||||
|
||||
| Impact Level | Focus | Examples |
|
||||
|--------------|-------|---------|
|
||||
|
|
@ -206,9 +180,20 @@ The feature opportunity scanner checks 25 dimensions and groups recommendations
|
|||
| **Worth Considering** | Workflow efficiency | Path-scoped rules, modular `@imports`, custom agents |
|
||||
| **Explore** | Nice-to-have | Keybindings, status line, output styles, agent teams |
|
||||
|
||||
Alongside the dimensions sit four **conditional levers** — recommendations that only make
|
||||
sense under a measured condition, so they stay silent otherwise. One of them is model/effort
|
||||
routing (`CA-GAP-028`): when you have your own subagents and *not one* of them names a
|
||||
`model:` or an `effort:`, every delegated task runs on the main conversation's model, because
|
||||
`model` defaults to `inherit`. It fires only when you actually have agents, and goes quiet the
|
||||
moment any of them routes either axis — so a deliberate everything-on-one-model setup is not
|
||||
nagged. Writing `model: inherit` out in full does not count as routing; it is the default
|
||||
spelled out.
|
||||
|
||||
Each recommendation is **context-aware** — it considers what your project actually contains. A solo TypeScript project gets different suggestions than a team Python monorepo. Recommendations include *why* (backed by Anthropic's official guidance) and *how* (concrete steps).
|
||||
|
||||
Run `/config-audit feature-gap` to see what's relevant to your project.
|
||||
Run `/config-audit feature-gap` to see what's relevant to your project. To see what each agent
|
||||
currently runs on, `/config-audit whats-active` lists `model` and `effort` per agent, and
|
||||
`/config-audit manifest` shows them on the agent rows.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -279,7 +264,9 @@ Your team configuration changes over time. Track it:
|
|||
| `/config-audit manifest` | Ranked table of every token source (CLAUDE.md, rules, agents, skills, output styles, MCP, hooks) sorted by estimated tokens — each tagged with its **load pattern** (always-loaded / on-demand / external) plus an **always-loaded subtotal** ("≈X tokens enter context every turn before you type"). Component-level: no coarse plugin roll-up (it would double-count) |
|
||||
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact |
|
||||
| `/config-audit optimize` | Optimization lens (mechanism-fit): config that works but fits a better mechanism — procedure→skill, lifecycle→hook, unscoped path→rule, "never"→permission. Hybrid motor (deterministic pre-filter + opus precision gate), every finding cites a best-practices-register rule |
|
||||
| `/config-audit optimize --subtract` | **Subtraction lens** — the inverse question no other command asks: what no longer earns its always-loaded rent? Ranks CLAUDE.md blocks that correct general model *behaviour* rather than stating a local fact, split into **dead** (never missed) and **earned** (returns if the model stumbles), with the token payoff (`BP-SUB-001`). **Load-bearing local facts are excluded deterministically before the judge sees anything** — remotes, versions, paths, filenames, policy invariants and unresolvable entity names are never candidates, and an ordered list is treated as a contract. Opt-in, proposes only, never writes. Pair with `--global` to reach the user-level CLAUDE.md, where the always-loaded cost actually sits |
|
||||
| `/config-audit optimize --subtract` | **Subtraction lens** — the inverse question no other command asks: what no longer earns its always-loaded rent? Ranks CLAUDE.md blocks that correct general model *behaviour* rather than stating a local fact, split into **dead** (never missed) and **earned** (returns if the model stumbles), with the token payoff (`BP-SUB-001`). **Load-bearing local facts are excluded deterministically before the judge sees anything** — remotes, versions, paths, filenames, policy invariants and unresolvable entity names are never candidates, and an ordered list is treated as a contract. Opt-in and proposes only; add `--apply` to execute the removals you pick. Pair with `--global` to reach the user-level CLAUDE.md, where the always-loaded cost actually sits |
|
||||
| `/config-audit optimize --subtract --apply` | **Execute approved removals.** You pick which blocks go by number; nothing is inferred. Every removal is checked against the file as it reads *now* — an approval that no longer matches is refused rather than applied to whatever moved into those lines — and the floor is re-asserted at write time, so a load-bearing block cannot be removed even by a hand-built approval. A dry run always precedes the write, the backup's manifest is verified to cover the file being written before a byte changes, and `/config-audit rollback` restores it. A removal targeting your machine-wide `~/.claude/CLAUDE.md` is **refused until you approve that scope explicitly** — it costs, and saves, in every project on every turn |
|
||||
| `/config-audit optimize --subtract --for-model <name>` | **Model-scoped subtraction.** Some instructions are dead weight only for a *particular* model: Anthropic documents that Claude Opus 5 verifies its own work and **over-verifies** when told to double-check or to delegate verification to a subagent, adding token cost with no quality gain (`BP-PROMPT-001`). This flag annotates the `--subtract` candidates that carry a reflexive or delegated verification target — "double-check your own work", "use a subagent to verify" — while leaving *external* verification ("check the CI status") untagged. It **never widens the candidate set**, and there is deliberately no auto-detection: a CLAUDE.md has no frontmatter and no resolvable target model, and the same file is read by whichever model the next session runs — so the citation is reported as **conditional**, and an unrecognized model name is reported as unrecognized rather than as a silent zero |
|
||||
| `/config-audit fix` | Auto-fix deterministic issues with backup + verification |
|
||||
| `/config-audit rollback` | Restore configuration from a previous backup |
|
||||
| `/config-audit plan` | Generate prioritized action plan from audit findings |
|
||||
|
|
@ -305,6 +292,28 @@ Your team configuration changes over time. Track it:
|
|||
|
||||
By default, `/config-audit` auto-detects scope from your git context. Override with: `/config-audit current`, `/config-audit repo`, `/config-audit home`, `/config-audit full`. Use `--delta` for incremental scanning (only new/changed findings).
|
||||
|
||||
### Where a write is allowed to land
|
||||
|
||||
Reading is machine-wide; **writing is not**. Every write target is classified against the project
|
||||
you are standing in, and the classification — not the command doing the asking — decides how
|
||||
strong the gate is. A change inside your project applies normally. A write into a *different*
|
||||
project is disclosed and then applied, because some commands (like `campaign export`) are
|
||||
cross-repo by design. A write to your machine-wide `~/.claude` configuration, or to a path in no
|
||||
project at all, is **withheld until you approve that scope explicitly** — it costs, and saves, in
|
||||
every project on every turn.
|
||||
|
||||
That gate now runs inside the engines, not only in the command prose that wraps them. It covers
|
||||
`/config-audit fix`, `/config-audit rollback`, `campaign export`, `--save-baseline`, and
|
||||
`optimize --subtract --apply`. For rollback that is now literal: the restore runs through
|
||||
`scanners/rollback-cli.mjs`, which verifies each file's checksum before and after writing it, so
|
||||
"restored and verified" is something the run reports rather than something the output template
|
||||
says. A restore that would land outside your project is refused with nothing written. When a run is withheld, nothing has been written: you get the
|
||||
reason and the affected paths, and you re-run with your approval (`--approve-scope` on the CLIs).
|
||||
Approval is always a separate act — classifying a target is not approving it. The plugin's own
|
||||
bookkeeping (backups, session state, ledgers, and the report file you named with `--output-file`)
|
||||
is deliberately exempt: a gate that fired on every run would be switched off, and then it would
|
||||
guard nothing.
|
||||
|
||||
---
|
||||
|
||||
## Deterministic Scanners
|
||||
|
|
@ -315,14 +324,14 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
|
||||
| Scanner | Prefix | What It Catches |
|
||||
|---------|--------|-----------------|
|
||||
| `claude-md-linter.mjs` | CML | Oversized files (line count **plus** a context-window-scaled char budget mirroring Claude Code's ~40.0k-char startup warning), missing sections, broken @imports, duplicates, stale TODOs |
|
||||
| `claude-md-linter.mjs` | CML | Oversized files (line count **plus** a context-window-scaled char budget mirroring Claude Code's ~40.0k-char startup warning), missing sections, broken @imports, duplicates, stale TODOs, and **dead prose references** (`CA-CML-013`) — backtick-quoted relative paths in prose that resolve to nothing, next to the file or from the scan root |
|
||||
| `settings-validator.mjs` | SET | Schema violations, unknown/deprecated keys, type mismatches, permission issues |
|
||||
| `hook-validator.mjs` | HKV | Invalid format, missing scripts, wrong event names, timeout risks, verbose-stdout scripts, and a low-precision **advisory** (info) when a hook injects un-grepped command output into `hookSpecificOutput.additionalContext` — that payload enters context on every fire (plain stdout does not) |
|
||||
| `rules-validator.mjs` | RUL | Bad glob patterns, orphaned rules, deprecated fields, unscoped rules |
|
||||
| `mcp-config-validator.mjs` | MCP | Invalid server types, exposed env vars, unknown fields |
|
||||
| `import-resolver.mjs` | IMP | Broken @imports, circular references, deep chains, tilde path issues |
|
||||
| `conflict-detector.mjs` | CNF | Settings contradictions across scopes, permission conflicts, hook duplicates |
|
||||
| `feature-gap-scanner.mjs` | GAP | 25 feature checks shown as opportunities, not grades — plus a conditional `disableBundledSkills` recommendation when the active skill listing is over budget, and a conditional **filter-before-Claude-reads** lever when a hook injects unfiltered output into `additionalContext` (companion to the HKV advisory; cites the documented `filter-test-output.sh` pattern) |
|
||||
| `feature-gap-scanner.mjs` | GAP | 24 feature checks shown as opportunities, not grades — plus four conditional levers: a `disableBundledSkills` recommendation when the active skill listing is over budget, a **CLI-over-MCP** lever when tool schemas are forced upfront, a **filter-before-Claude-reads** lever when a hook injects unfiltered output into `additionalContext` (companion to the HKV advisory; cites the documented `filter-test-output.sh` pattern), and **agent model/effort routing** (`CA-GAP-028`) when authored subagents exist and none pins either axis (cites `BP-MODEL-001/002`). Levers are not dimensions: they stay out of the utilization denominators |
|
||||
| `token-hotspots.mjs` | TOK | Cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascades, bloated skill descriptions, MCP tool-schema budget, and stale `~/.claude/plugins/cache` versions (disk-cleanup, zero live-context impact) — cache-aware ranking excludes superseded plugin versions by default (`--no-exclude-cache` to include) |
|
||||
| `cache-prefix-scanner.mjs` | CPS | Volatile content in lines 31–150 of the CLAUDE.md cascade — beyond Pattern A's top-30 window but still re-loaded every turn — **plus** volatile content inside `@import`-ed files (inlined into the cached prefix, one hop, otherwise invisible to per-file scans) |
|
||||
| `disabled-in-schema-scanner.mjs` | DIS | Dead/ineffective permission entries: (1) tools in BOTH `permissions.deny` and `permissions.allow` — deny wins (incl. the `Tool(*)` deny-all glob, equivalent to a bare deny); (2) unanchored allow wildcards (`*`, `B*`, `mcp__*`) that Claude Code silently skips — valid only as `mcp__<server>__*`; (3) `Tool(param:value)` rules whose key is the tool's own canonicalizing field (`command`/`file_path`/`path`/`notebook_path`/`url`) — CC ignores these and emits a startup warning |
|
||||
|
|
@ -332,6 +341,27 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
| `optimization-lens-scanner.mjs` | OPT | Optimization lens (mechanism-fit): a multi-step procedure in CLAUDE.md that would fit better as a skill (`CA-OPT-001`) — reads the machine-readable best-practices register, framed as an opportunity, not a failure. The deterministic half of the lens; prose-judgment cases (lifecycle→hook, unscoped path→rule, "never"→permission) are judged by the opus `optimization-lens-agent` via `/config-audit optimize` |
|
||||
| `agent-listing-scanner.mjs` | AGT | Always-loaded agent-listing budget: a per-agent description over the soft bloat cap (`CA-AGT-001`, advisory) and the summed active-agent name+description listing — re-sent every turn — exceeding the listing budget (`CA-AGT-002`). Both LOW and explicitly **inferred / upper-bound**: the agent-listing mechanism is undocumented, so the evidence discloses the estimate and heuristic budget rather than overstating certainty |
|
||||
|
||||
> **Dead prose references — a check designed around what it stays silent about.**
|
||||
> `import-resolver` follows `@import` targets; a path written in ordinary prose was
|
||||
> checked by nothing, so `CA-CML-013` (low) resolves those too. A reference has to be
|
||||
> unambiguously path-shaped to qualify: a separator, plus either a trailing `/` or a
|
||||
> known file extension, resolved both next to the CLAUDE.md and from the scan root —
|
||||
> a nested file may legitimately write repo-root-relative paths. One finding per file,
|
||||
> carrying the count and the first few paths.
|
||||
>
|
||||
> The design work is the silence list, and every entry on it was measured against 407
|
||||
> real CLAUDE.md files rather than argued for. Left alone: commands (`npm test`), URLs,
|
||||
> globs and placeholders (`CA-GAP-*`, `${CLAUDE_PLUGIN_ROOT}/…`), absolute and `~/`
|
||||
> paths, config keys and flags (`model:`, `--raw`), bare filenames (`README.md` — a
|
||||
> filename in prose is a concept, and admitting them tripled the output with name-drops
|
||||
> of tools that exist elsewhere), org/repo slugs and package names
|
||||
> (`ktg/some-repo`, `@anthropic-ai/claude-agent-sdk`), bare folder names (`docs/`,
|
||||
> `open/` — the same reasoning one level up), fenced code blocks, and anything resolving
|
||||
> outside the scanned tree. That last rule is not fussiness: without it a `../../../../etc/passwd`
|
||||
> resolved to the real file and silenced its own finding. A path carrying a trailing
|
||||
> `:54-56` locator is a known v1 miss — for a precision-first check, silence is the safe
|
||||
> failure direction.
|
||||
|
||||
> **Cross-scanner remediation — diagnosis meets the fix.** SKL diagnoses an over-budget
|
||||
> skill listing (`CA-SKL-002`); GAP prescribes the remedy. When the active skill listing
|
||||
> exceeds its ~2%-of-context budget and `disableBundledSkills` is not already set (in the
|
||||
|
|
@ -420,6 +450,7 @@ All tools work standalone — no Claude Code session needed:
|
|||
| **Tokens** | `node scanners/token-hotspots-cli.mjs <path> [--json] [--global] [--no-exclude-cache] [--output-file path] [--accurate-tokens] [--with-telemetry-recipe]` |
|
||||
| **Manifest** | `node scanners/manifest.mjs <path> [--json]` — ranked component-level source table with per-source load pattern + always-loaded subtotal |
|
||||
| **What's active** | `node scanners/whats-active.mjs <path> [--json] [--verbose] [--suggest-disables]` |
|
||||
| **Backup / restore** | `node scanners/rollback-cli.mjs [--list] [--restore <id>] [--delete <id>] [--create --target <path> …] [--created <path>] [--dry-run] [--approve-scope] [--repo <root>] [--json] [--output-file path]` |
|
||||
| **Self-audit** | `node scanners/self-audit.mjs [--json] [--fix] [--check-readme]` |
|
||||
| **Full scan** | `node scanners/scan-orchestrator.mjs <path> [--global] [--full-machine] [--no-suppress]` |
|
||||
|
||||
|
|
@ -491,7 +522,12 @@ Skills activate automatically when your question matches their trigger patterns.
|
|||
|
||||
### Finding ID Format
|
||||
|
||||
Every finding has a unique ID: `CA-{SCANNER}-{NNN}` — where `{SCANNER}` is the scanner prefix (see table above) and `{NNN}` is a sequential number. Examples: `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`.
|
||||
Every finding carries an ID of the form `CA-{SCANNER}-{NNN}` — where `{SCANNER}` is the scanner prefix (see table above) and `{NNN}` **names the check**, not the finding's position in a run. Examples: `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`.
|
||||
|
||||
Two consequences worth knowing before you pin one:
|
||||
|
||||
- **The same check always has the same ID.** It does not move when you fix an unrelated finding, when a check stops firing, or when a later release adds or retires one. That is what makes an ID safe to write into `.config-audit-ignore`. A number withdrawn from service is never reissued — a suppression naming a retired check goes dead rather than quietly matching a different one.
|
||||
- **An ID is not unique per finding.** One check failing in three files produces three findings that share an ID; `file` and `line` tell them apart. Suppressing by ID suppresses the check everywhere in scope.
|
||||
|
||||
### Suppression
|
||||
|
||||
|
|
@ -510,6 +546,8 @@ CA-PLH-*
|
|||
|
||||
Suppressed findings are tracked in the scan envelope's `suppressed_findings` array for audit trail — nothing is silently hidden. Use `--no-suppress` to see everything.
|
||||
|
||||
A pattern that names no known check is reported back in the envelope's `unknown_suppressions` array, so a pin that has gone stale (a typo, or a check retired in a later release) is visible instead of silently protecting nothing. Scanner-wide globs like `CA-GAP-*` are validated only down to the prefix, which is why a glob is the durable way to pin a whole scanner.
|
||||
|
||||
---
|
||||
|
||||
## Examples & Self-Audit
|
||||
|
|
@ -558,6 +596,7 @@ Shared modules used by all scanners — useful if you're reading the source or e
|
|||
| `suppression.mjs` | `.config-audit-ignore` parsing, finding suppression, audit trail |
|
||||
| `active-config-reader.mjs` | Read-only inventory of plugins/skills/MCP/hooks/CLAUDE.md cascade with token estimates |
|
||||
| `tokenizer-api.mjs` | Anthropic `count_tokens` wrapper for `--accurate-tokens` (v5 N5); 5s timeout, 429 backoff, key masking |
|
||||
| `write-scope.mjs` | Classifies a write target against the current project (`SCOPE_CLASSES`, `classifyWriteTarget()`); one source for class, gate strength and wording |
|
||||
|
||||
### Action Engines
|
||||
|
||||
|
|
@ -570,6 +609,8 @@ Shared modules used by all scanners — useful if you're reading the source or e
|
|||
| `manifest.mjs` | CLI: ranked component-level source table w/ load-pattern accounting (v5 N2; v5.6 B) |
|
||||
| `whats-active.mjs` | CLI: read-only active-config inventory (v3.1.0+) |
|
||||
| `token-hotspots-cli.mjs` | CLI: token hotspots ranking with optional `--accurate-tokens` |
|
||||
| `write-scope-cli.mjs` | CLI: classify write targets before an approval surface (`--target`, repeatable) |
|
||||
| `rollback-cli.mjs` | CLI: the runnable entry to backup and restore (`--list` / `--create` / `--restore` / `--delete`). Both pipelines back up through this one code path, so the manifest format is never written or read by hand |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -612,6 +653,39 @@ classification), or when it names a capitalized entity the mechanism cannot reso
|
|||
dictionary. That last rule is a deliberate conservative default: it declines to decide and
|
||||
keeps the block, paying in recall rather than risk.
|
||||
|
||||
### Model-scoped candidates (`--for-model`)
|
||||
|
||||
Some instructions are dead weight only for a *particular* model. Anthropic documents that
|
||||
Claude Opus 5 verifies its own work, and that explicit instructions to double-check or to
|
||||
delegate verification to a subagent cause **over-verification** — token cost with no gain in
|
||||
quality. `--subtract --for-model opus-5` annotates the subtraction candidates that carry a
|
||||
reflexive target ("double-check your own work", "før du svarer") or a delegated one ("verify
|
||||
with a subagent"), while leaving *external* verification ("check the CI status") untagged.
|
||||
|
||||
It is an annotation, never a detector: it tags a subset of what `--subtract` already found and
|
||||
can never widen the candidate set. There is deliberately **no auto-detection** — a CLAUDE.md
|
||||
has no frontmatter and no resolvable target model, and the same file is read by whichever model
|
||||
the next session happens to run. That is also why the citation is reported as *conditional*
|
||||
rather than as a settled fact about the file, both in the report and in the approval listing
|
||||
shown before anything is written.
|
||||
|
||||
Precision comes from requiring the target, not from a narrow verb list. Measured across 409
|
||||
real CLAUDE.md files: 392 subtraction candidates, **31 carry a verify verb, and 0 also carry a
|
||||
reflexive or delegated target** — a zero confirmed by two independent raw-text greps, so it is
|
||||
the corpus rather than an over-narrow rule. Those 31 verb-only blocks are precisely the false
|
||||
positives a looser version would have produced. A model name the register does not cover is
|
||||
reported as **unrecognized** rather than as a bare zero, so a typo never reads as "your config
|
||||
is already clean".
|
||||
|
||||
The write half (`--apply`) keeps the same asymmetry. It is not a `fix` action and not a
|
||||
`plan`/`implement` step, and both exclusions are measurements rather than preferences: the
|
||||
subtraction axis never enters the orchestrated envelope, so `fix`'s re-scan verification would
|
||||
report every removal as verified whether or not it happened — a success-shaped no-op — and the
|
||||
findings pipeline expects a finding code, which by invariant names a deterministic check, not a
|
||||
prose judgement. `scanners/lib/subtraction-write.mjs` owns the execution instead, and it
|
||||
re-asserts the floor rather than trusting that the pre-filter already did: the veto stays where
|
||||
it is, and is simply repeated as the last red line before the delete.
|
||||
|
||||
Granularity is the **leaf block** — one list item including its wrapped continuation lines, or
|
||||
one paragraph — with two structural exceptions: a paragraph ending in `:` merges with the list
|
||||
it introduces, and an *ordered* list is treated as a contract whose steps inherit floor from
|
||||
|
|
@ -630,7 +704,7 @@ the finding, and the reason precision-over-recall is the only defensible tuning.
|
|||
node --test 'tests/**/*.test.mjs'
|
||||
```
|
||||
|
||||
1168 tests across 67 test files (22 lib + 35 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Requires Node.js 18+ (`node:test`).
|
||||
That one command runs the whole suite from a clean clone: **1477 tests, 370 suites** (measured 2026-08-03, all passing). Nothing runs it automatically — there is no CI in this organisation, so the command above is the verification, not a badge. Test fixtures in `tests/fixtures/`. Requires Node.js 18+ (`node:test`).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -673,22 +747,74 @@ This plugin is cautious by design — configuration files are important, and a b
|
|||
| **Verification pass** | A separate agent confirms changes actually work |
|
||||
| **Human-in-the-loop** | You approve the plan before anything is implemented |
|
||||
| **Post-edit guard** | Hook blocks the session if a new critical/high finding is introduced |
|
||||
| **Scope disclosed before every write** | Each write target is classified against the project you are in, and the approval surface says when a change leaves it |
|
||||
|
||||
### Writes That Leave Your Project
|
||||
|
||||
config-audit reads configuration across projects and machine-wide, so some of what
|
||||
it proposes does not land where you are standing. A count of files cannot tell those
|
||||
cases apart: a plan that edits `~/.claude/CLAUDE.md` and one that edits your
|
||||
project's own `CLAUDE.md` are both "1 file".
|
||||
|
||||
Every write target is therefore classified before you are asked to approve it, and
|
||||
the class — not the command — decides how strong the gate is:
|
||||
|
||||
| Where the write lands | What happens |
|
||||
|---|---|
|
||||
| Inside the project you are working in | No extra gate; the usual confirmation applies |
|
||||
| config-audit's own session state and backups | No extra gate; this is the plugin's bookkeeping, not your configuration |
|
||||
| Your machine-wide Claude configuration (`~/.claude`) | Stated plainly, and it needs an explicit go-ahead — a change here affects every project you open |
|
||||
| A different project | Stated plainly, including that directories will be created there. `campaign export` does this deliberately, so this is disclosure, not refusal |
|
||||
| Anywhere else | Stated plainly, and it needs an explicit go-ahead |
|
||||
|
||||
Where a machine-wide or cross-project write is involved, the safe option is listed
|
||||
first — the default is never "proceed".
|
||||
|
||||
---
|
||||
|
||||
## What This Plugin Does Not Cover
|
||||
## Non-goals
|
||||
|
||||
- **Runtime behavior** — this plugin audits configuration files, not what Claude actually does at runtime. For runtime defense, see [claude-code-llm-security](https://git.fromaitochitta.com/open/claude-code-llm-security)
|
||||
- **Runtime behavior** — this plugin audits configuration files, not what Claude actually does at runtime. For runtime defense, see [llm-security](https://git.fromaitochitta.com/open/llm-security)
|
||||
- **Secret scanning** — config-audit checks for structural issues, not leaked credentials. Use llm-security for secret detection
|
||||
- **Custom scanner rules** — scanners check against known Claude Code configuration schemas. Custom rule definitions are not supported
|
||||
- **Remote/team configuration** — managed settings, SSO-provisioned config, and organization-level policies are detected as gaps but not managed
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
## config-audit vs. the built-in /doctor
|
||||
|
||||
Claude Code ships `/doctor` (alias `/checkup`, v2.1.205+): an agent-driven setup checkup that
|
||||
diagnoses and — with your confirmation — fixes issues. The overlap with config-audit was
|
||||
measured (2026-08-03, CC 2.1.220, prediction-before-measurement protocol), and the two tools
|
||||
divide cleanly:
|
||||
|
||||
| | Built-in `/doctor` | config-audit |
|
||||
|---|---|---|
|
||||
| Engine | Model-driven judgment, one machine, one run | Deterministic scanners, byte-stable, reproducible |
|
||||
| Cost | A full agent session per run (quota) | Free, seconds, scriptable/CI-able |
|
||||
| Scope | Current setup; trims **checked-in** CLAUDE.md only | All scopes incl. **local/private** files, plus cross-repo `campaign` |
|
||||
| Evidence | **Usage telemetry** (transcripts, lifetime counters) — its unique edge | Static analysis with provenance-stamped best-practices register |
|
||||
| Memory | None between runs | Baselines (`drift`), suppressions with audit trail, backup/rollback |
|
||||
|
||||
Division of labor per area: `/doctor` parses settings — the SET scanner validates the schema
|
||||
exhaustively (unknown/deprecated keys, types, whole cascade). `/doctor` measures hook
|
||||
*latency* — the HKV scanner validates hook *correctness*. `/doctor` judges conflicts per run —
|
||||
the CNF scanner detects them deterministically. `/doctor` trims derivable content from
|
||||
checked-in CLAUDE.md — `optimize --subtract` covers **all** scopes with a coded load-bearing
|
||||
floor (`/doctor` itself refers local-file trimming to `optimize --subtract`). `/doctor`
|
||||
estimates context weight once — `tokens`/`manifest` measure it deterministically and
|
||||
cache-aware. Run `/doctor` for a usage-weighted one-shot cleanup; run config-audit for
|
||||
reproducible, all-scope, zero-quota auditing.
|
||||
|
||||
---
|
||||
|
||||
## Changelog
|
||||
|
||||
Full detail in [CHANGELOG.md](CHANGELOG.md). Highlights per release:
|
||||
|
||||
| Version | Date | Highlights |
|
||||
|---------|------|-----------|
|
||||
| **6.0.0** | 2026-08-18 | "Prose is not a contract" — a MAJOR release whose subject is a *class* of defect rather than a feature area. Three sweeps (Q1, Q2, Q_AUDIT) kept surfacing the same shape: a command template stated an invariant in prose, code on the other side depended on it, and nothing checked that the two still agreed. **Breaking (`M-BUG-28`):** a finding ID's `{NNN}` names the **check**, not its emission position — so IDs are **not unique per finding** (one check failing in three files emits three findings sharing an ID) and any consumer keying on `id` alone must move to `(id, file, line)`. `finding-codes.mjs` is the single authority and **throws** on an undeclared code, with no counter fallback — a fallback lets a half-converted scanner ship IDs that look valid. **Q1 — the write gate runs in code:** `write-scope.mjs` existed for four releases while exactly *one* writer imported it and five templates paraphrased the policy; measured, 9 files under `scanners/` write to disk and 1 imported the gate. Every writer must now import it or hold an `EXEMPT` entry naming **where the bytes land** — four of them write the plugin's own bookkeeping and must stay ungated, because a gate that fires on every run gets switched off. **Q2 — the argv is checked against the CLI that receives it:** `--stale-after 30` reached its CLI as one argument under zsh, matched no flag, and the command reported "✓ all 14 entries re-verified within the last 90 days" about a threshold the user had just overridden. The probe builds argv from each template's **own text**, and proves itself per CLI by first being seen rejecting a flag that cannot exist. **Q_AUDIT — the third instance, measured not fixed:** data contracts (`state.yaml`, backup manifests) are hand-built by the model and parsed by engines knowing one frozen example; ranked R1–R9 with the **recovery path** on top — `rollback` has no CLI entry at all and runs as model prose that pre-renders "(checksum verified)". **Added:** `optimize --subtract --apply` (the subtraction axis can now remove what it proposes — validated against the ORIGINAL content, applied in **descending** line order, coverage asserted from the backup manifest before a byte changes, and the load-bearing floor re-checked in the engine so a hand-built approval cannot route around it); `--for-model <name>` (`BP-PROMPT-001`), an annotation on existing candidates that can never widen the set, reporting `recognized` separately from `matchedCount` so a typo'd model name cannot read as a clean config; a cross-repo write disclosure before approval (`M-BUG-41`); and model/effort routing as a **lever**, not a 25th GAP dimension — a dimension would move every user's utilization score. **Removed:** the `No autoMode classifier` GAP dimension as a `/doctor` duplicate (**25 → 24**); its title lived in **four** tables, not the two the removal was scoped against. **Fixed:** the CLI-argument class across all 14 CLIs, shell state assumed to survive between fenced blocks (**20 places across 9 files**), stdout discarded against a pipe, a nonexistent target path graded instead of erroring, and `drift` crashing on a moved finding humanized as if it were flat. **1752** tests, 0 failing; frozen `v5.0.0` untouched. No component-count change (scanners **16**, agents **7**, commands **21**, hooks **4**). |
|
||||
| **5.13.0** | 2026-07-31 | "Pipeline hardening" — the batch release of everything found by dogfooding the plugin against the maintainer's real machine and by walking the `analyze → plan → implement → rollback` pipeline end-to-end on a throwaway repo copy: one new lens mode plus **14 bugs** (`M-BUG-11`…`M-BUG-20`, `M-BUG-22`…`M-BUG-25`). **Added — `optimize --subtract` (`BP-SUB-001`):** the subtraction axis, asking what no longer earns its always-loaded rent. Opt-in, proposes only, and the only lens that removes config — so a **load-bearing block is never a candidate**, decided in code (`scanners/lib/floor-exclusion.mjs`) *before* the judge runs, never in prose. Verified against a hand-built ground truth written before any classifier existed: **zero load-bearing blocks proposed**, 11/18 groups, ~756 tok ≈ 18% of a ~4300-token file. **Fixed — `rollback` (`M-BUG-22/23/24/25`):** nothing agreed on where a backup lives; `listBackups()` returned 9 phantom test backups and 0 of 4 real ones, and `restoreBackup` returned `{restored:[],failed:[]}` — a **success-shaped no-op** — because `parseManifest` knew only one of the two manifest spellings in use. Canonical root now, legacy kept readable, unparseable manifests **throw**. **`M-BUG-19`:** `globToRegex` corrupted mid-pattern `/**/`, flagging live rules dead. **`M-BUG-18`/`M-BUG-20`:** the subagent harness won't write report-shaped `.md` (analyze now persists the returned report), and parallel agents clobbered the shared log with `Write` (pinned to Bash `>>`). **`M-BUG-11`/`M-BUG-13`:** `optimize` and `feature-gap` scanned vendored plugin config a user cannot act on — `optimize` candidates **454→45**, `feature-gap` **~0 (masked) → 18** opportunities. **`M-BUG-12`/`M-BUG-14`/`M-BUG-15`/`M-BUG-16`/`M-BUG-17`:** plain-language output that contradicted its own evidence (posture's `--output-file` never humanized; four finding types with no humanizer entry). Known, deliberately unfixed: `rollback` cannot delete files `implement` *created* — it now reports them (`createdNotRemoved`) instead of failing silently; automatic deletion of user files gets its own design. No count change (scanners **16**, agents **7**, commands **21**, hooks **4**); frozen v5.0.0 untouched, SC-5 regenerated once for two humanized titles. **1398** tests (+54). |
|
||||
| **5.12.5** | 2026-06-26 | "Dogfood denoise" — a samle-release of the Fase-3 scanner false-positive batch (`M-BUG-2/6/7/8/10`, all dogfooding finds on the maintainer's real machine). Five scanners stop counting non-user / non-live config as the user's: **CNF** (`M-BUG-2`) excludes files under `.claude/plugins/` from conflict analysis (`isPluginBundled`) — installed plugins' bundled settings/hooks/fixtures are not a user-resolvable cascade (dogfood **339→0**, Conflicts was an F on pure plugin noise). **file-discovery** (`M-BUG-8`) adds `backups` to `SKIP_DIRS` — a `backups/` tree holds frozen copies, never live config (dogfood files-under-`/backups/` **36→0**, 717 live retained). **token estimator** (`M-BUG-6`) strips block-level `<!-- -->` HTML comments from CLAUDE.md sizing — CC strips them before injection, so they were never always-loaded tokens (dogfood ~3386→3301, ~85 tok). **CPS** (`M-BUG-7`) skips fenced/inline code and whitelists CC-stable path vars (`${CLAUDE_PLUGIN_ROOT}`/`${CLAUDE_PROJECT_DIR}`) before cache-buster matching (dogfood **5→2**). **SET** (`M-BUG-10`) typo-gates the unknown-settings-key finding — the CC schema is passthrough (verified against the 2.1.193 binary), so an unknown key is forward-compatible, not an error; it now flags only a near-miss of a known key (levenshtein ≤2), severity medium→low, +6 binary-verified `KNOWN_KEYS` (dogfood **6→0**). No count change (scanners **16**, agents **7**, commands **21**, hooks **4**); all five are byte-stable — frozen v5.0.0 + SC-5 + default-output snapshots untouched, no re-seed (each fixture's findings are genuinely unchanged). **1344** tests (+37). |
|
||||
| **5.12.4** | 2026-06-26 | "Rooted rules" — fixes `M-BUG-9` (dogfooding find) in `scanners/rules-validator.mjs`: the RUL "Rule path pattern matches no files" check now resolves a rule's `paths:`/`globs:` pattern against the rule's **own project root** (the dir containing its `.claude/`), not the outer scan root. Previously `countGlobMatches` globbed against the scan target and `collectProjectFiles`' `depth>4` cutoff never reached deep matching files, so a live rule in a **nested repo** (e.g. a marketplace checkout under `~/.claude`) was wrongly flagged "never activates" (high) — a false F-grade for anyone with rules in a nested repo. The fix derives each rule's project root, collects+globs per root (cached), and skips the check for user-global rules (`root === $HOME`), which scope against the active project at runtime. Same scope-conflation family as `M-BUG-1/2`. No count change (scanners **16**, agents **7**, commands **21**); the fix is a no-op when `projectRoot === targetPath` (the common single-repo scan), so frozen v5.0.0 + default-output snapshots stay byte-stable. **1307** tests (+2 TDD: nested-repo false-positive + HOME guard). |
|
||||
|
|
@ -725,8 +851,6 @@ This plugin is cautious by design — configuration files are important, and a b
|
|||
| **1.0.0** | 2026-02-11 | Cross-platform support |
|
||||
| **0.7.0** | 2026-02-07 | Initial version (version reset from inflated 1.2.0) |
|
||||
|
||||
See [CHANGELOG.md](CHANGELOG.md) for full details.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
|
|
|||
34
SECURITY.md
Normal file
34
SECURITY.md
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
# Security policy
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Report privately to <security@fromaitochitta.com> - do not open a
|
||||
public issue.
|
||||
Canonical repository: https://git.fromaitochitta.com/open/config-audit
|
||||
|
||||
Please include the affected version or commit, a minimal reproduction,
|
||||
and the impact you see. We acknowledge every report within 5 working
|
||||
days, agree a fix and disclosure timeline with the reporter, and aim to
|
||||
disclose within 90 days of the initial report.
|
||||
|
||||
## Response process
|
||||
|
||||
1. Acknowledge within 5 working days.
|
||||
2. Triage and confirm severity within 10 working days.
|
||||
3. Develop and test a fix.
|
||||
4. Publish an advisory and credit the reporter unless they prefer
|
||||
to remain anonymous.
|
||||
|
||||
## Supported versions
|
||||
|
||||
| Version | Supported |
|
||||
|---------|-----------|
|
||||
| 5.13.x | Yes |
|
||||
| < 5.13 | No |
|
||||
|
||||
Only the latest tagged release receives security fixes. There is no
|
||||
long-term support line.
|
||||
|
||||
## Advisories
|
||||
|
||||
Security-relevant fixes are noted in [CHANGELOG.md](CHANGELOG.md).
|
||||
|
|
@ -38,8 +38,8 @@ whether the line is *really* that kind of instruction:
|
|||
## The subtraction lens (`--subtract` only)
|
||||
|
||||
Present only when the payload has a `subtract` block. It asks the inverse of
|
||||
every other lens: *what is no longer earning its always-loaded rent?* Three
|
||||
things make it different, and all three are non-negotiable.
|
||||
every other lens: *what is no longer earning its always-loaded rent?* Four
|
||||
things make it different, and all four are non-negotiable.
|
||||
|
||||
**1. The floor is not yours to decide.** A deterministic pre-step has already
|
||||
excluded every block carrying a local fact — a code span, path, domain, version
|
||||
|
|
@ -64,6 +64,15 @@ earned its place — its subject matter recurs in the repo's own history — is
|
|||
2 even when its classification is "compensatory". Reporting it as dead weight is
|
||||
wrong even though the label matches.
|
||||
|
||||
**4. A model-scoped citation is conditional, never authoritative.** When a
|
||||
candidate carries `modelScope` (only under `--for-model`), apply the SAME tier-2
|
||||
/ tier-3 judgement as any other `compensatory-instruction` candidate — the model
|
||||
tag sharpens the citation, it does not bypass precision rule 1. State it as
|
||||
conditional in the report copy ("redundant if targeting {model}; this operator
|
||||
may run other models in other sessions"), never as a settled fact for all future
|
||||
sessions. The tag is an annotation on a candidate the general detector already
|
||||
found; it is not evidence that the block is dead.
|
||||
|
||||
Rank kept candidates by always-loaded token cost, and state the total payoff.
|
||||
Frame it as *rent*, never as a mistake: this config was correct when written.
|
||||
|
||||
|
|
@ -82,6 +91,12 @@ You receive an `optimize-lens` payload (JSON) with:
|
|||
register, detectors }`. Each candidate spans `line`–`endLine` (a whole leaf
|
||||
block, not one line) and carries `signalText` plus the BP-SUB-001 register
|
||||
block. Everything load-bearing was already removed before you saw this.
|
||||
Under `--for-model <name>` the block also carries `forModel`
|
||||
(`{ requested, recognized, matchedCount }`), and a SUBSET of its candidates
|
||||
carry `modelScope` (`{ registerId, claim, requestedModel }`). If
|
||||
`recognized` is `false`, the operator named a model the register does not
|
||||
cover — say so, and do not treat the absence of tags as evidence of a clean
|
||||
config.
|
||||
|
||||
Always **Read the actual CLAUDE.md file(s)** named in the candidates before
|
||||
judging — `signalText` is one line out of context; the surrounding lines decide
|
||||
|
|
@ -150,6 +165,8 @@ to return) — ranked by token cost, with a payoff total. For each:}
|
|||
**{file}:{line}-{endLine}** — {what the block says, in one line} · ~{N} tok/turn
|
||||
Tier: {dead | earned — and why}
|
||||
Source: {register.source.url}
|
||||
Model-scoped: {only if the candidate carries `modelScope`} {claim}, for
|
||||
{requestedModel} (conditional — verify this is still the model you target)
|
||||
```
|
||||
|
||||
Omit any section with zero kept findings (except keep the "left alone" note when
|
||||
|
|
|
|||
|
|
@ -171,18 +171,10 @@ Total backup size: ~6.4 KB
|
|||
**Rationale**:
|
||||
Code style rules found in 3 projects are identical. Moving to global reduces duplication.
|
||||
|
||||
**Content**:
|
||||
```markdown
|
||||
# Code Style Rules
|
||||
|
||||
## Language Preferences
|
||||
- TypeScript > JavaScript
|
||||
- Explicit > implicit
|
||||
- Lesbarhet > cleverness
|
||||
|
||||
## Commit Format
|
||||
- Conventional Commits: `type(scope): description`
|
||||
```
|
||||
**Content outline** (describe it — do not inline the file):
|
||||
Language preferences, then commit format. The implementer reads the source
|
||||
files and writes the content itself; a full file body pasted here is what the
|
||||
200-line budget above forbids.
|
||||
|
||||
**Validation**:
|
||||
- File exists after creation
|
||||
|
|
|
|||
|
|
@ -140,7 +140,15 @@ Checking for secrets...
|
|||
|
||||
## Output Format
|
||||
|
||||
Append to: `~/.claude/config-audit/sessions/{session-id}/implementation-log.md`
|
||||
Return the report below as your final message. Do NOT write it to a file: this
|
||||
agent is read-only by design (`tools: Read, Glob, Grep`) and has no write tool,
|
||||
so a write instruction here would be a contract it cannot keep.
|
||||
|
||||
The orchestrator appends what you return to
|
||||
`~/.claude/config-audit/sessions/{session-id}/implementation-log.md` itself,
|
||||
with Bash `>>` (`commands/implement.md` Step 5) — never the Write tool. That log
|
||||
is shared with the implementer agents running in parallel, and a full-file Write
|
||||
on it silently clobbers their entries.
|
||||
|
||||
```markdown
|
||||
## Verification Report
|
||||
|
|
@ -243,8 +251,8 @@ Optional: Generate before/after comparison:
|
|||
|
||||
This agent:
|
||||
- Only uses Read, Glob, Grep tools
|
||||
- Never modifies any files
|
||||
- Reports findings without taking action
|
||||
- Never modifies any files, including the shared implementation log
|
||||
- Reports findings without taking action — every result is returned inline
|
||||
- Safe to run multiple times
|
||||
|
||||
## Model policy
|
||||
|
|
|
|||
|
|
@ -20,9 +20,16 @@ Generate comprehensive analysis report from discovery findings.
|
|||
|
||||
## Implementation
|
||||
|
||||
### Step 1: Verify session state
|
||||
### Step 1: Resolve the session and verify its state
|
||||
|
||||
Read `~/.claude/config-audit/sessions/{session-id}/state.yaml` using the Read tool and verify discovery phase completed. If not, tell the user: "Discovery hasn't been run yet. Start with `/config-audit discover` or just run `/config-audit` for a full audit."
|
||||
Find the session first — never guess which one `{session-id}` refers to:
|
||||
|
||||
```
|
||||
Glob: ~/.claude/config-audit/sessions/*/state.yaml
|
||||
Sort by modification time — the most recently modified session wins
|
||||
```
|
||||
|
||||
Every `{session-id}` below is that session's id. Read its `state.yaml` using the Read tool and verify discovery phase completed. If the Glob returns nothing, or discovery hasn't completed, tell the user: "Discovery hasn't been run yet. Start with `/config-audit discover` or just run `/config-audit` for a full audit."
|
||||
|
||||
### Step 2: Tell the user what's happening
|
||||
|
||||
|
|
@ -37,17 +44,16 @@ This includes hierarchy mapping, conflict detection, and prioritized recommendat
|
|||
|
||||
Tell the user: **"Generating analysis (this takes about 30 seconds)..."**
|
||||
|
||||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
```
|
||||
Check whether `$ARGUMENTS` contains `--raw`. Carry the answer yourself: the agent
|
||||
prompt below is **not** a shell, so a variable assigned in a bash block cannot be
|
||||
referenced from it. Substitute `{mode}` literally with `--raw` or `humanized`.
|
||||
|
||||
```
|
||||
Agent(subagent_type: "config-audit:analyzer-agent")
|
||||
model: sonnet
|
||||
prompt: |
|
||||
Analyze all findings in: ~/.claude/config-audit/sessions/{session-id}/findings/
|
||||
Mode: $RAW_FLAG (empty = humanized; "--raw" = v5.0.0 verbatim severity prefiks)
|
||||
Mode: {mode} ("humanized" = humanized; "--raw" = v5.0.0 verbatim severity prefiks)
|
||||
Generate comprehensive report covering:
|
||||
1. Executive summary with key metrics, grouped by userImpactCategory
|
||||
2. Hierarchy map visualization
|
||||
|
|
@ -95,4 +101,4 @@ Full report: `~/.claude/config-audit/sessions/{session-id}/analysis-report.md`
|
|||
|
||||
### Step 6: Update state
|
||||
|
||||
Update `state.yaml` with `current_phase: "analyze"`, `next_phase: "plan"`.
|
||||
Update `state.yaml` with all four fields `.claude/rules/state-management.md` requires: `current_phase: "analyze"`, `completed_phases` (append `analyze` to the existing array — read it first), `next_phase: "plan"`, and `updated_at`. A write that names only two of the four silently deletes the other two.
|
||||
|
|
|
|||
|
|
@ -62,7 +62,8 @@ From `$ARGUMENTS`, pick the mode:
|
|||
- `export <path>` → export a planned repo's action plan into that repo's own `docs/`.
|
||||
- `help` → show this surface and stop.
|
||||
|
||||
Set a shared date stamp for any write: `TODAY=$(date +%F)`.
|
||||
Every write step derives its own date stamp inside its own block — there is no shared one to
|
||||
set here, because each fenced block runs as a separate process.
|
||||
|
||||
### Step 2: Always report current state first
|
||||
|
||||
|
|
@ -147,6 +148,9 @@ If already initialized, say so and stop (no clobber). Otherwise tell the user wh
|
|||
then create it:
|
||||
|
||||
```bash
|
||||
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
|
||||
# in an earlier block is empty by the time this one runs.
|
||||
TODAY=$(date +%F)
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs init \
|
||||
--reference-date "$TODAY" \
|
||||
--output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?
|
||||
|
|
@ -170,13 +174,23 @@ at `~/.claude/config-audit/campaign-ledger.json`." Then suggest `add`.
|
|||
them in one call (idempotent — already-tracked repos are skipped, not reset):
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs add <path1> <path2> ... \
|
||||
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
|
||||
# in an earlier block is empty by the time this one runs.
|
||||
TODAY=$(date +%F)
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs add "<path1>" "<path2>" ... \
|
||||
--reference-date "$TODAY" \
|
||||
--output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
(For a single repo with a custom display name, add `--name "<name>"`.) Read the result file and
|
||||
report what was `added` vs `skipped`, then re-show the repo table.
|
||||
report what was `added`, `addedUnverified`, and `skipped`, then re-show the repo table.
|
||||
|
||||
`addedUnverified[]` holds paths that were tracked but could **not** be read right now (they do
|
||||
not exist, or are not directories). They are tracked deliberately — an unmounted volume is a
|
||||
legitimate reason for a repo to be missing today — but they must be named, not glossed over:
|
||||
"Tracked, but I couldn't read `<path>` — check for a typo, or mount it before the next token
|
||||
sweep." An unreported phantom row stays in the backlog forever and quietly widens every
|
||||
machine-wide total.
|
||||
|
||||
### Step 5 (mode `set-status`): Transition a repo — propose, approve, write
|
||||
|
||||
|
|
@ -194,9 +208,12 @@ roll-up stays meaningful. Two honest sources, in order of preference:
|
|||
On approval:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs set-status <path> <status> \
|
||||
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
|
||||
# in an earlier block is empty by the time this one runs.
|
||||
TODAY=$(date +%F)
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs set-status "<path>" "<status>" \
|
||||
--reference-date "$TODAY" \
|
||||
[--findings '{"critical":0,"high":0,"medium":0,"low":0}'] [--session <id>] \
|
||||
[--findings '{"critical":0,"high":0,"medium":0,"low":0}'] [--session "<id>"] \
|
||||
--output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
|
|
@ -215,6 +232,9 @@ replaces, never accumulates), and **skips — never aborts on** — any repo tha
|
|||
the user it will read each tracked repo's live config (a few seconds per repo), then on approval:
|
||||
|
||||
```bash
|
||||
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
|
||||
# in an earlier block is empty by the time this one runs.
|
||||
TODAY=$(date +%F)
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs refresh-tokens \
|
||||
--reference-date "$TODAY" \
|
||||
--output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?
|
||||
|
|
@ -235,6 +255,9 @@ not buried in a session dir. This step copies it there, byte-faithfully.
|
|||
that carries an `action-plan.md` (i.e. `/config-audit plan` has run there). Run without `--write`:
|
||||
|
||||
```bash
|
||||
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
|
||||
# in an earlier block is empty by the time this one runs.
|
||||
TODAY=$(date +%F)
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-export-cli.mjs --repo "<path>" \
|
||||
--reference-date "$TODAY" \
|
||||
--output-file ~/.claude/config-audit/sessions/campaign-export.json 2>/dev/null; echo $?
|
||||
|
|
@ -252,9 +275,24 @@ no/corrupt ledger). Read `~/.claude/config-audit/sessions/campaign-export.json`
|
|||
— the first ~12 lines of `document` only, never the whole file, never the raw JSON (UX rules).
|
||||
Ask for explicit approval to write it.
|
||||
|
||||
Showing the path is not the same as saying it leaves this repo. Classify it first:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/write-scope-cli.mjs --target "<targetPath>" --repo "$PWD" --output-file ~/.claude/config-audit/sessions/campaign-export-scope.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read that file and render each distinct string in `disclosures[]` verbatim before the
|
||||
approval question. Exporting into another repo is what this command is *for*, so the
|
||||
gate here **discloses and does not refuse** — say that the write lands in a different
|
||||
project and that a `docs/` directory will be created there if it is missing. Do not
|
||||
turn this into a refusal.
|
||||
|
||||
**On approval, write it** (the CLI does the faithful copy — do NOT hand-write the file):
|
||||
|
||||
```bash
|
||||
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
|
||||
# in an earlier block is empty by the time this one runs.
|
||||
TODAY=$(date +%F)
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-export-cli.mjs --repo "<path>" --write \
|
||||
--reference-date "$TODAY" \
|
||||
--output-file ~/.claude/config-audit/sessions/campaign-export.json 2>/dev/null; echo $?
|
||||
|
|
|
|||
|
|
@ -75,7 +75,13 @@ Manage and clean up accumulated config-audit sessions in `~/.claude/config-audit
|
|||
- Warn before deleting active sessions: "Session {id} is still active (phase: {phase}). Delete anyway?"
|
||||
|
||||
6. **Execute cleanup**:
|
||||
- For each session to delete: `rm -rf ~/.claude/config-audit/sessions/{session-id}/`
|
||||
- **Validate the id before it ever reaches `rm -rf`.** Each `{session-id}`
|
||||
must match `^[0-9]{8}_[0-9]{6}$` (the id format `discover` generates) or be
|
||||
an existing directory name read verbatim from the Glob in step 1. If an id
|
||||
is empty or fails to match, **refuse to delete it**, report it, and continue
|
||||
with the rest. An empty id expands the path to
|
||||
`~/.claude/config-audit/sessions//`, which deletes *every* session.
|
||||
- For each validated session: `rm -rf ~/.claude/config-audit/sessions/{session-id}/`
|
||||
- Track deleted count and freed space
|
||||
|
||||
7. **Output summary**:
|
||||
|
|
|
|||
|
|
@ -83,29 +83,41 @@ This is a silent infrastructure step — do NOT show output to the user.
|
|||
|
||||
### Step 3: Run scanners and posture assessment
|
||||
|
||||
Tell the user: **"Running 12 configuration scanners..."**
|
||||
Tell the user: **"Running 16 configuration scanners..."**
|
||||
|
||||
Run both scanners and posture in a single Bash command. Default mode runs the humanizer, so each finding in `scan-results.json` carries `userImpactCategory`, `userActionLanguage`, and `relevanceContext` alongside the v5.0.0 fields. If the user passed `--raw`, thread it through to both CLIs to get v5.0.0 verbatim output.
|
||||
|
||||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/scan-orchestrator.mjs <target-path> --output-file ~/.claude/config-audit/sessions/{session-id}/findings/scan-results.json [--full-machine] [--global] $RAW_FLAG 2>/dev/null; node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs <target-path> --output-file ~/.claude/config-audit/sessions/{session-id}/posture.json [--full-machine] [--global] $RAW_FLAG 2>/dev/null; echo $?
|
||||
if echo "$ARGUMENTS" | grep -qE -- '(^| )--raw( |$)'; then RAW_FLAG="--raw"; fi
|
||||
# Set to the ONE scope flag the detected scope calls for, otherwise leave empty.
|
||||
# Exactly one token: zsh does not word-split an unquoted expansion, so a variable
|
||||
# holding "--flag value" would reach argv as a single unrecognised argument.
|
||||
# A placeholder in square brackets does not start with a dash either, so both
|
||||
# CLIs' arg loops would take it as the TARGET PATH instead of a flag.
|
||||
SCOPE_FLAG="" # e.g. SCOPE_FLAG="--full-machine" or SCOPE_FLAG="--global"
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/scan-orchestrator.mjs "<target-path>" --output-file ~/.claude/config-audit/sessions/{session-id}/findings/scan-results.json $SCOPE_FLAG $RAW_FLAG >/dev/null 2>/dev/null; ORCH_STATUS=$?
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs "<target-path>" --output-file ~/.claude/config-audit/sessions/{session-id}/posture.json $SCOPE_FLAG $RAW_FLAG >/dev/null 2>~/.claude/config-audit/sessions/{session-id}/posture-stderr.txt; POSTURE_STATUS=$?
|
||||
echo "$ORCH_STATUS $POSTURE_STATUS"
|
||||
```
|
||||
|
||||
Use `--full-machine` for `full` scope, `--global` for `home` scope. For `repo` and `current`, pass the resolved path directly.
|
||||
|
||||
Check the echoed exit code:
|
||||
- `0`, `1`, or `2` → continue normally
|
||||
- `3` → tell user: "Scanner encountered an unexpected error. Try `/config-audit posture` for a quick check instead." and stop.
|
||||
Two exit codes are echoed — the orchestrator's first, posture's second. They must be read **independently**; a single trailing `echo $?` would report only the last command, hiding an orchestrator failure behind posture's success.
|
||||
|
||||
- both in `0`, `1`, `2` → continue normally
|
||||
- **either** is `3` → tell user: "Scanner encountered an unexpected error. Try `/config-audit posture` for a quick check instead." and stop.
|
||||
|
||||
Posture's stderr goes to a **file**, not `/dev/null`: it carries the humanized scorecard headline that step 6 renders, and that headline exists nowhere in the JSON payload. Writing it to a file keeps UX rule 2 intact (the user still never sees raw scanner output) while leaving the text readable.
|
||||
|
||||
### Step 4: Analyze results
|
||||
|
||||
Tell the user: **"Scanners complete. Preparing your results..."**
|
||||
|
||||
Read BOTH output files using the Read tool:
|
||||
Read all three output files using the Read tool:
|
||||
- `~/.claude/config-audit/sessions/{session-id}/findings/scan-results.json`
|
||||
- `~/.claude/config-audit/sessions/{session-id}/posture.json`
|
||||
- `~/.claude/config-audit/sessions/{session-id}/posture-stderr.txt` — the humanized scorecard. Take the `Health: {grade} ({score}/100) — {prose}` headline from it; that prose is not in either JSON payload.
|
||||
|
||||
Extract these metrics from the JSON:
|
||||
|
||||
|
|
@ -146,7 +158,7 @@ Present results using this template. The humanizer has already replaced jargon-h
|
|||
|
||||
**Health: {overallGrade}** | {qualityAreaCount} areas scanned
|
||||
|
||||
{Use the headline line from the humanized stderr scorecard — it carries grade-context prose already. Avoid hardcoding a separate per-grade prose ladder.}
|
||||
{Use the `Health: …` headline read from `posture-stderr.txt` in step 4 — it carries grade-context prose already. Avoid hardcoding a separate per-grade prose ladder. If that file is missing or empty, say the grade plainly without inventing prose for it.}
|
||||
|
||||
Scanned {files_scanned} files | {real_finding_count} findings ({severity_breakdown})
|
||||
{If test_fixture_count > 0: "({test_fixture_count} additional findings in test fixtures were excluded.)"}
|
||||
|
|
@ -160,9 +172,11 @@ Scanned {files_scanned} files | {real_finding_count} findings ({severity_breakdo
|
|||
| Settings | {grade} | {count} | {status} |
|
||||
| Hooks | {grade} | {count} | {status} |
|
||||
| Rules | {grade} | {count} | {status} |
|
||||
| MCP Servers | {grade} | {count} | {status} |
|
||||
| MCP | {grade} | {count} | {status} |
|
||||
| Imports | {grade} | {count} | {status} |
|
||||
| Conflicts | {grade} | {count} | {status} |
|
||||
| Token Efficiency | {grade} | {count} | {status} |
|
||||
| Plugin Hygiene | {grade} | {count} | {status} |
|
||||
|
||||
{For the status column, use the humanized title from the most-severe finding in that area, or a one-phrase plain-language summary. Findings carry userImpactCategory which already groups by impact bucket — use that vocabulary, not raw scanner names.}
|
||||
|
||||
|
|
|
|||
|
|
@ -72,14 +72,19 @@ Run the scan orchestrator silently to discover and scan files. Default mode emit
|
|||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/scan-orchestrator.mjs <target-path> --output-file ~/.claude/config-audit/sessions/{session-id}/findings/scan-results.json [--full-machine] [--global] $RAW_FLAG 2>/dev/null; echo $?
|
||||
# Set to the flag itself for full/home scope, otherwise leave empty. Never pass
|
||||
# a placeholder wrapped in square brackets: it does not start with a dash, so
|
||||
# the orchestrator's arg loop takes it as the SCAN TARGET and silently scans a
|
||||
# path that does not exist.
|
||||
SCOPE_FLAGS="" # e.g. SCOPE_FLAGS="--full-machine" or SCOPE_FLAGS="--global"
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/scan-orchestrator.mjs "<target-path>" --output-file ~/.claude/config-audit/sessions/{session-id}/findings/scan-results.json $SCOPE_FLAGS $RAW_FLAG >/dev/null 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Check exit code: 0/1/2 → normal. 3 → "Discovery encountered an error. Try a narrower scope."
|
||||
|
||||
### Step 6: Save scope and state
|
||||
|
||||
Write `scope.yaml` and `state.yaml` to session directory. Update state with `current_phase: "discover"`, `next_phase: "analyze"`.
|
||||
Write `scope.yaml` and `state.yaml` to session directory. Update state with all four fields `.claude/rules/state-management.md` requires: `current_phase: "discover"`, `completed_phases: [discover]`, `next_phase: "analyze"`, and `updated_at`. The last two are what make an interrupted run resumable.
|
||||
|
||||
### Step 7: Present summary
|
||||
|
||||
|
|
|
|||
|
|
@ -29,10 +29,10 @@ Tell the user: **"Saving current configuration as baseline..."**
|
|||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <path> --save --name <baseline-name> $RAW_FLAG 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs "<path>" --save --name "<baseline-name>" --json $RAW_FLAG 2>/dev/null
|
||||
```
|
||||
|
||||
Read stdout for confirmation. Tell the user:
|
||||
`--save` writes its human confirmation to **stderr**, which `2>/dev/null` discards — pass `--json` so the `{saved, name, path}` object lands on stdout. Read stdout for confirmation. Tell the user:
|
||||
|
||||
```markdown
|
||||
### Baseline Saved
|
||||
|
|
@ -50,10 +50,16 @@ Tell the user: **"Comparing current configuration against baseline..."**
|
|||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <path> --baseline <name> $RAW_FLAG 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs "<path>" --baseline "<name>" --output-file /tmp/config-audit-drift.json $RAW_FLAG >/dev/null 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read stdout. In default mode the diff sections are humanized — finding titles, descriptions, and recommendations have already been replaced with plain-language equivalents. New/resolved/changed finding lists carry `userImpactCategory`, `userActionLanguage`, and `relevanceContext` so you can group and prioritize without re-deriving severity prose. If `--raw` was passed, the v5.0.0 diff is verbatim — present it in a code block as-is.
|
||||
Exit codes: `0` = stable/improving, `1` = degrading (both normal — present the result either way), `3` = a real error.
|
||||
|
||||
Then read `/tmp/config-audit-drift.json` with the **Read tool**. The default-mode report itself goes to stderr, so `--output-file` is the only way this command sees the diff at all.
|
||||
|
||||
**Check `_baselineAnchor` first.** If the baseline was saved from a different directory than the one being scanned, the diff is not a drift signal — every baseline finding shows as "resolved" and every current finding as "new", which renders as a falsely reassuring "improving" trend. When the anchor differs, say so plainly and offer to re-anchor with `/config-audit drift --save` instead of presenting the numbers as drift.
|
||||
|
||||
In default mode the diff sections are humanized — finding titles, descriptions, and recommendations have already been replaced with plain-language equivalents. New/resolved/changed finding lists carry `userImpactCategory`, `userActionLanguage`, and `relevanceContext` so you can group and prioritize without re-deriving severity prose. If `--raw` was passed, the v5.0.0 diff is verbatim — present it in a code block as-is.
|
||||
|
||||
If baseline not found, tell the user:
|
||||
|
||||
|
|
@ -96,9 +102,14 @@ When iterating new/resolved findings, prefer `userActionLanguage` over raw `seve
|
|||
If `$ARGUMENTS` contains `--list`:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs --list 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs --list --output-file /tmp/config-audit-baselines.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
The human-readable listing goes to stderr, which `2>/dev/null` discards — read
|
||||
`/tmp/config-audit-baselines.json` with the Read tool and render the `baselines`
|
||||
array (`name`, `findingCount`, `savedAt`) as a table. If the array is empty, tell
|
||||
the user no baselines are saved yet and point at `/config-audit drift --save`.
|
||||
|
||||
### What's next
|
||||
|
||||
After viewing drift:
|
||||
|
|
|
|||
|
|
@ -42,7 +42,7 @@ Generate session ID (`YYYYMMDD_HHmmss`) if no active session exists.
|
|||
mkdir -p ~/.claude/config-audit/sessions/{session-id}/findings 2>/dev/null
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs <target-path> --output-file ~/.claude/config-audit/sessions/{session-id}/posture.json $RAW_FLAG 2>/dev/null; echo $?
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs "<target-path>" --output-file ~/.claude/config-audit/sessions/{session-id}/posture.json $RAW_FLAG >/dev/null 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
If exit code is non-zero: "Assessment couldn't run. Check that the path exists and contains configuration files."
|
||||
|
|
@ -55,8 +55,8 @@ Extract GAP findings from `scannerEnvelope.scanners` (find scanner with `scanner
|
|||
|
||||
Detect project context:
|
||||
```bash
|
||||
test -f <target-path>/package.json && echo "has_package_json" || echo "no_package_json"
|
||||
ls <target-path>/*.py <target-path>/requirements.txt <target-path>/pyproject.toml 2>/dev/null | head -3
|
||||
test -f "<target-path>"/package.json && echo "has_package_json" || echo "no_package_json"
|
||||
ls "<target-path>"/*.py "<target-path>"/requirements.txt "<target-path>"/pyproject.toml 2>/dev/null | head -3
|
||||
```
|
||||
|
||||
### Step 4: Build numbered recommendations
|
||||
|
|
@ -128,15 +128,23 @@ If the user picks numbers: parse the selection and proceed to Step 6.
|
|||
|
||||
For each selected recommendation:
|
||||
|
||||
1. **Create backup** of any files that will be modified:
|
||||
1. **Create backup** of any files that will be modified.
|
||||
|
||||
Do **not** reach for `fix-cli.mjs` here. It is dry-run by default, so calling
|
||||
it without `--apply` creates no backup at all and returns `backupId: null` —
|
||||
and calling it *with* `--apply` would execute unrelated auto-fixes that the
|
||||
user never selected. Copy the files yourself:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <target-path> --json 2>/dev/null
|
||||
BACKUP_DIR=~/.claude/config-audit/backups/$(date +%Y%m%d_%H%M%S)/files
|
||||
mkdir -p "$BACKUP_DIR" 2>/dev/null
|
||||
# repeat per file that will be touched:
|
||||
cp "<file-to-modify>" "$BACKUP_DIR/" 2>/dev/null; echo $?
|
||||
```
|
||||
Or create manual backup:
|
||||
```bash
|
||||
mkdir -p ~/.claude/config-audit/backups/$(date +%Y%m%d_%H%M%S)/files/ 2>/dev/null
|
||||
```
|
||||
Copy each file that will be touched.
|
||||
|
||||
Tell the user where the copies landed. These are plain file copies — they are
|
||||
restored by copying them back, **not** by `/config-audit rollback`, which only
|
||||
knows about backups written by `fix` and `implement`.
|
||||
|
||||
2. **Apply the template** from gap-closure-templates.md. Use the Write or Edit tool to create or modify the relevant configuration file.
|
||||
|
||||
|
|
@ -151,9 +159,12 @@ Implementing 3 recommendations...
|
|||
|
||||
4. **Verify** by re-running posture:
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs <target-path> --json --output-file /tmp/config-audit-verify-$$.json 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs "<target-path>" --json --output-file /tmp/config-audit-verify.json >/dev/null 2>/dev/null
|
||||
```
|
||||
|
||||
Use the Read tool on `/tmp/config-audit-verify.json` for the new `overallGrade`
|
||||
and score — stdout is discarded on purpose (the same envelope is 255 KB).
|
||||
|
||||
### Step 7: Show results
|
||||
|
||||
```markdown
|
||||
|
|
|
|||
|
|
@ -15,8 +15,13 @@ Auto-fix deterministic configuration issues. Scans, plans fixes, backs up origin
|
|||
- `$ARGUMENTS` may contain:
|
||||
- A target path (default: current working directory)
|
||||
- `--dry-run`: Show fix plan without applying
|
||||
- `--global`: Include user-scope config (`~/.claude`) in the scan **and** the fix run
|
||||
- `--raw`: Pass-through to scanners; produces v5.0.0 verbatim envelope (bypasses the humanizer) for byte-stable diff tooling
|
||||
|
||||
`--global` must be passed to **every** step below. The scan that builds the table and
|
||||
the scan that plans the fixes are two different runs; if only one of them sees the
|
||||
user scope, the plan and the table describe different config.
|
||||
|
||||
## Implementation
|
||||
|
||||
### Step 1: Greet and scan
|
||||
|
|
@ -34,7 +39,11 @@ Parse flags and run scanners silently. Default mode emits humanized JSON — eac
|
|||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/scan-orchestrator.mjs <path> --output-file /tmp/config-audit-fix-scan-$$.json [--global] $RAW_FLAG 2>/dev/null; echo $?
|
||||
# Set to --global when the user asked for global scope, otherwise leave empty.
|
||||
# A placeholder in square brackets does not start with a dash, so the arg loop
|
||||
# would take it as the scan/fix TARGET instead of a flag.
|
||||
GLOBAL_FLAG=""
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/scan-orchestrator.mjs "<path>" --output-file /tmp/config-audit-fix-scan.json $GLOBAL_FLAG $RAW_FLAG >/dev/null 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Exit code 3 → tell user: "Scanner error. Try `/config-audit posture` to check your configuration."
|
||||
|
|
@ -44,13 +53,31 @@ Exit code 3 → tell user: "Scanner error. Try `/config-audit posture` to check
|
|||
Run fix planner silently. The fix-cli emits humanized prose to stderr in default mode and v5.0.0-shape JSON to stdout when `--json` is set; we use `--json` here for structured data and let the humanizer-aware rendering layer (this command's prose output below) supply the plain-language wording from the scan envelope above:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> --json 2>/dev/null
|
||||
# Re-assign here: each fenced block is its own Bash call, so the value
|
||||
# set in Step 1 is empty by the time this block runs.
|
||||
GLOBAL_FLAG="" # --global when the user asked for global scope
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs "<path>" $GLOBAL_FLAG --output-file /tmp/config-audit-fix-plan.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read the JSON output using the Read tool. Cross-reference each fix-plan entry against the humanized scan envelope (`/tmp/config-audit-fix-scan-$$.json`) by finding ID to recover the humanized `title`/`description`/`recommendation` plus `userImpactCategory`/`userActionLanguage` for grouping.
|
||||
Exit codes: 0 = plan produced, 2 = one or more fixes failed (apply step only), 3 = argument or tool error. On 3, show the stderr message — an unknown flag is rejected by design, not silently ignored.
|
||||
|
||||
Read `/tmp/config-audit-fix-plan.json` using the Read tool. Cross-reference each fix-plan entry against the humanized scan envelope (`/tmp/config-audit-fix-scan.json`) by finding ID to recover the humanized `title`/`description`/`recommendation` plus `userImpactCategory`/`userActionLanguage` for grouping.
|
||||
|
||||
### Step 3: Present fix plan
|
||||
|
||||
First classify where the auto-fixable entries write. With `--global` the run
|
||||
takes `~/.claude` into the *fix* pass, so machine-wide and project rows land in
|
||||
one table; without a marker they read as equally local. Pass one `--target` per
|
||||
distinct file in the auto-fixable set:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/write-scope-cli.mjs --target "<file-1>" --target "<file-2>" --repo "$PWD" --output-file /tmp/config-audit-fix-scope.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Exit 0 = classified; 3 = argument error (show the stderr message). Read
|
||||
`/tmp/config-audit-fix-scope.json` and carry each file's `scopeClass` into the
|
||||
table below.
|
||||
|
||||
Show what will be fixed and what needs manual attention. Group by `userActionLanguage` so the urgency phrasing stays consistent with the rest of the toolchain:
|
||||
|
||||
```markdown
|
||||
|
|
@ -62,9 +89,9 @@ Show what will be fixed and what needs manual attention. Group by `userActionLan
|
|||
|
||||
#### {userActionLanguage}
|
||||
|
||||
| # | ID | Issue | File |
|
||||
|---|-----|-------|------|
|
||||
| 1 | {id} | {humanized title} | {file} |
|
||||
| # | ID | Issue | File | Scope |
|
||||
|---|-----|-------|------|-------|
|
||||
| 1 | {id} | {humanized title} | {file} | {scopeClass, or blank when "in-repo"} |
|
||||
|
||||
**Manual ({M} issues — require human judgment), grouped by impact:**
|
||||
|
||||
|
|
@ -77,7 +104,10 @@ Show what will be fixed and what needs manual attention. Group by `userActionLan
|
|||
|
||||
### Step 4: Confirm with user
|
||||
|
||||
If not `--dry-run`, ask for confirmation:
|
||||
If not `--dry-run`, ask for confirmation. Render each distinct string in the scope
|
||||
payload's `disclosures[]` verbatim first.
|
||||
|
||||
When `requiresApproval` is false:
|
||||
|
||||
```
|
||||
AskUserQuestion:
|
||||
|
|
@ -88,24 +118,59 @@ AskUserQuestion:
|
|||
- "Cancel"
|
||||
```
|
||||
|
||||
When `requiresApproval` is true — which is what `--global` produces, since
|
||||
`~/.claude` is machine-wide — the question MUST say so and the safe option MUST
|
||||
come first:
|
||||
|
||||
```
|
||||
AskUserQuestion:
|
||||
question: "{K} of {N} fixes change configuration outside this project. Apply all {N}?"
|
||||
options:
|
||||
- "Show dry-run only"
|
||||
- "Yes — apply all, including outside this project"
|
||||
- "Cancel"
|
||||
```
|
||||
|
||||
### Step 5: Apply fixes
|
||||
|
||||
If confirmed, apply:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> --apply --json 2>/dev/null
|
||||
# Re-assign here: each fenced block is its own Bash call, so the value
|
||||
# set in Step 1 is empty by the time this block runs.
|
||||
GLOBAL_FLAG="" # --global when the user asked for global scope
|
||||
# --approve-scope carries the answer the user just gave to the Step-4 question.
|
||||
# The engine runs the same scope gate as Step 3 and withholds a `require-ok`
|
||||
# write on its own, so leaving this empty after the user answered "Yes — apply
|
||||
# all, including outside this project" makes the run refuse the very fixes they
|
||||
# approved. Set it ONLY on that answer — never as a default, and never because
|
||||
# Step 3 already classified the targets: classifying is not approving.
|
||||
APPROVE_SCOPE="" # --approve-scope when the user approved the outside-project fixes
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs "<path>" --apply $GLOBAL_FLAG $APPROVE_SCOPE --output-file /tmp/config-audit-fix-applied.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read the JSON output to get applied/failed counts and backup location.
|
||||
Read `/tmp/config-audit-fix-applied.json` with the Read tool to get applied/failed counts and the backup ID. Exit code 2 means at least one fix failed — report it; `failed[]` carries the reason per fix.
|
||||
|
||||
The payload also carries the engine's own scope verdict. When it reads
|
||||
`"status": "refused"` with `"reason": "scope-gate"`, nothing was written: render
|
||||
each line of `disclosures` verbatim, then ask the Step-4 question again rather
|
||||
than re-running with the flag on the user's behalf. A refusal is a verdict about
|
||||
a config that WAS examined, so the exit code stays in the normal 0/1/2 range —
|
||||
do not report it as a tool error.
|
||||
|
||||
### Step 6: Show results
|
||||
|
||||
Run a quick posture check to measure improvement:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs <path> --json --output-file /tmp/config-audit-fix-posture-$$.json 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs "<path>" --json --output-file /tmp/config-audit-fix-posture.json >/dev/null 2>/dev/null
|
||||
```
|
||||
|
||||
Use the Read tool on `/tmp/config-audit-fix-posture.json` and take `overallGrade`
|
||||
and the score from there. That read is the only source for the numbers below:
|
||||
`--json` prints the same envelope to stdout, but 255 KB of raw JSON in the
|
||||
transcript to recover one grade is exactly the waste this plugin exists to find.
|
||||
|
||||
Present results:
|
||||
|
||||
```markdown
|
||||
|
|
@ -139,7 +204,8 @@ Run `/config-audit plan` to get a step-by-step guide for addressing these.
|
|||
|
||||
## Safety
|
||||
|
||||
- Backup is **mandatory** — every fix creates a backup first
|
||||
- Backup is **mandatory** — every fix creates a backup first, including file renames (the source file is backed up before the rename, so rollback can restore it at its original path)
|
||||
- Dry-run by default — user must confirm before changes
|
||||
- Verify after fix — re-scans to confirm findings resolved
|
||||
- Verify after fix — re-scans in the **same scope** the fix run used, so a `--global` run is verified against user scope too
|
||||
- Rollback always available — `/config-audit rollback <backup-id>`
|
||||
- A failed fix is reported, never swallowed — exit 2 plus a `failed[]` entry
|
||||
|
|
|
|||
|
|
@ -22,24 +22,47 @@ Execute the action plan with full backup, verification, and rollback support.
|
|||
|
||||
### Step 1: Parse flags, load and verify
|
||||
|
||||
Check whether `$ARGUMENTS` contains `--raw`. Carry the answer yourself: the agent
|
||||
prompt in Step 4 is **not** a shell, so a variable assigned in a bash block cannot
|
||||
be referenced from it. Substitute `{mode}` literally with `--raw` or `humanized`.
|
||||
|
||||
Find the most recent session with a plan (use the **Glob tool** for
|
||||
`~/.claude/config-audit/sessions/*/state.yaml`, then Read the newest match — Read
|
||||
does not expand `*`). If none: "No action plan found. Run `/config-audit plan` first."
|
||||
|
||||
Use the Read tool on the action plan and count actions.
|
||||
|
||||
Now classify where those actions actually write. A plan whose actions target
|
||||
`~/.claude/CLAUDE.md` and a plan whose actions target `./CLAUDE.md` are the same
|
||||
count of actions — presenting only the count made a machine-wide change look
|
||||
identical to a project-local one. Pass one `--target` per distinct file the plan
|
||||
touches (absolute paths, as written in the plan):
|
||||
|
||||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/write-scope-cli.mjs --target "<file-1>" --target "<file-2>" --repo "$PWD" --output-file /tmp/config-audit-implement-scope.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Find the most recent session with a plan. If none: "No action plan found. Run `/config-audit plan` first."
|
||||
|
||||
Use the Read tool on the action plan and count actions. Tell the user:
|
||||
Exit 0 = classified; 3 = argument error (show the stderr message). Read
|
||||
`/tmp/config-audit-implement-scope.json`. Tell the user:
|
||||
|
||||
```
|
||||
## Implementing Action Plan
|
||||
|
||||
Found {N} actions to execute across {M} files.
|
||||
A backup will be created before any changes are made.
|
||||
|
||||
{For each target whose `gate` is not "silent", one line:}
|
||||
- `{target}` — {scopeClass}
|
||||
```
|
||||
|
||||
### Step 2: Get user approval
|
||||
|
||||
Render each distinct string in `disclosures[]` verbatim before asking — they are
|
||||
already plain-language, and the payload carries them so this template never has
|
||||
to restate what a scope class means.
|
||||
|
||||
When `requiresApproval` is false, ask as before:
|
||||
|
||||
```
|
||||
AskUserQuestion:
|
||||
question: "Ready to implement {N} actions? Backup created automatically — you can roll back with one command."
|
||||
|
|
@ -49,30 +72,47 @@ AskUserQuestion:
|
|||
- "Cancel"
|
||||
```
|
||||
|
||||
When `requiresApproval` is true, the question MUST name the scope, and the
|
||||
safe option MUST come first — a plan that edits machine-wide configuration
|
||||
affects every project the user opens, so the default must not be "proceed":
|
||||
|
||||
```
|
||||
AskUserQuestion:
|
||||
question: "This plan changes configuration outside this project ({K} of {M} files). Proceed?"
|
||||
options:
|
||||
- "Review plan first" (then show the plan file path)
|
||||
- "Yes — change files outside this project too"
|
||||
- "Cancel"
|
||||
```
|
||||
|
||||
### Step 3: Create backup
|
||||
|
||||
Create backup silently:
|
||||
Create the backup through the code that owns the format. Pass one `--target` per
|
||||
pre-existing file the plan will MODIFY, and one `--created` per file the plan
|
||||
will CREATE — a backup cannot hold a file that does not exist yet, so those are
|
||||
recorded rather than copied, and `/config-audit rollback` reads them back to tell
|
||||
the user which files it is leaving behind.
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.claude/config-audit/backups/$(date +%Y%m%d_%H%M%S)/files/ 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/rollback-cli.mjs --create --target "<file-1>" --target "<file-2>" --created "<new-file-1>" --repo "$PWD" --output-file /tmp/config-audit-implement-backup.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Copy each file to be modified. Generate `manifest.yaml` with checksums.
|
||||
| Exit | Meaning |
|
||||
|------|---------|
|
||||
| 0 | every target was backed up |
|
||||
| 1 | at least one target was not there — read `skipped[]` before continuing |
|
||||
| 3 | the CLI could not run (show the stderr message); do not edit anything |
|
||||
|
||||
The manifest is what `/config-audit rollback` reads, so it MUST carry both lists:
|
||||
Read `/tmp/config-audit-implement-backup.json`. The payload's `backupId` is the
|
||||
ID to quote from here on — **never re-derive it**. Shell state does not survive
|
||||
to the next block, and a second `date` call that straddles a second boundary
|
||||
would hand the user a rollback ID that does not exist. Use it wherever
|
||||
`{backup-id}` appears below.
|
||||
|
||||
```yaml
|
||||
files: # pre-existing files this run will MODIFY
|
||||
- backup: files/root/CLAUDE.md
|
||||
original: /abs/path/CLAUDE.md
|
||||
sha256: <sha256 of the pre-change content>
|
||||
created: # files this run will CREATE (no backup can exist)
|
||||
- /abs/path/.claude/rules/post-quality.md
|
||||
```
|
||||
|
||||
Record every `create`-type action under `created:`. Rollback cannot restore a
|
||||
file that never existed, but it must be able to tell the user which files it is
|
||||
leaving behind — a half-restored target is only dangerous when it is silent.
|
||||
On exit 1, name the `skipped[]` paths before doing anything else: the plan is
|
||||
about to change files that have no backup behind them. If any skipped path is one
|
||||
the plan MODIFIES (rather than creates), stop and report — that action cannot be
|
||||
rolled back.
|
||||
|
||||
Tell the user: **"Backup created. Implementing actions..."**
|
||||
|
||||
|
|
@ -86,7 +126,7 @@ Agent(subagent_type: "config-audit:implementer-agent")
|
|||
prompt: |
|
||||
Execute action: {action-id}
|
||||
File: {file-path}, Type: {create|modify|delete}
|
||||
Mode: $RAW_FLAG (empty = humanized progress prose; "--raw" = v5.0.0 verbatim)
|
||||
Mode: {mode} ("humanized" = humanized progress prose; "--raw" = v5.0.0 verbatim)
|
||||
Details: {changes}
|
||||
Verify backup exists, make change, validate syntax.
|
||||
When logging progress, use the humanized title/userActionLanguage
|
||||
|
|
@ -117,7 +157,19 @@ Agent(subagent_type: "config-audit:verifier-agent")
|
|||
1. Modified files exist and are syntactically valid
|
||||
2. New files created correctly
|
||||
3. No new conflicts introduced
|
||||
Report to: ~/.claude/config-audit/sessions/{session-id}/implementation-log.md
|
||||
Return your findings as your final message. Do NOT write them to a file —
|
||||
this agent is read-only by design (tools: Read, Glob, Grep) and has no
|
||||
write tool; instructing it to write a report is a contract it cannot keep.
|
||||
```
|
||||
|
||||
Append the verifier's returned findings to the log yourself, with Bash `>>`
|
||||
(heredoc) — never the Write tool, for the same reason as Step 4:
|
||||
|
||||
```bash
|
||||
cat >> ~/.claude/config-audit/sessions/{session-id}/implementation-log.md <<'EOF'
|
||||
## Verification
|
||||
{verifier findings}
|
||||
EOF
|
||||
```
|
||||
|
||||
If verifier finds issues: one retry with implementer agent. If still failing: report and suggest rollback.
|
||||
|
|
@ -129,27 +181,51 @@ If verifier finds issues: one retry with implementer agent. If still failing: re
|
|||
|
||||
**{succeeded} succeeded** | {failed} failed | {skipped} skipped
|
||||
|
||||
{If score improved, run quick posture and show:}
|
||||
Score impact: {old_grade} → {new_grade} (+{delta} points)
|
||||
|
||||
{If failed > 0:}
|
||||
{failed} action(s) couldn't be completed — see log for details.
|
||||
|
||||
**Backup location:** `~/.claude/config-audit/backups/{timestamp}/`
|
||||
**Rollback:** `/config-audit rollback {timestamp}`
|
||||
**Backup location:** `~/.claude/config-audit/backups/{backup-id}/`
|
||||
**Rollback:** `/config-audit rollback {backup-id}`
|
||||
**Full log:** `~/.claude/config-audit/sessions/{session-id}/implementation-log.md`
|
||||
```
|
||||
|
||||
**On reporting a score.** Only quote a grade *change* if the pre-change grade was
|
||||
actually captured before Step 4 ran. Once the files are edited, only the new grade
|
||||
is measurable — a delta computed after the fact has no source and must not be
|
||||
invented. To offer one, measure first in Step 1 and again here:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs "<target-path>" --output-file /tmp/config-audit-implement-posture.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Then Read `/tmp/config-audit-implement-posture.json`. Both the `--output-file` and
|
||||
the `2>/dev/null` are required by the output rules — a bare scanner call would put
|
||||
diagnostic output in front of the user. If no pre-change grade was captured, report
|
||||
the new grade alone and say nothing about a delta.
|
||||
|
||||
### Step 7: Update state
|
||||
|
||||
Update `state.yaml` with `current_phase: "implement"`, `next_phase: null`.
|
||||
Update `state.yaml` with all four fields `.claude/rules/state-management.md` requires:
|
||||
|
||||
- `current_phase: "implement"`
|
||||
- `completed_phases`: append `implement` to the existing array (read it first; never replace it)
|
||||
- `next_phase: null`
|
||||
- `updated_at`: current timestamp
|
||||
|
||||
A full-file Write that names only two of the four silently deletes the other two.
|
||||
|
||||
## Rollback
|
||||
|
||||
If the user requests rollback at any point:
|
||||
1. Read `manifest.yaml` from backup
|
||||
2. Restore each file and verify checksums
|
||||
3. Delete newly created files
|
||||
1. Run `/config-audit rollback {backup-id}` — it drives `scanners/rollback-cli.mjs`,
|
||||
which verifies each checksum before and after writing. Do not restore by hand:
|
||||
a copy performs neither check, and the result cannot be reported as verified.
|
||||
2. Read the restore payload and report the per-file `status` it returns
|
||||
3. **Report — do not delete — the files this run created.** Rollback restores from
|
||||
backup, and no backup can exist for a file that did not exist before. Those
|
||||
paths stay on disk; `/config-audit rollback` lists them under "Left in place"
|
||||
so the user can remove them deliberately. Promising deletion here would leave a
|
||||
half-restored config that reads as a clean rollback.
|
||||
4. Update state to `rolled_back`
|
||||
|
||||
## Error Handling
|
||||
|
|
|
|||
|
|
@ -11,8 +11,8 @@ Gather user preferences to inform the action plan.
|
|||
|
||||
## IMPORTANT: Inline Execution Only
|
||||
|
||||
This command runs AskUserQuestion **directly in the main context** — NOT via a Task subagent.
|
||||
AskUserQuestion requires synchronous terminal interaction and does not work when delegated to a Task subagent.
|
||||
This command runs AskUserQuestion **directly in the main context** — NOT via an `Agent` subagent.
|
||||
AskUserQuestion requires synchronous terminal interaction and does not work when delegated to an `Agent` subagent.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -32,10 +32,29 @@ AskUserQuestion requires synchronous terminal interaction and does not work when
|
|||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
```
|
||||
|
||||
1. **Load session state**: Verify analysis phase completed, read analysis report for context
|
||||
2. **Conduct interview inline**: Use AskUserQuestion tool directly (NOT via Task). Adapt questions based on analysis findings.
|
||||
1. **Resolve the session, then load its state**:
|
||||
|
||||
```
|
||||
Glob: ~/.claude/config-audit/sessions/*/state.yaml
|
||||
Sort by modification time — the most recently modified session wins
|
||||
```
|
||||
|
||||
Every path below substitutes that session's id for `{session-id}`. Never guess
|
||||
it: if the Glob returns nothing, say "No audit session found — run
|
||||
`/config-audit discover` first" and exit. Read the session's `state.yaml` and
|
||||
verify `completed_phases` contains `analyze`; if it doesn't, tell the user
|
||||
analysis hasn't run yet and exit. Then read the analysis report for context.
|
||||
2. **Conduct interview inline**: Use AskUserQuestion tool directly (never delegate it to a subagent via `Agent` — a subagent cannot hold the interactive turn). Adapt questions based on analysis findings.
|
||||
3. **Save interview results**: Write to `~/.claude/config-audit/sessions/{session-id}/interview.md`
|
||||
4. **Update state** (see state-management rule)
|
||||
4. **Update state** (see state-management rule), with one bound specific to this
|
||||
command: interview is optional and can be run against a session that already
|
||||
moved past it. If `completed_phases` already contains a later phase (`plan`,
|
||||
`implement`, `verify`), do **not** rewind `current_phase` and do not re-add a
|
||||
phase already in `completed_phases` — append `interview` only if it is absent,
|
||||
leave `current_phase`/`next_phase` pointing at the furthest phase reached, and
|
||||
tell the user the preferences will apply the next time `/config-audit plan`
|
||||
runs. Rewinding a finished session is how its progress gets lost. Always set
|
||||
`updated_at` to the current timestamp, whichever branch above applies.
|
||||
5. **Output summary**
|
||||
|
||||
## Interview Questions
|
||||
|
|
|
|||
|
|
@ -44,14 +44,21 @@ re-verification) and polling for new Claude Code practices...
|
|||
### Step 2: Run the stale-check CLI
|
||||
|
||||
```bash
|
||||
# Pass the threshold as its OWN quoted argument. Building "--stale-after 30" into
|
||||
# one variable and expanding it unquoted only works if the shell word-splits —
|
||||
# bash does, zsh (the macOS default) does not, and there the flag silently
|
||||
# reverted to the 90-day default while the command reported success.
|
||||
TODAY=$(date +%F)
|
||||
STALE_AFTER=""
|
||||
if echo "$ARGUMENTS" | grep -qE -- '--stale-after'; then
|
||||
STALE_AFTER="--stale-after $(echo "$ARGUMENTS" | sed -nE 's/.*--stale-after[ =]+([0-9]+).*/\1/p')"
|
||||
STALE_AFTER_DAYS=$(echo "$ARGUMENTS" | sed -nE 's/.*--stale-after[ =]+([0-9]+).*/\1/p')
|
||||
if [ -n "$STALE_AFTER_DAYS" ]; then
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/knowledge-refresh-cli.mjs \
|
||||
--reference-date "$TODAY" --stale-after "$STALE_AFTER_DAYS" \
|
||||
--output-file ~/.claude/config-audit/sessions/knowledge-refresh.json 2>/dev/null; echo $?
|
||||
else
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/knowledge-refresh-cli.mjs \
|
||||
--reference-date "$TODAY" \
|
||||
--output-file ~/.claude/config-audit/sessions/knowledge-refresh.json 2>/dev/null; echo $?
|
||||
fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/knowledge-refresh-cli.mjs \
|
||||
--reference-date "$TODAY" $STALE_AFTER \
|
||||
--output-file ~/.claude/config-audit/sessions/knowledge-refresh.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Exit code **0** = all fresh, **1** = some stale (advisory, normal), **3** = real error →
|
||||
|
|
@ -100,16 +107,27 @@ Be explicit: **"I will not change any file until you approve specific items."**
|
|||
|
||||
### Step 6: Apply approved writes (only the approved ones)
|
||||
|
||||
**Where the register lives — say this before writing.** The register is part of the plugin,
|
||||
and the stale check above read it from `${CLAUDE_PLUGIN_ROOT}/knowledge/best-practices.json`
|
||||
(the `registerPath` field in the payload names the exact file). For a marketplace install that
|
||||
is the plugin cache, so **an edit there is discarded by the next plugin upgrade** — the durable
|
||||
home for an approved change is the plugin's own checkout. Tell the user which of the two they
|
||||
are about to write to, using the `registerPath` they can see, before asking for approval.
|
||||
|
||||
For each approved item:
|
||||
1. Edit `knowledge/best-practices.json` — bump `source.verified`, update the `claim`/
|
||||
`recommendation`, or append the new entry. Keep the file's 2-space JSON formatting.
|
||||
2. If a `knowledge/*.md` mirror states the same fact, update it too so the human-readable
|
||||
mirror doesn't drift from the register.
|
||||
1. Edit `${CLAUDE_PLUGIN_ROOT}/knowledge/best-practices.json` — bump `source.verified`, update
|
||||
the `claim`/`recommendation`, or append the new entry. Keep the file's 2-space JSON
|
||||
formatting. Use the anchored path, never a bare `knowledge/…` — a relative path resolves
|
||||
against the user's current repo, which is not the file the CLI read.
|
||||
2. If a `${CLAUDE_PLUGIN_ROOT}/knowledge/*.md` mirror states the same fact, update it too so
|
||||
the human-readable mirror doesn't drift from the register.
|
||||
3. **Validate before declaring done** — re-run the register schema check and confirm zero errors:
|
||||
```bash
|
||||
node --test ${CLAUDE_PLUGIN_ROOT}/tests/lib/best-practices-register.test.mjs 2>&1 | tail -5
|
||||
```
|
||||
If validation fails, revert that edit and report it — never leave the register invalid.
|
||||
This test loads the register through the same anchored path, so it validates the file you
|
||||
just edited — that only holds while step 1 uses the anchored path too. If validation fails,
|
||||
revert that edit and report it — never leave the register invalid.
|
||||
|
||||
Report exactly what changed (ids + fields), and what was deferred to manual review.
|
||||
|
||||
|
|
|
|||
|
|
@ -39,10 +39,9 @@ First non-flag argument is the path (default `.`). Recognized flags:
|
|||
Tell the user: **"Building token-source manifest for `<path>`..."**
|
||||
|
||||
```bash
|
||||
TMPFILE="/tmp/ca-manifest-$$.json"
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/manifest.mjs <path> --output-file "$TMPFILE" $RAW_FLAG 2>/dev/null; echo $?
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/manifest.mjs "<path>" --output-file /tmp/config-audit-manifest.json $RAW_FLAG >/dev/null 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
**Exit code handling:**
|
||||
|
|
@ -52,14 +51,14 @@ node ${CLAUDE_PLUGIN_ROOT}/scanners/manifest.mjs <path> --output-file "$TMPFILE"
|
|||
### Step 3: If `--json` was requested, cat the file and stop
|
||||
|
||||
```bash
|
||||
cat "$TMPFILE"
|
||||
cat /tmp/config-audit-manifest.json
|
||||
```
|
||||
|
||||
Do NOT render the table in JSON mode.
|
||||
|
||||
### Step 4: Read JSON and render
|
||||
|
||||
Use the Read tool on `$TMPFILE`. Extract `meta.repoPath`, `total`, `summary`, and `sources[]`. Lead with the **always-loaded subtotal** (the headline), then render the top 20 sources (or fewer if the manifest is shorter):
|
||||
Use the Read tool on `/tmp/config-audit-manifest.json`. Extract `meta.repoPath`, `total`, `summary`, and `sources[]`. Lead with the **always-loaded subtotal** (the headline), then render the top 20 sources (or fewer if the manifest is shorter):
|
||||
|
||||
```markdown
|
||||
**Token-source manifest for `<repoPath>`** — ~{total} tokens total
|
||||
|
|
@ -70,10 +69,11 @@ Use the Read tool on `$TMPFILE`. Extract `meta.repoPath`, `total`, `summary`, an
|
|||
|
||||
| Rank | Kind | Name | Source | Tokens | Load |
|
||||
|------|------|------|--------|--------|------|
|
||||
| 1 | {kind} | `<name>` | {source} | ~{estimated_tokens} | {load} |
|
||||
| 1 | {kind} | `<name>` | {source} | ~{estimated_tokens} | {loadPattern} |
|
||||
| ... | ... | ... | ... | ... | ... |
|
||||
|
||||
_Load column: **always** / **on-demand** / **external**. Append `°` when `derivationConfidence` is `inferred` (no primary-doc row pins it exactly)._
|
||||
_Agent rows carry `model` and `effort`. When either is set, append it to the name — `` `reviewer` (haiku/low) `` — using `inherit` / `default` for the unset side. Leave the suffix off entirely when both are null; a row of "inherit/default" on every agent is noise, and `/config-audit feature-gap` is where that becomes a finding._
|
||||
_Estimates assume ~4 chars/token (Claude ballpark). Real token count varies ±15%._
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -25,6 +25,8 @@ is hybrid: a cheap deterministic pre-filter finds candidates, then the opus
|
|||
- **Unscoped path-specific instructions → path-scoped rules** (BP-MECH-002)
|
||||
- **Absolute "never" prohibitions → permissions / hooks** (BP-MECH-004)
|
||||
- **`--subtract`:** instructions that no longer earn their always-loaded rent (BP-SUB-001)
|
||||
- **`--subtract --for-model <name>`:** of those, the ones a named model documents
|
||||
as redundant — e.g. self-verification instructions on Claude Opus 5 (BP-PROMPT-001)
|
||||
|
||||
Each finding cites its register rule + source URL. A clean CLAUDE.md returns "no
|
||||
opportunities" — that is a good result, not a failure.
|
||||
|
|
@ -35,7 +37,25 @@ opportunities" — that is a good result, not a failure.
|
|||
|
||||
Split `$ARGUMENTS` into a path (first non-flag argument; default: current working
|
||||
directory) and flags. Recognized flags: `--global` (include the user `~/.claude`
|
||||
cascade in discovery) and `--subtract` (add the subtraction axis, below).
|
||||
cascade in discovery), `--subtract` (add the subtraction axis, below),
|
||||
`--for-model <name>` (annotate model-scoped candidates, below) and `--apply`
|
||||
(execute approved removals — Step 7).
|
||||
|
||||
`--apply` only means anything alongside `--subtract`. If it is present without
|
||||
it, say so and continue with the ordinary lens run:
|
||||
|
||||
```
|
||||
`--apply` executes approved subtraction removals, so it needs `--subtract` too.
|
||||
Running the ordinary lens; re-run with `--subtract --apply` to remove anything.
|
||||
```
|
||||
|
||||
`--for-model <name>` has the same dependency. If it is present without
|
||||
`--subtract`, say so and continue with the ordinary lens run:
|
||||
|
||||
```
|
||||
`--for-model` annotates subtraction candidates, so it needs `--subtract` too.
|
||||
Running the ordinary lens; re-run with `--subtract --for-model <name>`.
|
||||
```
|
||||
|
||||
**`--subtract` — the inverse question.** Every other lens asks what to *add* or
|
||||
*move*; this one asks what no longer earns its always-loaded rent. It is opt-in
|
||||
|
|
@ -43,6 +63,15 @@ because it asks something different, and because deleting is not undoable by
|
|||
reading. Pair it with `--global` to reach the user-level CLAUDE.md, where the
|
||||
always-loaded cost actually sits (it loads in every repo, every session).
|
||||
|
||||
**`--for-model <name>` — whose redundancy?** Some instructions are only dead
|
||||
weight for a *particular* model. Anthropic documents that Claude Opus 5 verifies
|
||||
its own work and over-verifies when told to double-check or to delegate
|
||||
verification to a subagent. That claim is model-scoped, so the model must be
|
||||
named: a CLAUDE.md carries no frontmatter and no target model, and the same file
|
||||
is read by whichever model the next session happens to run. The flag never adds
|
||||
candidates — it annotates ones `--subtract` already found. Example:
|
||||
`--subtract --for-model opus-5`.
|
||||
|
||||
If `--subtract` is present, say so up front:
|
||||
|
||||
```
|
||||
|
|
@ -69,7 +98,17 @@ GLOBAL_FLAG=""
|
|||
if echo "$ARGUMENTS" | grep -q -- "--global"; then GLOBAL_FLAG="--global"; fi
|
||||
SUBTRACT_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--subtract"; then SUBTRACT_FLAG="--subtract"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/optimize-lens-cli.mjs <target-path> --output-file ~/.claude/config-audit/sessions/{session-id}/optimize-lens.json $GLOBAL_FLAG $SUBTRACT_FLAG 2>/dev/null; echo $?
|
||||
# --for-model takes a VALUE, so flag and value are two separate variables. Never
|
||||
# pack them into one ("--for-model x"): the shell here is zsh, which does not
|
||||
# word-split an unquoted expansion, so one variable would reach the CLI as a
|
||||
# single malformed argv entry and be silently ignored (M-BUG-45).
|
||||
FOR_MODEL_FLAG=""
|
||||
FOR_MODEL_VALUE=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--for-model "; then
|
||||
FOR_MODEL_FLAG="--for-model"
|
||||
FOR_MODEL_VALUE=$(echo "$ARGUMENTS" | sed -n 's/.*--for-model \([^ ]*\).*/\1/p')
|
||||
fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/optimize-lens-cli.mjs "<target-path>" --output-file ~/.claude/config-audit/sessions/{session-id}/optimize-lens.json $GLOBAL_FLAG $SUBTRACT_FLAG $FOR_MODEL_FLAG $FOR_MODEL_VALUE 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Exit code 0 is normal. Only exit code 3 is a real error → "The lens couldn't run.
|
||||
|
|
@ -85,6 +124,17 @@ Under `--subtract` it also has a `subtract` block (`candidates`, `register`) and
|
|||
`counts.subtractCandidates`. Each subtraction candidate spans `line`–`endLine`
|
||||
(a whole block). Include the whole `subtract` block when spawning the agent.
|
||||
|
||||
Under `--for-model` the `subtract` block additionally has `forModel`
|
||||
(`{ requested, recognized, matchedCount }`), and a subset of its candidates carry
|
||||
`modelScope` (`{ registerId, claim, requestedModel }`). If `recognized` is
|
||||
`false`, the register does not cover that model name — tell the user before
|
||||
showing results, so a zero is never read as "your config is already clean":
|
||||
|
||||
```
|
||||
I don't have a model-specific rule for "{requested}", so nothing was annotated
|
||||
for it. The ordinary subtraction results below are unaffected.
|
||||
```
|
||||
|
||||
**Early exit:** if `counts.deterministic === 0` and `counts.candidates === 0`
|
||||
(and, under `--subtract`, `counts.subtractCandidates === 0`), skip the agent and
|
||||
tell the user plainly:
|
||||
|
|
@ -125,7 +175,88 @@ If the agent kept nothing from the candidates (all dropped) but there were
|
|||
deterministic findings, show those; if it kept nothing at all, show the clean
|
||||
result from Step 3.
|
||||
|
||||
### Step 6: Next steps
|
||||
### Step 7: Apply approved removals (`--subtract --apply` only)
|
||||
|
||||
Skip this step entirely unless BOTH flags are present and the agent kept at
|
||||
least one subtraction finding. Removal is the only thing this plugin does that
|
||||
takes configuration away, so nothing here happens without a named choice.
|
||||
|
||||
**7a — show what is on the table, with honest sizing.** List the kept
|
||||
subtraction findings numbered, each with its file, line span and first line of
|
||||
text. A finding carrying `modelScope` must show its condition here too — this
|
||||
is the last point a human sees it before it reaches an approval file, so the
|
||||
citation must not read as unconditional:
|
||||
|
||||
```
|
||||
{n}. {file}:{line}-{endLine} — {first line of text}
|
||||
Model-scoped: redundant if you are targeting {requestedModel}. Other
|
||||
sessions on this config may run a different model.
|
||||
```
|
||||
|
||||
Do not imply a bigger win than there is:
|
||||
|
||||
```
|
||||
Removing all of these saves roughly {n} tokens per turn — on a typical
|
||||
always-loaded CLAUDE.md that is around a fifth of the file, not most of it.
|
||||
```
|
||||
|
||||
Ask which to remove: numbers, `all`, or `none`. `none` ends the command.
|
||||
|
||||
**7b — write the approval file.** With the **Write** tool, write the operator's
|
||||
choice to `~/.claude/config-audit/sessions/{session-id}/subtraction-approved.json`
|
||||
(absolute path — a relative one resolves against the user's CWD). Take `file`,
|
||||
`line`, `endLine` and `signalText` verbatim from the Step 3 payload; `text` must
|
||||
be the `signalText` byte-for-byte, because the engine refuses a removal whose
|
||||
text no longer matches the file:
|
||||
|
||||
```json
|
||||
{ "sessionId": "{session-id}",
|
||||
"removals": [ { "file": "...", "line": 12, "endLine": 15, "text": "..." } ] }
|
||||
```
|
||||
|
||||
**7c — dry run first.** Always. It costs one call and proves the spans still
|
||||
match before anything is written.
|
||||
|
||||
`--repo` is the **session's own root (`$PWD`), never the scanned path**. It is
|
||||
what the target is classified *against*: pass the scan target and
|
||||
`~/.claude/CLAUDE.md` classifies as `in-repo`, which drops the gate to `silent`
|
||||
on the one target that most needs it (measured — the same silent downgrade as a
|
||||
naive `.git`-upward walk, arriving through a different door).
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/subtraction-write-cli.mjs --approved ~/.claude/config-audit/sessions/{session-id}/subtraction-approved.json --repo "$PWD" --dry-run --output-file ~/.claude/config-audit/sessions/{session-id}/subtraction-dryrun.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read the payload. Exit 3 is a real error (bad or unreadable approval file).
|
||||
Report any `refused` entry with its `reason` before going further —
|
||||
`block-mismatch` means the file changed since the scan (re-run the lens),
|
||||
`floor` means the block is load-bearing and will never be removable.
|
||||
|
||||
**7d — the scope gate.** If the dry-run payload has `requiresApproval: true`,
|
||||
show every line in `disclosures` verbatim and ask for an explicit go-ahead. This
|
||||
is the machine-wide case (`~/.claude/CLAUDE.md`): the change costs — and saves —
|
||||
in every project, on every turn, so it is not the same decision as editing the
|
||||
CLAUDE.md in front of you. Without a clear yes, stop here.
|
||||
|
||||
**7e — apply.** Same command without `--dry-run`, adding `--approve-scope` only
|
||||
if the operator gave that go-ahead in 7d:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/subtraction-write-cli.mjs --approved ~/.claude/config-audit/sessions/{session-id}/subtraction-approved.json --repo "$PWD" --output-file ~/.claude/config-audit/sessions/{session-id}/subtraction-result.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
**7f — report.** From the result payload, tell the user: what was removed (file
|
||||
+ line span + the text, from `applied`), what was refused and why (`refused`
|
||||
with `reason`), and how to undo it:
|
||||
|
||||
```
|
||||
Backed up as {backupId} — `/config-audit rollback {backupId}` restores every
|
||||
file exactly as it was.
|
||||
```
|
||||
|
||||
Never report a run as successful when `counts.applied` is 0.
|
||||
|
||||
### Step 8: Next steps
|
||||
|
||||
End with context-sensitive next steps, explaining WHY each is useful:
|
||||
|
||||
|
|
@ -139,12 +270,26 @@ End with context-sensitive next steps, explaining WHY each is useful:
|
|||
|
||||
- This command is **agent-driven and not byte-stable** — its output is a
|
||||
human-facing report, deliberately outside the deterministic snapshot suite.
|
||||
- `--subtract` **proposes, never writes.** Nothing is deleted; act on a finding
|
||||
via `/config-audit plan` → `/config-audit implement` (backup + rollback).
|
||||
- `--subtract` **proposes; only `--apply` writes**, and only blocks the operator
|
||||
named. Every removal is preceded by a backup whose manifest is verified to
|
||||
cover the file being written, and `/config-audit rollback` restores it.
|
||||
- **`--for-model` annotates; it never detects.** It tags a subset of the
|
||||
candidates `--subtract` already produced, and there is no auto-detection by
|
||||
design: a CLAUDE.md has no frontmatter and no resolvable target model, and the
|
||||
same file is read by whichever model the next session runs. So the citation is
|
||||
always **conditional** — it must be shown that way in the report and in the
|
||||
Step 7a approval listing, never as a settled fact about the file.
|
||||
- **Removal is not a `fix` action and not a `plan`/`implement` step**, by
|
||||
measurement rather than preference: the subtraction axis never enters the
|
||||
orchestrated envelope, so `fix`'s re-scan verification would mark every
|
||||
removal verified whether or not it happened, and the findings pipeline would
|
||||
require a finding code — which names a deterministic check, not a prose
|
||||
judgement. `subtraction-write-cli.mjs` owns the execution instead.
|
||||
- The subtraction floor is deterministic and runs *before* the agent, so a
|
||||
load-bearing block is never a candidate. It errs toward keeping: on a
|
||||
well-maintained config this axis is mostly a no-op, and that is a good result.
|
||||
- The deterministic half (CA-OPT-001) also rides in the normal orchestrated
|
||||
audit; this command adds the prose-judgment half on top.
|
||||
- No files are modified. To act on a finding, use `/config-audit plan` →
|
||||
`/config-audit implement` (backup + rollback) or edit by hand.
|
||||
- Without `--apply`, no files are modified. To act on a mechanism-fit finding,
|
||||
use `/config-audit plan` → `/config-audit implement` (backup + rollback) or
|
||||
edit by hand.
|
||||
|
|
|
|||
|
|
@ -22,7 +22,11 @@ Generate a prioritized action plan based on analysis results.
|
|||
|
||||
### Step 1: Verify session state
|
||||
|
||||
Find the most recent session with analysis completed using the Read tool on `~/.claude/config-audit/sessions/*/state.yaml`. If none found: "No analysis results found. Run `/config-audit` first to scan your configuration."
|
||||
Find the most recent session with analysis completed using the **Glob tool** on `~/.claude/config-audit/sessions/*/state.yaml`, then Read the newest match. The Read tool takes one literal path and does not expand `*` — pointing it at the glob makes this step report "no analysis results" even when a valid session exists.
|
||||
|
||||
If no session is found: "No analysis results found. Run `/config-audit` first to scan your configuration."
|
||||
|
||||
Then confirm the report itself exists — a session can carry a valid `state.yaml` and still be missing its report. Read `~/.claude/config-audit/sessions/{session-id}/analysis-report.md`. If it is absent: "Session {session-id} has no analysis report. Run `/config-audit analyze` to generate it." Stop — the planner agent has nothing to read.
|
||||
|
||||
### Step 2: Tell the user what's happening
|
||||
|
||||
|
|
@ -35,10 +39,10 @@ Actions are ordered by impact, with risk assessment and dependency tracking.
|
|||
|
||||
### Step 3: Parse flags and spawn planner agent
|
||||
|
||||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
```
|
||||
Check whether `$ARGUMENTS` contains `--raw`. Carry the answer yourself: the agent
|
||||
prompt below is **not** a shell, so a variable assigned in a bash block cannot be
|
||||
referenced from it. Substitute `{mode}` literally with `--raw` or with `humanized`
|
||||
when writing the prompt.
|
||||
|
||||
Tell the user: **"Generating your action plan (this takes about 30 seconds)..."**
|
||||
|
||||
|
|
@ -49,7 +53,7 @@ Agent(subagent_type: "config-audit:planner-agent")
|
|||
Generate action plan based on:
|
||||
- Analysis: ~/.claude/config-audit/sessions/{session-id}/analysis-report.md
|
||||
- Interview: ~/.claude/config-audit/sessions/{session-id}/interview.md (if exists)
|
||||
Mode: $RAW_FLAG (empty = humanized; "--raw" = v5.0.0 verbatim severity prefiks)
|
||||
Mode: {mode} ("humanized" = humanized; "--raw" = v5.0.0 verbatim severity prefiks)
|
||||
Create a prioritized plan that consumes the humanized finding fields:
|
||||
- Group actions by userImpactCategory (e.g., "Configuration mistake",
|
||||
"Conflict", "Wasted tokens", "Missed opportunity", "Dead config")
|
||||
|
|
@ -69,7 +73,22 @@ Agent(subagent_type: "config-audit:planner-agent")
|
|||
|
||||
### Step 4: Present the plan summary
|
||||
|
||||
Read the generated plan and show a concise overview:
|
||||
Read the generated plan, then classify the files its actions target. This summary
|
||||
IS the approval surface — there is no separate confirmation step here, so a plan
|
||||
that proposes writing to machine-wide configuration has to say so where the user
|
||||
reads it. Pass one `--target` per distinct file the plan touches:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/write-scope-cli.mjs --target "<file-1>" --target "<file-2>" --repo "$PWD" --output-file /tmp/config-audit-plan-scope.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Exit 0 = classified; 3 = argument error (show the stderr message). Read
|
||||
`/tmp/config-audit-plan-scope.json`. If `gate` is not `"silent"`, render each
|
||||
distinct string in `disclosures[]` verbatim directly under the action table, and
|
||||
mark the affected rows — not in a footnote further down, where a user scanning the
|
||||
table would miss it.
|
||||
|
||||
Show a concise overview:
|
||||
|
||||
```markdown
|
||||
### Action Plan Ready
|
||||
|
|
@ -94,7 +113,14 @@ You can edit the plan file to remove, reorder, or modify actions before implemen
|
|||
|
||||
### Step 5: Update state
|
||||
|
||||
Update `state.yaml` with `current_phase: "plan"`, `next_phase: "implement"`.
|
||||
Update `state.yaml` with all four fields `.claude/rules/state-management.md` requires — a partial write drops the fields that make an interrupted run resumable:
|
||||
|
||||
- `current_phase: "plan"`
|
||||
- `completed_phases`: append `plan` to the existing array (read it first; never overwrite it with a fresh list)
|
||||
- `next_phase: "implement"`
|
||||
- `updated_at`: current timestamp
|
||||
|
||||
The planner agent may already have written these. Read the file before writing and preserve whichever fields it set — a full-file Write that names only two fields silently deletes the other two.
|
||||
|
||||
## Plan Modification
|
||||
|
||||
|
|
|
|||
|
|
@ -32,15 +32,21 @@ Auditing {N} plugin(s) for structure, frontmatter quality, and cross-plugin conf
|
|||
|
||||
### Step 2: Run scanner
|
||||
|
||||
Run silently for each plugin. Default mode emits a humanized JSON envelope where each PLH finding carries `userImpactCategory`, `userActionLanguage`, and `relevanceContext` alongside the v5.0.0 fields. `--raw` is passed through verbatim when present.
|
||||
Run silently for each plugin. Default mode writes a humanized JSON payload to `--output-file` where each PLH finding carries `userImpactCategory`, `userActionLanguage`, and `relevanceContext` alongside the v5.0.0 fields. `--raw` is passed through verbatim when present, and prints the byte-stable v5.0.0 envelope on stdout instead.
|
||||
|
||||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs <path> $RAW_FLAG 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs "<path>" --output-file /tmp/config-audit-plugin-health.json $RAW_FLAG >/dev/null 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read stdout output (JSON) using the Read tool. Parse findings.
|
||||
Read `/tmp/config-audit-plugin-health.json` with the Read tool. Exit codes 0, 1 and 2 are normal; only 3 is a real error.
|
||||
|
||||
The payload carries three things the report needs:
|
||||
|
||||
- `plugins[]` — one row per plugin: `name`, `declaredName`, `commandCount`, `agentCount`, `findingCount`, `score`, `grade`. Use these for the table; never estimate a grade yourself.
|
||||
- `cross_plugin_findings[]` — the namespace-collision and shared-command-name findings, already separated from the per-plugin ones (they also carry `crossPlugin: true` in `findings`).
|
||||
- `findings[]` — every finding, humanized.
|
||||
|
||||
### Step 3: Present results
|
||||
|
||||
|
|
@ -49,7 +55,7 @@ Read stdout output (JSON) using the Read tool. Parse findings.
|
|||
|
||||
| Plugin | Grade | Commands | Agents | Status |
|
||||
|--------|-------|----------|--------|--------|
|
||||
| {name} | {grade} ({score}) | {cmd_count} | {agent_count} | {Good/Issues found} |
|
||||
| {plugins[].name} | {plugins[].grade} ({plugins[].score}) | {plugins[].commandCount} | {plugins[].agentCount} | {Good/Issues found} |
|
||||
| ... | ... | ... | ... | ... |
|
||||
|
||||
{If cross-plugin issues:}
|
||||
|
|
|
|||
|
|
@ -42,27 +42,35 @@ Run silently — JSON goes to a file, the humanized scorecard prints to stderr (
|
|||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs <target-path> --output-file /tmp/config-audit-posture-$$.json $RAW_FLAG 2>/tmp/config-audit-posture-stderr-$$.txt; echo $?
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs "<target-path>" --output-file /tmp/config-audit-posture.json $RAW_FLAG >/dev/null 2>/tmp/config-audit-posture-stderr.txt; echo $?
|
||||
```
|
||||
|
||||
Both paths are fixed literals, repeated literally in every later step: each
|
||||
```bash fence is its own process, so a `$$`-derived path could never be named
|
||||
again. `>/dev/null` is required, not cosmetic — with `--raw` the scanner writes
|
||||
the full envelope to stdout *as well as* the file (`posture.mjs:101`), which is
|
||||
255 KB on a real repo.
|
||||
|
||||
If exit code is non-zero, tell the user: "Assessment couldn't complete. Check that the path exists and contains Claude Code configuration files."
|
||||
|
||||
If `--raw` was passed, treat the captured stderr as v5.0.0-shape verbatim text and present it as-is in a code block; skip the humanized rendering steps below.
|
||||
|
||||
### Step 3: Read and interpret results
|
||||
|
||||
Read the JSON output file using the Read tool. Extract:
|
||||
Use the Read tool on `/tmp/config-audit-posture.json`. Extract:
|
||||
|
||||
- `overallGrade`, `opportunityCount`
|
||||
- `areas[]` — each with `name`, `grade`, `score`, `findingCount`
|
||||
- `scannerEnvelope.scanners[].findings[]` — when surfacing individual findings, prefer the humanizer-provided fields: `userImpactCategory` (e.g., "Configuration mistake", "Wasted tokens"), `userActionLanguage` (e.g., "Fix this now", "Fix soon", "Optional cleanup"), and `relevanceContext` ("affects-everyone", "affects-this-machine-only", "test-fixture-no-impact"). These let you group and prioritize without hardcoded severity-to-prose mappings.
|
||||
|
||||
Also Read the captured stderr file — its body is the humanized scorecard (grade headline, area-score block, opportunity hint). You can present it verbatim or interleave its lines with the JSON-driven table.
|
||||
Also use the Read tool on `/tmp/config-audit-posture-stderr.txt` — its body is the humanized scorecard (grade headline, area-score block, opportunity hint). You can present it verbatim or interleave its lines with the JSON-driven table.
|
||||
|
||||
### Step 4: Present the scorecard
|
||||
|
||||
```markdown
|
||||
**Health: {overallGrade}** | {qualityAreaCount} areas scanned
|
||||
**Health: {overallGrade}** | (area count: take it from the humanized scorecard's
|
||||
"N areas reviewed" line — do NOT use `areas.length`, which counts Feature
|
||||
Coverage; the table below excludes it, so the two would disagree)
|
||||
|
||||
{Use the headline line from the humanized stderr scorecard — it carries grade-context prose already (e.g., " Health: A (97/100) — Healthy setup, only minor polish needed"). Do not re-derive an A/B/C/D prose table here; the humanizer owns that vocabulary.}
|
||||
|
||||
|
|
@ -93,19 +101,19 @@ Avoid hardcoded grade-to-prose ladders here — the humanized scorecard headline
|
|||
|
||||
Run drift comparison silently:
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <target-path> 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs "<target-path>" --output-file /tmp/config-audit-posture-drift.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read stdout output and append a "Configuration Drift" section showing what changed since the last baseline.
|
||||
Use the Read tool on `/tmp/config-audit-posture-drift.json` and append a "Configuration Drift" section showing what changed since the last baseline. Both scanners report to stderr in default mode, which `2>/dev/null` discards — the payload is the only readable output.
|
||||
|
||||
**If `--plugin-health` flag is present:**
|
||||
|
||||
Run plugin health scanner silently:
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs <target-path> 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs "<target-path>" --output-file /tmp/config-audit-posture-plh.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read stdout output and append a "Plugin Health" section.
|
||||
Use the Read tool on `/tmp/config-audit-posture-plh.json` and append a "Plugin Health" section, using its `plugins[]` rows for per-plugin grades.
|
||||
|
||||
**If both flags:** Use `scanners/lib/report-generator.mjs` to produce a unified markdown report.
|
||||
|
||||
|
|
@ -113,5 +121,9 @@ Read stdout output and append a "Plugin Health" section.
|
|||
|
||||
If a config-audit session exists, save results:
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs <target-path> --json --output-file ~/.claude/config-audit/sessions/<session-id>/posture.json 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs "<target-path>" --json --output-file ~/.claude/config-audit/sessions/<session-id>/posture.json >/dev/null 2>/dev/null
|
||||
```
|
||||
|
||||
This is a second scan on purpose: the session file stores the raw v5.0.0 shape,
|
||||
while step 2 wrote the humanized one. `>/dev/null` matters most here — `--json`
|
||||
sends the same envelope to stdout regardless of `--output-file`.
|
||||
|
|
|
|||
|
|
@ -10,6 +10,12 @@ model: sonnet
|
|||
|
||||
Restore configuration files from a previous backup. Without arguments, lists available backups. With a backup ID, restores files from that backup.
|
||||
|
||||
Every step below runs `scanners/rollback-cli.mjs`, which drives the rollback
|
||||
engine: it verifies each file's checksum before AND after writing it, resolves
|
||||
backups made under the pre-v2.2.0 root, and reports the files a restore cannot
|
||||
undo. Never restore by copying files back by hand — a copy performs none of
|
||||
those checks, and the result cannot honestly be reported as verified.
|
||||
|
||||
## Arguments
|
||||
|
||||
- `$ARGUMENTS` may contain a backup ID (format: `YYYYMMDD_HHMMSS`)
|
||||
|
|
@ -19,14 +25,18 @@ Restore configuration files from a previous backup. Without arguments, lists ava
|
|||
|
||||
### List mode (no argument)
|
||||
|
||||
Parse flags and list available backups from `~/.claude/config-audit/backups/`:
|
||||
|
||||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
ls -1 ~/.claude/config-audit/backups/
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/rollback-cli.mjs --list --output-file /tmp/config-audit-rollback-list.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Exit 0 = listed; 3 = the CLI could not do its job (show the stderr message).
|
||||
Read `/tmp/config-audit-rollback-list.json` and render one row per entry in
|
||||
`backups[]` — `{id}`, how many `{files}` it holds, and `{createdAt}`. An entry
|
||||
whose `{legacy}` is true was made under the pre-v2.2.0 backup root; say so, since
|
||||
its path differs from the one printed below.
|
||||
|
||||
```
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
Available Backups
|
||||
|
|
@ -40,12 +50,26 @@ ls -1 ~/.claude/config-audit/backups/
|
|||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
```
|
||||
|
||||
Use the Read tool on each backup's `manifest.yaml` (the list of changes captured at backup time) to extract the file list and timestamps.
|
||||
If `{count}` is 0, say there are no backups yet and stop — do not offer a restore.
|
||||
|
||||
### Restore mode (with backup ID)
|
||||
|
||||
1. Read the list of changes from `~/.claude/config-audit/backups/{backup-id}/manifest.yaml` using the Read tool
|
||||
2. Show files that will be restored — ask for confirmation:
|
||||
1. The `backups[]` entry for that ID (from the list payload above) carries the
|
||||
`files[]` this restore would write. Classify those `{originalPath}` values
|
||||
before showing them. A restore writes to the absolute path recorded at backup
|
||||
time, which may be machine-wide even when the backup was taken from a project
|
||||
— so the file list must be rendered as the absolute originals, never shortened
|
||||
to a repo-relative-looking form that implies the write stays local:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/write-scope-cli.mjs --target "<original-1>" --target "<original-2>" --repo "$PWD" --output-file /tmp/config-audit-rollback-scope.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Exit 0 = classified; 3 = argument error (show the stderr message). Read
|
||||
`/tmp/config-audit-rollback-scope.json`, render each distinct string in
|
||||
`disclosures[]` verbatim, then ask for confirmation.
|
||||
|
||||
When `requiresApproval` is false:
|
||||
```
|
||||
AskUserQuestion:
|
||||
question: "Restore 3 files from backup 20260403_163045?"
|
||||
|
|
@ -53,21 +77,54 @@ Use the Read tool on each backup's `manifest.yaml` (the list of changes captured
|
|||
- "Yes, restore"
|
||||
- "Cancel"
|
||||
```
|
||||
3. For each file in the list of changes:
|
||||
a. Read the backup file from `~/.claude/config-audit/backups/{backup-id}/files/{safeName}`
|
||||
b. Write to the original path
|
||||
c. Verify the checksum matches the recorded value in the list of changes
|
||||
4. Show result:
|
||||
|
||||
When `requiresApproval` is true, name the scope and put the safe option first:
|
||||
```
|
||||
Restored 3 files from backup 20260403_163045
|
||||
- .claude/settings.json (checksum verified)
|
||||
- hooks/hooks.json (checksum verified)
|
||||
- .claude/rules/typescript.md (checksum verified)
|
||||
AskUserQuestion:
|
||||
question: "This restores {K} of 3 files to locations outside this project. Restore all 3?"
|
||||
options:
|
||||
- "Cancel"
|
||||
- "Yes — restore, including outside this project"
|
||||
```
|
||||
5. **Report what rollback cannot undo.** A backup only holds files that already
|
||||
existed, so files the implement step CREATED survive the restore. If the
|
||||
manifest has a `created:` section (or `restoreBackup()` returns a non-empty
|
||||
`createdNotRemoved`), list those paths and say plainly that they remain:
|
||||
|
||||
2. Run the restore. Classifying is not approving: add `--approve-scope` only
|
||||
after the user has answered yes to the question above, and only then.
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/rollback-cli.mjs --restore "<backup-id>" --repo "$PWD" --output-file /tmp/config-audit-rollback-restore.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Append ` --approve-scope` to that command when the user approved a restore
|
||||
that leaves this project. To preview without writing anything, append
|
||||
` --dry-run` instead — a dry run reports what would happen and touches no file.
|
||||
|
||||
| Exit | Meaning |
|
||||
|------|---------|
|
||||
| 0 | every file restored |
|
||||
| 1 | nothing was written — the restore needs approval it was not given |
|
||||
| 2 | at least one file failed; read `failed[]` before saying anything else |
|
||||
| 3 | the CLI could not run (bad ID, unreadable manifest) — show the stderr message |
|
||||
|
||||
3. Read `/tmp/config-audit-rollback-restore.json` and report what the run
|
||||
actually did. Render one line per entry in `restored[]` and `failed[]`, each
|
||||
showing its own `{status}` from the payload — never a fixed verification
|
||||
phrase, because the outcome differs per file and only the payload knows it:
|
||||
|
||||
```
|
||||
Restored 2 of 3 files from backup 20260403_163045
|
||||
- /abs/path/.claude/settings.json — {status}
|
||||
- /abs/path/hooks/hooks.json — {status}
|
||||
- /abs/path/.claude/rules/typescript.md — {status}
|
||||
```
|
||||
|
||||
When `{requiresApproval}` is true and nothing was restored, the run was
|
||||
refused: list the `refused[]` paths, say plainly that no file was changed, and
|
||||
offer to re-run with approval.
|
||||
|
||||
4. **Report what rollback cannot undo.** A backup only holds files that already
|
||||
existed, so files the implement step CREATED survive the restore. When
|
||||
`createdNotRemoved` is non-empty, list those paths and say plainly that they
|
||||
remain:
|
||||
```
|
||||
Left in place — created by implement, no backup exists:
|
||||
- .claude/rules/post-quality.md
|
||||
|
|
@ -79,29 +136,24 @@ Use the Read tool on each backup's `manifest.yaml` (the list of changes captured
|
|||
|
||||
### Delete mode
|
||||
|
||||
If user says "delete" after listing, confirm and remove the backup directory.
|
||||
If the user says "delete" after listing, confirm, then:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/rollback-cli.mjs --delete "<backup-id>" --output-file /tmp/config-audit-rollback-delete.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Exit 0 = removed (`deleted` is true in the payload); 3 = no backup with that ID
|
||||
in either root — show the stderr message rather than reporting a deletion.
|
||||
|
||||
## Implementation
|
||||
|
||||
Use the backup and rollback libraries directly:
|
||||
```javascript
|
||||
import { listBackups, restoreBackup, deleteBackup } from '../scanners/rollback-engine.mjs';
|
||||
import { parseManifest, getBackupDir } from '../scanners/lib/backup.mjs';
|
||||
```
|
||||
`scanners/rollback-cli.mjs` is the only entry point. It reads
|
||||
`~/.claude/config-audit/backups` and falls back to the pre-v2.2.0
|
||||
`~/.config-audit/backups`, so a backup made before the move still resolves; the
|
||||
list payload flags those with `legacy: true`, and both roots are echoed in
|
||||
`meta` so a report can name the one it used.
|
||||
|
||||
Both read `~/.claude/config-audit/backups` and fall back to the pre-v2.2.0
|
||||
`~/.config-audit/backups`, so a backup made before the move still resolves;
|
||||
`listBackups()` flags those with `legacy: true`. Prefer this API over ad-hoc
|
||||
`cp` — it verifies the checksum before and after each write.
|
||||
|
||||
Or via Bash:
|
||||
```bash
|
||||
# List backups
|
||||
ls -1 ~/.claude/config-audit/backups/
|
||||
|
||||
# Read manifest
|
||||
cat ~/.claude/config-audit/backups/{id}/manifest.yaml
|
||||
|
||||
# Restore (copy back)
|
||||
cp ~/.claude/config-audit/backups/{id}/files/{safeName} {originalPath}
|
||||
```
|
||||
The same CLI creates backups (`--create --target <path> …`), which is how
|
||||
`/config-audit implement` records what it is about to change. Backup and restore
|
||||
therefore share one manifest format, owned by `scanners/lib/backup.mjs` — the
|
||||
format is never written or read by hand.
|
||||
|
|
|
|||
|
|
@ -37,8 +37,13 @@ When `--raw` is in `$ARGUMENTS`, render the raw `current_phase` field value verb
|
|||
```bash
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
ALL_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -qw -- "all"; then ALL_FLAG="all"; fi
|
||||
```
|
||||
|
||||
When `ALL_FLAG` is set, skip steps 2–4 and render the **List All Sessions**
|
||||
table below instead of a single session's status.
|
||||
|
||||
2. **Find active session**:
|
||||
```
|
||||
Glob: ~/.claude/config-audit/sessions/*/state.yaml
|
||||
|
|
@ -128,11 +133,13 @@ All config-audit sessions:
|
|||
| 20250120_160000 | implement | 2025-01-20 16:00 |
|
||||
```
|
||||
|
||||
## Resume Session
|
||||
## Resuming a session
|
||||
|
||||
If multiple sessions exist:
|
||||
```
|
||||
/config-audit resume {session-id}
|
||||
```
|
||||
There is no `resume` command. Sessions are selected by recency: every
|
||||
session-aware command globs `~/.claude/config-audit/sessions/*/state.yaml` and
|
||||
takes the most recently modified one.
|
||||
|
||||
Sets that session as active and continues from last phase.
|
||||
To continue an older session, run its next phase directly — `/config-audit plan`,
|
||||
`/config-audit implement`, and so on read `next_phase` from the state file. If
|
||||
the wrong session keeps winning, delete the stale ones with
|
||||
`/config-audit cleanup`.
|
||||
|
|
|
|||
|
|
@ -40,12 +40,25 @@ Tell the user: **"Analysing token hotspots for `<path>`..."**
|
|||
Default mode (no `--json`, no `--raw`) emits a humanized JSON envelope: each finding carries `userImpactCategory`, `userActionLanguage`, and `relevanceContext` in addition to the v5.0.0 fields. Pass `--raw` through verbatim if the user requested it.
|
||||
|
||||
```bash
|
||||
TMPFILE="/tmp/config-audit-tokens-$$.json"
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/token-hotspots-cli.mjs <path> --output-file "$TMPFILE" [--global] [--no-exclude-cache] $RAW_FLAG 2>/dev/null; echo $?
|
||||
# Set each to the flag itself when the user asked for it, otherwise leave empty.
|
||||
# A placeholder in square brackets does not start with a dash, so the CLI's arg
|
||||
# loop would take it as the TARGET PATH instead of a flag.
|
||||
GLOBAL_FLAG="" # --global
|
||||
CACHE_FLAG="" # --no-exclude-cache
|
||||
JSON_FLAG="" # --json
|
||||
TELEMETRY_FLAG="" # --with-telemetry-recipe
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/token-hotspots-cli.mjs "<path>" --output-file /tmp/config-audit-tokens.json $GLOBAL_FLAG $CACHE_FLAG $JSON_FLAG $TELEMETRY_FLAG $RAW_FLAG >/dev/null 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
`--json` and `--with-telemetry-recipe` must be threaded here, not just
|
||||
documented: the CLI is what turns them on. `--json`/`--raw` make the payload
|
||||
byte-stable v5.0.0 (the humanizer is skipped, `token-hotspots-cli.mjs:127`), and
|
||||
`--with-telemetry-recipe` is what adds `telemetry_recipe_path`. `>/dev/null` is
|
||||
required because those two modes also print the payload to stdout even with
|
||||
`--output-file` set (`token-hotspots-cli.mjs:137`).
|
||||
|
||||
**Exit code handling:**
|
||||
- `0` → continue
|
||||
- `3` → tell user: "Couldn't analyse tokens. Check that the path exists and is a directory." Stop.
|
||||
|
|
@ -53,14 +66,14 @@ node ${CLAUDE_PLUGIN_ROOT}/scanners/token-hotspots-cli.mjs <path> --output-file
|
|||
### Step 3: If `--json` was requested, cat the file and stop
|
||||
|
||||
```bash
|
||||
cat "$TMPFILE"
|
||||
cat /tmp/config-audit-tokens.json
|
||||
```
|
||||
|
||||
Do NOT render tables in JSON mode.
|
||||
|
||||
### Step 4: Read JSON and render
|
||||
|
||||
Use the Read tool on `$TMPFILE`. Extract:
|
||||
Use the Read tool on `/tmp/config-audit-tokens.json`. Extract:
|
||||
|
||||
- `total_estimated_tokens` — top-line number
|
||||
- `hotspots[]` — top 10 ranked sources; each carries a **load pattern** (`loadPattern` ∈ always / on-demand / external, plus `survivesCompaction` / `derivationConfidence`)
|
||||
|
|
@ -109,7 +122,7 @@ _Estimates assume ~4 chars/token (Claude ballpark). Real token count varies ±20
|
|||
### Step 5: Cleanup and next steps
|
||||
|
||||
```bash
|
||||
rm -f "$TMPFILE"
|
||||
rm -f /tmp/config-audit-tokens.json
|
||||
```
|
||||
|
||||
```markdown
|
||||
|
|
|
|||
|
|
@ -33,10 +33,14 @@ Split `$ARGUMENTS` into a path and flags. Path is the first non-flag argument. D
|
|||
Tell the user: **"Reading active configuration for `<path>`..."**
|
||||
|
||||
```bash
|
||||
TMPFILE="/tmp/ca-whats-active-$$.json"
|
||||
RAW_FLAG=""
|
||||
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/whats-active.mjs <path> --output-file "$TMPFILE" [--verbose] [--suggest-disables] $RAW_FLAG 2>/dev/null; echo $?
|
||||
# Set each to the flag itself when the user asked for it, otherwise leave empty.
|
||||
# A placeholder in square brackets does not start with a dash, so the scanner's
|
||||
# arg loop would take it as the TARGET PATH instead of a flag.
|
||||
VERBOSE_FLAG="" # --verbose
|
||||
SUGGEST_FLAG="" # --suggest-disables
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/whats-active.mjs "<path>" --output-file /tmp/config-audit-whats-active.json $VERBOSE_FLAG $SUGGEST_FLAG $RAW_FLAG >/dev/null 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
**Exit code handling:**
|
||||
|
|
@ -46,14 +50,14 @@ node ${CLAUDE_PLUGIN_ROOT}/scanners/whats-active.mjs <path> --output-file "$TMPF
|
|||
### Step 3: If `--json` was requested, cat the file and stop
|
||||
|
||||
```bash
|
||||
cat "$TMPFILE"
|
||||
cat /tmp/config-audit-whats-active.json
|
||||
```
|
||||
|
||||
Do NOT render tables in JSON mode.
|
||||
|
||||
### Step 4: Read JSON and render
|
||||
|
||||
Use the Read tool on `$TMPFILE`. Extract:
|
||||
Use the Read tool on `/tmp/config-audit-whats-active.json`. Extract:
|
||||
|
||||
- `meta.repoPath`, `meta.durationMs`, `meta.gitRoot`, `meta.projectKey`
|
||||
- `totals.estimatedTokens.grandTotal` (and subtotals)
|
||||
|
|
@ -90,7 +94,17 @@ Render as markdown:
|
|||
|-------|--------|--------|
|
||||
| {name} | {source}{if pluginName: ` (${pluginName})`} | ~{estimatedTokens} |
|
||||
|
||||
### MCP Servers ({mcpServers.length}, ~{mcpServers subtotal} tokens)
|
||||
### Agents ({agents.length}, ~{agents subtotal} tokens)
|
||||
|
||||
| Agent | Source | Model | Effort | Tokens |
|
||||
|-------|--------|-------|--------|--------|
|
||||
| {name} | {source}{if pluginName: ` (${pluginName})`} | {model or "inherit"} | {effort or "session default"} | ~{estimatedTokens} |
|
||||
|
||||
Skip this section entirely when `agents` is empty. `model: null` is rendered as
|
||||
*inherit* and `effort: null` as *session default* — both are the documented
|
||||
defaults, so a blank cell would read as missing data rather than as the choice
|
||||
it is. If no agent pins either column, say so in one sentence: every delegated
|
||||
task then costs what the session costs.
|
||||
|
||||
| Server | Source | Status | Command |
|
||||
|--------|--------|--------|---------|
|
||||
|
|
@ -149,7 +163,7 @@ Do NOT suggest items you can't name concrete redundancy for. If you can't find 3
|
|||
### Step 7: Cleanup and next steps
|
||||
|
||||
```bash
|
||||
rm -f "$TMPFILE"
|
||||
rm -f /tmp/config-audit-whats-active.json
|
||||
```
|
||||
|
||||
```markdown
|
||||
|
|
|
|||
244
docs/q-audit-prose-invariants.md
Normal file
244
docs/q-audit-prose-invariants.md
Normal file
|
|
@ -0,0 +1,244 @@
|
|||
# Q_AUDIT — the prose-invariant sweep (v6 quality plan §3)
|
||||
|
||||
**Session #68, 2026-08-12, Fable 5/xhigh (no advisor — every number below is
|
||||
command-produced; the commands are in the appendix).** Q1 closed the write gate
|
||||
in code, Q2 closed the argv contract in code. This sweep asked what ELSE is
|
||||
enforced only by prose across the three surfaces the plan names: **21** command
|
||||
templates, **7** agent prompts, **4** `.claude/rules/` files (4,950 lines), against
|
||||
the **17** guard files that already exist under `tests/commands/` + `tests/agents/`.
|
||||
|
||||
Output is a rated list, not code. Nothing here was fixed in this session.
|
||||
|
||||
## 1. The taxonomy — what counts as a prose invariant
|
||||
|
||||
The open decision this session owned. A statement counts when all three hold:
|
||||
|
||||
1. **Something else relies on it** — another component (code, hook, agent,
|
||||
downstream command) behaves correctly only while the statement is true.
|
||||
2. **Its violation is silent** — nothing fails loudly when it stops holding.
|
||||
3. **No code or test detects the violation.**
|
||||
|
||||
What deliberately does NOT count (the negative list matters — Q1's own lesson is
|
||||
that a gate firing on legitimate writes gets switched off):
|
||||
|
||||
- **Judgment rubrics** given to agents (analyzer's 100-point CLAUDE.md rubric,
|
||||
planner's risk formula) — prose is the *medium* of a judgment task, not a bug.
|
||||
- **Narration/UX rules** (ux-rules.md) — degraded output, self-correcting.
|
||||
- **Output budgets** ("MUST NOT exceed 300 lines") — worst case is a long report.
|
||||
- **Harness facts** ("the Write tool requires a prior Read") — enforced upstream.
|
||||
|
||||
Marker-grep is a non-detector here: only **18** MUST/NEVER/ALWAYS-class markers
|
||||
exist across all 28 command+agent files. The invariants are procedural steps
|
||||
whose omission is silent, not shouted rules. They are found by reading, which is
|
||||
why this was a session, not a script.
|
||||
|
||||
## 2. The class finding — the THIRD instance
|
||||
|
||||
Q1 was *gate-in-prose* (policy paraphrased in five templates). Q2 was
|
||||
*argv-in-prose* (caller contract unchecked, 54 pairs). The third instance this
|
||||
sweep asked for is:
|
||||
|
||||
**Class 3 — data-contract-in-prose: prose WRITES what code READS.**
|
||||
|
||||
Q2's mirror image. Templates instruct the model to hand-build files —
|
||||
`manifest.yaml`, `state.yaml`, `scope.yaml`, register edits — that engines,
|
||||
hooks, and later commands then parse. The writer side is a prose schema; the
|
||||
reader side either trusts it or has quietly learned one measured variant of it.
|
||||
The class has already bitten once: `parseManifest` grew its second format branch
|
||||
*after* implement-produced backups made `restoreBackup` "a success-shaped no-op"
|
||||
(comment in `scanners/lib/backup.mjs:172`).
|
||||
|
||||
Two further classes surfaced (the sweep found things it wasn't looking for,
|
||||
again): **Class 4 — contracts an agent cannot honor** (instructions colliding
|
||||
with harness behavior or the agent's own declared tools), and **Class 5 —
|
||||
knowledge tables duplicated between prose and code**.
|
||||
|
||||
## 3. The rated list
|
||||
|
||||
Rated by what breaks if the invariant silently stops holding. R1/R2 outrank
|
||||
everything because they sit on the *recovery* path — the surface that runs
|
||||
exactly when the user is already in trouble, and the least-exercised one.
|
||||
|
||||
### R1 — the restore flow is model-executed prose; the code engine has no CLI entry
|
||||
**CLOSED in #74** — `scanners/rollback-cli.mjs`. The measurement below is kept as written.
|
||||
**Where:** `commands/rollback.md` §Implementation; `scanners/rollback-engine.mjs`.
|
||||
**Measured:** 16 scanner CLIs carry a `process.argv` entry — `rollback-engine.mjs`
|
||||
is not one of them. The template's "Implementation" shows an ESM `import` block a
|
||||
command template cannot execute, then offers ad-hoc `cp` as the runnable
|
||||
alternative. The engine's `restoreBackup()` verifies checksums before AND after
|
||||
each write and returns `createdNotRemoved` — none of it reachable from the
|
||||
command without `node -e`. The success template pre-renders "`(checksum
|
||||
verified)`" — a claim the runnable path never establishes.
|
||||
**What breaks:** the recovery path for every other write the plugin makes. A
|
||||
half-restore or a stale-backup restore lands on user config at the worst
|
||||
possible moment, reported as verified.
|
||||
**Also unguardable as-is:** `command-cli-contract.test.mjs` probes CLIs; with no
|
||||
CLI here, the whole Q2 guard class is structurally blind to this command.
|
||||
**Guard shape:** a thin `rollback-cli.mjs` over `listBackups`/`restoreBackup`/
|
||||
`deleteBackup`; template calls it like every other command; the contract test
|
||||
then covers it for free. The "(checksum verified)" line becomes payload-driven.
|
||||
|
||||
### R2 — implement's backup manifest is hand-built prose; the parser knows one frozen sample of it
|
||||
**CLOSED in #74** — the real fix, not the minimum: `implement.md` Step 3 calls `rollback-cli.mjs --create`, so the prose format has no author left. The measurement below is kept as written.
|
||||
**Where:** `commands/implement.md` Step 3 (mkdir/cp + hand-written
|
||||
`manifest.yaml` with sha256 lines); `scanners/lib/backup.mjs` `parseManifest`;
|
||||
`tests/scanners/rollback-paths.test.mjs:157-186`.
|
||||
**Measured:** the fix pipeline's backups go through code (`fix-engine.mjs:10`
|
||||
imports `createBackup`); the implement pipeline's backup is template prose —
|
||||
two copies of the backup policy, one per pipeline. `parseManifest`'s
|
||||
implement-format branch exists because the seam already failed silently once.
|
||||
The test fixture pinning that format is **hand-written**, not derived from the
|
||||
template's own example block — the exact #63 defect shape ("a hand-typed call is
|
||||
a path no user takes").
|
||||
**What breaks:** implement.md's example drifts (a key renamed, quoting added) →
|
||||
`parseManifest` finds 0 files → every implement backup is unrestorable while
|
||||
`rollback` reports success. Second occurrence of a failure that already happened.
|
||||
**Guard shape:** minimum — derive the parser fixture from `implement.md`'s own
|
||||
fenced YAML (the Q2 extractor trick pointed at an example block). Real fix —
|
||||
implement's Step 3 calls the same `createBackup` code path fix already uses
|
||||
(via the R1 CLI), and the prose format dies entirely.
|
||||
|
||||
### R3 — optimize + feature-gap still instruct agents to write reports the harness blocks
|
||||
**Where:** `commands/optimize.md:121-128`, `commands/feature-gap.md:123`;
|
||||
agents `optimization-lens-agent.md` §Output, `feature-gap-agent.md` §Output.
|
||||
**Measured:** both templates tell the agent to write `*-report.md` to the
|
||||
session dir, then Read that file. The harness note measured on `analyze`
|
||||
(M-BUG-18) blocks agent writes of exactly the report/findings file class;
|
||||
`analyze` was converted to orchestrator-writes, these two arms were left open
|
||||
(tracked in STATE as the open M-BUG-18 class — this rating is its severity call).
|
||||
**What breaks:** the Read step fails or the model improvises a rescue; the
|
||||
command's documented artifact (`optimization-lens-report.md`) may never exist.
|
||||
User-visible flow breakage, no data corruption.
|
||||
**Guard shape:** the `analyze` pattern, already proven: agent returns inline,
|
||||
command persists; `analyze-report-persistence.test.mjs` is the template to copy.
|
||||
|
||||
### R4 — verifier-agent contradicts itself, and its "Read-Only Guarantee" is unenforced
|
||||
**Where:** `agents/verifier-agent.md` — §Output Format says "Append to:
|
||||
implementation-log.md"; §Read-Only Guarantee says "only uses Read, Glob, Grep /
|
||||
never modifies any files"; frontmatter `tools:` lists no write tool.
|
||||
**Measured:** `implement.md` Step 5 was already fixed to return-inline and
|
||||
append orchestrator-side — so the agent's system prompt and the spawn prompt now
|
||||
give OPPOSITE instructions to the same agent. Harness enforcement of `tools:`
|
||||
is measured absent (memory: verifier/implement-log writes went through live).
|
||||
**What breaks:** which instruction wins is nondeterministic; if the agent
|
||||
improvises a write to satisfy its own §Output Format, a full-file Write on the
|
||||
shared log clobbers parallel implementer entries — the precise defect
|
||||
`implement-log-append.test.mjs` exists to prevent, entering through the file
|
||||
that test does not read.
|
||||
**Guard shape:** rewrite verifier-agent's Output section to return-inline (one
|
||||
file), and extend `implement-log-append`/`agent-prompt-shape` to assert no agent
|
||||
prompt instructs appending to the shared log. Cheap.
|
||||
|
||||
### R5 — session state (`state.yaml`, `scope.yaml`) is a model-written machine contract with no schema anywhere
|
||||
**Where:** every phase template ("Write scope.yaml and state.yaml", "append —
|
||||
never replace — completed_phases"), `.claude/rules/state-management.md`,
|
||||
readers in `hooks/scripts/session-start.mjs` + `stop-session-reminder.mjs` +
|
||||
every session-aware command.
|
||||
**Measured:** the phase vocabulary (`discover`…`verify`) appears as a shared
|
||||
constant in **zero** code files — it lives only in prose copies (status.md's
|
||||
table, state-management.md, each template). Hooks parse with a line-grep
|
||||
(`parseYamlValue`) and print whatever they find. The existing guard
|
||||
(`command-shell-state-shape`: "phase commands name all four fields") checks the
|
||||
template *text*, not the written *file*.
|
||||
**What breaks:** resume-by-recency picks wrong sessions, status misnarrates,
|
||||
session-start reminders go quiet — degradation, not corruption, but it erodes
|
||||
exactly the "can resume if interrupted" promise the rule exists for.
|
||||
**Guard shape:** either a state-write CLI (heavy) or a defensive reader: a lib
|
||||
that validates phase tokens + required fields and *flags* malformed state, used
|
||||
by hooks and dogfooded in a test. The plugin flags drift in everyone else's
|
||||
config; its own session state deserves the same reader.
|
||||
|
||||
### R6 — both "Required Frontmatter" contracts are unguarded (currently compliant)
|
||||
**Where:** `.claude/rules/agent-development.md`, `.claude/rules/command-development.md`.
|
||||
**Measured:** no test outside fixtures matches `allowed-tools`;
|
||||
`agent-prompt-shape` asserts only `name:` on **3 of 7** agents. Measured today:
|
||||
7/7 agent frontmatters match the CLAUDE.md table; duplicate colors: **0**. So —
|
||||
compliant, unwatched. Every MUST in those two rules is enforced by nothing.
|
||||
**What breaks:** a new agent/command ships with missing `allowed-tools` or a
|
||||
duplicate color; nothing fails; the rules files become fiction one file at a
|
||||
time (the exemption-table lesson from Q1: what nothing declares, everyone
|
||||
re-answers by reading).
|
||||
**Guard shape:** near-free shape test walking `agents/*.md` + `commands/*.md`
|
||||
asserting the two rules' required keys, name conventions, color uniqueness.
|
||||
Note the irony budget: `plugin-health-scanner` already audits *other* plugins'
|
||||
frontmatter — pointing it at its own repo in a test is the dogfood version.
|
||||
|
||||
### R7 — secret detection exists only as agent prose, in a domain STATE has parked elsewhere
|
||||
**Where:** `agents/scanner-agent.md` §Secret Detection Patterns (xoxb/sk-/ghp_
|
||||
regexes); `agents/verifier-agent.md` Check 7 ("Secrets Scan ✓").
|
||||
**Measured:** `xoxb`/`ghp_` appear in **zero** files under `scanners/`;
|
||||
`mcp-config-validator.mjs` contains the string "secret" **zero** times. The
|
||||
deterministic pipeline has no secret scanning at all; the agent path claims it
|
||||
in prose, and the verifier's report template renders "Secrets Scan ✓ Pass" as a
|
||||
table row regardless. STATE parks secrets as the `llm-security` plugin's domain.
|
||||
**What breaks:** a user reads "Secrets Scan ✓" as an executed check. The lie is
|
||||
in the reporting, not in a missing feature — the feature is deliberately owned
|
||||
elsewhere.
|
||||
**Guard shape:** this is a *removal* candidate, not a gate (measurement can
|
||||
decline the feature): strip the prose secret patterns + the verifier's Check 7,
|
||||
say "secrets: out of scope, see llm-security" where the row used to be. If the
|
||||
capability is ever wanted deterministically, it starts life as a scanner with a
|
||||
finding code, not as agent prose.
|
||||
|
||||
### R8 — knowledge tables duplicated between prose and code
|
||||
**Where/measured:** managed-path table — **three** copies (scanner-agent prose +
|
||||
`file-discovery.mjs` + `active-config-reader.mjs`). Precedence — analyzer prose
|
||||
("global beats managed (user preference)") vs `conflict-detector.mjs:138`
|
||||
("local > project > user"): different vocabularies for the same claim, no link.
|
||||
Optimization-lens agent's mechanism table restates register entries
|
||||
(BP-MECH-001/002/004, BP-SUB-001) that live as data in
|
||||
`knowledge/best-practices.json`.
|
||||
**What breaks:** slow divergence — an agent narrates precedence or hierarchy the
|
||||
deterministic scanners no longer implement. Confusing, not corrupting.
|
||||
**Guard shape:** two-copies rule applies but *measure first* (#67): the two code
|
||||
copies may legitimately differ; the prose copies should cite the code as owner
|
||||
("hierarchy per `file-discovery.mjs`") rather than restate values.
|
||||
|
||||
### R9 — knowledge-refresh applies approved writes by model edit, validated only afterwards
|
||||
**Where:** `commands/knowledge-refresh.md` Step 6 (model `Edit` of
|
||||
`best-practices.json`, then run the schema test and revert on failure).
|
||||
**Measured:** already tracked open in STATE ("knowledge-refresh skrive-CLI");
|
||||
path anchoring is guarded (`knowledge-refresh-write-target.test.mjs`), the write
|
||||
itself is not. The post-hoc validation step is real mitigation — this ranks
|
||||
last *because* the failure is loud (a failing schema test in the same flow).
|
||||
**Guard shape:** the campaign pattern, which this command's own sibling already
|
||||
implements: every mutation a subcommand of a write-CLI. `campaign.md` is the
|
||||
in-repo proof that "human-approved writes" and "CLI-executed writes" compose.
|
||||
|
||||
## 4. What this changes in the plan
|
||||
|
||||
- **Q3 (severity axis)** should carry R1/R2 as its first two rows — recovery-path
|
||||
items never live in a backlog paragraph (plan §2 property 2).
|
||||
- **Q4 (v6.0.0 release)** is NOT blocked by this list (the release gate is Q1 +
|
||||
green suite); but R1+R2 are the strongest candidates for the first post-v6
|
||||
chunk, as one chunk: a rollback/backup CLI closes both, and converts R2's
|
||||
guard from "pin the prose format" to "delete the prose format".
|
||||
- **R4 + R6** are lunch-sized; they can ride along with any adjacent session the
|
||||
way M-BUG fixes have.
|
||||
- **R7** is an operator decision (removing a claimed capability): propose, don't do.
|
||||
|
||||
## Appendix — measurement log
|
||||
|
||||
Every number above, and the command that produced it (run at `30c78ae`):
|
||||
|
||||
| # | Number | Command |
|
||||
|---|--------|---------|
|
||||
| 1 | 21 / 7 / 4 files | `ls commands/*.md \| wc -l` etc. |
|
||||
| 2 | 4,950 lines | `wc -l commands/*.md agents/*.md .claude/rules/*.md` |
|
||||
| 3 | 17 guard files | `find tests/commands tests/agents -name '*.test.mjs' \| wc -l` |
|
||||
| 4 | 18 markers | `grep -cE '\b(MUST\|NEVER\|ALWAYS\|DO NOT\|…)\b' commands/*.md agents/*.md` |
|
||||
| 5 | 16 CLIs, rollback-engine absent | `grep -ln "process.argv" scanners/*.mjs` |
|
||||
| 6 | parseManifest dual-format + its cause | `sed -n '140,220p' scanners/lib/backup.mjs` (comment at :172) |
|
||||
| 7 | hand-written fixture | `grep -n "sha256" tests/scanners/rollback-paths.test.mjs` (:159-186) |
|
||||
| 8 | fix uses code backup | `grep -n "backup" scanners/fix-engine.mjs` (:10) |
|
||||
| 9 | 0 secret patterns in code | `grep -rln "xoxb\|ghp_" scanners/` (empty); `grep -n "secret" scanners/mcp-config-validator.mjs` (empty) |
|
||||
| 10 | 3 copies managed-path table | `grep -rln "Library/Application Support" scanners/` (2) + scanner-agent prose (1) |
|
||||
| 11 | 0 phase-vocabulary constants | `grep -rn "'discover'" scanners/lib/*.mjs hooks/scripts/*.mjs` (empty) |
|
||||
| 12 | 3/7 agents in shape guard | `AGENT_FILES` array read in `tests/agents/agent-prompt-shape.test.mjs` |
|
||||
| 13 | 0 frontmatter guards | `grep -rln "allowed-tools" tests/` (fixtures + yaml-parser only) |
|
||||
| 14 | 0 duplicate colors | `grep -h "^color:" agents/*.md \| sort \| uniq -d \| wc -l` |
|
||||
| 15 | verifier self-contradiction | Read `agents/verifier-agent.md` (:143 append vs :242 read-only) |
|
||||
|
||||
Not covered by this sweep (deliberate): `knowledge/*.md` content freshness
|
||||
(knowledge-refresh's domain), README/plugin.json surface (repo-standard's
|
||||
domain), the deterministic scanners themselves (Q1/Q2 territory, already coded).
|
||||
|
|
@ -17,7 +17,7 @@ Scanner CLI: `node scanners/scan-orchestrator.mjs <path> [--global] [--full-mach
|
|||
| `mcp-config-validator.mjs` | MCP | Server types, env vars, unknown fields |
|
||||
| `import-resolver.mjs` | IMP | Broken @imports, circular refs, deep chains, tilde paths |
|
||||
| `conflict-detector.mjs` | CNF | Settings conflicts, permission contradictions, hook duplicates |
|
||||
| `feature-gap-scanner.mjs` | GAP | 25 feature checks across 4 tiers — shown as opportunities, not grades |
|
||||
| `feature-gap-scanner.mjs` | GAP | 24 feature checks across 4 tiers — shown as opportunities, not grades |
|
||||
| `token-hotspots.mjs` | TOK | Cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascade, bloated SKILL.md descriptions, MCP tool-schema budget, MCP tool-schema deferral (CA-TOK-006), stale plugin-cache disk-cleanup (prompt-cache patterns) |
|
||||
| `cache-prefix-scanner.mjs` | CPS | Volatile content in lines 31–150 of CLAUDE.md cascade (beyond Pattern A's top-30 window); plus volatile content inside `@import`-ed files (v5.10 B6, one hop) |
|
||||
| `disabled-in-schema-scanner.mjs` | DIS | Dead/ineffective permission entries (low). (1) Tools in BOTH `permissions.deny` AND `permissions.allow` — deny wins; dominance is param-aware and treats the `Tool(*)` deny-all glob as equivalent to a bare deny (covers a bare allow). (2) Unanchored allow wildcards (`*`, `B*`, `mcp__*`) that Claude Code silently skips — CC accepts allow globs only after a literal glob-free `mcp__<server>__` prefix. Predicates shared with CNF live in `lib/permission-rules.mjs` |
|
||||
|
|
@ -44,6 +44,8 @@ Scanner CLI: `node scanners/scan-orchestrator.mjs <path> [--global] [--full-mach
|
|||
| `active-config-reader.mjs` | Read-only inventory: readActiveConfig(), detectGitRoot(), walkClaudeMdCascade(), readClaudeJsonProjectSlice() (longest-prefix match), enumeratePlugins(), enumerateSkills(), readActiveHooks(), readActiveMcpServers() (with cache → package.json tool-count fallback), estimateTokens() (v5: `'mcp'` kind = 500 + toolCount × 200) |
|
||||
| `tokenizer-api.mjs` | Anthropic `count_tokens` wrapper for `--accurate-tokens` (v5 N5); 5s AbortController timeout, exponential 429 backoff, key masking |
|
||||
| `humanizer.mjs` | Plain-language output translator (v5.1.0): `humanizeFinding`, `humanizeFindings`, `humanizeEnvelope`, `computeRelevanceContext`. Pure functions; never mutate inputs. Adds `userImpactCategory`, `userActionLanguage`, `relevanceContext` fields and replaces title/description/recommendation when a translation exists. Bypassed by `--raw` and `--json` paths. |
|
||||
| `cli-args.mjs` | Argv precondition shared by the CLIs: `findArgError(args, spec)` / `requireValidArgs(args, spec)`. Rejects an unknown flag, and a value-taking flag whose next token is missing or is itself a flag — exit 3, never a verdict. Runs BEFORE each CLI's own parse loop, so valid argv reaches the existing parser unchanged (see Implementation notes → arg-sluk) |
|
||||
| `require-target-dir.mjs` | Target-path precondition: a scan root that does not exist, or is not a directory, is exit 3 rather than a graded verdict (#56) |
|
||||
| `humanizer-data.mjs` | TRANSLATIONS table for 16 scanner prefixes (CML/SET/HKV/RUL/MCP/IMP/CNF/COL/TOK/CPS/DIS/GAP/PLH/SKL/OST/OPT). Three-step lookup: exact title → regex pattern → `_default` → fall through to original |
|
||||
|
||||
## Action Engines (`scanners/`)
|
||||
|
|
@ -219,6 +221,46 @@ returns ≥1 chatty hook — surfaces the documented **filter-before-Claude-read
|
|||
grep ERROR and return only matches instead of a 10,000-line log). No chatty hook → silent (opportunity,
|
||||
not noise — same contract as the cliOverMcp / bundledSkills levers).
|
||||
|
||||
### feature-gap — agent model/effort routing lever (v5.14 C4, `CA-GAP-028`)
|
||||
|
||||
`agentModelRoutingLeverFinding` fires only when the target has **authored** subagents (the same
|
||||
`isAuthoredConfig` set the presence checks use, so plugin-bundled and fixture agents cannot make a
|
||||
machine look routed — M-BUG-13) and **not one of them** names `model:` or `effort:`. Cites
|
||||
`BP-MODEL-001` (a subagent's `model` defaults to `inherit`, so omitting it is a choice to pay the
|
||||
session's rate) and `BP-MODEL-002` (effort is a separate axis with its own frontmatter field).
|
||||
|
||||
**Why a lever and not a 25th dimension — decided by measurement, not taste.** A dimension is always
|
||||
evaluated, so "no agents at all" would have to read as *present*, and present weight feeds the
|
||||
utilization score. Measured on `tests/fixtures/marketplace-medium` (hermetic HOME) before the change:
|
||||
`utilization.score` 44, `segment` "Developing", where the "Competent" boundary is 45. As a t3
|
||||
dimension the denominators move 41→42 and the vacuous present pushes 18/41→19/42 = **45** — flipping
|
||||
`segment` in the frozen `v5.0.0/posture.json`, which `strip-retired-gap.mjs` does **not** mask (it
|
||||
drops only `utilization.score`/`overhang` and `feature_coverage.score`). A lever leaves every
|
||||
denominator alone and cannot move a score it never enters. The general rule now lives in CLAUDE.md
|
||||
(*GAP dimensions vs. levers*).
|
||||
|
||||
**One check across both axes, not one per axis.** It fires only when *neither* axis is used anywhere,
|
||||
so a deliberate everything-on-one-model policy stays silent. The cost is recall: a config that pins
|
||||
`model:` everywhere but never `effort:` gets no nudge. That is the v1 boundary, chosen for precision.
|
||||
|
||||
**`model: inherit` is not routing** — found by dogfooding, where installed agents write it out
|
||||
explicitly. `inherit` is the documented default, so spelling it out changes nothing about what the
|
||||
agent costs; counting it as a pin would let a config opt out of the opportunity without changing
|
||||
anything real. Effort has no documented sentinel of this kind, so it has no counterpart rule.
|
||||
|
||||
**Two silences that must not be conflated.** "No authored agents" (owned by dimension `t2_6`,
|
||||
*No custom subagents*) and "the only agents on disk are plugin-bundled" produce the same quiet
|
||||
output for different reasons. `tests/scanners/gap-agent-model-routing.test.mjs` P5 pins the second
|
||||
one specifically — it asserts the agent file *was* discovered before asserting silence, so the arm
|
||||
cannot pass for P4's reason.
|
||||
|
||||
**Humanizer coverage is now a blanket invariant.** The old guard asserted `TRANSLATIONS.GAP.static`
|
||||
keys *equal* `GAP_CHECKS` titles, which forbade humanizing any lever — so all three existing levers
|
||||
fell through to the generic GAP `_default` ("You have a feature opportunity worth a look"), wrong for
|
||||
a budget lever. The guard now requires a static entry for **every title GAP can emit** (dimensions ∪
|
||||
levers), and it was seen red against those three before the four entries were written. `TITLE_TO_ID`
|
||||
keeps strict equality with `GAP_CHECKS`: levers are not dimensions and must stay out of scoring.
|
||||
|
||||
### cache-prefix-scanner — @import extension (v5.10 B6)
|
||||
|
||||
CPS originally scanned only the files discovery classifies as `claude-md`. But a CLAUDE.md can pull
|
||||
|
|
@ -397,6 +439,43 @@ claimed CC "suggests the parent directory when an entry points at a file"; that
|
|||
primary-source-verified conditions. `skills` is deliberately *not* in `SHADOWING_PATH_FIELDS`
|
||||
(it adds to the default scan, never shadows).
|
||||
|
||||
### PLH scanner — `scanDetailed`, `--output-file`, and the marketplace.json exemption (økt #46)
|
||||
|
||||
`scan()` returns the **frozen v5.0.0 envelope** (`scanner, status, files_scanned, duration_ms,
|
||||
findings, counts`) and nothing else — `--raw`/`--json` print it verbatim and are snapshot-gated. Two
|
||||
things the `/config-audit plugin-health` report requires therefore cannot live there: one row per
|
||||
plugin (the `| Plugin | Grade | Commands | Agents |` table) and the cross-plugin/per-plugin split.
|
||||
Both were computed inside `scan()` and discarded at the return: `pluginResults` never escaped, and
|
||||
the only grade code — `formatPluginHealthReport` — had no caller anywhere in the repo.
|
||||
|
||||
`scanDetailed(targetPath)` is the seam. It returns `{ result, plugins, crossPluginFindings }`;
|
||||
`scan()` is now `(await scanDetailed(p)).result`, so the byte-stable envelope is unchanged by
|
||||
construction. `plugins[]` carries `name, declaredName, path, commandCount, agentCount, findingCount`
|
||||
plus `score`/`grade` from the shared `pluginGrade(issueCount)` helper (which
|
||||
`formatPluginHealthReport` now also calls, so the formula has exactly one home).
|
||||
|
||||
Cross-plugin findings are identified **positionally**, not by predicate: `crossPluginStart =
|
||||
allFindings.length` is taken immediately before the namespace/command-name sections, and the tail is
|
||||
sliced off at the end. A predicate would have to key on `category: 'plugin-hygiene'`, which the
|
||||
per-plugin shadow and skills findings share. The marker (`crossPlugin: true`) is stamped only on the
|
||||
**humanized copies** in the `--output-file` payload — never inside `finding()`, which would add a key
|
||||
to the frozen envelope.
|
||||
|
||||
`--output-file` follows the `drift-cli` contract: humanized payload in default mode, stdout
|
||||
untouched. This matters because default mode writes its report to **stderr**, and ux-rules rule 2
|
||||
requires the command to run under `2>/dev/null` — before this, `commands/plugin-health.md` (and both
|
||||
optional scanner calls in `commands/posture.md`) captured zero bytes. Argument parsing uses the same
|
||||
`BOOL_FLAGS`/`VALUE_FLAGS` + unknown-flag-throws shape as `drift-cli`/`fix-cli` (M-BUG-21, third
|
||||
arm); here the swallowed-flag failure mode was *worse than an error* — scanning the dropped flag's
|
||||
value found no plugins, so the scanner answered `No plugins found` (info) with exit `0`.
|
||||
|
||||
**`marketplace.json` exemption:** `.claude-plugin/`'s known-file set is `plugin.json` **and**
|
||||
`marketplace.json`. The catalog's location is documented and required (*"Create
|
||||
`.claude-plugin/marketplace.json` in your repository root"*), and a marketplace entry with
|
||||
`"source": "./"` makes the repo root its own plugin — so one `.claude-plugin/` legitimately holds
|
||||
both. Verified against the primary docs before the change; the check was a false positive, latent in
|
||||
this marketplace only because `catalog/` ships no `plugin.json` and is thus not scanned as a plugin.
|
||||
|
||||
### SET scanner — autoMode validation (`CA-SET`)
|
||||
|
||||
Per-file check in `settings-validator.mjs` (`autoMode` was in `KNOWN_KEYS` but had no nested
|
||||
|
|
@ -690,3 +769,107 @@ The second half of Block 4. **Asymmetric:** plan export is the new testable code
|
|||
commands **21** (export is a *mode*, not a new command), snapshot/backcompat suite untouched.
|
||||
suite 1150→1168 (lib +8, export-cli +10). **Block 4a (`migrateLedger`) still deferred** to the
|
||||
first breaking schema change (export needs no schema bump).
|
||||
|
||||
### arg-sluk — the CLI argument class, measured across all fourteen CLIs (v5.14, #57)
|
||||
|
||||
`scanners/lib/cli-args.mjs`. Every CLI in `scanners/` parses argv with a chain of
|
||||
`if (a === '--x') … else if …`, and two things fell through that chain in silence.
|
||||
|
||||
**Arm 1 — the unknown flag.** With no `else` branch, `--zzz` left no trace: exit 0, full
|
||||
payload, a confident answer to a question the caller did not ask. First costed in #51, when
|
||||
`knowledge-refresh`'s only knob reached the CLI malformed and the command reported "all 14
|
||||
register entries were re-verified within the last 90 days" — about a threshold the user had
|
||||
just overridden.
|
||||
|
||||
**Arm 2 — the value that was really a flag.** `a === '--output-file' && args[i + 1]` asks
|
||||
only whether a next token *exists*, never whether it is a value. So `--output-file --json`
|
||||
took `--json` as the filename. Measured: `manifest`, `campaign-cli` and
|
||||
`knowledge-refresh-cli` each **wrote a file literally named `--json`** into the caller's
|
||||
working directory, exit 0, with `--json` mode silently dropped. A wrong answer is bad; an
|
||||
unintended file on disk is worse.
|
||||
|
||||
**Width — the deferral list was a prediction, not a measurement.** `KNOWN_OPEN` in
|
||||
`tests/scanners/cli-unknown-flag-rejection.test.mjs` named **two** CLIs. Measuring all
|
||||
fourteen found **7** open on arm 1 and **10** on arm 2 — including `campaign-cli` and
|
||||
`knowledge-refresh-cli`, which were already in `GUARDED` and *passing* the arm-1 test while
|
||||
arm 2 stood open a few lines away. Three CLIs (`drift-cli`, `fix-cli`,
|
||||
`plugin-health-scanner`) were already correct on both arms because they use a different
|
||||
parse form; they were moved into `GUARDED` rather than left unguarded.
|
||||
|
||||
**Why a gate and not a rewrite.** The module runs *before* each CLI's existing loop and does
|
||||
not replace it. Valid argv therefore reaches the existing parser byte-for-byte unchanged, so
|
||||
no frozen snapshot can move — the byte-stability argument is structural, not empirical. The
|
||||
three CLIs that carried a bespoke `else if (a.startsWith('--')) fail(…)` branch had it removed
|
||||
once the gate made it unreachable, along with the now-redundant `&& args[i + 1]` guards.
|
||||
|
||||
Exit code is **3** by the exit-code contract: a malformed argument means the scanner never got
|
||||
to do its job, which is categorically different from 0/1/2 — verdicts about a configuration
|
||||
that *was* examined.
|
||||
|
||||
---
|
||||
|
||||
### rollback-cli — the recovery path gets a runnable entry (R1+R2, #74)
|
||||
|
||||
`scanners/rollback-cli.mjs` is the entry to `rollback-engine.mjs` and, via
|
||||
`--create`, to `lib/backup.mjs`. It closes the two KRITISK rows of the Q3
|
||||
severity table as one chunk, because neither half holds alone.
|
||||
|
||||
**R1 — the restore was model prose over a code engine.** Measured at the head of
|
||||
the chunk: 16 files under `scanners/` carried a `process.argv` entry;
|
||||
`rollback-engine.mjs` was not one of them, though it had verified every file's
|
||||
checksum *before and after* each write since M-BUG-22 and reported
|
||||
`createdNotRemoved` since M-BUG-25. `commands/rollback.md` §Implementation
|
||||
showed an ESM `import` block a command template cannot execute and offered
|
||||
ad-hoc `cp` underneath as the runnable alternative, then pre-rendered
|
||||
"`(checksum verified)`" three times in the success output. `cp` establishes no
|
||||
checksum, so the verification was a property of the template, not of the run —
|
||||
on the one surface that runs when the user is already in trouble.
|
||||
|
||||
**R2 — the backup format had two authors, one of them prose.** The fix pipeline
|
||||
backed up through `createBackup`; the implement pipeline hand-built its own —
|
||||
`mkdir`, `cp`, a `date +%Y%m%d_%H%M%S` id, and a manifest typed out in the
|
||||
template. `parseManifest` grew a second branch for that format because the seam
|
||||
had already failed silently once, and the fixture pinning it was **hand-written**
|
||||
rather than derived from the template's own text: the #63 shape on the data
|
||||
side. Rename a key in the template and `parseManifest` returns zero files while
|
||||
`rollback` reports success.
|
||||
|
||||
**Why one chunk.** Fix only R1 and the new CLI still parses a prose format. Fix
|
||||
only R2 and the format is clean with no runnable entry behind it.
|
||||
|
||||
**Exit contract.** 0 done · 1 outstanding (a restore the scope gate will not
|
||||
perform without `--approve-scope`, nothing written; or a backup that covered
|
||||
fewer targets than it was given) · 2 at least one file failed · 3 the CLI could
|
||||
not do its job. A gated restore is **1, not 3**: "this write leaves your project"
|
||||
is a verdict about a write that *was* examined, and it rides in the payload,
|
||||
where a command running under `2>/dev/null` can act on it. That is F3's class,
|
||||
avoided rather than repeated.
|
||||
|
||||
**`--created` records, it does not copy.** No backup can hold a file that does
|
||||
not exist yet. Those paths go into the manifest so `rollback` can list what it is
|
||||
leaving in place. `serializeManifest` emits the bare `created:` key; the pre-R2
|
||||
implement format used `created: <timestamp>` with a VALUE, meaning the backup id,
|
||||
and `parseManifest` tells them apart on exactly that — which is why the two never
|
||||
collide in one file.
|
||||
|
||||
**The implement-format branch stays.** Nothing writes that shape any more, but
|
||||
every backup implement made before this chunk is on disk in it and must remain
|
||||
restorable — the same reasoning that keeps `getLegacyBackupDir()` readable. The
|
||||
hand-written fixture in `tests/scanners/rollback-paths.test.mjs` therefore
|
||||
changed meaning rather than becoming obsolete: it is now a golden sample of
|
||||
historical bytes, which is a legitimate thing to write by hand, instead of a
|
||||
stand-in for a template's own text, which is not.
|
||||
|
||||
**What replaced "(checksum verified)".** `tests/commands/backup-restore-contract.test.mjs`
|
||||
runs the CLI and checks every field `rollback.md` renders against the real
|
||||
payload. A renamed payload key now fails a test instead of turning into a
|
||||
confident sentence. Seen red against its own defect by renaming `{status}`.
|
||||
|
||||
**A guard hole found by mutation.** The first version of the implement-side
|
||||
assertion matched `--create` as a substring, and the same invocation carries
|
||||
`--created` — so replacing the `--create` call with `--list` left the guard
|
||||
green. It matches `--create(?![a-z])` now. Separately, mutating
|
||||
`if (!requireValidArgs(...)) return;` into a bare call showed that
|
||||
`requireValidArgs` sets exit 3 *by itself*: a CLI can report "I could not parse
|
||||
my arguments" and still run the restore underneath. `rollback-cli.test.mjs`
|
||||
asserts on the bytes for that case, not on the exit code.
|
||||
|
|
|
|||
|
|
@ -1,161 +1,228 @@
|
|||
# v5.13 Plan — Model Routing, Effort Awareness, Dead References
|
||||
# v5.14 Plan — Doctor Overlap, Model Routing, Effort Awareness, Dead References
|
||||
|
||||
> **Target version is now v5.14.** The `5.13.0` slot was consumed by the pipeline-hardening batch
|
||||
> (`optimize --subtract` is a feature, so that release could not be a patch). The filename is kept so
|
||||
> existing references resolve; only the target moved.
|
||||
> **Filename note:** kept as `v5.13-…` so existing references resolve; the target has been
|
||||
> v5.14 since the `5.13.0` slot was consumed by the pipeline-hardening batch. Rewritten
|
||||
> 2026-08-03 (session #53) to merge the `/doctor`-overlap decision (Oppgave A) and the
|
||||
> context-engineering follow-ups (B1–B4) from `docs/v5.14-doctor-overlap-brief.md` into the
|
||||
> pre-existing chunks. One plan, top-to-bottom, no relitigation.
|
||||
|
||||
Derived from an external video analysis ("The Model Isn't the Moat", 2026-07) cross-checked
|
||||
against primary sources and against what config-audit already encodes. Every claim acted on
|
||||
here was verified against Anthropic's own docs; video-only claims are explicitly rejected below.
|
||||
## Inputs and their status
|
||||
|
||||
## Source verification (done 2026-07-14)
|
||||
| Input | Status |
|
||||
|---|---|
|
||||
| `/doctor`-overlap measurement (session #53, fasit-first) | **DONE** — decisions binding, recorded in `docs/doctor-overlap-results.local.md` |
|
||||
| B1 register freshness defect (evidence-age / supersededBy / sources[]) | **DONE** — landed as `d66035e` (TDD, 1477/1477 green) |
|
||||
| Video-derived model/effort chunks (verified 2026-07-14) | Open — carried below unchanged |
|
||||
| Open posts from dogfood sessions #45–#51 | Open — detail lives in `STATE.md`; referenced here by ID only |
|
||||
|
||||
## A. The `/doctor` verdict (measured 2026-08-03, CC 2.1.220 — binding)
|
||||
|
||||
`/doctor` (alias `/checkup`, v2.1.205+) is an **agent-driven, usage-data-backed, quota-priced,
|
||||
non-reproducible** checkup: 10 checks incl. unused skills/plugins/MCP vs. context cost,
|
||||
CLAUDE.md dedup/contradiction judgment, derivable-content trim (checked-in files only),
|
||||
lazy-loading migration, hook latency, and a chars÷4 context manifest. It overlaps our
|
||||
**judgment lenses and alarms — not the deterministic validators.** Full per-scanner table in
|
||||
`docs/doctor-overlap-results.local.md`.
|
||||
|
||||
Binding outcomes (0 whole scanners removed; 2 measured function-duplicates removed;
|
||||
5 surfaces repositioned):
|
||||
|
||||
| ID | Chunk | What |
|
||||
|---|---|---|
|
||||
| **D1** | GAP autoMode removal | Delete the «No autoMode classifier» gap dimension (`feature-gap-scanner.mjs:485`) — `/doctor` Sjekk 8 checks AND fixes it. Byte-stability: existing-scanner finding-removal variant of [[adding-scanner-byte-stability]]; humanizer entries for the removed title go too. |
|
||||
| **D2** | SKL alarm re-scope | Remove CA-SKL-002's aggregate-over-budget **alarm** role (native startup warning v2.1.105/2.1.181 + `/doctor` Sjekk 6 both do it better-placed); re-position CA-SKL-001 as per-description **attribution** (author lens, repo scope); CA-SKL-003 untouched. Exact form (delete vs. re-scope 002) decided inside the chunk against the frozen-baseline cost. |
|
||||
| **D3** | Positioning rewrite | **DONE in #53** — README section «config-audit vs. the built-in /doctor» (division-of-labor table + per-area split for SET/HKV/CNF/OPT/tokens/manifest, citing `/doctor`'s own referral to `optimize --subtract`) + CLAUDE.md invariant line. Remaining refinement (per-command copy in `commands/*.md`, if wanted) rides along with D1/D2. |
|
||||
|
||||
**Strategic line (constrains all copy):** our defensible identity = determinism/byte-stability,
|
||||
all scopes incl. LOCAL, cross-repo campaign breadth, zero quota. `/doctor`'s unmatchable edge =
|
||||
usage telemetry. Long-term (v5.15+ candidate, NOT this release): transcript/usage telemetry as a
|
||||
scanner *input*, so the axes compose instead of competing.
|
||||
|
||||
## B. Context-engineering follow-ups (article 2026-07-24, brief §2)
|
||||
|
||||
- **B1 — DONE** (`d66035e`): `sources[]` + `published` + `supersededBy` + evidence-age rule
|
||||
(green-but-outdated is now expressible and flagged; re-verifying the old source no longer
|
||||
clears it). BP-SUB-001 carries the article as verified corroborating source.
|
||||
**Deviation from brief, verified 2026-08-03:** the article contains NO mechanism-choice or
|
||||
size-limit content → it was NOT added to BP-MECH-*/BP-SIZE-001 (that would be false
|
||||
provenance). If a future read finds real coverage, add it then.
|
||||
- **B2 — DONE, and the detector was DECLINED by measurement.** `BP-JUDG-001` ships as
|
||||
register knowledge with `lensCheck: null`; **no `CA-OPT-002`** (OPT next-free stays 2).
|
||||
Measured over **409 real CLAUDE.md files** (38 488 lines, 8 689 prose blocks): the caging
|
||||
class fires **7 times, all 7 false positives**, confirmed along an independent grep path in
|
||||
both word orders (5 lines / 1 line, none an instruction). Where the shape *does* occur — 45
|
||||
lines across 4 755 skill/agent/command files — it is the author's **editorial policy** (emoji,
|
||||
sentence length, slide titles), and nothing in the text separates that from a vendor's
|
||||
over-tight guardrail: the article's reasoning does not transfer, because the model is not the
|
||||
author of a user's config. Precision-first ⇒ silence. Two premises the chunk falsified: the
|
||||
brief's «these blocks are inside the floor» (the article's own example carries **no** floor
|
||||
marker; the corpus tendency is 76 %, which is not a mechanism), and the §4 form-noun
|
||||
vocabulary (`name`/`format` alone were 97 % of fires). Full record:
|
||||
`docs/b2-judgment-lens-fasit.local.md` §9.
|
||||
- **B3 — two-layer duplication/contradiction (CNF extension):** article rule 4 + `/doctor`'s
|
||||
measured Sjekk 2 catch (global CLAUDE.md model-policy ↔ agent frontmatter) prove the class
|
||||
exists and is catchable. Extend `conflict-detector` with cross-layer checks where the pair is
|
||||
structurable (CLAUDE.md/rule, rule/skill-description, CLAUDE.md-keyword ↔ frontmatter field).
|
||||
This **supersedes the old "Explicitly rejected #5"** below — it now has evidence.
|
||||
- **B4 — thinner gaps (backlog, after everything above):** rule 2 (skills/agents leaning on
|
||||
examples where a parameter enum would do) and rule 6 (reference *form*: prefer in-code files;
|
||||
`import-resolver` is the natural owner). Park until D/B2/B3 land.
|
||||
- **Rules 3 and 5 need nothing** (verified in brief §B4): progressive disclosure = BP-LOAD-001..006
|
||||
+ BP-MECH-003 + `token-hotspots`/`manifest`; router pattern + auto-memory covered.
|
||||
|
||||
## C. Carried chunks (verified 2026-07-14 — unchanged specs)
|
||||
|
||||
Source-verification table, rejected-claims list, and full chunk specs below are carried
|
||||
verbatim from the 2026-07-14 revision; only numbering context changed.
|
||||
|
||||
### C1 — Register entries: model routing + effort (dogfoods `knowledge-refresh`)
|
||||
|
||||
- **BP-MODEL-001** (`model-fit`): mechanical/read-only subagents can pin a cheaper model via
|
||||
`model:` frontmatter; orchestrator keeps the strong model. Source: code.claude.com/docs/en/sub-agents
|
||||
→ `confirmed`.
|
||||
- **BP-MODEL-002** (`model-fit`): reasoning effort tunable at five levels in five places;
|
||||
default `high`; higher is not universally better. Source: code.claude.com/docs/en/model-config
|
||||
→ `confirmed`.
|
||||
- Schema per `scanners/lib/best-practices-register.mjs:42-102`. New sources SHOULD carry
|
||||
`published` so the B1 evidence-age rule has teeth.
|
||||
|
||||
### C2 — fix-engine effort hygiene (tiny, TDD)
|
||||
|
||||
`fix-engine.mjs:26` `VALID_EFFORT_LEVELS` missing `xhigh` → nearest-match "fix" for `xhig`
|
||||
corrects to `high`. Red test first: `findNearestEffortLevel('xhig') === 'xhigh'`. Align with
|
||||
`settings-validator.mjs:75`.
|
||||
|
||||
### C3 — CA-CML dead prose references (new deterministic check)
|
||||
|
||||
Flag backtick-quoted relative file paths in CLAUDE.md prose that do not exist on disk (today
|
||||
only `@import` targets are checked). Conservative v1: skip URLs, globs, placeholders,
|
||||
absolute/`~/` paths. Severity low. New CA-CML-NNN (verify next free NNN at implementation).
|
||||
Byte-stability per [[adding-scanner-byte-stability]] incl. humanizer step 7.
|
||||
|
||||
### C4 — feature-gap + inventory: model/effort awareness
|
||||
|
||||
New T3 opportunity check: authored agents where NO agent sets `model:`/`effort:` → routing
|
||||
opportunity citing BP-MODEL-001/002. Fires only when authored agents exist; opportunity
|
||||
framing, suppressable. `whats-active`/`manifest` surface `model`/`effort` per agent.
|
||||
Humanizer step 7; verify via direct `scan()` ([[agent-commands-need-scanner-scoping]]).
|
||||
Known tension with the operator's own Opus-for-everything policy stands as written 2026-07-14:
|
||||
the check serves general users; on this machine it gets suppressed.
|
||||
|
||||
### C5 — planner-agent adversarial gate (AFTER DEL B 3.2 dogfood)
|
||||
|
||||
Required "Failure modes" section in `agents/planner-agent.md`'s action-plan contract.
|
||||
Sequencing: only after the DEL B fasit pass that judges planner-agent, or the fasit target
|
||||
moves mid-evaluation.
|
||||
|
||||
## D. Open posts from dogfooding (detail in STATE.md — not restated here)
|
||||
|
||||
C-SKL1 (#37) · M-BUG-26 · **M-BUG-28** (suppression-ID positional instability — ID-semantics
|
||||
change touching all scanners + frozen snapshots, own chunk) · **M-BUG-41** (no scope-gate from
|
||||
scan to write, two arms — design change, own chunk) · **arg-sluk CLI arm**
|
||||
(`optimize-lens-cli` + `token-hotspots-cli`, `KNOWN_OPEN` in
|
||||
`tests/scanners/cli-unknown-flag-rejection.test.mjs`) · **P6/M-BUG-44** (scanner-side stdout
|
||||
with `--output-file`) · knowledge-refresh write-CLI · cleanup-invisible session files ·
|
||||
web-poll candidates (nothing written; primary sources unread).
|
||||
|
||||
## Priority order for v5.14
|
||||
|
||||
Cheap-and-loud first, judgment-heavy later; D-chunks early because they DELETE code the rest
|
||||
must not build on. **Re-ordered 2026-08-10 (operator decision, session #61):** the operator wants
|
||||
to *use* the subtraction axis, so its write half — and the scope-gate it depends on — move ahead
|
||||
of the remaining additive work. Everything below step 4 is unchanged in content, only in position.
|
||||
|
||||
1. ~~**C2**~~ ✅ · 2. ~~**Arg-sluk CLI arm**~~ ✅ · 3. ~~**D1 + D2**~~ ✅ · 4. **D3** (rest dropped,
|
||||
`stop-at-meaningful-value`) · 5. ~~**M-BUG-28**~~ ✅ · 6. ~~**C1**~~ ✅ · 7a. ~~**C4**~~ ✅
|
||||
8. **M-BUG-41** (scope-gate design) — **promoted from 10.** Prerequisite for anything that writes
|
||||
outside the repo the session stands in, which subtraction-write does by definition
|
||||
(`~/.claude/CLAUDE.md`).
|
||||
9. **SUB-WRITE** (new) — the write half of `optimize --subtract`; see §C6 below.
|
||||
10. **C3** (CA-CML dead prose references) — was 7b.
|
||||
11. **B2** (new lens axis)
|
||||
12. **B3** (CNF two-layer extension)
|
||||
13. **P6/M-BUG-44**, knowledge-refresh write-CLI, cleanup glob (small batch)
|
||||
14. **C5** (after DEL B 3.2), **B4** backlog last
|
||||
15. Release-cut via `release-plugin.mjs` when the batch is coherent. **Level is MAJOR — v6.0.0:**
|
||||
M-BUG-28 shipped as `fix(scanners)!` with a `BREAKING CHANGE:` footer, which outranks the
|
||||
minor the D1/D2 removals plus C3/C4 additions would have implied. `release-plugin.mjs` does
|
||||
not derive the level — pass `--version` explicitly.
|
||||
|
||||
### C6 — SUB-WRITE: the write half of `optimize --subtract`
|
||||
|
||||
`--subtract` proposes and never writes (`commands/optimize.md`), which is correct for a v1 whose
|
||||
judge is an agent. The operator now wants the removal executed. Scope: apply an approved
|
||||
subtraction candidate to the CLAUDE.md it came from, with backup and rollback.
|
||||
|
||||
Non-negotiable frames, all inherited rather than invented here:
|
||||
|
||||
- **The floor is not the judge's decision.** `floor-exclusion.mjs` runs deterministically before
|
||||
anything is proposed, and that ordering must not migrate into the write path either.
|
||||
- **`~/.claude` is git-tracked with a `.gitignore` of `*`** — archive by `mv` into `_archive/`,
|
||||
never `rm`. Machine-side config writes need operator approval.
|
||||
- **User level is mandatory in v1**: that is where the cost is (~4 300 tokens every turn in every
|
||||
repo). Project level follows.
|
||||
- **Honest sizing:** the #40 fasit measured deletable ≈1 400 always-loaded tokens, realistically
|
||||
≈850 after tier-2 earn-backs, against a ≈4 300-token file — **≈20 %, not 80 %.** Do not let the
|
||||
command's copy imply more.
|
||||
- Verify the backup covers the file the write actually touched, not merely that a backup exists
|
||||
(M-BUG-31's shape).
|
||||
|
||||
Open decision, to be settled in the chunk's fasit before code: whether removal is a `fix`-engine
|
||||
action, a `plan`/`implement` step, or its own flag — decided against M-BUG-41's gate, not before it.
|
||||
|
||||
**Rejected 2026-08-10, do not revive:** a sibling `/repo-reinit` skill that rewrites a CLAUDE.md
|
||||
from scratch. It would be a third implementation of one judgement (this axis, plus `/doctor`
|
||||
Check 3) and fails the binding `/doctor` positioning; and regenerating destroys exactly the floor
|
||||
— local facts, gotchas, policy invariants — that a mature repo's CLAUDE.md is most valuable for.
|
||||
`repo-init` already owns the fresh-repo case.
|
||||
|
||||
## Source verification (done 2026-07-14 — carried)
|
||||
|
||||
| Claim from video | Verdict | Source |
|
||||
|---|---|---|
|
||||
| Orchestrator + cheaper worker models is a supported, recommended pattern | VERIFIED | code.claude.com/docs/en/sub-agents ("Control costs by routing tasks to faster, cheaper models like Haiku"), code.claude.com/docs/en/workflows |
|
||||
| Reasoning effort is tunable per settings / session / launch / **per-agent frontmatter** / SDK; levels `low, medium, high, xhigh, max` | VERIFIED | code.claude.com/docs/en/model-config#adjust-effort-level, sub-agents doc |
|
||||
| Leaked Fable 5 system prompt principles ("partial recognition ≠ current knowledge"; "a prompt implying a file is present doesn't mean one is"; answer-first-then-one-question; tool-call scaling 1 / 3–5 / 5–10) | VERIFIED near-verbatim, **provenance unconfirmed** (third-party leak repo, not Anthropic-confirmed) | github.com/asgeirtj/system_prompts_leaks `Anthropic/claude-fable-5.md` |
|
||||
| "Fable 5 on low ≈ Opus 4.8 on high, slightly higher cost/quality" score-vs-cost chart | **CONTRADICTED** — no such chart/statement on Anthropic's pages; GPT-5.5 appears only in a testimonial | anthropic.com/news/claude-fable-5-mythos-5 |
|
||||
| Orchestrator + cheaper worker models is supported/recommended | VERIFIED | code.claude.com/docs/en/sub-agents, /workflows |
|
||||
| Effort tunable per settings/session/launch/agent-frontmatter/SDK; `low..max` | VERIFIED | code.claude.com/docs/en/model-config#adjust-effort-level |
|
||||
| Leaked Fable 5 system-prompt principles | VERIFIED near-verbatim, provenance unconfirmed | github.com/asgeirtj/system_prompts_leaks |
|
||||
| "Fable low ≈ Opus high" chart | **CONTRADICTED** | anthropic.com/news/claude-fable-5-mythos-5 |
|
||||
|
||||
## Already covered — no action
|
||||
## Explicitly rejected (unchanged unless noted)
|
||||
|
||||
| Video idea | Existing coverage |
|
||||
|---|---|
|
||||
| "Process is the moat" (config/harness > raw model) | The plugin's entire thesis |
|
||||
| Extract repeated procedure into a skill | BP-MECH-003 + CA-OPT-001 (`optimization-lens-scanner.mjs:121`) |
|
||||
| CLAUDE.md size/ownership discipline | BP-SIZE-001 + CA-CML line/size checks (`claude-md-linter.mjs:109/:120/:140`) |
|
||||
| Check that referenced imports exist | CA-IMP broken `@import` (`lib/import-resolver.mjs:88`) — but **only** `@import`, see Chunk 3 |
|
||||
| Plugin's own agents are model-routed | Agents table already pins sonnet for mechanical, opus for judgment |
|
||||
1. "Fable low ≈ Opus high" framing — never encode.
|
||||
2. Tool-call-count effort scaling as register entry — unconfirmed leak, not carried.
|
||||
3. Cost/intelligence/"taste" routing-table generator — subjective, doesn't fit provenance-gated design.
|
||||
4. "Fable mode" skill — out of plugin scope.
|
||||
5. ~~CLAUDE.md prose contradiction detection~~ — **superseded by B3** (2026-08-03: article
|
||||
rule 4 + `/doctor` Sjekk 2 measurement supplied the evidence the 2026-07-14 rejection lacked).
|
||||
|
||||
## Gaps → chunks
|
||||
## Verification (per chunk, unchanged discipline)
|
||||
|
||||
Verified gap summary (register-mapper sweep, 2026-07-14): no scanner audits per-agent
|
||||
`model:`/`effort:` frontmatter; effort has validity-check only (`settings-validator.mjs:195`),
|
||||
no recommendation; no dead-reference check for prose file mentions in CLAUDE.md; no
|
||||
adversarial/failure-mode requirement in planner-agent; `fix-engine.mjs:26` effort list
|
||||
omits `xhigh` (settings-validator has all five).
|
||||
- Full suite green (`node --test 'tests/**/*.test.mjs'`; baseline 2026-08-03: 1477/0), frozen
|
||||
`tests/snapshots/v5.0.0/` untouched (`git status --porcelain` empty), red test before every
|
||||
production change.
|
||||
- **D1:** GAP fixture with autoMode absent → no finding; humanizer has no orphaned entries
|
||||
(M-16/M-17 checks reversed for removal); frozen baselines untouched or consciously re-seeded
|
||||
per [[adding-scanner-byte-stability]].
|
||||
- **D2:** over-budget fixture → no 002-alarm (or re-scoped payload per in-chunk decision);
|
||||
001 fires per oversized description with attribution copy; 003 unchanged.
|
||||
- **D3:** README/CLAUDE.md name `/doctor` explicitly; `self-audit --check-readme` PASS.
|
||||
- **C1–C5:** criteria as specified 2026-07-14 (C2 red-first nearest-match; C3 fixture
|
||||
missing-path fires / URL-glob-placeholder silent; C4 authored-agent matrix; C5 failure-modes
|
||||
section present).
|
||||
- **B2 (superseded by the measurement above):** the criterion was written for a detector that
|
||||
measurement declined. What was verified instead: `BP-JUDG-001` present, `confirmed`, primary
|
||||
source dated `2026-07-24`, `lensCheck` **absent** — plus a guard that every `lensCheck` in the
|
||||
register is backed by a real detector, so no later session can "complete" the entry by wiring
|
||||
one. Both assertions seen RED against their own defect before landing.
|
||||
- **B3:** fixture with same instruction in CLAUDE.md + rule → CNF finding; single-layer only →
|
||||
silent ([[guard-can-be-green-on-its-own-defect]]: assert the blanket invariant).
|
||||
|
||||
### Chunk 1 — Register entries: model routing + effort (dogfoods `knowledge-refresh`)
|
||||
## Key assumptions (test at implementation)
|
||||
|
||||
Add to `knowledge/best-practices.json` via the knowledge-refresh flow (human-approved write):
|
||||
|
||||
- **BP-MODEL-001** (`category: model-fit`): subagents doing mechanical/read-only work can pin a
|
||||
cheaper model via `model:` frontmatter; orchestrator keeps the strong model. Source:
|
||||
code.claude.com/docs/en/sub-agents → `confidence: confirmed`.
|
||||
- **BP-MODEL-002** (`category: model-fit`): reasoning effort is tunable at five levels in five
|
||||
places (settings `effortLevel`, `/effort`, `--effort`, per-agent `effort` frontmatter, SDK);
|
||||
default `high`; higher effort is not universally better for simple tasks. Source:
|
||||
code.claude.com/docs/en/model-config → `confidence: confirmed`.
|
||||
|
||||
Schema per `scanners/lib/best-practices-register.mjs:42-102` (id/claim/confidence/source.url/
|
||||
source.verified required). This chunk doubles as the DEL B dogfood of `/config-audit
|
||||
knowledge-refresh` (each chunk is also a plugin test).
|
||||
|
||||
### Chunk 2 — fix-engine effort hygiene (tiny, TDD)
|
||||
|
||||
`fix-engine.mjs:26` `VALID_EFFORT_LEVELS = ['low','medium','high','max']` — missing `xhigh`.
|
||||
Consequence: nearest-match "fix" for a typo like `xhig` corrects to `high`, not `xhigh`.
|
||||
Red test first: `findNearestEffortLevel('xhig') === 'xhigh'`. Align list with
|
||||
`settings-validator.mjs:75`.
|
||||
|
||||
### Chunk 3 — CA-CML dead prose references (new deterministic check)
|
||||
|
||||
The strongest video-derived principle ("a prompt implying a file is present doesn't mean one
|
||||
is") applied to CLAUDE.md quality: flag file paths mentioned in CLAUDE.md **prose** that do not
|
||||
exist on disk. Today only `@import` targets are existence-checked; stale pointers like
|
||||
`docs/foo.md` or `scripts/bar.sh` rot silently and burn always-loaded tokens on misdirection.
|
||||
|
||||
Conservative v1 to control false positives:
|
||||
- Only backtick-quoted tokens that look like relative file paths (contain `/` or a known
|
||||
extension), resolved against the CLAUDE.md's own directory.
|
||||
- Skip URLs, globs (`*`), placeholders (`{...}`, `<...>`, `$VAR`, `${...}`), absolute and
|
||||
`~/` paths (machine-specific), and paths under `.gitignore`d dirs if cheap to determine.
|
||||
- Severity: low. New CA-CML-NNN (verify next free NNN at implementation — IDs are dynamic).
|
||||
|
||||
Byte-stability: follow [[adding-scanner-byte-stability]] steps for a new finding type in an
|
||||
EXISTING scanner — frozen `tests/snapshots/v5.0.0/` must stay untouched; default-output
|
||||
snapshots regenerate (`UPDATE_SNAPSHOT=1`) only if a fixture actually carries the new type;
|
||||
humanizer step 7 (M-16/M-17 lessons): `TRANSLATIONS`-static entry for the new RAW title
|
||||
(CML category mapping already exists).
|
||||
|
||||
### Chunk 4 — feature-gap + inventory: model/effort awareness
|
||||
|
||||
- New T3 opportunity check in `feature-gap-scanner.mjs`: authored agents
|
||||
(`isAuthoredConfig`, M-BUG-13 lesson) where **no** agent sets `model:` or `effort:` →
|
||||
"all agents inherit the session model/effort — mechanical agents can be routed cheaper /
|
||||
effort-calibrated" citing BP-MODEL-001/002. Fires only when authored agents exist
|
||||
(M-BUG-15 lesson: no enhancement-check on empty collections). Opportunity framing, never
|
||||
failure — deliberate max-model setups are a valid choice; finding is suppressable
|
||||
(`.config-audit-ignore`).
|
||||
- `whats-active` / `manifest`: surface `model`/`effort` per agent in the inventory tables.
|
||||
- Humanizer wiring step 7 for the new GAP finding; verify via direct `scan()` output, not the
|
||||
self-suppressed default output ([[agent-commands-need-scanner-scoping]]).
|
||||
- feat commit → docs-gate: README + CLAUDE.md diffs required.
|
||||
|
||||
### Chunk 5 — planner-agent adversarial gate (do AFTER DEL B pipeline dogfood)
|
||||
|
||||
Add a required "Failure modes" section to `agents/planner-agent.md`'s action-plan contract:
|
||||
before an action plan is emitted, list what could go wrong per change + rollback trigger.
|
||||
Mirrors the video's scoping-vs-devil's-advocate distinction; currently absent (zero
|
||||
adversarial requirements in agents/). **Sequencing constraint:** DEL B step 3.2 judges
|
||||
planner-agent against a fasit — change the agent only after that dogfood pass, or the
|
||||
fasit target moves mid-evaluation.
|
||||
|
||||
## Explicitly rejected (do not revisit without new evidence)
|
||||
|
||||
1. **"Fable low ≈ Opus high" cost/score framing** — contradicted by Anthropic's own pages.
|
||||
Never encode in register, copy, or recommendations.
|
||||
2. **Tool-call-count effort scaling (1 / 3–5 / 5–10) as a register entry** — source is an
|
||||
unconfirmed third-party leak → would be `confidence: inferred`, never surfaced. Not worth
|
||||
carrying.
|
||||
3. **Cost/intelligence/"taste" model-routing table generator** — subjective scores don't fit
|
||||
the deterministic, provenance-gated design. BP-MODEL-001 covers the actionable core.
|
||||
4. **"Fable mode" skill** — a user-level skill, not configuration auditing. Out of plugin scope.
|
||||
5. **CLAUDE.md prose contradiction detection** — real gap (CA-CNF only covers
|
||||
settings/permissions/hooks) but not video-driven; keep this plan surgical.
|
||||
|
||||
## Known tension (named, not resolved here)
|
||||
|
||||
The operator's own global policy is Opus/max-effort for ALL subagents, never Haiku — the
|
||||
opposite of Chunk 4's recommendation. Both are legitimate: the docs-backed routing advice
|
||||
optimizes cost at equal quality for the general user; the operator deliberately buys maximum
|
||||
quality. Chunk 4's copy must respect that (opportunity framing + suppressability), and on this
|
||||
machine the finding will simply be suppressed or ignored. The plugin serves general users;
|
||||
the operator's setup is not the target of the check.
|
||||
|
||||
## Verification
|
||||
|
||||
Global, after every chunk:
|
||||
- `node --test 'tests/**/*.test.mjs'` → green (baseline 1359/0; count grows with new tests)
|
||||
- `git status --porcelain tests/snapshots/v5.0.0/` → empty (frozen untouched)
|
||||
- TDD: red test exists and fails BEFORE each production change
|
||||
|
||||
Per chunk:
|
||||
- **C1:** `node --test tests/lib/best-practices-register.test.mjs` green;
|
||||
`node scanners/knowledge-refresh-cli.mjs` classifies BP-MODEL-001/002 as fresh
|
||||
- **C2:** `findNearestEffortLevel('xhig')` → `xhigh` (red first); `grep xhigh scanners/fix-engine.mjs` non-empty
|
||||
- **C3:** fixture CLAUDE.md referencing `docs/missing.md` → finding fires; existing file /
|
||||
URL / glob / placeholder / `~/` path → silent; humanized output has non-contradictory copy
|
||||
- **C4:** authored-agent fixture without model/effort → opportunity fires; with either set →
|
||||
silent; zero authored agents → silent; `userImpactCategory` ≠ `Other` end-to-end via direct scan()
|
||||
- **C5:** dogfood plan run produces a Failure-modes section; DEL B 3.2 fasit judged BEFORE the change
|
||||
|
||||
## Key assumptions (test before/at implementation)
|
||||
|
||||
1. **Per-agent `effort` frontmatter is official** — verified 2026-07-14 against
|
||||
code.claude.com/docs/en/sub-agents + /model-config; re-fetch both pages at implementation
|
||||
(docs move).
|
||||
2. **New finding type in existing scanner leaves frozen snapshots untouched** — M-17 precedent
|
||||
says yes when no v5.0.0 fixture carries the type; verify by running the suite and inspecting
|
||||
which snapshots differ before committing.
|
||||
3. **Next free CA-CML/CA-GAP finding numbers** — IDs are built dynamically; grep tests +
|
||||
snapshots for the highest used NNN before assigning.
|
||||
|
||||
## Sequencing vs DEL B (one plan, no relitigation)
|
||||
|
||||
This plan does NOT preempt the active DEL B sequence. Recommended order:
|
||||
1. DEL B step 3 pipeline dogfood (`analyze → plan → implement → rollback`) — unchanged, next.
|
||||
2. Batch patch release M-11→M-17 — unchanged.
|
||||
3. v5.13 chunks 1→5 (chunk 1 doubles as the `knowledge-refresh` dogfood already queued in
|
||||
DEL B "Resten"; chunk 5 explicitly waits for step 3.2). Release as minor v5.13.0 via
|
||||
`release-plugin.mjs` when all chunks land.
|
||||
1. Per-agent `effort` frontmatter still official — re-fetch sub-agents + model-config pages.
|
||||
2. Finding-type REMOVAL in an existing scanner leaves frozen v5.0.0 snapshots untouched only if
|
||||
no frozen fixture carries the type — verify per D1/D2 before committing; re-seed consciously
|
||||
if not.
|
||||
3. Next free CA-CML/CA-GAP/CA-OPT NNN — grep tests + snapshots before assigning.
|
||||
4. `/doctor`'s check set is version-fluid (2.1.205→220 changed it materially) — re-run the
|
||||
overlap measurement cheaply (CLI + one in-session run) before executing D1–D3 if CC has
|
||||
moved significantly past 2.1.220.
|
||||
|
|
|
|||
136
docs/v5.14-doctor-overlap-brief.md
Normal file
136
docs/v5.14-doctor-overlap-brief.md
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
# Brief — `/doctor`-overlapp og ny kontekst-doktrine (v5.14-inngang)
|
||||
|
||||
**Skrevet:** 2026-08-03, økt #52 (Opus 5/high)
|
||||
**Skrevet for:** neste økt — **Fable 5 / high** (operatørens modellvalg; rubrikken ga `Opus 5/high`, `rule=path=partial`)
|
||||
**Status ved overlevering:** ingenting implementert. Denne økten leverte kun analyse + denne briefen.
|
||||
|
||||
---
|
||||
|
||||
## 0. Hva denne økten gjorde (så du slipper å gjenta det)
|
||||
|
||||
Fant og leste originalartikkelen bak videoen operatøren limte inn:
|
||||
|
||||
> **«The new rules of context engineering for Claude 5 generation models»**
|
||||
> Thariq Shihipar, Member of Technical Staff, Anthropic — **24. juli 2026**
|
||||
> https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models
|
||||
|
||||
Verifisert direkte mot artikkelen (`WebFetch`, to pass). Tre ting videoen tok feil på — ikke gjenta dem:
|
||||
|
||||
1. Forfatteren heter **Thariq Shihipar**, ikke «Tariq».
|
||||
2. **Artikkelen sier ingenting om tokens, kostnad, caching eller måling.** Videoens token-budsjett-argument er skaperens påbygg. Det er riktig, men det er ikke Anthropics påstand — og det betyr at vi **ikke kan sitere artikkelen som hjemmel** for token-argumenter i registeret (Verifiseringsplikt).
|
||||
3. **Videoens `/doctor`-liste (fem ting) står ikke i artikkelen.** Artikkelen har én setning: *«We rolled out a new command called `claude doctor,` which will help you do this automatically.»* Alt annet i videoen er skaperens observasjon fra sin egen kjøring. **Behandle den som uverifisert.**
|
||||
|
||||
Målt ground truth (CC **2.1.220**, denne maskinen):
|
||||
|
||||
```
|
||||
$ claude doctor --help
|
||||
Check the health of your Claude Code installation. Reads settings files in the
|
||||
current directory without a trust prompt. For a full checkup that can also fix
|
||||
issues, run /doctor in a session.
|
||||
```
|
||||
|
||||
⇒ **CLI-en `claude doctor` = kun install-helse. Det er `/doctor` i sesjon som overlapper oss** («a full checkup that can also fix issues»). Ikke bland dem.
|
||||
|
||||
---
|
||||
|
||||
## 1. OPPGAVE A — mål `/doctor`, og fjern det vi dupliserer
|
||||
|
||||
**Dette er økten sin hovedoppgave, og den har en beslutning i seg som skal tas, ikke utsettes.**
|
||||
|
||||
### Premisset som skal falsifiseres
|
||||
|
||||
Anthropic leverer nå en innebygd, gratis kommando som gjør noe av det config-audit finnes for. Vi vet ikke hvor mye. Vi har aldri målt det. **Hver funksjon i config-audit som `/doctor` gjør like godt eller bedre, skal ut** — ikke omdøpes, ikke «beholdes for kompletthet», ikke pakkes inn i en flagg. Ut.
|
||||
|
||||
Begrunnelsen er pluginens egen doktrine, snudd mot oss selv: `BP-SUB-001` sier at en blokk som ikke lenger tjener sin plass skal vurderes fjernet. En scanner som duplikerer en innebygd kommando tjener ikke sin plass — den koster vedlikehold, tester, byte-stabile baselines og operatørens oppmerksomhet, for et svar hen kan få gratis.
|
||||
|
||||
### Metode (samme mal som de ti dogfood-chunkene — fasit FØR kjøring)
|
||||
|
||||
1. **Skriv `docs/doctor-overlap-fasit.local.md` FØR du kjører noe.** Nummererte prediksjoner: for hver av våre 16 scannere, forutsi om `/doctor` dekker den (JA / DELVIS / NEI) og hvorfor. Eksplisitte avkreftelser: hvilke scannere du er *sikker* på at `/doctor` ikke rører. Forpliktende breddetall: «jeg forventer at N av 16 overlapper». **Aldri rediger fasiten for å matche utfallet** — [[judge-the-judge-build-fasit-first]].
|
||||
2. **Kjør `/doctor` i en sesjon** og fang hele outputen. Merk: `/doctor` kan *fikse* ting — kjør den der en utilsiktet fiks er billig, og les hva den foreslår før du godtar noe.
|
||||
3. **Kjør `claude doctor` (CLI) også**, separat, så de to ikke smelter sammen i notatene.
|
||||
4. **Sammenlign mot vår faktiske inventar** — ikke mot README-ens beskrivelse av den. Kildene er `scanners/*.mjs` (16) og CA-ID-rommet.
|
||||
5. **Premiss-verifiser fasiten mot utfallet**, og rapporter avviket før du handler.
|
||||
|
||||
### Beslutningen som skal tas i denne økten
|
||||
|
||||
For hver scanner/kommando, ett av tre utfall — skrevet ned med begrunnelse:
|
||||
|
||||
| Utfall | Betyr | Handling |
|
||||
|---|---|---|
|
||||
| **FJERNES** | `/doctor` gjør dette like godt eller bedre | Egen v5.14-chunk: slett scanner + tester + CA-ID + README/CLAUDE.md-rader. Frosne baselines må re-seedes bevisst. |
|
||||
| **BEHOLDES, SKJERPES** | Vi gjør noe `/doctor` ikke gjør, men README selger det ikke slik | Omskriv posisjoneringen så forskjellen er eksplisitt |
|
||||
| **BEHOLDES** | Ingen overlapp | Ingen handling |
|
||||
|
||||
### Hypotesen min (skriv din egen fasit først — les denne etterpå)
|
||||
|
||||
Der jeg tror vi faktisk skiller oss, som *bør* overleve målingen:
|
||||
determinisme + byte-stabile baselines, `drift` mot lagret baseline, suppressions med revisjonsspor, backup/rollback, `campaign` på tvers av repo, `plugin-health` + polyrepo-katalog, og det provenansstemplede registeret med `confidence`-nivåer. Der jeg tror overlappet er reelt: deler av `posture`, og deler av `feature-gap`s t1-nivå.
|
||||
|
||||
**Men dette er en hypotese fra en økt som ikke har kjørt `/doctor`. Den er ikke fasit, og den skal ikke få lov til å ankre din.**
|
||||
|
||||
### Utfall som ikke er lov
|
||||
|
||||
- «Vi beholder alt, men dokumenterer forskjellen bedre.» Det er svaret man gir når man ikke vil måle.
|
||||
- Å utsette fjerningen til «senere». Beslutningen tas her; *utførelsen* er en egen chunk (byte-stabilitet + frosne baselines gjør sletting til flerfilsarbeid).
|
||||
|
||||
---
|
||||
|
||||
## 2. OPPGAVE B — oppdater de videre planene
|
||||
|
||||
Når oppgave A har landet en beslutning, skal planverket reflektere den. **Ikke før** — rekkefølgen er poenget, ellers planlegger vi rundt et overlapp vi ikke har målt.
|
||||
|
||||
### B1. Registeret (`knowledge/best-practices.json`) — høyest verdi, lavest risiko
|
||||
|
||||
To ting, og de er forskjellige:
|
||||
|
||||
**(a) Provenans.** Alle `BP-MECH-*`, `BP-SIZE-001` og `BP-SUB-001` siterer «Steering Claude Code»-bloggen. Den nye artikkelen bekrefter og forsterker dem — legg den til som kilde. Særlig `BP-SUB-001`, som nå har nesten ordrett dekning: *«briefly describe what your repo is for, but spend most of the tokens on gotchas inside of the codebase»* og *«Avoid stating 'the obvious' things Claude should know by looking at your file system or your repo»*.
|
||||
|
||||
**(b) En defekt i vår egen ferskhetsgaranti — dette er det viktigste funnet.**
|
||||
`BP-SUB-001` er stemplet `verified: 2026-07-31`. Artikkelen kom **24. juli**. Vi re-verifiserte altså den *gamle* kilden en uke etter at den nye lå ute, og fanget den ikke.
|
||||
|
||||
Årsaken er strukturell: `knowledge-refresh-cli.mjs` gjør kun `assessFreshness` — den **aldrer eksisterende oppføringer etter dato**. Den har ingen måte å uttrykke «en ny kilde har supersedert en gammel». **En oppføring kan derfor være grønn og substansielt foreldet samtidig.** Det er en garanti vi gir som ikke holder.
|
||||
|
||||
Fiksen er en datamodell-endring (`sources[]` og/eller `supersededBy`) + en freshness-regel som ser på kildens alder, ikke bare oppføringens. Dette er den eneste posten her som er nær-ren TDD og har sterk verifikasjon.
|
||||
|
||||
### B2. Ny lens-akse for artikkelens **regel 1** (skjønn over regler)
|
||||
|
||||
`BP-SUB-001` fanger blokker som *gjentar generell ingeniør-atferd*. Artikkelens regel 1 er en **annen defektklasse**: instruksjoner som er lokale og spesifikke, men som **overspesifiserer og burer skjønnet**. Anthropics eget eksempel var ikke redundant — «never write multi-paragraph docstrings, one short line max» er presis og lokal. Den ble slettet fordi den *begrenset* en modell som nå dømmer bedre selv.
|
||||
|
||||
**Viktig, og lett å gjøre feil:** dette kan ikke bli en utvidelse av `--subtract`. `scanners/lib/floor-exclusion.mjs` beskytter «policy invariants» og «local facts» fra å bli slettekandidater — nettopp kategorien artikkelen sier ofte er for stram. Gulvet gjør riktig jobb for `--subtract`; regel 1 trenger sin **egen akse**: *behold innholdet, løsne formuleringen*, ikke *fjern blokka*. Å blande dem er ÅS#5-defektklassen om igjen (to akser presset inn i ett vokabular).
|
||||
|
||||
Foreslått: `BP-JUDG-001` + `CA-OPT-002`, med samme presisjonsgate som `optimization-lens-agent` allerede har (siter regel + kilde, ti stille når usikker).
|
||||
|
||||
### B3. Regel 4 — vi måler feil akse
|
||||
|
||||
Vi har duplikatdeteksjon (`claude-md-linter` 3+ repetisjon, `conflict-detector` hook-duplikater, `CA-TOK-002` permissions). **Alt er innenfor ett lag.** Artikkelens regel 4 handler om samme instruksjon i **to lag** — hos oss: CLAUDE.md *og* en rule, rule *og* en skill-beskrivelse, CLAUDE.md *og* en agent-prompt. Sannsynligvis en utvidelse av `conflict-detector` (den kjenner allerede flere lag), ikke en ny scanner.
|
||||
|
||||
### B4. Regel 2 og 6 — ekte hull, men tynnere
|
||||
|
||||
- **Regel 2** (eksempler → grensesnitt): ingen scanner ser på om en skill/agent lener seg på eksempler der en parameter-enum ville gjort jobben. `skill-listing-scanner` teller tegn, ikke form.
|
||||
- **Regel 6** (rike referanser): *«prefer files that are in code as it provides clear, high-fidelity instructions»*, *«a HTML mockup of a design will generally produce better results than a description or screenshot»*. Vi har ingen oppfatning om referanse-*form*. Som eier av `@`-referanser (`import-resolver.mjs`) er vi det naturlige stedet.
|
||||
|
||||
**Regel 3 og 5 krever ingenting.** Progressiv avdekking *er* `BP-LOAD-001..006` + `BP-MECH-003` + `token-hotspots` + `manifest`; router-mønsteret dekkes av `t2_2`/`t2_3`; auto-memory av `t2_4`. Bekreftet, ikke endret.
|
||||
|
||||
### B5. Skriv om `docs/v5.13-model-routing-effort-deadref-plan.md` (v5.14-planen)
|
||||
|
||||
Den bærer i dag: C-SKL1 (#37), M-BUG-26, M-BUG-28, M-BUG-41 (to armer), arg-sluk-klassens CLI-arm (`optimize-lens-cli` + `token-hotspots-cli`), P6/M-BUG-44 scanner-siden. **Alt dette står fortsatt.** Oppgave A og B1–B4 skal flettes inn og prioriteres mot det — ikke legges oppå som en parallell plan. Detaljene på de åpne postene ligger i `STATE.md`; ikke gjenskap dem her.
|
||||
|
||||
---
|
||||
|
||||
## 3. Rekkefølge
|
||||
|
||||
1. Fasit for `/doctor`-overlappet → kjør → premiss-verifiser → **beslutning per scanner**
|
||||
2. B1 registerfiksen (TDD, sterk verifikasjon — den eneste posten her som har det)
|
||||
3. B5 omskriving av v5.14-planen med A + B1–B4 innflettet
|
||||
4. Oppdater `STATE.md` + `README`/`CLAUDE.md`-posisjonering mot `/doctor`
|
||||
|
||||
**Scope-grense:** ingen sletting av scannere i denne økten. Beslutningen tas og skrives ned; utførelsen er egne chunks, fordi frosne baselines og byte-stabilitet gjør sletting til flerfilsarbeid med egen verifikasjon.
|
||||
|
||||
---
|
||||
|
||||
## 4. Om modellvalget
|
||||
|
||||
Rubrikken ga `Opus 5/high` (rad 3, `rule=path=partial`). Operatøren valgte **Fable 5** for denne økten. Konsekvenser å være klar over:
|
||||
|
||||
- **Ingen advisor.** Fable godtar kun Fable-advisor, og Fable er ikke valgbar som advisor i CC 2.1.220. Fallback-raden (`Sonnet 5/xhigh --advisor opus`) er derfor *ikke* tilgjengelig som billigere utvei i denne økten.
|
||||
- `route-last` for neste økt skal registrere `model=Fable 5; effort=high` + om oppgave A faktisk ble lukket. Det er dataene som avgjør om Fable-radene i rubrikken noen gang blir levende policy.
|
||||
96
docs/v6-quality-plan.md
Normal file
96
docs/v6-quality-plan.md
Normal file
|
|
@ -0,0 +1,96 @@
|
|||
# v6 quality plan — from "green suite" to A+
|
||||
|
||||
**Written 2026-08-12 (session #65), after a gate defect surfaced by accident rather than by
|
||||
process.** This plan is about the *process* that missed it, not about the one defect.
|
||||
|
||||
## 0. Root cause — and why a bigger model is not the fix
|
||||
|
||||
The defect (`fix-engine.mjs` writes files without ever consulting the scope gate) was **already
|
||||
written down**. It sat in `STATE.md`'s ÅPNE POSTER paragraph, mid-sentence, between "P6/M-BUG-44"
|
||||
and "M-BUG-26":
|
||||
|
||||
> *scope-gaten for de FEM andre armene er fortsatt prosa-kontrakt — `fix-engine` leser ikke `gate`*
|
||||
|
||||
It was read at session start and not acted on. **A stronger model reading the same paragraph
|
||||
reaches the same conclusion**, because the paragraph gives it no reason to: an unenforced safety
|
||||
gate is formatted identically to "4 inline copies of a target guard" and "cleanup of invisible
|
||||
session files". The tracking system has **no severity axis**, so nothing in the data says one of
|
||||
these can let a write reach `~/.claude/CLAUDE.md` unapproved and the others cannot.
|
||||
|
||||
That is the finding. Model choice does not fix a missing severity axis.
|
||||
|
||||
**Second occurrence, same class.** #63 found a gate silently not firing because the *command
|
||||
layer* was untested (`--repo <scan-target>` dropped the gate to `silent`, 29 removals, no
|
||||
approval asked). #65 finds a gate not firing because the *engine* never reads it. Two instances
|
||||
of "only prose stood behind a write gate" is a class, not luck ([[defect-found-in-one-file-is-a-class]]).
|
||||
|
||||
## 1. The class, measured
|
||||
|
||||
| Question | Measured 2026-08-12 |
|
||||
|---|---|
|
||||
| Files in `scanners/` that write to disk | **9** |
|
||||
| …that import the scope gate | **1** (`lib/subtraction-write.mjs`) |
|
||||
| Command templates | **21** |
|
||||
| …that name a write action | **17** |
|
||||
| …that invoke `write-scope-cli.mjs` | **5** |
|
||||
|
||||
**The 8 ungated writers are not 8 bugs.** `write-output.mjs`, `backup.mjs`, `baseline.mjs`,
|
||||
`scan-orchestrator.mjs` write plugin-managed artefacts and are legitimately exempt. The defect is
|
||||
that **nothing declares which**: "does this write path need the gate?" is answered by reading
|
||||
code, never by a guard. That is precisely what let `fix-engine` sit unguarded next to
|
||||
`subtraction-write`, which does it right.
|
||||
|
||||
## 2. What A+ means here, concretely
|
||||
|
||||
Not "more care". Three falsifiable properties:
|
||||
|
||||
1. **No invariant is enforced only by prose.** Every contract a command template states about a
|
||||
write, a gate, or a scope is asserted by a test that fails when the code stops honouring it.
|
||||
2. **Every open item carries a severity and a consequence sentence.** "What breaks if this stays
|
||||
open" is written next to it, and anything touching a write, a gate, or user-scope config never
|
||||
lives in the backlog paragraph.
|
||||
3. **Shipped ≠ committed.** Work that is not released is not quality: the machine runs the
|
||||
released plugin, so 29 unreleased commits are 29 fixes nobody has.
|
||||
|
||||
## 3. Chunks, in order
|
||||
|
||||
### Q1 — the gate moves from prose into code (BLOCKS the release)
|
||||
`fix-engine` calls `classifyWriteTarget` + `strongestGate`, exactly as `subtraction-write` already
|
||||
does — share the constant, do not copy it ([[two copies of one table drift]]). Add an **explicit
|
||||
exemption table** naming every plugin-managed writer and *why* it is exempt.
|
||||
**Verify:** a guard that walks `scanners/` for write calls and fails on any writer that neither
|
||||
imports the gate nor appears in the exemption table. Seen RED against today's tree first.
|
||||
|
||||
### Q2 — the command layer gets contract tests
|
||||
The 17 templates that name a write are today verified by nothing. Build the argv **from the
|
||||
template's own text** ([[dogfood-the-command-not-the-cli]]) and assert: the command a template
|
||||
tells the agent to run parses, targets the file the gate classified, and calls the gate before
|
||||
any write.
|
||||
**Verify:** delete the `write-scope-cli` line from one template → its test goes red.
|
||||
|
||||
### Q_AUDIT — one Fable session: find the rest of the class
|
||||
A cross-cutting sweep for other invariants that exist only in prose (agent prompts, command
|
||||
templates, `.claude/rules/`), each rated by what breaks if it silently stops holding. This is
|
||||
review/big-picture work — Fable's documented form strength and a **first choice**, not a fallback.
|
||||
Output: a rated list, not code. A Fable session runs **without advisor**.
|
||||
|
||||
### Q3 — severity axis in the tracking (cheap, rides along)
|
||||
`STATE.md` open items become a table with `severity` + consequence. Rule: safety/write/user-scope
|
||||
items are never in the backlog paragraph. This is the fix for the actual root cause.
|
||||
|
||||
### Q4 — release v6.0.0
|
||||
29 commits, 21 of them `feat`/`fix`, including breaking ID semantics (`7a794b4`). Only after Q1.
|
||||
Gate: `self-audit --check-readme` + full suite + `check-versions.mjs` 0 ERROR.
|
||||
|
||||
### Then B3 (two-layer CNF), as planned.
|
||||
|
||||
## 4. Model routing for this plan
|
||||
|
||||
| Chunk | Model | Why |
|
||||
|---|---|---|
|
||||
| Q1, Q2, Q3 | **Opus 5 / high** | implementation with strong verification (tests fail loudly) |
|
||||
| Q_AUDIT | **Fable 5 / xhigh** | cross-cutting review + planning; deliberate override of the rubric, recorded in STATE as an override, no advisor |
|
||||
| Q4 release | Opus 5 / high | mechanical but one-way (a pushed tag) |
|
||||
|
||||
**Not a blanket model upgrade.** Escalating every session to compensate for a missing guard is the
|
||||
expensive way to not fix the guard.
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"version": 1,
|
||||
"note": "Machine-readable best-practices register. SOURCE OF TRUTH for the optimization lens (v5.7 CA-OPT). Human-readable mirror lives in knowledge/*.md. Every entry is provenance-stamped (source.url + source.verified) and carries a confidence; only CONFIRMED claims are consumed user-facing (Verifiseringsplikt). Curated manually + by /config-audit knowledge-refresh (human-approved). Seeded from docs/v5.5-steering-model-plan.md V-rows + the Anthropic 'Steering Claude Code' blog.",
|
||||
"note": "Machine-readable best-practices register. SOURCE OF TRUTH for the optimization lens (v5.7 CA-OPT). Human-readable mirror lives in knowledge/*.md. Every entry is provenance-stamped (source.url + source.verified; optional corroborating sources[] with published dates feed the evidence-age freshness rule; source.supersededBy marks a source replaced by a newer one) and carries a confidence; only CONFIRMED claims are consumed user-facing (Verifiseringsplikt). Curated manually + by /config-audit knowledge-refresh (human-approved). Seeded from docs/v5.5-steering-model-plan.md V-rows + the Anthropic 'Steering Claude Code' blog.",
|
||||
"entries": [
|
||||
{
|
||||
"id": "BP-MECH-001",
|
||||
|
|
@ -12,7 +12,11 @@
|
|||
"severity": "low",
|
||||
"category": "mechanism-fit",
|
||||
"lensCheck": "claude-md-lifecycle-phrasing",
|
||||
"source": { "url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more", "title": "Steering Claude Code: skills, hooks, rules, subagents and more", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
||||
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-MECH-002",
|
||||
|
|
@ -24,7 +28,11 @@
|
|||
"severity": "low",
|
||||
"category": "mechanism-fit",
|
||||
"lensCheck": "unscoped-path-specific-instruction",
|
||||
"source": { "url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more", "title": "Steering Claude Code: skills, hooks, rules, subagents and more", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
||||
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-MECH-003",
|
||||
|
|
@ -36,7 +44,11 @@
|
|||
"severity": "low",
|
||||
"category": "mechanism-fit",
|
||||
"lensCheck": "procedure-in-claude-md",
|
||||
"source": { "url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more", "title": "Steering Claude Code: skills, hooks, rules, subagents and more", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
||||
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-MECH-004",
|
||||
|
|
@ -48,7 +60,11 @@
|
|||
"severity": "low",
|
||||
"category": "mechanism-fit",
|
||||
"lensCheck": "never-instruction",
|
||||
"source": { "url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more", "title": "Steering Claude Code: skills, hooks, rules, subagents and more", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
||||
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-MECH-005",
|
||||
|
|
@ -60,7 +76,11 @@
|
|||
"severity": "medium",
|
||||
"category": "mechanism-fit",
|
||||
"lensCheck": "CA-OST-001",
|
||||
"source": { "url": "https://code.claude.com/docs/en/output-styles", "title": "Output styles", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/output-styles",
|
||||
"title": "Output styles",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-LOAD-001",
|
||||
|
|
@ -69,7 +89,11 @@
|
|||
"confidence": "confirmed",
|
||||
"category": "loading-model",
|
||||
"lensCheck": null,
|
||||
"source": { "url": "https://code.claude.com/docs/en/context-window", "title": "Context window — what survives compaction", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/context-window",
|
||||
"title": "Context window — what survives compaction",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-LOAD-002",
|
||||
|
|
@ -78,7 +102,11 @@
|
|||
"confidence": "confirmed",
|
||||
"category": "loading-model",
|
||||
"lensCheck": null,
|
||||
"source": { "url": "https://code.claude.com/docs/en/memory", "title": "Memory — path-specific rules", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/memory",
|
||||
"title": "Memory — path-specific rules",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-LOAD-003",
|
||||
|
|
@ -87,7 +115,11 @@
|
|||
"confidence": "confirmed",
|
||||
"category": "loading-model",
|
||||
"lensCheck": null,
|
||||
"source": { "url": "https://code.claude.com/docs/en/context-window", "title": "Context window — what survives compaction", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/context-window",
|
||||
"title": "Context window — what survives compaction",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-LOAD-004",
|
||||
|
|
@ -96,7 +128,11 @@
|
|||
"confidence": "confirmed",
|
||||
"category": "loading-model",
|
||||
"lensCheck": null,
|
||||
"source": { "url": "https://code.claude.com/docs/en/skills", "title": "Skills", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/skills",
|
||||
"title": "Skills",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-LOAD-005",
|
||||
|
|
@ -105,7 +141,11 @@
|
|||
"confidence": "confirmed",
|
||||
"category": "loading-model",
|
||||
"lensCheck": null,
|
||||
"source": { "url": "https://code.claude.com/docs/en/hooks", "title": "Hooks", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/hooks",
|
||||
"title": "Hooks",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-LOAD-006",
|
||||
|
|
@ -114,7 +154,11 @@
|
|||
"confidence": "confirmed",
|
||||
"category": "loading-model",
|
||||
"lensCheck": null,
|
||||
"source": { "url": "https://code.claude.com/docs/en/sub-agents", "title": "Subagents", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/sub-agents",
|
||||
"title": "Subagents",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-SIZE-001",
|
||||
|
|
@ -125,7 +169,11 @@
|
|||
"severity": "medium",
|
||||
"category": "size-budget",
|
||||
"lensCheck": "CA-CML-001",
|
||||
"source": { "url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more", "title": "Steering Claude Code: skills, hooks, rules, subagents and more", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
||||
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-SIZE-002",
|
||||
|
|
@ -135,7 +183,11 @@
|
|||
"severity": "low",
|
||||
"category": "size-budget",
|
||||
"lensCheck": "CA-SKL-002",
|
||||
"source": { "url": "https://code.claude.com/docs/en/skills", "title": "Skills", "verified": "2026-06-20" }
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/skills",
|
||||
"title": "Skills",
|
||||
"verified": "2026-06-20"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-SUB-001",
|
||||
|
|
@ -147,7 +199,104 @@
|
|||
"severity": "low",
|
||||
"category": "subtraction",
|
||||
"lensCheck": "compensatory-instruction",
|
||||
"source": { "url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more", "title": "Steering Claude Code: skills, hooks, rules, subagents and more", "verified": "2026-07-31" }
|
||||
"source": {
|
||||
"url": "https://claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more",
|
||||
"title": "Steering Claude Code: skills, hooks, rules, subagents and more",
|
||||
"verified": "2026-07-31"
|
||||
},
|
||||
"sources": [
|
||||
{
|
||||
"url": "https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models",
|
||||
"title": "The new rules of context engineering for Claude 5 generation models",
|
||||
"published": "2026-07-24",
|
||||
"verified": "2026-08-03",
|
||||
"note": "Near-verbatim coverage: 'briefly describe what your repo is for, but spend most of the tokens on gotchas inside of the codebase'; 'Avoid stating the obvious things Claude should know by looking at your file system or your repo.'"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "BP-MODEL-001",
|
||||
"claim": "A subagent's `model` frontmatter field defaults to `inherit`, so a subagent that names no model runs on the main conversation's model. Routing mechanical or read-only subagents to a cheaper alias (`haiku`, `sonnet`) while the orchestrating session keeps the stronger model is the documented way to control cost. The pin is not absolute: Claude Code resolves the model as CLAUDE_CODE_SUBAGENT_MODEL, then a per-invocation `model` parameter, then the frontmatter, then the main conversation's model.",
|
||||
"mechanism": "model",
|
||||
"appliesTo": "agent",
|
||||
"recommendation": "Set `model:` explicitly on subagents whose work is mechanical or read-only (search, extraction, summarisation) and leave the orchestrator on the stronger model. Omitting the field is not a neutral default — it inherits, so every subagent costs what the session costs.",
|
||||
"confidence": "confirmed",
|
||||
"severity": "low",
|
||||
"category": "model-fit",
|
||||
"lensCheck": null,
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/sub-agents",
|
||||
"title": "Create custom subagents — supported frontmatter fields / choose a model",
|
||||
"verified": "2026-08-10"
|
||||
},
|
||||
"sources": [
|
||||
{
|
||||
"url": "https://claude.com/blog/claude-model-and-effort-level-in-claude-code",
|
||||
"title": "Choosing a Claude model and effort level in Claude Code",
|
||||
"published": "2026-07-07",
|
||||
"verified": "2026-08-10",
|
||||
"note": "Verbatim: 'Pick a smaller model when the work is routine. For example, edits you can describe precisely, mechanical changes, or questions about code that's already in context.'"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "BP-MODEL-002",
|
||||
"claim": "Reasoning effort is an axis separate from model choice: five levels (`low`, `medium`, `high`, `xhigh`, `max`) on current models, four on Opus 4.6 and Sonnet 4.6, which omit `xhigh`; the default is `high` on every model that supports effort except Opus 4.7, which defaults to `xhigh`. Higher is not universally better — `max` \"can improve performance on demanding tasks but may show diminishing returns and is prone to overthinking\". Effort is settable in six places: `/effort`, the slider in `/model`, the `--effort` flag, CLAUDE_CODE_EFFORT_LEVEL, `effortLevel` in settings, and `effort:` in skill or subagent frontmatter; the environment variable takes precedence over all of them.",
|
||||
"mechanism": "effort",
|
||||
"appliesTo": "agent",
|
||||
"recommendation": "Treat effort as a per-task dial rather than a global maximum: pin a lower `effort:` in the frontmatter of mechanical skills and subagents, and reserve `xhigh`/`max` for work whose product is judgment. The scale is calibrated per model, so the same level name is not the same amount of thinking across models — and CLAUDE_CODE_EFFORT_LEVEL silently overrides every other source, so verify which level is actually in force.",
|
||||
"confidence": "confirmed",
|
||||
"severity": "low",
|
||||
"category": "model-fit",
|
||||
"lensCheck": null,
|
||||
"source": {
|
||||
"url": "https://code.claude.com/docs/en/model-config",
|
||||
"title": "Model configuration — adjust effort level / set the effort level",
|
||||
"verified": "2026-08-10"
|
||||
},
|
||||
"sources": [
|
||||
{
|
||||
"url": "https://claude.com/blog/claude-model-and-effort-level-in-claude-code",
|
||||
"title": "Choosing a Claude model and effort level in Claude Code",
|
||||
"published": "2026-07-07",
|
||||
"verified": "2026-08-10",
|
||||
"note": "Verbatim: 'Claude will be more predisposed to double-checking additional hypotheses or verifying correctness at higher effort levels, but it generally won't artificially inflate usage for simple tasks at higher effort levels.'; 'In fact, our team pays close attention to \"overthinking\" during model training as it degrades effectiveness.'"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "BP-JUDG-001",
|
||||
"claim": "An instruction that fixes a form decision — how code or prose should look (comment density, docstring length, naming shape, sentence or paragraph count) — as an absolute rule buys a guardrail current models no longer need, and is wrong for the cases the rule did not anticipate. Anthropic removed its own example from the Claude Code system prompt: \"Never write multi-paragraph docstrings or multi-line comment blocks — one short line max\" was replaced by \"Write code that reads like the surrounding code: match its comment density, naming, and idiom.\"",
|
||||
"appliesTo": "claude-md",
|
||||
"recommendation": "Where an absolute governs a form decision rather than a local fact, state the outcome you want and let the model judge the instance. This does not apply to safety rules, tool or version facts, or a house style you hold deliberately — those are the reason the claim is not machine-checkable.",
|
||||
"confidence": "confirmed",
|
||||
"category": "judgment-fit",
|
||||
"lensCheck": null,
|
||||
"note": "KNOWLEDGE ONLY — no detector, by measurement (docs/b2-judgment-lens-fasit.local.md §9). Across 409 real CLAUDE.md files (38488 lines, 8689 prose blocks) the class fired 7 times and all 7 were false positives; an independent grep in both word orders found 5 lines and 1 line respectively, none an instruction. Where the shape does occur — 45 matching lines across 4755 skill/agent/command files — it is the author's editorial policy (emoji, sentence length, slide titles), which nothing in the text separates from a vendor's over-tight guardrail. Precision-first: no CA-OPT code was allocated and OPT next-free stays 2.",
|
||||
"source": {
|
||||
"url": "https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models",
|
||||
"title": "The new rules of context engineering for Claude 5 generation models",
|
||||
"published": "2026-07-24",
|
||||
"verified": "2026-08-12"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "BP-PROMPT-001",
|
||||
"claim": "Claude Opus 5 catches and fixes its own mistakes without being told to, and over-verifies when explicitly instructed to double-check or to delegate verification to a subagent — this adds token cost without improving output quality. The source states both halves: \"Avoid instructing re-checks it already performs ('double-check your answer,' 're-verify before responding')\" and \"If your prompt contains explicit verification instructions ('include a final verification step for any non-trivial task,' 'use a subagent to verify'), remove them\".",
|
||||
"mechanism": "deletion",
|
||||
"appliesTo": "claude-md",
|
||||
"recommendation": "Remove explicit self-verification or delegate-to-subagent-for-verification instructions when targeting Claude Opus 5; re-add only if the model actually stumbles. The claim is model-scoped, not universal — a config that runs a different model in a different session still needs them.",
|
||||
"confidence": "confirmed",
|
||||
"severity": "low",
|
||||
"category": "prompting-fit",
|
||||
"modelScope": ["opus-5"],
|
||||
"lensCheck": null,
|
||||
"note": "lensCheck deliberately null, same discipline as BP-JUDG-001: this is a knowledge-cited ANNOTATION on an existing BP-SUB-001 candidate (scanners/lib/prompting-model-scope.mjs), gated behind an explicit --for-model flag, not a second competing detector. It never widens the candidate set. Auto-detection is not viable — a CLAUDE.md carries no frontmatter and no statically-resolvable target model. source.published is absent because the page carries no visible publish or last-updated date (re-checked 2026-08-12); both quoted sentences were verified verbatim on that date. MEASURED 2026-08-12 across 409 real CLAUDE.md files: 392 BP-SUB-001 candidates, 31 of them (7.9%) carry a verify verb, and 0 also carry a reflexive or delegated target — so the annotation fires 0 times on this corpus. Two independent raw-text greps (reflexive phrasings; subagent-near-verify in both word orders) also found 0, so the zero is the corpus, not an over-narrow regex. The TARGET requirement is what earns its place: without it the same 31 verb-only blocks — 'sjekk relevante config-filer', 'Type-sjekk: pyright', 'To verify plugin functionality' — would all have been tagged, which is the BP-JUDG-001 failure mode (7/7 false positives) arriving one lens over. Precision on the corpus is 0 wrong out of 31 chances to be wrong; recall is untested there because the class is absent — the detector's true positives are the source doc's own example phrasings, pinned in tests/lib/prompting-model-scope.test.mjs.",
|
||||
"source": {
|
||||
"url": "https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5",
|
||||
"title": "Prompting Claude Opus 5 — Self-correction / Task scope and over-verification",
|
||||
"verified": "2026-08-12"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -65,6 +65,7 @@ export async function scan(_targetPath, _discovery) {
|
|||
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'description-bloat',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Agent description is long (re-sent every turn in the always-loaded listing)',
|
||||
description:
|
||||
|
|
@ -91,6 +92,7 @@ export async function scan(_targetPath, _discovery) {
|
|||
if (aggregate.overBudget) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'aggregate-listing-budget',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Aggregate agent listing may exceed the always-loaded budget',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -157,6 +157,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'volatile-in-prefix',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Volatile content inside cached prefix breaks reuse',
|
||||
description:
|
||||
|
|
@ -199,6 +200,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'volatile-in-import',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Volatile content in @imported file breaks cached prefix',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -23,7 +23,8 @@
|
|||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeFile } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { findArgError } from './lib/cli-args.mjs';
|
||||
import {
|
||||
loadLedger,
|
||||
validateLedger,
|
||||
|
|
@ -32,20 +33,32 @@ import {
|
|||
defaultLedgerPath,
|
||||
} from './lib/campaign-ledger.mjs';
|
||||
|
||||
/**
|
||||
* Usage error. Throws rather than calling process.exit(): exit() discards
|
||||
* unflushed stdout when stdout is a pipe. The top-level catch prints the same
|
||||
* `Error: ` text and sets the same exit code 3, so callers see no difference.
|
||||
*/
|
||||
class CliUsageError extends Error {}
|
||||
|
||||
function fail(message) {
|
||||
process.stderr.write(`Error: ${message}\n`);
|
||||
process.exit(3);
|
||||
throw new CliUsageError(message);
|
||||
}
|
||||
|
||||
/** Flag surface, measured 2026-08-09. The gate runs BEFORE the loop below, so the
|
||||
* loop no longer needs to re-check that a value followed its flag. */
|
||||
const ARG_SPEC = { value: ['--ledger-file', '--output-file'] };
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const argError = findArgError(args, ARG_SPEC);
|
||||
if (argError) fail(argError);
|
||||
let ledgerFile = null;
|
||||
let outputFile = null;
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--ledger-file' && args[i + 1]) ledgerFile = args[++i];
|
||||
else if (a === '--output-file' && args[i + 1]) outputFile = args[++i];
|
||||
if (a === '--ledger-file') ledgerFile = args[++i];
|
||||
else if (a === '--output-file') outputFile = args[++i];
|
||||
}
|
||||
|
||||
const ledgerPath = resolve(ledgerFile || defaultLedgerPath());
|
||||
|
|
@ -96,17 +109,18 @@ async function main() {
|
|||
}
|
||||
|
||||
const json = JSON.stringify(payload, null, 2);
|
||||
if (outputFile) await writeFile(outputFile, json, 'utf-8');
|
||||
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
|
||||
else process.stdout.write(json + '\n');
|
||||
|
||||
process.exit(exitCode);
|
||||
process.exitCode = exitCode;
|
||||
}
|
||||
|
||||
const isDirectRun =
|
||||
process.argv[1] && resolve(process.argv[1]) === resolve(new URL(import.meta.url).pathname);
|
||||
if (isDirectRun) {
|
||||
main().catch((err) => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
const prefix = err instanceof CliUsageError ? 'Error' : 'Fatal';
|
||||
process.stderr.write(`${prefix}: ${err.message}\n`);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -35,18 +35,41 @@
|
|||
import { resolve, join, dirname } from 'node:path';
|
||||
import { homedir } from 'node:os';
|
||||
import { readFile, writeFile, mkdir } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import {
|
||||
loadLedger,
|
||||
validateLedger,
|
||||
defaultLedgerPath,
|
||||
} from './lib/campaign-ledger.mjs';
|
||||
import { planExportPath, buildPlanExportDocument } from './lib/campaign-export.mjs';
|
||||
import { evaluateWriteTargets } from './lib/write-scope.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
|
||||
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
|
||||
|
||||
/**
|
||||
* Flag surface. This was the last hand-rolled parser of the fifteen CLIs the
|
||||
* command layer calls, and the only one that misclassified its own argv: the
|
||||
* chain below guards each value flag with `argv[i + 1] !== undefined`, so a
|
||||
* trailing `--repo` fell past every branch to the `startsWith('--')` catch-all
|
||||
* and was reported as an **unknown flag** — about the one flag this CLI
|
||||
* requires. The shared gate runs first and names the real fault; valid argv
|
||||
* reaches the loop below byte-for-byte unchanged.
|
||||
*/
|
||||
const ARG_SPEC = {
|
||||
boolean: ['--write', '--approve-scope'],
|
||||
value: ['--repo', '--ledger-file', '--sessions-dir', '--reference-date', '--output-file', '--session-root'],
|
||||
};
|
||||
|
||||
/**
|
||||
* Usage error. Throws rather than calling process.exit(): exit() discards
|
||||
* unflushed stdout when stdout is a pipe. The top-level catch prints the same
|
||||
* `Error: ` text and sets the same exit code 3, so callers see no difference.
|
||||
*/
|
||||
class CliUsageError extends Error {}
|
||||
|
||||
function fail(message) {
|
||||
process.stderr.write(`Error: ${message}\n`);
|
||||
process.exit(3);
|
||||
throw new CliUsageError(message);
|
||||
}
|
||||
|
||||
/** Default session store: next to the ledger, OUTSIDE the plugin dir. */
|
||||
|
|
@ -55,7 +78,7 @@ function defaultSessionsDir() {
|
|||
}
|
||||
|
||||
function parseArgs(argv) {
|
||||
const flags = { repo: null, ledgerFile: null, sessionsDir: null, referenceDate: null, outputFile: null, write: false };
|
||||
const flags = { repo: null, ledgerFile: null, sessionsDir: null, referenceDate: null, outputFile: null, write: false, approveScope: false, sessionRoot: null };
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
if (a === '--repo' && argv[i + 1] !== undefined) flags.repo = argv[++i];
|
||||
|
|
@ -64,6 +87,8 @@ function parseArgs(argv) {
|
|||
else if (a === '--reference-date' && argv[i + 1] !== undefined) flags.referenceDate = argv[++i];
|
||||
else if (a === '--output-file' && argv[i + 1] !== undefined) flags.outputFile = argv[++i];
|
||||
else if (a === '--write') flags.write = true;
|
||||
else if (a === '--approve-scope') flags.approveScope = true;
|
||||
else if (a === '--session-root' && argv[i + 1] !== undefined) flags.sessionRoot = argv[++i];
|
||||
else if (a.startsWith('--')) fail(`unknown flag "${a}"`);
|
||||
else fail(`unexpected argument "${a}"`);
|
||||
}
|
||||
|
|
@ -72,13 +97,16 @@ function parseArgs(argv) {
|
|||
|
||||
async function emit(payload, outputFile, exitCode) {
|
||||
const json = JSON.stringify(payload, null, 2);
|
||||
if (outputFile) await writeFile(outputFile, json, 'utf-8');
|
||||
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
|
||||
else process.stdout.write(json + '\n');
|
||||
process.exit(exitCode);
|
||||
process.exitCode = exitCode;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const flags = parseArgs(process.argv.slice(2));
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
|
||||
const flags = parseArgs(args);
|
||||
if (!flags.repo) fail('--repo <path> is required');
|
||||
if (flags.referenceDate && !DATE_RE.test(flags.referenceDate)) fail('--reference-date must be YYYY-MM-DD');
|
||||
|
||||
|
|
@ -140,6 +168,32 @@ async function main() {
|
|||
now,
|
||||
});
|
||||
|
||||
// Q1 — the scope gate, in code rather than in commands/campaign.md's prose.
|
||||
//
|
||||
// `--repo` here names the repo being EXPORTED TO, which is the write target's
|
||||
// repo, not the session's. The session root is where the operator stands, so
|
||||
// it comes from `--session-root` (default cwd) — reading it off `--repo`
|
||||
// would make every export look "in-repo" and silence the gate by
|
||||
// construction (#63).
|
||||
//
|
||||
// Export into another project is `cross-repo`, whose gate is `disclose`, NOT
|
||||
// `require-ok`: campaign export is cross-repo BY DESIGN, and tightening it
|
||||
// into a refusal breaks the feature. So the disclosure always rides in the
|
||||
// payload, and only a `require-ok` class (machine-wide config, or a path in
|
||||
// no project at all) actually withholds the write.
|
||||
const scope = evaluateWriteTargets([targetPath], resolve(flags.sessionRoot ?? process.cwd()));
|
||||
|
||||
if (flags.write && scope.requiresApproval && !flags.approveScope) {
|
||||
return emit(
|
||||
{ status: 'refused', action: 'export', reason: 'scope-gate', repo: repoInfo,
|
||||
sessionId: repo.sessionId, sourcePlanPath, exportable: true, problems: [],
|
||||
written: false, targetPath, gate: scope.gate, requiresApproval: true,
|
||||
disclosures: scope.disclosures },
|
||||
flags.outputFile,
|
||||
0,
|
||||
);
|
||||
}
|
||||
|
||||
let written = false;
|
||||
if (flags.write) {
|
||||
await mkdir(dirname(targetPath), { recursive: true });
|
||||
|
|
@ -149,7 +203,9 @@ async function main() {
|
|||
|
||||
return emit(
|
||||
{ status: 'ok', action: 'export', repo: repoInfo, sessionId: repo.sessionId, sourcePlanPath,
|
||||
exportable: true, problems: [], written, targetPath, document },
|
||||
exportable: true, problems: [], written, targetPath, document,
|
||||
gate: scope.gate, requiresApproval: scope.requiresApproval,
|
||||
disclosures: scope.disclosures },
|
||||
flags.outputFile,
|
||||
0,
|
||||
);
|
||||
|
|
@ -159,7 +215,8 @@ const isDirectRun =
|
|||
process.argv[1] && resolve(process.argv[1]) === resolve(new URL(import.meta.url).pathname);
|
||||
if (isDirectRun) {
|
||||
main().catch((err) => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
const prefix = err instanceof CliUsageError ? 'Error' : 'Fatal';
|
||||
process.stderr.write(`${prefix}: ${err.message}\n`);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -43,7 +43,8 @@
|
|||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeFile } from 'node:fs/promises';
|
||||
import { stat } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import {
|
||||
createLedger,
|
||||
addRepo,
|
||||
|
|
@ -57,32 +58,63 @@ import {
|
|||
} from './lib/campaign-ledger.mjs';
|
||||
import { readActiveConfig } from './lib/active-config-reader.mjs';
|
||||
import { buildManifest, splitManifestByOwnership } from './manifest.mjs';
|
||||
import { findArgError } from './lib/cli-args.mjs';
|
||||
|
||||
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
|
||||
|
||||
/**
|
||||
* Usage error. Throws rather than calling process.exit(): exit() discards
|
||||
* unflushed stdout when stdout is a pipe. The top-level catch prints the same
|
||||
* `Error: ` text and sets the same exit code 3, so callers see no difference.
|
||||
*/
|
||||
class CliUsageError extends Error {}
|
||||
|
||||
function fail(message) {
|
||||
process.stderr.write(`Error: ${message}\n`);
|
||||
process.exit(3);
|
||||
throw new CliUsageError(message);
|
||||
}
|
||||
|
||||
/** Flag surface, measured 2026-08-09. The gate runs BEFORE the loop below, so the
|
||||
* loop no longer needs to re-check that a value followed its flag. */
|
||||
const ARG_SPEC = {
|
||||
value: ['--ledger-file', '--reference-date', '--output-file', '--name', '--findings', '--session'],
|
||||
};
|
||||
|
||||
/** Parse argv into a subcommand, positional args, and the flag map. */
|
||||
function parseArgs(argv) {
|
||||
const argError = findArgError(argv, ARG_SPEC);
|
||||
if (argError) fail(argError);
|
||||
const positionals = [];
|
||||
const flags = { ledgerFile: null, referenceDate: null, outputFile: null, name: null, findings: null, session: null };
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
if (a === '--ledger-file' && argv[i + 1] !== undefined) flags.ledgerFile = argv[++i];
|
||||
else if (a === '--reference-date' && argv[i + 1] !== undefined) flags.referenceDate = argv[++i];
|
||||
else if (a === '--output-file' && argv[i + 1] !== undefined) flags.outputFile = argv[++i];
|
||||
else if (a === '--name' && argv[i + 1] !== undefined) flags.name = argv[++i];
|
||||
else if (a === '--findings' && argv[i + 1] !== undefined) flags.findings = argv[++i];
|
||||
else if (a === '--session' && argv[i + 1] !== undefined) flags.session = argv[++i];
|
||||
else if (a.startsWith('--')) fail(`unknown flag "${a}"`);
|
||||
if (a === '--ledger-file') flags.ledgerFile = argv[++i];
|
||||
else if (a === '--reference-date') flags.referenceDate = argv[++i];
|
||||
else if (a === '--output-file') flags.outputFile = argv[++i];
|
||||
else if (a === '--name') flags.name = argv[++i];
|
||||
else if (a === '--findings') flags.findings = argv[++i];
|
||||
else if (a === '--session') flags.session = argv[++i];
|
||||
else positionals.push(a);
|
||||
}
|
||||
return { subcommand: positionals[0], rest: positionals.slice(1), flags };
|
||||
}
|
||||
|
||||
/**
|
||||
* Can this path actually be read as a repo right now?
|
||||
*
|
||||
* `readActiveConfig` resolves any string and its sub-readers all tolerate ENOENT, so a repo
|
||||
* that does not exist yields an EMPTY config instead of an error. Without this check the
|
||||
* sweep records a phantom repo as successfully swept with 0 tokens, and the machine-wide
|
||||
* bill claims coverage it does not have. A missing path is reported, never rejected — an
|
||||
* unmounted volume is a legitimate reason for a tracked repo to be absent today.
|
||||
*/
|
||||
async function isReadableRepoDir(path) {
|
||||
try {
|
||||
return (await stat(path)).isDirectory();
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/** Load an existing ledger, treating a parse error as a hard failure (never clobber corrupt data). */
|
||||
async function loadOrFail(path) {
|
||||
try {
|
||||
|
|
@ -94,9 +126,9 @@ async function loadOrFail(path) {
|
|||
|
||||
async function emit(payload, outputFile, exitCode) {
|
||||
const json = JSON.stringify(payload, null, 2);
|
||||
if (outputFile) await writeFile(outputFile, json, 'utf-8');
|
||||
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
|
||||
else process.stdout.write(json + '\n');
|
||||
process.exit(exitCode);
|
||||
process.exitCode = exitCode;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
|
|
@ -140,6 +172,7 @@ async function main() {
|
|||
let ledger = loaded === null ? createLedger({ now }) : loaded;
|
||||
|
||||
const added = [];
|
||||
const addedUnverified = [];
|
||||
const skipped = [];
|
||||
for (const p of paths) {
|
||||
const resolved = resolve(p);
|
||||
|
|
@ -147,13 +180,17 @@ async function main() {
|
|||
// --name applies only to a lone path; multi-add lets the lib derive each basename.
|
||||
const name = paths.length === 1 ? flags.name || undefined : undefined;
|
||||
ledger = addRepo(ledger, { path: p, name }, { now });
|
||||
(present ? skipped : added).push(resolved);
|
||||
if (present) skipped.push(resolved);
|
||||
else if (await isReadableRepoDir(resolved)) added.push(resolved);
|
||||
// Tracked either way, but never silently vouched for: the command reports these
|
||||
// separately so a typo does not become a permanent phantom row in the backlog.
|
||||
else addedUnverified.push(resolved);
|
||||
}
|
||||
await saveLedger(ledgerPath, ledger);
|
||||
return emit(
|
||||
{
|
||||
status: 'ok', action: 'add', written: true, autoInitialized, ledgerPath,
|
||||
added, skipped, repos: ledger.repos, rollUp: rollUp(ledger),
|
||||
added, addedUnverified, skipped, repos: ledger.repos, rollUp: rollUp(ledger),
|
||||
},
|
||||
flags.outputFile,
|
||||
0,
|
||||
|
|
@ -210,6 +247,13 @@ async function main() {
|
|||
let sharedSummary = null;
|
||||
|
||||
for (const repo of ledger0.repos) {
|
||||
// Check readability FIRST: readActiveConfig returns an empty config for a path that
|
||||
// does not exist rather than throwing, so the catch below would never see it and the
|
||||
// repo would be recorded as swept with a 0-token delta — a bill that looks complete.
|
||||
if (!(await isReadableRepoDir(repo.path))) {
|
||||
skipped.push({ path: repo.path, reason: 'repo path is not readable (does not exist or is not a directory)' });
|
||||
continue;
|
||||
}
|
||||
let split;
|
||||
try {
|
||||
const activeConfig = await readActiveConfig(repo.path, { verbose: false });
|
||||
|
|
@ -246,7 +290,8 @@ const isDirectRun =
|
|||
process.argv[1] && resolve(process.argv[1]) === resolve(new URL(import.meta.url).pathname);
|
||||
if (isDirectRun) {
|
||||
main().catch((err) => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
const prefix = err instanceof CliUsageError ? 'Error' : 'Fatal';
|
||||
process.stderr.write(`${prefix}: ${err.message}\n`);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -5,12 +5,13 @@
|
|||
*/
|
||||
|
||||
import { readTextFile } from './lib/file-discovery.mjs';
|
||||
import { finding, scannerResult, resetCounter } from './lib/output.mjs';
|
||||
import { finding, scannerResult } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
import { parseFrontmatter, extractSections, findImports } from './lib/yaml-parser.mjs';
|
||||
import { lineCount, truncate } from './lib/string-utils.mjs';
|
||||
import { CONTEXT_WINDOW_ANCHOR, LARGE_CONTEXT_WINDOW, LARGE_CONTEXT_SCALE, scaleForWindow, withCommas } from './lib/context-window.mjs';
|
||||
import { dirname } from 'node:path';
|
||||
import { dirname, resolve as resolvePath, sep } from 'node:path';
|
||||
import { stat } from 'node:fs/promises';
|
||||
|
||||
const SCANNER = 'CML';
|
||||
const MAX_RECOMMENDED_LINES = 200;
|
||||
|
|
@ -30,6 +31,134 @@ const CHAR_BUDGET_RECOMMENDATION =
|
|||
const CLAUDE_MD_CHAR_WARN_ANCHOR = 40_000; // chars @ 200k context (CC startup warning)
|
||||
const CLAUDE_MD_CHAR_WARN_LARGE = CLAUDE_MD_CHAR_WARN_ANCHOR * LARGE_CONTEXT_SCALE; // 200,000 @ 1M
|
||||
|
||||
// ── C3: dead prose references ───────────────────────────────────────────────
|
||||
// `import-resolver` resolves @import targets; a path written in prose is not
|
||||
// checked by anything. The whole design here is the SILENCE taxonomy — a
|
||||
// precision-first check whose failure mode must be a miss, never a false alarm.
|
||||
// Each rule below was measured against 407 real CLAUDE.md files, not reasoned
|
||||
// about; the numbers live in docs/c3-deadref-fasit.local.md §2.
|
||||
const KNOWN_EXTENSIONS = /\.(?:md|mjs|js|ts|tsx|jsx|json|ya?ml|sh|py|toml|txt|html|css)$/i;
|
||||
|
||||
// How many dead references the evidence names before it summarises the rest.
|
||||
const MAX_LISTED_DEAD_REFS = 5;
|
||||
|
||||
/**
|
||||
* Inline-code spans that sit in prose, i.e. outside fenced code blocks.
|
||||
* Fenced code is illustrative — a dead path in a `bash` sample is a sample,
|
||||
* not a reference (silence class S1).
|
||||
* @param {string} content
|
||||
* @returns {Array<{text: string, line: number}>}
|
||||
*/
|
||||
export function extractInlineSpans(content) {
|
||||
const spans = [];
|
||||
const lines = String(content == null ? '' : content).split('\n');
|
||||
let inFence = false;
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const raw = lines[i];
|
||||
if (/^\s*(?:```|~~~)/.test(raw)) {
|
||||
inFence = !inFence;
|
||||
continue;
|
||||
}
|
||||
if (inFence) continue;
|
||||
|
||||
const re = /`([^`\n]+)`/g;
|
||||
let m;
|
||||
while ((m = re.exec(raw)) !== null) {
|
||||
const text = m[1].trim();
|
||||
if (text) spans.push({ text, line: i + 1 });
|
||||
}
|
||||
}
|
||||
return spans;
|
||||
}
|
||||
|
||||
/**
|
||||
* Lexical half of the taxonomy: is this token even a path reference?
|
||||
* Order is load-bearing — the FIRST matching rule is the reported reason, so
|
||||
* `npm test` is silenced as `whitespace` (a command) rather than as
|
||||
* `no-separator`, and the taxonomy keeps describing what actually happened.
|
||||
*
|
||||
* @param {string} token - the text inside one backtick span
|
||||
* @returns {{rule: string|null}} rule name, or null when the token is a
|
||||
* candidate that still needs resolving against the filesystem
|
||||
*/
|
||||
export function classifyProseReference(token) {
|
||||
const t = String(token == null ? '' : token);
|
||||
|
||||
// S2 — a command invocation, not a path.
|
||||
if (/\s/.test(t)) return { rule: 'whitespace' };
|
||||
// S3 — an external resource; on-disk existence is meaningless.
|
||||
if (/^[a-z][a-z0-9+.-]*:\/\//i.test(t) || /^(?:www\.|mailto:)/i.test(t)) return { rule: 'url' };
|
||||
// S4 — a pattern or template: resolves to many, or to nothing until expanded.
|
||||
if (/[*?[\]{}<>$]/.test(t)) return { rule: 'glob-or-placeholder' };
|
||||
// S5 — outside project scope, and machine-dependent.
|
||||
if (t.startsWith('/') || t.startsWith('~')) return { rule: 'absolute-or-home' };
|
||||
// S6 — a config key or a CLI flag.
|
||||
if (t.endsWith(':') || t.startsWith('-')) return { rule: 'key-or-flag' };
|
||||
// S7 — a bare filename in prose is a concept or a tool name, not a reference.
|
||||
// Measured: admitting bare names triples the output, and its top entries are
|
||||
// name-drops of tools that exist elsewhere on the machine.
|
||||
if (!t.includes('/')) return { rule: 'no-separator' };
|
||||
// S8 — has a separator but no unambiguous path shape: org/repo slugs, npm
|
||||
// packages, pytest node ids, prose enumerations. Also swallows S10, a path
|
||||
// carrying a trailing `:54-56` locator — a known v1 gap, and a miss rather
|
||||
// than a false alarm.
|
||||
if (!t.endsWith('/') && !KNOWN_EXTENSIONS.test(t)) return { rule: 'ambiguous-slug' };
|
||||
// S8b — a BARE folder name is a concept one level up from a bare filename,
|
||||
// and the same D-A reasoning applies. Measured on the same corpus: 183 of 699
|
||||
// fires (26 %) are single-segment directory tokens, led by `open/` (39x, a
|
||||
// remote namespace prefix) and generic names — `tests/`, `src/`, `docs/`,
|
||||
// `scripts/` — that prose almost always MENTIONS rather than references. A
|
||||
// specific path like `tools/wiki_ingest/` still qualifies.
|
||||
if (t.endsWith('/') && t.replace(/^\.\//, '').split('/').filter(Boolean).length === 1) {
|
||||
return { rule: 'single-segment-directory' };
|
||||
}
|
||||
|
||||
return { rule: null };
|
||||
}
|
||||
|
||||
/** @returns {Promise<boolean>} */
|
||||
async function pathExists(p) {
|
||||
try {
|
||||
await stat(p);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/** Is `p` the root itself or below it? */
|
||||
function isInside(p, root) {
|
||||
return p === root || p.startsWith(root.endsWith(sep) ? root : root + sep);
|
||||
}
|
||||
|
||||
/**
|
||||
* Filesystem half of the taxonomy. Two bases, because a nested CLAUDE.md
|
||||
* routinely writes repo-root-relative paths.
|
||||
*
|
||||
* Containment is checked against the SCAN ROOT, not the file's own directory:
|
||||
* a legitimate `../docs/x.md` inside the same repo must still resolve, while a
|
||||
* `..` chain that leaves the tree must not. Measured: without this,
|
||||
* `../../../../etc/passwd` resolved to the real /etc/passwd and silenced the
|
||||
* finding by accident. A base a `..` chain can escape is not a base.
|
||||
*
|
||||
* @param {string} token
|
||||
* @param {{fileDir: string, scanRoot: string}} bases
|
||||
* @returns {Promise<{rule: string|null}>} null means the reference is dead
|
||||
*/
|
||||
export async function resolveProseReference(token, { fileDir, scanRoot }) {
|
||||
const root = resolvePath(scanRoot);
|
||||
const ownAbs = resolvePath(fileDir, token);
|
||||
|
||||
if (!isInside(ownAbs, root)) return { rule: 'outside-scan-tree' };
|
||||
if (await pathExists(ownAbs)) return { rule: 'resolves-own-dir' };
|
||||
|
||||
const rootAbs = resolvePath(root, token);
|
||||
if (isInside(rootAbs, root) && await pathExists(rootAbs)) return { rule: 'resolves-scan-root' };
|
||||
|
||||
return { rule: null };
|
||||
}
|
||||
|
||||
/** Recommended sections for a project CLAUDE.md */
|
||||
const RECOMMENDED_SECTIONS = [
|
||||
{ pattern: /project|overview|description|what/i, label: 'Project overview' },
|
||||
|
|
@ -62,6 +191,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
return scannerResult(SCANNER, 'ok', [
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-claude-md',
|
||||
severity: SEVERITY.high,
|
||||
title: 'No CLAUDE.md found',
|
||||
description: 'No CLAUDE.md files were discovered. This is the primary configuration surface for Claude Code.',
|
||||
|
|
@ -91,6 +221,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (file.scope === 'project' && relDir !== '.' && relDir !== '.claude' && lines > 5) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'nested-not-reinjected',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Nested CLAUDE.md is not re-injected after compaction',
|
||||
description: `${file.relPath} is a nested (subdirectory) CLAUDE.md. It loads when Claude reads a file in that directory, but after a context compaction it is not re-injected (only the project-root CLAUDE.md is) — its instructions silently drop until a file in that directory is read again.`,
|
||||
|
|
@ -109,6 +240,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (lines > MAX_ABSOLUTE_LINES) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'over-500-lines',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md exceeds 500 lines',
|
||||
description: `${file.relPath} has ${lines} lines. A file this size loads in full on every turn (token cost) and, on smaller-context models, can crowd out instructions. Large-context models tolerate longer files when the cache prefix stays stable — raw line count is no longer an absolute adherence threshold (CC 2.1.169 scales it by context window).`,
|
||||
|
|
@ -120,6 +252,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
} else if (lines > MAX_RECOMMENDED_LINES) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'over-200-lines',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md exceeds recommended 200 lines',
|
||||
description: `${file.relPath} has ${lines} lines. Under ~200 lines is the safe default across models; larger is fine on large-context models when the cache prefix stays stable. A long file still costs tokens every turn.`,
|
||||
|
|
@ -142,6 +275,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
// Conservative 200k anchor — byte-identical to the pre-B8 finding.
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'over-char-budget',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md exceeds Claude Code\'s performance-warning threshold',
|
||||
description: `${file.relPath} is ${withCommas(chars)} chars. Claude Code shows a startup warning ("Large CLAUDE.md will impact performance ... chars > 40.0k") once a CLAUDE.md passes ~40.0k chars on a 200k-context model — it loads in full on every turn. CC 2.1.169 scales that threshold with the context window, so on a ${withCommas(LARGE_CONTEXT_WINDOW)}-token model it relaxes to ~${withCommas(CLAUDE_MD_CHAR_WARN_LARGE)} chars and you are likely within it.`,
|
||||
|
|
@ -156,6 +290,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
const threshLabel = withCommas(charThreshold);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'over-char-budget',
|
||||
severity: advisory ? SEVERITY.info : SEVERITY.medium,
|
||||
title: 'CLAUDE.md exceeds Claude Code\'s performance-warning threshold',
|
||||
description: `${file.relPath} is ${withCommas(chars)} chars, over the ~${threshLabel}-char performance-warning threshold Claude Code applies at a ${winLabel}-token context window (it scales the ~40.0k-char @ 200k warning by the context window, CC 2.1.169) — it loads in full on every turn.` +
|
||||
|
|
@ -172,6 +307,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (lines < 3) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'nearly-empty',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md is nearly empty',
|
||||
description: `${file.relPath} has only ${lines} lines.`,
|
||||
|
|
@ -197,6 +333,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (missingSections.length > 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-sections',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Missing recommended sections',
|
||||
description: `${file.relPath} is missing: ${missingSections.join(', ')}`,
|
||||
|
|
@ -212,6 +349,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (sections.length === 0 && lines > 10) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-headings',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md has no markdown headings',
|
||||
description: `${file.relPath} has ${lines} lines but no ## headings. Structured content with headers improves Claude's ability to find and follow instructions.`,
|
||||
|
|
@ -228,6 +366,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (imp.path.includes('..') && imp.path.split('..').length > 3) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deep-relative-import',
|
||||
severity: SEVERITY.low,
|
||||
title: '@import with deep relative path',
|
||||
description: `${file.relPath}:${imp.line} imports "${truncate(imp.path, 60)}" with multiple parent traversals.`,
|
||||
|
|
@ -245,6 +384,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (htmlComments > 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'html-comments',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Uses HTML comments',
|
||||
description: `${file.relPath} uses ${htmlComments} HTML comment(s). These are stripped before injection, saving tokens.`,
|
||||
|
|
@ -266,6 +406,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (duplicates.length > 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'repeated-content',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Repeated content detected',
|
||||
description: `${file.relPath} has ${duplicates.length} line(s) repeated 3+ times.`,
|
||||
|
|
@ -281,6 +422,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (todos.length > 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'todo-markers',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Contains TODO/FIXME markers',
|
||||
description: `${file.relPath} has ${todos.length} TODO/FIXME/HACK marker(s).`,
|
||||
|
|
@ -288,6 +430,40 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
evidence: truncate(todos[0].trim(), 80),
|
||||
}));
|
||||
}
|
||||
|
||||
// --- Dead prose references (C3) ---
|
||||
// One finding per FILE, matching the idiom of the two checks above: a
|
||||
// machine-wide scan measured 699 dead references across 128 files, and
|
||||
// per-token emission would bury the file that has ten of them.
|
||||
const deadRefs = [];
|
||||
for (const span of extractInlineSpans(content)) {
|
||||
if (classifyProseReference(span.text).rule !== null) continue;
|
||||
const { rule } = await resolveProseReference(span.text, {
|
||||
fileDir: dirname(file.absPath),
|
||||
scanRoot: targetPath,
|
||||
});
|
||||
if (rule === null) deadRefs.push(span);
|
||||
}
|
||||
|
||||
if (deadRefs.length > 0) {
|
||||
const listed = deadRefs
|
||||
.slice(0, MAX_LISTED_DEAD_REFS)
|
||||
.map(r => `${r.text} (line ${r.line})`)
|
||||
.join(', ');
|
||||
const rest = deadRefs.length - Math.min(deadRefs.length, MAX_LISTED_DEAD_REFS);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'dead-prose-reference',
|
||||
severity: SEVERITY.low,
|
||||
title: 'CLAUDE.md points at files that are not there',
|
||||
description: `${file.relPath} has ${deadRefs.length} backtick-quoted path reference(s) in prose that resolve to nothing — neither next to the file nor from the scan root. Anyone following them, human or Claude, finds nothing.`,
|
||||
file: file.absPath,
|
||||
line: deadRefs[0].line,
|
||||
evidence: `${listed}${rest > 0 ? `, +${rest} more` : ''}`,
|
||||
recommendation: 'Point each reference at where the file actually lives, or drop it. Only unambiguous relative paths are checked — URLs, globs, absolute paths and bare filenames are left alone.',
|
||||
autoFixable: false,
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
||||
return scannerResult(SCANNER, 'ok', findings, filesScanned, Date.now() - start);
|
||||
|
|
|
|||
|
|
@ -74,6 +74,7 @@ export async function scan(_targetPath, _discovery) {
|
|||
];
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'skill-user-vs-plugin',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Skill name "${name}" collides between user-level and plugin sources`,
|
||||
description:
|
||||
|
|
@ -97,6 +98,7 @@ export async function scan(_targetPath, _discovery) {
|
|||
const pluginNames = pluginSkills.map(s => s.pluginName);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'skill-multi-plugin',
|
||||
severity: SEVERITY.low,
|
||||
title: `Skill name "${name}" used by multiple plugins`,
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -129,6 +129,7 @@ export async function scan(targetPath, discovery) {
|
|||
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'settings-key-conflict',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Settings key conflict: "${key}"`,
|
||||
description: `Key "${key}" has different values across scopes. ${details}`,
|
||||
|
|
@ -160,6 +161,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (rulesIntersect(allowRule, denyRule)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'permission-allow-deny',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Permission allow/deny conflict',
|
||||
description: `"${allowRule}" is allowed in ${a.scope} (${a.file}) but denied in ${b.scope} (${b.file}).`,
|
||||
|
|
@ -177,6 +179,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (rulesIntersect(allowRule, denyRule)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'permission-allow-deny',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Permission allow/deny conflict',
|
||||
description: `"${allowRule}" is allowed in ${b.scope} (${b.file}) but denied in ${a.scope} (${a.file}).`,
|
||||
|
|
@ -227,6 +230,7 @@ export async function scan(targetPath, discovery) {
|
|||
const [event, matcher] = key.split(':');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'duplicate-hook',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Duplicate hook definition',
|
||||
description: `Hook "${event}" with matcher "${matcher}" is defined in ${uniqueSources.length} sources.`,
|
||||
|
|
|
|||
|
|
@ -113,6 +113,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deny-and-allow',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Tool listed in both permissions.deny and permissions.allow',
|
||||
description:
|
||||
|
|
@ -134,6 +135,7 @@ export async function scan(targetPath, discovery) {
|
|||
const evidence = `allow: ${ineffective.slice(0, 5).map(e => `"${e}"`).join(', ')}`;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'ineffective-allow-wildcard',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Ineffective allow wildcard — Claude Code ignores this rule',
|
||||
description:
|
||||
|
|
@ -160,6 +162,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'forbidden-param-deny',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Permission rule silently ignored — deny/ask uses a forbidden param key',
|
||||
description:
|
||||
|
|
@ -184,6 +187,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'forbidden-param-allow',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Permission rule silently ignored — allow uses a forbidden param key (dead config)',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -5,16 +5,22 @@
|
|||
* Compare current configuration against a saved baseline.
|
||||
* Usage:
|
||||
* node drift-cli.mjs <path> --save [--name my-baseline]
|
||||
* node drift-cli.mjs <path> [--baseline my-baseline] [--json]
|
||||
* node drift-cli.mjs <path> [--baseline my-baseline] [--json] [--output-file path]
|
||||
* node drift-cli.mjs --list
|
||||
* Unknown options and value-less --name/--baseline/--output-file exit 3.
|
||||
* Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireTargetDir } from './lib/require-target-dir.mjs';
|
||||
import { runAllScanners } from './scan-orchestrator.mjs';
|
||||
import { diffEnvelopes, formatDiffReport } from './lib/diff-engine.mjs';
|
||||
import { saveBaseline, loadBaseline, listBaselines } from './lib/baseline.mjs';
|
||||
import { humanizeFindings } from './lib/humanizer.mjs';
|
||||
import { humanizeFindings, humanizeFinding } from './lib/humanizer.mjs';
|
||||
|
||||
const BOOL_FLAGS = ['--save', '--list', '--json', '--raw', '--global'];
|
||||
const VALUE_FLAGS = ['--name', '--baseline', '--output-file'];
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
|
|
@ -25,30 +31,50 @@ async function main() {
|
|||
let jsonMode = false;
|
||||
let rawMode = false;
|
||||
let includeGlobal = false;
|
||||
let outputFile = null;
|
||||
|
||||
// M-BUG-21: this loop used to end in `else if (!arg.startsWith('-')) targetPath = arg`,
|
||||
// with no unknown-flag branch. An unrecognised flag was dropped silently and its
|
||||
// VALUE fell through to targetPath — so `--output-file /tmp/x.json` scanned
|
||||
// /tmp/x.json, a path that does not exist, yielding a near-empty scan and
|
||||
// therefore permanent phantom drift. A missing value for --name was equally
|
||||
// silent and destructive: it left baselineName at 'default' and OVERWROTE the
|
||||
// default baseline. Both now fail loudly (exit 3) instead.
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--save') {
|
||||
save = true;
|
||||
} else if (args[i] === '--name' && args[i + 1]) {
|
||||
baselineName = args[++i];
|
||||
} else if (args[i] === '--baseline' && args[i + 1]) {
|
||||
baselineName = args[++i];
|
||||
} else if (args[i] === '--list') {
|
||||
list = true;
|
||||
} else if (args[i] === '--json') {
|
||||
jsonMode = true;
|
||||
} else if (args[i] === '--raw') {
|
||||
rawMode = true;
|
||||
} else if (args[i] === '--global') {
|
||||
includeGlobal = true;
|
||||
} else if (!args[i].startsWith('-')) {
|
||||
targetPath = args[i];
|
||||
const arg = args[i];
|
||||
|
||||
if (BOOL_FLAGS.includes(arg)) {
|
||||
if (arg === '--save') save = true;
|
||||
else if (arg === '--list') list = true;
|
||||
else if (arg === '--json') jsonMode = true;
|
||||
else if (arg === '--raw') rawMode = true;
|
||||
else if (arg === '--global') includeGlobal = true;
|
||||
} else if (VALUE_FLAGS.includes(arg)) {
|
||||
const value = args[i + 1];
|
||||
if (value === undefined || value.startsWith('-')) {
|
||||
throw new Error(`Option ${arg} requires a value.`);
|
||||
}
|
||||
if (arg === '--name' || arg === '--baseline') baselineName = value;
|
||||
else outputFile = value;
|
||||
i++;
|
||||
} else if (arg.startsWith('-')) {
|
||||
throw new Error(
|
||||
`Unknown option: ${arg}\n` +
|
||||
`Valid options: ${[...BOOL_FLAGS, ...VALUE_FLAGS].join(' ')}`
|
||||
);
|
||||
} else {
|
||||
targetPath = arg;
|
||||
}
|
||||
}
|
||||
|
||||
// --- List mode ---
|
||||
if (list) {
|
||||
const result = await listBaselines();
|
||||
// commands/drift.md runs this with `2>/dev/null` (ux-rules rule 2). The
|
||||
// human listing below goes to stderr, so without --output-file the command
|
||||
// received 0 bytes and could render nothing. The flag was already accepted
|
||||
// by the arg parser; only list mode ignored it.
|
||||
if (outputFile) await writeOutputFile(outputFile, JSON.stringify(result, null, 2) + '\n', 'utf-8');
|
||||
if (jsonMode || rawMode) {
|
||||
process.stdout.write(JSON.stringify(result, null, 2) + '\n');
|
||||
} else {
|
||||
|
|
@ -65,7 +91,12 @@ async function main() {
|
|||
process.stderr.write('\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n');
|
||||
}
|
||||
}
|
||||
process.exit(0);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!(await requireTargetDir(resolve(targetPath)))) {
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
// --- Save mode ---
|
||||
|
|
@ -84,7 +115,7 @@ async function main() {
|
|||
process.stderr.write(`\nBaseline "${result.name}" saved to ${result.path}\n`);
|
||||
process.stderr.write(`Findings: ${envelope.aggregate.total_findings}\n`);
|
||||
}
|
||||
process.exit(0);
|
||||
return;
|
||||
}
|
||||
|
||||
// --- Drift mode (default) ---
|
||||
|
|
@ -103,7 +134,28 @@ async function main() {
|
|||
process.stderr.write(`Baseline "${baselineName}" not found.\n`);
|
||||
process.stderr.write(`Save one first: node drift-cli.mjs <path> --save\n`);
|
||||
}
|
||||
process.exit(1);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
// M-BUG-27: a baseline carries the path it was saved from, but nothing ever
|
||||
// compared it against the current target. Diffing a repo against a baseline
|
||||
// anchored elsewhere produced 100% phantom drift — every baseline finding
|
||||
// "resolved", every current finding "new" — and reported it as trend
|
||||
// "improving": a reassuring and entirely false signal, on the DEFAULT
|
||||
// baseline. The warning goes to stderr in every mode; stdout stays
|
||||
// byte-identical to the frozen v5.0.0 shape.
|
||||
const baselineTarget = baseline._baseline?.target_path || '';
|
||||
const currentTarget = resolve(targetPath);
|
||||
const anchorMatches = !baselineTarget || baselineTarget === currentTarget;
|
||||
if (baselineTarget && baselineTarget !== currentTarget) {
|
||||
process.stderr.write(
|
||||
`\nWarning: baseline "${baselineName}" was saved from a different target path.\n` +
|
||||
` baseline: ${baselineTarget}\n` +
|
||||
` current: ${currentTarget}\n` +
|
||||
` The two scans cover different trees, so this diff is not a drift signal.\n` +
|
||||
` Re-anchor with: drift-cli.mjs ${currentTarget} --save --name ${baselineName}\n\n`
|
||||
);
|
||||
}
|
||||
|
||||
// Run current scan
|
||||
|
|
@ -115,25 +167,49 @@ async function main() {
|
|||
// Diff
|
||||
const diff = diffEnvelopes(baseline, current);
|
||||
|
||||
// Default mode: humanize finding-bearing diff fields before report rendering.
|
||||
// `_baselineAnchor` rides here and NOT in the raw shape: commands/drift.md runs
|
||||
// the CLI under `2>/dev/null`, so the stderr warning above never reaches the
|
||||
// caller that has to act on it. --json/--raw stay v5.0.0-shaped.
|
||||
//
|
||||
// movedFindings holds {from, to} PAIRS, not flat findings — humanizeFindings()
|
||||
// builds a brand-new object from named finding fields only, so running it
|
||||
// directly over the pairs silently drops `from`/`to` and formatDiffReport's
|
||||
// `m.from.severity` crashes on undefined. Humanize each side of the pair.
|
||||
const humanizedDiff = {
|
||||
...diff,
|
||||
_baselineAnchor: { matches: anchorMatches, baselineTarget, currentTarget },
|
||||
newFindings: humanizeFindings(diff.newFindings || []),
|
||||
resolvedFindings: humanizeFindings(diff.resolvedFindings || []),
|
||||
unchangedFindings: humanizeFindings(diff.unchangedFindings || []),
|
||||
movedFindings: (diff.movedFindings || []).map((m) => ({
|
||||
from: humanizeFinding(m.from),
|
||||
to: humanizeFinding(m.to),
|
||||
})),
|
||||
};
|
||||
|
||||
if (jsonMode || rawMode) {
|
||||
// --json and --raw both write the raw v5.0.0-shape diff (byte-identical).
|
||||
process.stdout.write(JSON.stringify(diff, null, 2) + '\n');
|
||||
} else {
|
||||
// Default mode: humanize finding-bearing diff fields before report rendering.
|
||||
const humanizedDiff = {
|
||||
...diff,
|
||||
newFindings: humanizeFindings(diff.newFindings || []),
|
||||
resolvedFindings: humanizeFindings(diff.resolvedFindings || []),
|
||||
unchangedFindings: humanizeFindings(diff.unchangedFindings || []),
|
||||
movedFindings: humanizeFindings(diff.movedFindings || []),
|
||||
};
|
||||
const report = formatDiffReport(humanizedDiff);
|
||||
process.stderr.write('\n' + report + '\n');
|
||||
}
|
||||
|
||||
// ux-rules rule 2: every scanner Bash call uses `--output-file <path>` and the
|
||||
// command reads the file with the Read tool. drift-cli had no such flag, and
|
||||
// its default-mode report goes to stderr — which commands/drift.md discarded
|
||||
// via `2>/dev/null` while instructing the agent to "read stdout". The command
|
||||
// captured nothing. Matches posture.mjs: raw diff in --json/--raw, humanized
|
||||
// otherwise; stdout is unaffected.
|
||||
if (outputFile) {
|
||||
const fileDiff = (jsonMode || rawMode) ? diff : humanizedDiff;
|
||||
await writeOutputFile(outputFile, JSON.stringify(fileDiff, null, 2), 'utf-8');
|
||||
process.stderr.write(`\nResults written to ${outputFile}\n`);
|
||||
}
|
||||
|
||||
// Exit code: 0=stable/improving, 1=degrading
|
||||
if (diff.summary.trend === 'degrading') process.exit(1);
|
||||
process.exit(0);
|
||||
process.exitCode = diff.summary.trend === 'degrading' ? 1 : 0;
|
||||
}
|
||||
|
||||
// Only run CLI if invoked directly
|
||||
|
|
@ -141,6 +217,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,9 +1,10 @@
|
|||
/**
|
||||
* GAP Scanner — Feature Gap Scanner
|
||||
* Compares actual configuration against complete Claude Code feature register.
|
||||
* 25 gap dimensions across 4 tiers, plus a conditional disableBundledSkills
|
||||
* budget-lever check (remediation companion to SKL CA-SKL-002, fires only under
|
||||
* measured skill-listing pressure). Always runs with includeGlobal: true.
|
||||
* 24 gap dimensions across 4 tiers, plus four conditional levers (bundled-skills
|
||||
* budget, CLI-over-MCP, hook-output filtering, agent model/effort routing) which
|
||||
* fire only under a measured condition and are therefore NOT dimensions: they
|
||||
* stay out of the scoring denominators. Always runs with includeGlobal: true.
|
||||
* Finding IDs: CA-GAP-NNN
|
||||
*/
|
||||
|
||||
|
|
@ -115,6 +116,36 @@ const TIER_SEVERITY = {
|
|||
t4: SEVERITY.info,
|
||||
};
|
||||
|
||||
/**
|
||||
* Titles of the conditional levers — findings this scanner emits that are NOT
|
||||
* dimensions in GAP_CHECKS. They fire only under a measured condition, so they
|
||||
* carry no tier and never enter the scoring denominators (TIER_COUNTS /
|
||||
* TOTAL_DIMENSIONS) or the scoring TITLE_TO_ID map.
|
||||
*
|
||||
* Exported as the single source of both the code and the title: the
|
||||
* finding-code registry guard needs the codes, the humanizer coverage guard
|
||||
* needs the titles, and a hand-maintained copy of either list in a test is the
|
||||
* two-copies-drift class. One object so the two cannot disagree.
|
||||
*/
|
||||
export const LEVERS = {
|
||||
bundledSkills: {
|
||||
code: 'bundled-skills-lever',
|
||||
title: 'Bundled skills add to an over-budget skill listing',
|
||||
},
|
||||
cliOverMcp: {
|
||||
code: 'cli-over-mcp-lever',
|
||||
title: 'Prefer CLI over MCP for common operations',
|
||||
},
|
||||
filterHookOutput: {
|
||||
code: 'filter-hook-output-lever',
|
||||
title: 'Filter hook output before it enters context',
|
||||
},
|
||||
agentModelRouting: {
|
||||
code: 'agent-model-routing-lever',
|
||||
title: 'Subagents pin neither model nor effort',
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Lazily read and cache file content.
|
||||
* @param {CheckContext} ctx
|
||||
|
|
@ -177,7 +208,8 @@ export function bundledSkillsLeverFinding({ leverPulled, aggregate }) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.low,
|
||||
title: 'Bundled skills add to an over-budget skill listing',
|
||||
code: LEVERS.bundledSkills.code,
|
||||
title: LEVERS.bundledSkills.title,
|
||||
description:
|
||||
`Your ${aggregate.scanned} active skills already carry ~${aggregate.aggregateTokens} tokens of ` +
|
||||
`description text, over the ${aggregate.budgetTokens}-token listing budget Claude Code allots the ` +
|
||||
|
|
@ -221,7 +253,8 @@ export function cliOverMcpLeverFinding({ assessment } = {}) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.low,
|
||||
title: 'Prefer CLI over MCP for common operations',
|
||||
code: LEVERS.cliOverMcp.code,
|
||||
title: LEVERS.cliOverMcp.title,
|
||||
description:
|
||||
`Your active project MCP tool schemas (~${assessment.aggregateTokens} tokens) are forced into the ` +
|
||||
'always-loaded prefix every turn rather than deferred (see CA-TOK-006). CLI tools (gh, aws, gcloud, …) ' +
|
||||
|
|
@ -258,7 +291,8 @@ export function filterHookLeverFinding({ flaggedHooks } = {}) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
title: 'Filter hook output before it enters context',
|
||||
code: LEVERS.filterHookOutput.code,
|
||||
title: LEVERS.filterHookOutput.title,
|
||||
description:
|
||||
`${hooks.length} active hook${hooks.length === 1 ? '' : 's'} build hookSpecificOutput.additionalContext ` +
|
||||
"from un-grepped command output (see HKV advisory). That field enters Claude's context on every fire, " +
|
||||
|
|
@ -273,8 +307,103 @@ export function filterHookLeverFinding({ flaggedHooks } = {}) {
|
|||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Agent model/effort routing lever (C4) — cites BP-MODEL-001/002.
|
||||
*
|
||||
* A LEVER rather than a GAP_CHECKS dimension, and deliberately so. The question
|
||||
* "do your subagents route model/effort?" has no meaningful reading on a config
|
||||
* with no subagents — the `No custom subagents` dimension (t2_6) owns that case,
|
||||
* and firing here too would just double-report it. A dimension can only express
|
||||
* "not applicable" as "present", which would also inflate the utilization
|
||||
* denominator for every agent-less config.
|
||||
*
|
||||
* ONE check across BOTH axes, not two: it fires only when NEITHER `model:` nor
|
||||
* `effort:` appears on ANY authored agent. A deliberate all-on-one-model setup
|
||||
* therefore stays silent, which is the precision the opportunity framing needs.
|
||||
* The cost is recall — a config that pins `model:` everywhere but never uses
|
||||
* `effort:` gets no nudge. That trade is the v1 boundary, not an oversight.
|
||||
*
|
||||
* Pure and exported for unit testing.
|
||||
*
|
||||
* @param {{ agentCount: number, modelPinned: number, effortPinned: number }} counts
|
||||
* @returns {object|null} a GAP finding, or null when there is no opportunity
|
||||
*/
|
||||
export function agentModelRoutingLeverFinding({ agentCount, modelPinned, effortPinned }) {
|
||||
if (!agentCount) return null;
|
||||
if (modelPinned > 0 || effortPinned > 0) return null;
|
||||
|
||||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
code: LEVERS.agentModelRouting.code,
|
||||
title: LEVERS.agentModelRouting.title,
|
||||
description:
|
||||
`All ${agentCount} of your subagents name neither a \`model:\` nor an \`effort:\` in their frontmatter. ` +
|
||||
'The `model` field defaults to `inherit`, so each one runs on the main conversation\'s model — omitting ' +
|
||||
'it is not a neutral default but a choice to pay the session\'s rate for every delegated task ' +
|
||||
'(BP-MODEL-001, https://code.claude.com/docs/en/sub-agents). Reasoning effort is a separate axis with ' +
|
||||
'its own frontmatter field and its own default, so a subagent can be routed on either or both ' +
|
||||
'(BP-MODEL-002, https://code.claude.com/docs/en/model-config).',
|
||||
evidence:
|
||||
`authored_agents=${agentCount}; model_pinned=${modelPinned}; effort_pinned=${effortPinned}; ` +
|
||||
'lever=agent frontmatter `model:` / `effort:` (plugin-bundled and fixture agents excluded)',
|
||||
recommendation:
|
||||
'Pin a cheaper `model:` on the subagents whose work is mechanical or read-only (search, extraction, ' +
|
||||
'summarisation) and leave the orchestrating session on the stronger model; pin a lower `effort:` on the ' +
|
||||
'same ones and reserve the high levels for work whose product is judgement. If running everything on one ' +
|
||||
'model is a deliberate policy, suppress this with `CA-GAP-028` in `.config-audit-ignore`.',
|
||||
category: 'model-fit',
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Count authored agents and how many pin each routing axis.
|
||||
* Frontmatter-only read; an unparseable or frontmatter-less file counts as an
|
||||
* agent that pins nothing, matching what Claude Code would load.
|
||||
* @param {CheckContext} ctx
|
||||
* @returns {Promise<{ agentCount: number, modelPinned: number, effortPinned: number }>}
|
||||
*/
|
||||
async function countAgentRouting(ctx) {
|
||||
let agentCount = 0;
|
||||
let modelPinned = 0;
|
||||
let effortPinned = 0;
|
||||
for (const file of ctx.files.filter(f => f.type === 'agent-md')) {
|
||||
agentCount++;
|
||||
const content = await getContent(ctx, file.absPath);
|
||||
if (!content) continue;
|
||||
const { frontmatter } = parseFrontmatter(content);
|
||||
if (!frontmatter) continue;
|
||||
if (isRoutingValue(frontmatter.model) && !isDefaultModel(frontmatter.model)) modelPinned++;
|
||||
if (isRoutingValue(frontmatter.effort)) effortPinned++;
|
||||
}
|
||||
return { agentCount, modelPinned, effortPinned };
|
||||
}
|
||||
|
||||
/**
|
||||
* True for a frontmatter value that actually names something. An empty or
|
||||
* whitespace-only `model:` is a no-op in Claude Code, so it must not read as a pin.
|
||||
* @param {*} v
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function isRoutingValue(v) {
|
||||
return typeof v === 'string' ? v.trim().length > 0 : v != null && v !== false;
|
||||
}
|
||||
|
||||
/**
|
||||
* `inherit` IS the documented default for a subagent's `model` (BP-MODEL-001),
|
||||
* so writing it explicitly routes nothing — the agent still runs on the main
|
||||
* conversation's model. Spelling out a default must not buy silence, or a config
|
||||
* can opt out of the opportunity without changing a single thing about cost.
|
||||
* Effort has no documented sentinel of this kind, so it has no counterpart here.
|
||||
* @param {*} v
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function isDefaultModel(v) {
|
||||
return typeof v === 'string' && v.trim().toLowerCase() === 'inherit';
|
||||
}
|
||||
|
||||
/** @type {GapCheck[]} */
|
||||
const GAP_CHECKS = [
|
||||
export const GAP_CHECKS = [
|
||||
// --- Tier 1: Foundation ---
|
||||
{
|
||||
id: 't1_1', tier: 't1',
|
||||
|
|
@ -480,12 +609,11 @@ const GAP_CHECKS = [
|
|||
return false;
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 't3_8', tier: 't3',
|
||||
title: 'No autoMode classifier',
|
||||
recommendation: 'Configure autoMode in user/local settings with environment context and allow/deny rules.',
|
||||
check: async (ctx) => anySettingsHas(ctx, 'autoMode'),
|
||||
},
|
||||
// t3_8 ('No autoMode classifier') retired in v5.14: CC 2.1.226's /doctor
|
||||
// Check 8 covers auto mode with usage-weighted judgement, so an "adopt this
|
||||
// feature" nudge is a pure duplicate under the binding /doctor positioning.
|
||||
// The DETERMINISTIC side stays: SET still validates autoMode structure and
|
||||
// flags it as dead config in shared project settings.
|
||||
|
||||
// --- Tier 4: Team/Enterprise ---
|
||||
{
|
||||
|
|
@ -575,6 +703,7 @@ export async function scan(targetPath, sharedDiscovery) {
|
|||
if (!present) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: gap.id,
|
||||
severity: TIER_SEVERITY[gap.tier],
|
||||
title: gap.title,
|
||||
description: `Feature gap: ${gap.title}. ${gap.recommendation}`,
|
||||
|
|
@ -605,6 +734,13 @@ export async function scan(targetPath, sharedDiscovery) {
|
|||
const hookLever = filterHookLeverFinding({ flaggedHooks });
|
||||
if (hookLever) findings.push(hookLever);
|
||||
|
||||
// Agent model/effort routing lever (C4) — fires only when authored agents
|
||||
// exist and not one of them uses either routing axis. Reads the SAME authored
|
||||
// set as the presence checks, so plugin-bundled and fixture agents cannot
|
||||
// make a machine look routed (M-BUG-13).
|
||||
const routingLever = agentModelRoutingLeverFinding(await countAgentRouting(ctx));
|
||||
if (routingLever) findings.push(routingLever);
|
||||
|
||||
const filesScanned = discovery.files.length;
|
||||
return scannerResult(SCANNER, 'ok', findings, filesScanned, Date.now() - start);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -9,11 +9,19 @@
|
|||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireTargetDir } from './lib/require-target-dir.mjs';
|
||||
import { runAllScanners } from './scan-orchestrator.mjs';
|
||||
import { planFixes, applyFixes, verifyFixes } from './fix-engine.mjs';
|
||||
import { createBackup } from './lib/backup.mjs';
|
||||
import { humanizeFinding } from './lib/humanizer.mjs';
|
||||
|
||||
// `--dry-run` is a no-op alias: dry-run is already the default. It exists because
|
||||
// commands/fix.md documents it in argument-hint, and a documented flag that the
|
||||
// CLI silently drops is the same fail-silent class as the unknown-flag sink below.
|
||||
const BOOL_FLAGS = ['--apply', '--dry-run', '--json', '--raw', '--global', '--approve-scope'];
|
||||
const VALUE_FLAGS = ['--output-file', '--repo'];
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
let targetPath = '.';
|
||||
|
|
@ -21,18 +29,44 @@ async function main() {
|
|||
let jsonMode = false;
|
||||
let rawMode = false;
|
||||
let includeGlobal = false;
|
||||
let outputFile = null;
|
||||
let approveScope = false;
|
||||
// The session's root, never the scan target (#63). `--global` fixes files
|
||||
// under `~/.claude` while the session still stands somewhere else, so reading
|
||||
// the root off the target would classify a machine-wide write as "in-repo"
|
||||
// and silence the strongest gate exactly where it matters.
|
||||
let repoRoot = process.cwd();
|
||||
|
||||
// Same defect class as M-BUG-21 in drift-cli: this loop used to end in
|
||||
// `else if (!args[i].startsWith('-')) targetPath = args[i]` with no
|
||||
// unknown-flag branch, so an unrecognised flag was dropped silently and its
|
||||
// VALUE became the scan target. Here that is worse than in drift: combined
|
||||
// with --apply it silently moves the WRITE target to another tree.
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--apply') {
|
||||
apply = true;
|
||||
} else if (args[i] === '--json') {
|
||||
jsonMode = true;
|
||||
} else if (args[i] === '--raw') {
|
||||
rawMode = true;
|
||||
} else if (args[i] === '--global') {
|
||||
includeGlobal = true;
|
||||
} else if (!args[i].startsWith('-')) {
|
||||
targetPath = args[i];
|
||||
const arg = args[i];
|
||||
|
||||
if (BOOL_FLAGS.includes(arg)) {
|
||||
if (arg === '--apply') apply = true;
|
||||
else if (arg === '--json') jsonMode = true;
|
||||
else if (arg === '--raw') rawMode = true;
|
||||
else if (arg === '--global') includeGlobal = true;
|
||||
else if (arg === '--approve-scope') approveScope = true;
|
||||
// --dry-run: default behaviour, accepted so it is not silently dropped.
|
||||
} else if (VALUE_FLAGS.includes(arg)) {
|
||||
const value = args[i + 1];
|
||||
if (value === undefined || value.startsWith('-')) {
|
||||
throw new Error(`Option ${arg} requires a value.`);
|
||||
}
|
||||
if (arg === '--repo') repoRoot = value;
|
||||
else outputFile = value;
|
||||
i++;
|
||||
} else if (arg.startsWith('-')) {
|
||||
throw new Error(
|
||||
`Unknown option: ${arg}\n` +
|
||||
`Valid options: ${[...BOOL_FLAGS, ...VALUE_FLAGS].join(' ')}`
|
||||
);
|
||||
} else {
|
||||
targetPath = arg;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -41,6 +75,11 @@ async function main() {
|
|||
|
||||
const resolvedPath = resolve(targetPath);
|
||||
|
||||
if (!(await requireTargetDir(resolvedPath))) {
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
if (!machineMode) {
|
||||
process.stderr.write(`Config-Audit Fix CLI v2.1.0\n`);
|
||||
process.stderr.write(`Target: ${resolvedPath}\n`);
|
||||
|
|
@ -103,18 +142,29 @@ async function main() {
|
|||
let verified = [];
|
||||
let regressions = [];
|
||||
let backupId = null;
|
||||
// The engine's scope verdict, carried out to the payload. A `disclose` class
|
||||
// writes without withholding anything, so the ONLY place the command can
|
||||
// learn that a write left the project is here — stderr is discarded by
|
||||
// `2>/dev/null` (ux-rules rule 2), which is F3's defect class.
|
||||
let scopeGate = null;
|
||||
let scopeDisclosures = [];
|
||||
|
||||
if (fixes.length === 0) {
|
||||
const output = { planned: [], applied: [], failed: [], verified: [], regressions: [], manual, backupId: null };
|
||||
if (machineMode) {
|
||||
const output = { planned: [], applied: [], failed: [], verified: [], regressions: [], manual, backupId: null };
|
||||
process.stdout.write(JSON.stringify(output, null, 2) + '\n');
|
||||
}
|
||||
process.exit(0);
|
||||
if (outputFile) await writeOutputFile(outputFile, JSON.stringify(output, null, 2) + '\n', 'utf-8');
|
||||
return;
|
||||
}
|
||||
|
||||
if (apply) {
|
||||
// Create backup first
|
||||
const filesToBackup = [...new Set(fixes.filter(f => f.type !== 'file-rename').map(f => f.file))];
|
||||
// Create backup first. file-rename used to be excluded here, so a rule file
|
||||
// whose only defect was its extension was renamed with NO backup entry —
|
||||
// while commands/fix.md promised "every fix creates a backup first" and
|
||||
// handed the user a backupId that could not restore it. The source file is
|
||||
// backed up like any other; rollback recreates it at its original path.
|
||||
const filesToBackup = [...new Set(fixes.map(f => f.file))];
|
||||
const backup = createBackup(filesToBackup);
|
||||
backupId = backup.backupId;
|
||||
|
||||
|
|
@ -123,9 +173,43 @@ async function main() {
|
|||
process.stderr.write(` Applying ${fixes.length} fixes...\n\n`);
|
||||
}
|
||||
|
||||
const result = await applyFixes(fixes, { dryRun: false, backupDir: backup.backupPath });
|
||||
const result = await applyFixes(fixes, {
|
||||
dryRun: false,
|
||||
backupDir: backup.backupPath,
|
||||
repoRoot,
|
||||
approveScope,
|
||||
});
|
||||
applied = result.applied;
|
||||
failed = result.failed;
|
||||
scopeGate = result.gate ?? null;
|
||||
scopeDisclosures = result.disclosures ?? [];
|
||||
|
||||
// A refused set is a verdict about a config that WAS examined, not a tool
|
||||
// failure — so it rides in the payload and keeps the normal exit contract
|
||||
// (#62). Anything a command must act on has to reach it through
|
||||
// `--output-file`; stderr alone is invisible to the command layer (F3).
|
||||
if (result.requiresApproval && result.refused.length > 0) {
|
||||
const payload = {
|
||||
status: 'refused',
|
||||
reason: 'scope-gate',
|
||||
gate: result.gate,
|
||||
requiresApproval: true,
|
||||
disclosures: result.disclosures,
|
||||
refused: result.refused,
|
||||
backupId,
|
||||
};
|
||||
const json = JSON.stringify(payload, null, 2) + '\n';
|
||||
if (machineMode) process.stdout.write(json);
|
||||
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
|
||||
if (!machineMode) {
|
||||
for (const line of result.disclosures) process.stderr.write(`\n ${line}\n`);
|
||||
process.stderr.write(
|
||||
`\n Refused ${result.refused.length} fix(es) pending your go-ahead.`
|
||||
+ ' Re-run with --approve-scope to apply them.\n',
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (!machineMode) {
|
||||
process.stderr.write(` Results: ${applied.length} applied, ${failed.length} failed\n`);
|
||||
|
|
@ -142,7 +226,10 @@ async function main() {
|
|||
process.stderr.write(`\n Verifying...\n`);
|
||||
}
|
||||
|
||||
const verification = await verifyFixes(envelope, applied);
|
||||
// Verification must re-scan the scope the fix run used. It hardcoded
|
||||
// includeGlobal:false, so with --global every untouched global-scope
|
||||
// finding fell out of the re-scan and was reported as verified.
|
||||
const verification = await verifyFixes(envelope, applied, { includeGlobal });
|
||||
verified = verification.verified;
|
||||
regressions = verification.regressions;
|
||||
|
||||
|
|
@ -151,7 +238,11 @@ async function main() {
|
|||
if (regressions.length > 0) {
|
||||
process.stderr.write(` Regressions: ${regressions.join(', ')}\n`);
|
||||
}
|
||||
process.stderr.write(`\n Rollback: node scanners/rollback-cli.mjs ${backupId}\n`);
|
||||
// The user-facing entry, not the CLI: `/config-audit rollback` renders
|
||||
// the scope disclosures and asks before a restore that leaves the repo.
|
||||
// `scanners/rollback-cli.mjs` exists now (R1) and is what the command
|
||||
// runs, but naming it here would hand the user the ungated half.
|
||||
process.stderr.write(`\n Rollback: /config-audit rollback ${backupId}\n`);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
|
|
@ -165,7 +256,7 @@ async function main() {
|
|||
}
|
||||
|
||||
// JSON output (both --json and --raw write byte-equal v5.0.0-shape stdout)
|
||||
if (machineMode) {
|
||||
{
|
||||
const output = {
|
||||
planned: fixes.map(f => ({
|
||||
findingId: f.findingId,
|
||||
|
|
@ -192,8 +283,20 @@ async function main() {
|
|||
recommendation: m.recommendation,
|
||||
})),
|
||||
backupId,
|
||||
gate: scopeGate,
|
||||
disclosures: scopeDisclosures,
|
||||
};
|
||||
process.stdout.write(JSON.stringify(output, null, 2) + '\n');
|
||||
const serialized = JSON.stringify(output, null, 2) + '\n';
|
||||
if (machineMode) process.stdout.write(serialized);
|
||||
// --output-file carries the same payload to disk. ux-rules rule 2 requires
|
||||
// it: commands run scanners with `2>/dev/null`, so anything the command has
|
||||
// to act on must ride in a file, not in stdout or stderr.
|
||||
if (outputFile) await writeOutputFile(outputFile, serialized, 'utf-8');
|
||||
|
||||
// Exit code follows the convention the other scanners use: 0 PASS,
|
||||
// 2 FAIL, 3 tool error. A failed fix used to exit 0, so a caller could not
|
||||
// tell a clean run from one that silently lost a fix.
|
||||
if (failed.length > 0) process.exitCode = 2;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -202,6 +305,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,7 +8,9 @@ import { readFile, writeFile, rename, stat } from 'node:fs/promises';
|
|||
import { dirname } from 'node:path';
|
||||
import { parseJson, parseFrontmatter } from './lib/yaml-parser.mjs';
|
||||
import { createBackup } from './lib/backup.mjs';
|
||||
import { evaluateWriteTargets } from './lib/write-scope.mjs';
|
||||
import { runAllScanners } from './scan-orchestrator.mjs';
|
||||
import { VALID_EFFORT_LEVELS as SETTINGS_EFFORT_LEVELS } from './settings-validator.mjs';
|
||||
|
||||
/**
|
||||
* Fix type constants.
|
||||
|
|
@ -22,8 +24,8 @@ const FIX_TYPES = {
|
|||
FILE_RENAME: 'file-rename',
|
||||
};
|
||||
|
||||
/** Valid effortLevel values for nearest-match */
|
||||
const VALID_EFFORT_LEVELS = ['low', 'medium', 'high', 'max'];
|
||||
/** Valid effortLevel values for nearest-match — the validator's list, not a copy. */
|
||||
const VALID_EFFORT_LEVELS = [...SETTINGS_EFFORT_LEVELS];
|
||||
|
||||
/**
|
||||
* Plan fixes from a scanner envelope.
|
||||
|
|
@ -56,9 +58,21 @@ export function planFixes(envelope) {
|
|||
}
|
||||
}
|
||||
|
||||
// Sort fixes by severity weight (critical first)
|
||||
// Sort fixes by severity weight (critical first), but a file-rename always
|
||||
// sorts after every other fix. A rename moves the file out from under any
|
||||
// later fix that still addresses the old path: a rule file with both
|
||||
// `globs:` and a non-.md extension had the rename applied first, and the
|
||||
// frontmatter fix then failed with ENOENT while the run still exited 0.
|
||||
// `?? 4`, not `|| 4`: critical weighs 0, and `0 || 4` evaluates to 4 — so
|
||||
// critical fixes sorted LAST, the exact opposite of this function's contract
|
||||
// (M-BUG-30). The old test used the same falsy fallback and agreed with the bug.
|
||||
const severityOrder = { critical: 0, high: 1, medium: 2, low: 3, info: 4 };
|
||||
fixes.sort((a, b) => (severityOrder[a.severity] || 4) - (severityOrder[b.severity] || 4));
|
||||
fixes.sort((a, b) => {
|
||||
const aRename = a.type === FIX_TYPES.FILE_RENAME ? 1 : 0;
|
||||
const bRename = b.type === FIX_TYPES.FILE_RENAME ? 1 : 0;
|
||||
if (aRename !== bRename) return aRename - bRename;
|
||||
return (severityOrder[a.severity] ?? 4) - (severityOrder[b.severity] ?? 4);
|
||||
});
|
||||
|
||||
return { fixes, skipped, manual };
|
||||
}
|
||||
|
|
@ -224,6 +238,42 @@ export async function applyFixes(fixPlans, opts = {}) {
|
|||
throw new Error('backupDir is required when not in dryRun mode');
|
||||
}
|
||||
|
||||
// Q1 — the scope gate, in code rather than in the command template's prose.
|
||||
//
|
||||
// A file-rename writes TWO paths: the source disappears and `newPath`
|
||||
// appears. Classifying only `plan.file` would let a rename move a repo file
|
||||
// to a machine-wide destination under a `silent` gate.
|
||||
//
|
||||
// `!dryRun` is load-bearing and is the same rule the subtraction axis
|
||||
// settled (#63): the gate guards a WRITE, and a dry run is not one.
|
||||
// `requiresApproval` is reported either way, so a caller planning a run still
|
||||
// learns that approval will be owed before anything is applied.
|
||||
const scope = evaluateWriteTargets(
|
||||
fixPlans.flatMap((p) => (p.newPath ? [p.file, p.newPath] : [p.file])),
|
||||
opts.repoRoot ?? null,
|
||||
opts.home ? { home: opts.home } : {},
|
||||
);
|
||||
|
||||
if (scope.requiresApproval && !opts.approveScope && !opts.dryRun) {
|
||||
// A refused write is a VERDICT about a config that was examined, not a
|
||||
// tool failure (#62) — the caller renders the disclosure and asks. Nothing
|
||||
// is applied, and this is not an exit-3 situation.
|
||||
return {
|
||||
applied: [],
|
||||
failed: [],
|
||||
gate: scope.gate,
|
||||
requiresApproval: true,
|
||||
disclosures: scope.disclosures,
|
||||
refused: fixPlans.map((plan) => ({
|
||||
findingId: plan.findingId,
|
||||
file: plan.file,
|
||||
status: 'refused',
|
||||
reason: 'scope-gate',
|
||||
type: plan.type,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
for (const plan of fixPlans) {
|
||||
if (opts.dryRun) {
|
||||
applied.push({
|
||||
|
|
@ -256,7 +306,14 @@ export async function applyFixes(fixPlans, opts = {}) {
|
|||
}
|
||||
}
|
||||
|
||||
return { applied, failed };
|
||||
return {
|
||||
applied,
|
||||
failed,
|
||||
gate: scope.gate,
|
||||
requiresApproval: scope.requiresApproval,
|
||||
disclosures: scope.disclosures,
|
||||
refused: [],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -600,20 +657,29 @@ function extractEventFromDescription(description) {
|
|||
* Verify fixes by re-running affected scanners.
|
||||
* @param {object} originalEnvelope - Original scanner envelope
|
||||
* @param {object[]} appliedResults - Results from applyFixes()
|
||||
* @param {object} [opts]
|
||||
* @param {boolean} [opts.includeGlobal=false] - Must match the scope the fix run scanned
|
||||
* @returns {Promise<{ verified: string[], regressions: string[], newFindings: object[] }>}
|
||||
*/
|
||||
export async function verifyFixes(originalEnvelope, appliedResults) {
|
||||
export async function verifyFixes(originalEnvelope, appliedResults, opts = {}) {
|
||||
const targetPath = originalEnvelope.meta.target;
|
||||
const verified = [];
|
||||
const regressions = [];
|
||||
const newFindings = [];
|
||||
|
||||
// Re-scan the target
|
||||
const newEnvelope = await runAllScanners(targetPath, { includeGlobal: false });
|
||||
// Re-scan the target in the SAME scope the fix run used. This was hardcoded
|
||||
// to includeGlobal:false: after a --global run, every global-scope finding
|
||||
// was absent from the re-scan and therefore counted as verified — a clean
|
||||
// "fixed" report for files nothing had touched.
|
||||
const newEnvelope = await runAllScanners(targetPath, { includeGlobal: opts.includeGlobal === true });
|
||||
|
||||
// Build set of original finding IDs that were fixed
|
||||
const fixedIds = new Set(
|
||||
appliedResults.filter(r => r.status === 'applied').map(r => r.findingId),
|
||||
// Build the set of fixed finding INSTANCES. A finding ID names the check, so
|
||||
// one check failing in two files yields two findings sharing an ID; keying on
|
||||
// the ID alone marks both fixed when one was, and the untouched sibling — still
|
||||
// present in the re-scan — is then reported as a regression (M-BUG-28).
|
||||
const instanceKey = (findingId, file) => `${findingId}::${file || ''}`;
|
||||
const fixedInstances = new Set(
|
||||
appliedResults.filter(r => r.status === 'applied').map(r => instanceKey(r.findingId, r.file)),
|
||||
);
|
||||
|
||||
// Build set of new finding titles for comparison
|
||||
|
|
@ -627,11 +693,13 @@ export async function verifyFixes(originalEnvelope, appliedResults) {
|
|||
// Check that fixed findings are gone
|
||||
for (const scanner of originalEnvelope.scanners) {
|
||||
for (const f of scanner.findings) {
|
||||
if (!fixedIds.has(f.id)) continue;
|
||||
if (!fixedInstances.has(instanceKey(f.id, f.file))) continue;
|
||||
|
||||
const key = `${f.scanner}:${f.title}:${f.file}`;
|
||||
// For file-rename fixes, the original file path won't exist anymore
|
||||
const fixResult = appliedResults.find(r => r.findingId === f.id);
|
||||
const fixResult = appliedResults.find(
|
||||
r => instanceKey(r.findingId, r.file) === instanceKey(f.id, f.file),
|
||||
);
|
||||
if (fixResult && fixResult.type === 'file-rename') {
|
||||
// Check that the finding doesn't reappear at the new path
|
||||
verified.push(f.id);
|
||||
|
|
|
|||
|
|
@ -70,6 +70,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed === null) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Invalid JSON in hooks.json',
|
||||
description: `${file.relPath} contains invalid JSON. All hooks in this file will be ignored.`,
|
||||
|
|
@ -120,6 +121,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (typeof hooks !== 'object' || Array.isArray(hooks)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-not-object',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Hooks must be an object with event keys',
|
||||
description: `${file.relPath}: hooks is ${Array.isArray(hooks) ? 'an array' : typeof hooks}. Expected object with event names as keys.`,
|
||||
|
|
@ -135,6 +137,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (!VALID_EVENTS.has(event)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unknown-event',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Unknown hook event',
|
||||
description: `${file.relPath}: "${event}" is not a valid hook event. This hook will never fire.`,
|
||||
|
|
@ -149,6 +152,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (!Array.isArray(handlers)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'handlers-not-array',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Hook handlers must be an array',
|
||||
description: `${file.relPath}: handlers for "${event}" is not an array.`,
|
||||
|
|
@ -166,6 +170,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (typeof handlerGroup.matcher === 'object') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'matcher-not-string',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Matcher must be a string, not an object',
|
||||
description: `${file.relPath}: "${event}" has a matcher that is an object. Matcher should be a simple string like "Bash" or "Edit|Write".`,
|
||||
|
|
@ -180,6 +185,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (!handlerGroup.hooks || !Array.isArray(handlerGroup.hooks)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-hooks-array',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Missing hooks array in handler group',
|
||||
description: `${file.relPath}: "${event}" handler group is missing the "hooks" array.`,
|
||||
|
|
@ -195,6 +201,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (!hook.type || !VALID_TYPES.has(hook.type)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-handler-type',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Invalid hook handler type',
|
||||
description: `${file.relPath}: "${event}" has handler with type "${hook.type || '(missing)'}".`,
|
||||
|
|
@ -216,6 +223,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'script-not-found',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Hook script not found',
|
||||
description: `${file.relPath}: "${event}" references script that does not exist.`,
|
||||
|
|
@ -232,6 +240,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (verboseCount > VERBOSE_HOOK_LINE_THRESHOLD) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'verbose-output',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Verbose hook output (loud script)',
|
||||
description:
|
||||
|
|
@ -259,6 +268,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (ac.flagged) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unfiltered-additional-context',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Hook injects unfiltered output into context',
|
||||
description:
|
||||
|
|
@ -287,6 +297,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (typeof hook.timeout !== 'number') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'timeout-not-number',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Hook timeout must be a number',
|
||||
description: `${file.relPath}: "${event}" has non-numeric timeout.`,
|
||||
|
|
@ -298,6 +309,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
} else if (hook.timeout < MIN_TIMEOUT || hook.timeout > MAX_TIMEOUT) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'timeout-out-of-range',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Hook timeout outside recommended range',
|
||||
description: `${file.relPath}: "${event}" timeout is ${hook.timeout}ms. Recommended range: ${MIN_TIMEOUT}-${MAX_TIMEOUT}ms.`,
|
||||
|
|
|
|||
|
|
@ -74,6 +74,7 @@ async function walkImports(file, chain, reported, findings) {
|
|||
reported.add(`tilde::${resolved}`);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'tilde-path',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Tilde path in @import',
|
||||
description: `@${imp.path} uses ~ which may not expand correctly in all contexts.`,
|
||||
|
|
@ -91,6 +92,7 @@ async function walkImports(file, chain, reported, findings) {
|
|||
reported.add(reportKey);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'broken-link',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Broken @import link',
|
||||
description: `@${imp.path} references a file that does not exist.`,
|
||||
|
|
@ -111,6 +113,7 @@ async function walkImports(file, chain, reported, findings) {
|
|||
const cycle = chain.slice(cycleStart).map(f => basename(f)).join(' → ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'circular-reference',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Circular @import reference',
|
||||
description: `@${imp.path} creates a circular import chain.`,
|
||||
|
|
@ -129,6 +132,7 @@ async function walkImports(file, chain, reported, findings) {
|
|||
reported.add(`deep::${resolved}`);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deep-chain',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Deep @import chain',
|
||||
description: `@${imp.path} is at depth ${chain.length} (>${MAX_CHAIN_DEPTH} hops).`,
|
||||
|
|
|
|||
|
|
@ -24,19 +24,32 @@
|
|||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeFile } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { loadRegister, REGISTER_PATH } from './lib/best-practices-register.mjs';
|
||||
import { assessFreshness, STALE_AFTER_DAYS_DEFAULT } from './lib/knowledge-refresh.mjs';
|
||||
import { findArgError } from './lib/cli-args.mjs';
|
||||
|
||||
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
|
||||
|
||||
/**
|
||||
* Usage error. Throws rather than calling process.exit(): exit() discards
|
||||
* unflushed stdout when stdout is a pipe. The top-level catch prints the same
|
||||
* `Error: ` text and sets the same exit code 3, so callers see no difference.
|
||||
*/
|
||||
class CliUsageError extends Error {}
|
||||
|
||||
function fail(message) {
|
||||
process.stderr.write(`Error: ${message}\n`);
|
||||
process.exit(3);
|
||||
throw new CliUsageError(message);
|
||||
}
|
||||
|
||||
/** Flag surface, measured 2026-08-09. The gate runs BEFORE the loop below, so the
|
||||
* loop no longer needs to re-check that a value followed its flag. */
|
||||
const ARG_SPEC = { boolean: ['--dry-run'], value: ['--output-file', '--stale-after', '--reference-date'] };
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const argError = findArgError(args, ARG_SPEC);
|
||||
if (argError) fail(argError);
|
||||
let outputFile = null;
|
||||
let staleAfterDays = STALE_AFTER_DAYS_DEFAULT;
|
||||
let referenceDate = null; // null → today
|
||||
|
|
@ -45,12 +58,12 @@ async function main() {
|
|||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--dry-run') dryRun = true;
|
||||
else if (a === '--output-file' && args[i + 1]) outputFile = args[++i];
|
||||
else if (a === '--stale-after' && args[i + 1] !== undefined) {
|
||||
else if (a === '--output-file') outputFile = args[++i];
|
||||
else if (a === '--stale-after') {
|
||||
const n = Number.parseInt(args[++i], 10);
|
||||
if (!Number.isInteger(n) || n < 0) fail('--stale-after must be a non-negative integer (days)');
|
||||
staleAfterDays = n;
|
||||
} else if (a === '--reference-date' && args[i + 1]) {
|
||||
} else if (a === '--reference-date') {
|
||||
referenceDate = args[++i];
|
||||
if (!DATE_RE.test(referenceDate)) fail('--reference-date must be YYYY-MM-DD');
|
||||
}
|
||||
|
|
@ -87,17 +100,18 @@ async function main() {
|
|||
};
|
||||
|
||||
const json = JSON.stringify(payload, null, 2);
|
||||
if (outputFile) await writeFile(outputFile, json, 'utf-8');
|
||||
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
|
||||
else process.stdout.write(json + '\n');
|
||||
|
||||
process.exit(assessment.counts.stale > 0 ? 1 : 0);
|
||||
process.exitCode = assessment.counts.stale > 0 ? 1 : 0;
|
||||
}
|
||||
|
||||
const isDirectRun =
|
||||
process.argv[1] && resolve(process.argv[1]) === resolve(new URL(import.meta.url).pathname);
|
||||
if (isDirectRun) {
|
||||
main().catch((err) => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
const prefix = err instanceof CliUsageError ? 'Error' : 'Fatal';
|
||||
process.stderr.write(`${prefix}: ${err.message}\n`);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -817,7 +817,7 @@ export async function enumerateRules(repoPath, pluginList = []) {
|
|||
*
|
||||
* @param {string} repoPath
|
||||
* @param {Array<{name:string, path:string}>} [pluginList]
|
||||
* @returns {Promise<Array<{name:string, source:string, pluginName:string|null, path:string, bytes:number, estimatedTokens:number, loadPattern:string, survivesCompaction:string, derivationConfidence:string}>>}
|
||||
* @returns {Promise<Array<{name:string, source:string, pluginName:string|null, path:string, bytes:number, estimatedTokens:number, model:string|null, effort:string|null, loadPattern:string, survivesCompaction:string, derivationConfidence:string}>>}
|
||||
*/
|
||||
export async function enumerateAgents(repoPath, pluginList = []) {
|
||||
const out = [];
|
||||
|
|
@ -842,6 +842,11 @@ export async function enumerateAgents(repoPath, pluginList = []) {
|
|||
path: f.path,
|
||||
bytes: f.size,
|
||||
estimatedTokens: estimateTokens(f.size, 'frontmatter'),
|
||||
// Routing axes (C4). Explicit null rather than an absent key: `model`
|
||||
// defaults to `inherit` and `effort` to the session level, so a consumer
|
||||
// must be able to read "not pinned" without guessing (BP-MODEL-001/002).
|
||||
model: hasText(frontmatter && frontmatter.model) ? frontmatter.model.trim() : null,
|
||||
effort: hasText(frontmatter && frontmatter.effort) ? frontmatter.effort.trim() : null,
|
||||
...lp,
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -70,10 +70,19 @@ export function checksum(content) {
|
|||
|
||||
/**
|
||||
* Create a backup of the specified files.
|
||||
*
|
||||
* `opts.created` records paths the caller is about to CREATE. No backup can
|
||||
* hold a file that does not exist yet, so these are not copied — they are
|
||||
* written into the manifest so `rollback` can tell the user which files it is
|
||||
* leaving behind. That list used to exist only in `commands/implement.md`,
|
||||
* typed out by hand next to a manifest the template also typed out by hand;
|
||||
* moving it here is what lets the template stop owning the format (R2).
|
||||
*
|
||||
* @param {string[]} files - Array of absolute file paths to back up
|
||||
* @param {object} [opts]
|
||||
* @param {string} [opts.backupId] - Override backup ID (for testing)
|
||||
* @returns {{ backupId: string, backupPath: string, manifest: object }}
|
||||
* @param {string[]} [opts.created] - Paths this run will create (recorded, not copied)
|
||||
* @returns {{ backupId: string, backupPath: string, manifest: object, skipped: string[] }}
|
||||
*/
|
||||
export function createBackup(files, opts = {}) {
|
||||
const backupId = opts.backupId || generateBackupId();
|
||||
|
|
@ -83,9 +92,13 @@ export function createBackup(files, opts = {}) {
|
|||
mkdirSync(filesDir, { recursive: true });
|
||||
|
||||
const manifestFiles = [];
|
||||
const skipped = [];
|
||||
|
||||
for (const file of files) {
|
||||
if (!existsSync(file)) continue;
|
||||
// A target that is not there is reported, never silently dropped: the
|
||||
// caller asked for a backup of N files and must be able to learn it got
|
||||
// fewer, before it edits anything.
|
||||
if (!existsSync(file)) { skipped.push(file); continue; }
|
||||
|
||||
const safeName = safeFileName(file);
|
||||
copyFileSync(file, join(filesDir, safeName));
|
||||
|
|
@ -106,6 +119,7 @@ export function createBackup(files, opts = {}) {
|
|||
created_at: new Date().toISOString(),
|
||||
backup_id: backupId,
|
||||
files: manifestFiles,
|
||||
created: [...(opts.created || [])],
|
||||
};
|
||||
|
||||
// Write manifest as YAML-like format
|
||||
|
|
@ -115,7 +129,7 @@ export function createBackup(files, opts = {}) {
|
|||
// Cleanup old backups
|
||||
cleanupOldBackups();
|
||||
|
||||
return { backupId, backupPath, manifest };
|
||||
return { backupId, backupPath, manifest, skipped };
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -133,6 +147,14 @@ function serializeManifest(manifest) {
|
|||
yaml += ` checksum: "${f.checksum}"\n`;
|
||||
yaml += ` size_bytes: ${f.sizeBytes}\n`;
|
||||
}
|
||||
// Emitted only when non-empty, and read back by `parseManifest`'s `created:`
|
||||
// branch — the bare-key form, which is why the implement flow's
|
||||
// `created: <timestamp>` (a VALUE, meaning the backup id) never collides
|
||||
// with it.
|
||||
if (manifest.created && manifest.created.length > 0) {
|
||||
yaml += `created:\n`;
|
||||
for (const c of manifest.created) yaml += ` - ${c}\n`;
|
||||
}
|
||||
return yaml;
|
||||
}
|
||||
|
||||
|
|
@ -168,10 +190,13 @@ export function parseManifest(content) {
|
|||
}
|
||||
}
|
||||
|
||||
// Parse file entries — implement-flow format. `commands/implement.md` has the
|
||||
// agent hand-build the backup dir, so real manifests on disk use unquoted
|
||||
// `- backup:` / `original:` / `sha256:`. Reading only the engine format made
|
||||
// restoreBackup a success-shaped no-op on every backup implement produced.
|
||||
// Parse file entries — implement-flow format. Until R2, `commands/implement.md`
|
||||
// had the agent hand-build the backup dir, so manifests written by that flow
|
||||
// use unquoted `- backup:` / `original:` / `sha256:`. Reading only the engine
|
||||
// format made restoreBackup a success-shaped no-op on every backup implement
|
||||
// produced (M-BUG-25). The template no longer writes this format, but the
|
||||
// branch stays: backups already on disk in it must remain restorable — the
|
||||
// same reason `getLegacyBackupDir()` is still read.
|
||||
if (result.files.length === 0) {
|
||||
const implBlocks = content.split(/\n\s+-\s+backup:/).slice(1);
|
||||
for (const block of implBlocks) {
|
||||
|
|
|
|||
85
scanners/lib/cli-args.mjs
Normal file
85
scanners/lib/cli-args.mjs
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
/**
|
||||
* Argv precondition shared by the CLIs — the companion to `require-target-dir`.
|
||||
*
|
||||
* Every CLI here parses argv with a chain of `if (a === '--x') … else if …`.
|
||||
* Two things fall through that chain silently:
|
||||
*
|
||||
* 1. **An unknown flag.** With no `else` branch, `--zzz` leaves no trace: the
|
||||
* CLI exits 0 with a full payload — a confident answer to a question the
|
||||
* caller did not ask. Measured live (#51): `knowledge-refresh`'s only knob
|
||||
* reached the CLI malformed, was ignored, and the command reported "all 14
|
||||
* entries re-verified within the last 90 days" about a threshold the user
|
||||
* had just overridden.
|
||||
*
|
||||
* 2. **A value-taking flag whose value is another flag.** The guard
|
||||
* `a === '--output-file' && args[i + 1]` asks only whether a next token
|
||||
* exists, never whether it is a *value*. Measured live (#57):
|
||||
* `manifest --output-file --json` wrote a file literally named `--json`
|
||||
* into the caller's working directory, exit 0, with `--json` mode silently
|
||||
* dropped. A wrong answer is bad; an unintended file on disk is worse.
|
||||
*
|
||||
* This runs BEFORE the CLI's own loop and does not replace it. That is
|
||||
* deliberate: valid argv reaches the existing parser byte-for-byte unchanged, so
|
||||
* no frozen snapshot can move. Malformed argv never reaches it at all.
|
||||
*
|
||||
* By the exit-code contract, a malformed argument is exit **3** — the scanner
|
||||
* did not get to do its job — never 0/1/2, which are verdicts about a
|
||||
* configuration that WAS examined.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Find the first thing wrong with `args`.
|
||||
*
|
||||
* @param {string[]} args - argv slice (no node/script entries).
|
||||
* @param {{ boolean?: string[], value?: string[] }} spec - the CLI's flag surface.
|
||||
* @returns {string|null} diagnostic, or null when argv is well-formed.
|
||||
*/
|
||||
export function findArgError(args, spec) {
|
||||
const booleanFlags = new Set(spec.boolean || []);
|
||||
const valueFlags = new Set(spec.value || []);
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
|
||||
// Positionals and subcommands are the CLI's own business.
|
||||
if (!a.startsWith('-')) continue;
|
||||
if (booleanFlags.has(a)) continue;
|
||||
|
||||
if (valueFlags.has(a)) {
|
||||
const next = args[i + 1];
|
||||
if (next === undefined) {
|
||||
return `flag "${a}" needs a value, but nothing followed it`;
|
||||
}
|
||||
if (next.startsWith('-')) {
|
||||
return `flag "${a}" needs a value, but the next argument was another flag: "${next}"`;
|
||||
}
|
||||
i++; // consume the value so it is never re-examined as a positional
|
||||
continue;
|
||||
}
|
||||
|
||||
return `unknown flag "${a}"`;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Gate a CLI on well-formed argv. Writes the diagnostic and sets the exit code
|
||||
* itself, so callers stay a two-line guard:
|
||||
*
|
||||
* if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
*
|
||||
* Never throws, and never calls `process.exit()` — an abrupt exit discards
|
||||
* unflushed stdout when the CLI is on a pipe.
|
||||
*
|
||||
* @param {string[]} args - argv slice.
|
||||
* @param {{ boolean?: string[], value?: string[] }} spec - the CLI's flag surface.
|
||||
* @returns {boolean} true when the CLI may proceed.
|
||||
*/
|
||||
export function requireValidArgs(args, spec) {
|
||||
const error = findArgError(args, spec);
|
||||
if (error === null) return true;
|
||||
process.stderr.write(`Error: ${error}\n`);
|
||||
process.exitCode = 3;
|
||||
return false;
|
||||
}
|
||||
321
scanners/lib/finding-codes.mjs
Normal file
321
scanners/lib/finding-codes.mjs
Normal file
|
|
@ -0,0 +1,321 @@
|
|||
/**
|
||||
* Finding-code registry — the authority for the {NNN} in `CA-{SCANNER}-{NNN}`.
|
||||
*
|
||||
* A finding ID names the CHECK, not the finding's position in a run (M-BUG-28).
|
||||
* Before this registry, `{NNN}` came from an emission counter, so the same check
|
||||
* carried different IDs on different configurations: fixing an unrelated earlier
|
||||
* gap silently renumbered every later one, and a `.config-audit-ignore` entry
|
||||
* retargeted to a neighbouring finding without the user changing anything.
|
||||
*
|
||||
* Rules for editing this file:
|
||||
*
|
||||
* 1. A number, once published, belongs to its check forever. Adding a check
|
||||
* takes the next free number for that scanner — never the next source-order
|
||||
* position, and never a number listed in RETIRED.
|
||||
* 2. Removing a check moves its key to RETIRED. The number is never reissued;
|
||||
* a user's suppression must go dead rather than quietly point at a
|
||||
* different finding. (D1 retired GAP `t3_8` under the old scheme, which is
|
||||
* the incident that motivated the registry.)
|
||||
* 3. Several call sites may share one code when they are arms of one check —
|
||||
* e.g. the forward/reverse arms of a permission conflict. Duplicate
|
||||
* emission is legal; a finding is identified by (id, file, line).
|
||||
* 4. Numbers below are NOT all source order: the ones marked "documented"
|
||||
* are pinned by README / command copy that shipped before the registry.
|
||||
*
|
||||
* GAP keys are the `GAP_CHECKS[].id` values from `feature-gap-scanner.mjs`,
|
||||
* which were already stable. They are declared here rather than derived, so
|
||||
* numbers live in exactly one place; `tests/lib/finding-codes.test.mjs` binds
|
||||
* the two together instead of a second copy of the table drifting.
|
||||
*/
|
||||
|
||||
/**
|
||||
* @type {Record<string, Record<string, number>>}
|
||||
* scanner prefix → check key → number
|
||||
*/
|
||||
export const FINDING_CODES = {
|
||||
// ── CML: claude-md-linter (source order) ────────────────────────────────
|
||||
// `over-char-budget` has two call sites: the conservative 200k anchor and the
|
||||
// `--context-window` calibrated variant. One check, one code.
|
||||
CML: {
|
||||
'no-claude-md': 1,
|
||||
'nested-not-reinjected': 2,
|
||||
'over-500-lines': 3,
|
||||
'over-200-lines': 4,
|
||||
'over-char-budget': 5,
|
||||
'nearly-empty': 6,
|
||||
'missing-sections': 7,
|
||||
'no-headings': 8,
|
||||
'deep-relative-import': 9,
|
||||
'html-comments': 10,
|
||||
'repeated-content': 11,
|
||||
'todo-markers': 12,
|
||||
'dead-prose-reference': 13,
|
||||
},
|
||||
|
||||
// ── SET: settings-validator (source order) ──────────────────────────────
|
||||
SET: {
|
||||
'invalid-json': 1,
|
||||
'key-typo': 2,
|
||||
'deprecated-key': 3,
|
||||
'type-mismatch': 4,
|
||||
'invalid-effort-level': 5,
|
||||
'missing-schema': 6,
|
||||
'no-deny-rules': 7,
|
||||
'no-allow-rules': 8,
|
||||
'many-additional-dirs': 9,
|
||||
'automode-not-object': 10,
|
||||
'automode-unknown-subkey': 11,
|
||||
'automode-subkey-not-string-array': 12,
|
||||
'automode-in-shared-settings': 13,
|
||||
'hooks-as-array': 14,
|
||||
},
|
||||
|
||||
// ── HKV: hook-validator (source order) ──────────────────────────────────
|
||||
HKV: {
|
||||
'invalid-json': 1,
|
||||
'hooks-not-object': 2,
|
||||
'unknown-event': 3,
|
||||
'handlers-not-array': 4,
|
||||
'matcher-not-string': 5,
|
||||
'missing-hooks-array': 6,
|
||||
'invalid-handler-type': 7,
|
||||
'script-not-found': 8,
|
||||
'verbose-output': 9,
|
||||
'unfiltered-additional-context': 10,
|
||||
'timeout-not-number': 11,
|
||||
'timeout-out-of-range': 12,
|
||||
},
|
||||
|
||||
// ── RUL: rules-validator (source order) ─────────────────────────────────
|
||||
RUL: {
|
||||
'no-frontmatter': 1,
|
||||
'globs-instead-of-paths': 2,
|
||||
'pattern-matches-nothing': 3,
|
||||
'nearly-empty': 4,
|
||||
'large-unscoped': 5,
|
||||
'large-scoped-lost-after-compaction': 6,
|
||||
'not-markdown': 7,
|
||||
},
|
||||
|
||||
// ── MCP: mcp-config-validator (source order) ────────────────────────────
|
||||
MCP: {
|
||||
'invalid-json': 1,
|
||||
'unknown-server-type': 2,
|
||||
'sse-transport': 3,
|
||||
'unreferenced-env-var': 4,
|
||||
'unknown-server-field': 5,
|
||||
},
|
||||
|
||||
// ── IMP: import-resolver (source order) ─────────────────────────────────
|
||||
IMP: {
|
||||
'tilde-path': 1,
|
||||
'broken-link': 2,
|
||||
'circular-reference': 3,
|
||||
'deep-chain': 4,
|
||||
},
|
||||
|
||||
// ── CNF: conflict-detector ──────────────────────────────────────────────
|
||||
// `permission-allow-deny` covers both arms (allow-in-A/deny-in-B and reverse).
|
||||
CNF: {
|
||||
'settings-key-conflict': 1,
|
||||
'permission-allow-deny': 2,
|
||||
'duplicate-hook': 3,
|
||||
},
|
||||
|
||||
// ── DIS: disabled-in-schema-scanner (source order) ──────────────────────
|
||||
DIS: {
|
||||
'deny-and-allow': 1,
|
||||
'ineffective-allow-wildcard': 2,
|
||||
'forbidden-param-deny': 3,
|
||||
'forbidden-param-allow': 4,
|
||||
},
|
||||
|
||||
// ── CPS: cache-prefix-scanner (CPS-001 documented) ──────────────────────
|
||||
CPS: {
|
||||
'volatile-in-prefix': 1,
|
||||
'volatile-in-import': 2,
|
||||
},
|
||||
|
||||
// ── COL: collision-scanner (source order) ───────────────────────────────
|
||||
COL: {
|
||||
'skill-user-vs-plugin': 1,
|
||||
'skill-multi-plugin': 2,
|
||||
},
|
||||
|
||||
// ── AGT: agent-listing-scanner (both documented; source order matches) ──
|
||||
AGT: {
|
||||
'description-bloat': 1,
|
||||
'aggregate-listing-budget': 2,
|
||||
},
|
||||
|
||||
// ── OST: output-style-scanner (all three documented) ────────────────────
|
||||
OST: {
|
||||
'strips-coding-instructions': 1,
|
||||
'plugin-forces-style': 2,
|
||||
'style-not-found': 3,
|
||||
},
|
||||
|
||||
// ── OPT: optimization-lens-scanner (documented) ─────────────────────────
|
||||
OPT: {
|
||||
'procedure-should-be-skill': 1,
|
||||
},
|
||||
|
||||
// ── SKL: skill-listing-scanner (all three documented) ───────────────────
|
||||
// `aggregate-listing-budget` has two call sites: the conservative 200k anchor
|
||||
// and the calibrated `--context-window` variant. One check, one code.
|
||||
SKL: {
|
||||
'description-over-cap': 1,
|
||||
'aggregate-listing-budget': 2,
|
||||
'oversized-body': 3,
|
||||
},
|
||||
|
||||
// ── TOK: token-hotspots ─────────────────────────────────────────────────
|
||||
// 1/2/3/5/6 are documented (README + commands/tokens.md). `mcp-schema-deferral`
|
||||
// is documented as 006 although it is the 8th call site in source order, so
|
||||
// `cascade-over-budget` and `stale-plugin-cache` take the free 7 and 8.
|
||||
TOK: {
|
||||
'volatile-top': 1,
|
||||
'redundant-permissions': 2,
|
||||
'deep-import-chain': 3,
|
||||
'bloated-skill-description': 4,
|
||||
'mcp-schema-budget': 5,
|
||||
'mcp-schema-deferral': 6,
|
||||
'cascade-over-budget': 7,
|
||||
'stale-plugin-cache': 8,
|
||||
},
|
||||
|
||||
// ── PLH: plugin-health-scanner ──────────────────────────────────────────
|
||||
// 15 and 16 are documented (README v5.4.0 entry) but sit at source positions
|
||||
// 3 and 4; the remaining checks take {1…14, 17, 18, 19} in source order.
|
||||
PLH: {
|
||||
'invalid-plugin-json': 1,
|
||||
'missing-required-field': 2,
|
||||
'missing-plugin-json': 3,
|
||||
'claude-md-missing-section': 4,
|
||||
'missing-claude-md': 5,
|
||||
'command-missing-frontmatter': 6,
|
||||
'command-missing-field': 7,
|
||||
'agent-missing-frontmatter': 8,
|
||||
'agent-missing-field': 9,
|
||||
'agent-ignored-key': 10,
|
||||
'hooks-json-invalid-structure': 11,
|
||||
'hooks-json-array': 12,
|
||||
'hooks-json-invalid': 13,
|
||||
'unknown-plugin-file': 14,
|
||||
'plugin-json-shadows-default': 15,
|
||||
'skills-array-entry': 16,
|
||||
'no-plugins-found': 17,
|
||||
'command-name-collision': 18,
|
||||
'namespace-collision': 19,
|
||||
},
|
||||
|
||||
// ── GAP: feature-gap-scanner ────────────────────────────────────────────
|
||||
// Keys are GAP_CHECKS[].id for dimensions, and the lever code for the
|
||||
// conditional levers the scanner emits after the loop. Numbers 1–24 happen to
|
||||
// follow the current table order because that is how the dimensions were first
|
||||
// published — NOT because position determines the number. A new check takes the
|
||||
// next free number wherever it sits in the file (M-BUG-28).
|
||||
GAP: {
|
||||
t1_1: 1,
|
||||
t1_2: 2,
|
||||
t1_3: 3,
|
||||
t1_4: 4,
|
||||
t1_5: 5,
|
||||
t2_1: 6,
|
||||
t2_2: 7,
|
||||
t2_3: 8,
|
||||
t2_4: 9,
|
||||
t2_5: 10,
|
||||
t2_6: 11,
|
||||
t2_7: 12,
|
||||
t3_1: 13,
|
||||
t3_2: 14,
|
||||
t3_3: 15,
|
||||
t3_4: 16,
|
||||
t3_5: 17,
|
||||
t3_6: 18,
|
||||
t3_7: 19,
|
||||
t4_1: 20,
|
||||
t4_2: 21,
|
||||
t4_3: 22,
|
||||
t4_4: 23,
|
||||
t4_5: 24,
|
||||
'bundled-skills-lever': 25,
|
||||
'cli-over-mcp-lever': 26,
|
||||
'filter-hook-output-lever': 27,
|
||||
'agent-model-routing-lever': 28,
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Keys withdrawn from a scanner. Their numbers are never reissued, so a stale
|
||||
* suppression goes dead instead of silently naming a different check.
|
||||
* @type {Record<string, string[]>}
|
||||
*/
|
||||
export const RETIRED_CODES = {
|
||||
// D1 (4027cdc, 2026-08-09): "No autoMode classifier" — /doctor Check 8 covers
|
||||
// auto mode with usage-weighted judgement, so the nudge went. The number it
|
||||
// occupied under the old counter scheme is not reused.
|
||||
GAP: ['t3_8'],
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve a check key to its published number.
|
||||
* Throws rather than falling back: a fallback would let a half-converted scanner
|
||||
* ship IDs that look valid, which is the silent-degradation class this registry
|
||||
* exists to remove.
|
||||
* @param {string} scanner - scanner prefix, e.g. 'GAP'
|
||||
* @param {string} code - check key, e.g. 't3_7'
|
||||
* @returns {number}
|
||||
*/
|
||||
export function codeNumber(scanner, code) {
|
||||
const table = FINDING_CODES[scanner];
|
||||
if (!table) {
|
||||
throw new Error(`finding(): unknown scanner "${scanner}" — add it to FINDING_CODES`);
|
||||
}
|
||||
if (code === undefined || code === null || code === '') {
|
||||
throw new Error(`finding(): missing "code" for scanner ${scanner} — every finding must name its check`);
|
||||
}
|
||||
if (!Object.prototype.hasOwnProperty.call(table, code)) {
|
||||
const retired = (RETIRED_CODES[scanner] || []).includes(code);
|
||||
throw new Error(
|
||||
retired
|
||||
? `finding(): check "${code}" is RETIRED for ${scanner} — retired numbers are never reissued`
|
||||
: `finding(): undeclared check "${code}" for ${scanner} — add it to FINDING_CODES with the next free number`
|
||||
);
|
||||
}
|
||||
return table[code];
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a finding ID from a check key.
|
||||
* @param {string} scanner
|
||||
* @param {string} code
|
||||
* @returns {string} e.g. 'CA-GAP-019'
|
||||
*/
|
||||
export function findingId(scanner, code) {
|
||||
return `CA-${scanner}-${String(codeNumber(scanner, code)).padStart(3, '0')}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every declared ID, as a flat set — used to validate suppression patterns so a
|
||||
* stale pin is reported instead of silently matching nothing.
|
||||
* @returns {Set<string>}
|
||||
*/
|
||||
export function allFindingIds() {
|
||||
const ids = new Set();
|
||||
for (const [scanner, table] of Object.entries(FINDING_CODES)) {
|
||||
for (const n of Object.values(table)) {
|
||||
ids.add(`CA-${scanner}-${String(n).padStart(3, '0')}`);
|
||||
}
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scanner prefixes the registry knows about.
|
||||
* @returns {string[]}
|
||||
*/
|
||||
export function knownScanners() {
|
||||
return Object.keys(FINDING_CODES);
|
||||
}
|
||||
|
|
@ -77,6 +77,11 @@ export const TRANSLATIONS = {
|
|||
description: 'HTML comments still count as text sent to Claude on every turn — they don\'t actually hide anything.',
|
||||
recommendation: 'Delete the comment text if you don\'t want it sent, or convert it to a regular note.',
|
||||
},
|
||||
'CLAUDE.md points at files that are not there': {
|
||||
title: 'Your instructions file links to files that are not there',
|
||||
description: 'Some file paths written in `CLAUDE.md` point at files that do not exist — not next to the file, and not from your project root. Anyone following them finds nothing.',
|
||||
recommendation: 'Point each path at where the file actually lives, or drop the reference. Only clear relative paths are checked; web links, wildcards and plain file names are left alone.',
|
||||
},
|
||||
'Contains TODO/FIXME markers': {
|
||||
title: 'Your file has TODO or FIXME notes',
|
||||
description: 'These notes are sent to Claude on every turn even when they\'re internal reminders.',
|
||||
|
|
@ -499,11 +504,6 @@ export const TRANSLATIONS = {
|
|||
description: 'Dynamic context lets a skill see fresh information (file contents, command output) at the moment it runs, not at the time it was written.',
|
||||
recommendation: 'Use the dynamic-context block in skills that need up-to-date information.',
|
||||
},
|
||||
'No autoMode classifier': {
|
||||
title: 'You haven\'t set up auto-mode classification',
|
||||
description: 'Auto-mode classification helps Claude decide when to act on its own vs. ask you, based on the kind of task.',
|
||||
recommendation: 'Add an auto-mode classifier in your settings if you want this nuance.',
|
||||
},
|
||||
'No project .mcp.json in git': {
|
||||
title: 'Your team has no shared list of connected services',
|
||||
description: 'Without a project-level connected-services file, every teammate has to set up their own connections.',
|
||||
|
|
@ -529,6 +529,30 @@ export const TRANSLATIONS = {
|
|||
description: 'Language-server connections let Claude see types, error messages, and definitions the same way your editor does.',
|
||||
recommendation: 'Set up LSP integration if you work in a typed language.',
|
||||
},
|
||||
// Conditional levers. These are not "a feature you haven't set up" — they
|
||||
// fire only under a measured condition, so the generic _default would
|
||||
// misdescribe them. Every title the scanner can emit needs an entry here
|
||||
// (guarded in tests/scanners/feature-gap-scanner.test.mjs).
|
||||
'Bundled skills add to an over-budget skill listing': {
|
||||
title: 'Built-in skills are crowding an already-full skill list',
|
||||
description: 'Claude Code loads its own built-in skills into the same limited list as yours. Your list is already over budget, so entries risk being cut off and Claude may miss the right skill.',
|
||||
recommendation: 'Turn off the built-in skills to free up room — unless you use them, in which case shorten your own skill descriptions instead.',
|
||||
},
|
||||
'Prefer CLI over MCP for common operations': {
|
||||
title: 'Some connected services load their full tool list every turn',
|
||||
description: 'Most connected services only cost tokens when used, but yours are set to load everything upfront. That weight is there whether you use them or not.',
|
||||
recommendation: 'For services with a command-line equivalent (like `gh` or `aws`), the command line costs nothing until you run it.',
|
||||
},
|
||||
'Filter hook output before it enters context': {
|
||||
title: 'An automation is pasting its full output into the conversation',
|
||||
description: 'An automation that injects its output adds it to every turn that follows. Unfiltered command output can be much larger than the part that actually matters.',
|
||||
recommendation: 'Trim the output inside the script itself, so only the useful lines reach the conversation.',
|
||||
},
|
||||
'Subagents pin neither model nor effort': {
|
||||
title: 'Your helper agents all run at the same cost as your main session',
|
||||
description: 'A subagent that names no model inherits the one you are using, so routine delegated work costs the same as your hardest work. Reasoning effort is a separate dial with the same default.',
|
||||
recommendation: 'Give mechanical agents (search, extraction, summarizing) a smaller model or a lower effort level, and keep the strong settings for the work that needs judgement.',
|
||||
},
|
||||
},
|
||||
patterns: [],
|
||||
_default: {
|
||||
|
|
|
|||
|
|
@ -16,6 +16,14 @@
|
|||
/** Default re-verify cadence: a confirmed best-practice older than this needs a re-check. */
|
||||
export const STALE_AFTER_DAYS_DEFAULT = 90;
|
||||
|
||||
/**
|
||||
* Default evidence cadence: when the NEWEST `published` date across an entry's
|
||||
* sources is older than this, the entry is flagged even if recently re-verified.
|
||||
* Re-verifying the old source does not clear it — only a newer source does
|
||||
* (the BP-SUB-001 defect class: green stamp, substantially outdated evidence).
|
||||
*/
|
||||
export const EVIDENCE_STALE_AFTER_DAYS_DEFAULT = 365;
|
||||
|
||||
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
|
||||
const DAY_MS = 86_400_000;
|
||||
|
||||
|
|
@ -46,15 +54,40 @@ function verifiedMs(entry) {
|
|||
return Number.isNaN(ms) ? null : ms;
|
||||
}
|
||||
|
||||
/** Parse a YYYY-MM-DD string to UTC-midnight ms, or null. */
|
||||
function dateMs(v) {
|
||||
if (typeof v !== 'string' || !DATE_RE.test(v)) return null;
|
||||
const ms = Date.parse(`${v}T00:00:00Z`);
|
||||
return Number.isNaN(ms) ? null : ms;
|
||||
}
|
||||
|
||||
/**
|
||||
* Newest `published` across the primary `source` and any corroborating `sources[]`.
|
||||
* Null when no source carries a parseable published date (the evidence-age rule
|
||||
* is then silent for that entry — it cannot judge evidence it cannot date).
|
||||
*/
|
||||
function newestEvidenceMs(entry) {
|
||||
const candidates = [];
|
||||
if (entry && entry.source) candidates.push(entry.source);
|
||||
if (entry && Array.isArray(entry.sources)) candidates.push(...entry.sources);
|
||||
let newest = null;
|
||||
for (const s of candidates) {
|
||||
const ms = dateMs(s && s.published);
|
||||
if (ms !== null && (newest === null || ms > newest)) newest = ms;
|
||||
}
|
||||
return newest;
|
||||
}
|
||||
|
||||
/**
|
||||
* Classify every register entry as fresh or stale by the age of its source.verified stamp.
|
||||
*
|
||||
* @param {{entries:object[]}} register
|
||||
* @param {{ referenceDate: string|Date, staleAfterDays?: number }} opts
|
||||
* @param {{ referenceDate: string|Date, staleAfterDays?: number, evidenceStaleAfterDays?: number }} opts
|
||||
* @returns {{
|
||||
* referenceDate: string,
|
||||
* staleAfterDays: number,
|
||||
* stale: Array<{id:string, verified:string|undefined, ageDays:number|null, url:string|undefined, claim:string|undefined}>,
|
||||
* evidenceStaleAfterDays: number,
|
||||
* stale: Array<{id:string, verified:string|undefined, ageDays:number|null, url:string|undefined, claim:string|undefined, reasons:string[]}>,
|
||||
* fresh: Array<{id:string, verified:string|undefined, ageDays:number}>,
|
||||
* counts: { total:number, stale:number, fresh:number }
|
||||
* }}
|
||||
|
|
@ -63,6 +96,10 @@ export function assessFreshness(register, opts = {}) {
|
|||
const ref = normalizeReferenceDate(opts.referenceDate);
|
||||
const staleAfterDays =
|
||||
typeof opts.staleAfterDays === 'number' ? opts.staleAfterDays : STALE_AFTER_DAYS_DEFAULT;
|
||||
const evidenceStaleAfterDays =
|
||||
typeof opts.evidenceStaleAfterDays === 'number'
|
||||
? opts.evidenceStaleAfterDays
|
||||
: EVIDENCE_STALE_AFTER_DAYS_DEFAULT;
|
||||
|
||||
const entries = (register && Array.isArray(register.entries)) ? register.entries : [];
|
||||
const stale = [];
|
||||
|
|
@ -71,14 +108,30 @@ export function assessFreshness(register, opts = {}) {
|
|||
for (const e of entries) {
|
||||
const verified = e && e.source ? e.source.verified : undefined;
|
||||
const vms = verifiedMs(e);
|
||||
const reasons = [];
|
||||
let ageDays = null;
|
||||
|
||||
if (vms === null) {
|
||||
// No re-checkable date → needs attention. Stale with ageDays null.
|
||||
stale.push({ id: e && e.id, verified, ageDays: null, url: e && e.source && e.source.url, claim: e && e.claim });
|
||||
continue;
|
||||
// No re-checkable date → needs attention.
|
||||
reasons.push('no-verified-date');
|
||||
} else {
|
||||
ageDays = Math.floor((ref.ms - vms) / DAY_MS);
|
||||
if (ageDays > staleAfterDays) reasons.push('verified-age');
|
||||
}
|
||||
const ageDays = Math.floor((ref.ms - vms) / DAY_MS);
|
||||
if (ageDays > staleAfterDays) {
|
||||
stale.push({ id: e.id, verified, ageDays, url: e.source && e.source.url, claim: e.claim });
|
||||
|
||||
// A source explicitly marked as superseded is stale no matter how fresh the
|
||||
// verified stamp is — the stamp certifies the OLD source.
|
||||
if (e && e.source && e.source.supersededBy) reasons.push('superseded');
|
||||
|
||||
// Evidence age: keyed on the newest published date across all sources, so a
|
||||
// re-read of the old source never clears it — only newer evidence does.
|
||||
const evMs = newestEvidenceMs(e);
|
||||
if (evMs !== null && Math.floor((ref.ms - evMs) / DAY_MS) > evidenceStaleAfterDays) {
|
||||
reasons.push('evidence-age');
|
||||
}
|
||||
|
||||
if (reasons.length > 0) {
|
||||
stale.push({ id: e && e.id, verified, ageDays, url: e && e.source && e.source.url, claim: e && e.claim, reasons });
|
||||
} else {
|
||||
fresh.push({ id: e.id, verified, ageDays });
|
||||
}
|
||||
|
|
@ -87,6 +140,7 @@ export function assessFreshness(register, opts = {}) {
|
|||
return {
|
||||
referenceDate: ref.iso,
|
||||
staleAfterDays,
|
||||
evidenceStaleAfterDays,
|
||||
stale,
|
||||
fresh,
|
||||
counts: { total: entries.length, stale: stale.length, fresh: fresh.length },
|
||||
|
|
|
|||
|
|
@ -5,18 +5,13 @@
|
|||
*/
|
||||
|
||||
import { riskScore, riskBand, verdict } from './severity.mjs';
|
||||
|
||||
let findingCounter = 0;
|
||||
|
||||
/** Reset the finding counter. Call in beforeEach of tests and before each scanner run. */
|
||||
export function resetCounter() {
|
||||
findingCounter = 0;
|
||||
}
|
||||
import { findingId } from './finding-codes.mjs';
|
||||
|
||||
/**
|
||||
* Create a finding object with auto-incremented ID.
|
||||
* Create a finding object. The ID names the CHECK — see `finding-codes.mjs`.
|
||||
* @param {object} opts
|
||||
* @param {string} opts.scanner - 3-letter scanner prefix (CML, SET, HKV, RUL, etc.)
|
||||
* @param {string} opts.code - check key declared in FINDING_CODES for this scanner
|
||||
* @param {string} opts.severity - critical | high | medium | low | info
|
||||
* @param {string} opts.title
|
||||
* @param {string} opts.description
|
||||
|
|
@ -30,10 +25,8 @@ export function resetCounter() {
|
|||
* @returns {object}
|
||||
*/
|
||||
export function finding(opts) {
|
||||
findingCounter++;
|
||||
const id = `CA-${opts.scanner}-${String(findingCounter).padStart(3, '0')}`;
|
||||
const result = {
|
||||
id,
|
||||
id: findingId(opts.scanner, opts.code),
|
||||
scanner: opts.scanner,
|
||||
severity: opts.severity,
|
||||
title: opts.title,
|
||||
|
|
|
|||
109
scanners/lib/prompting-model-scope.mjs
Normal file
109
scanners/lib/prompting-model-scope.mjs
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
/**
|
||||
* Model-scoped ANNOTATION on top of the subtraction lens.
|
||||
*
|
||||
* This module generates no candidates of its own. `optimize --subtract` already
|
||||
* surfaces compensatory instructions through the single `compensatory-instruction`
|
||||
* detector (`BP-SUB-001`); this answers a narrower question over text that
|
||||
* already passed it: is this specifically the class of instruction a named model
|
||||
* documents as redundant — self-verification, or verification delegated to a
|
||||
* subagent?
|
||||
*
|
||||
* A second competing detector is deliberately NOT what this is. The register
|
||||
* entry it cites carries `lensCheck: null`, the same discipline `BP-JUDG-001`
|
||||
* shipped under: a plausible-looking detector for "instruction a model no longer
|
||||
* needs" measured 7/7 false positives across 409 real CLAUDE.md files, so this
|
||||
* claim class rides an existing measured detector rather than widening the
|
||||
* candidate set.
|
||||
*
|
||||
* Precision comes from requiring a reflexive/delegate TARGET alongside the
|
||||
* verify verb, never from narrowing the verb list. A bare "check"/"verify" also
|
||||
* matches EXTERNAL verification ("check the CI status") — which stays a true
|
||||
* negative here even though it is, correctly, still a `BP-SUB-001` candidate
|
||||
* upstream.
|
||||
*
|
||||
* Pure: text → annotation or null. Zero external dependencies.
|
||||
*/
|
||||
import { LB, RB } from './subtraction-prefilter.mjs';
|
||||
|
||||
/**
|
||||
* The reflexive self-verification target. This is what narrows a bare
|
||||
* verify/check imperative down to the specific claim the model's documented
|
||||
* self-correction contradicts.
|
||||
*/
|
||||
const SELF_TARGET_RE = new RegExp(
|
||||
LB +
|
||||
'(?:your (?:own )?(?:work|output|answer|changes)|yourself|' +
|
||||
'before (?:responding|submitting|finalizing)|dine egne?|deg selv|før du svarer)' +
|
||||
RB,
|
||||
'i',
|
||||
);
|
||||
|
||||
/**
|
||||
* Verify verbs — deliberately as broad as `subtraction-prefilter.mjs`'s
|
||||
* `IMPERATIVE_RE`. Breadth here is safe because a co-occurring target is
|
||||
* required; narrowing the list would only lose true positives.
|
||||
*/
|
||||
const VERIFY_VERB_RE = new RegExp(
|
||||
LB +
|
||||
'(?:double-check|re-verify|re-check|confirm|verify|review|check|' +
|
||||
'dobbeltsjekk|verifiser|sjekk)' +
|
||||
RB,
|
||||
'i',
|
||||
);
|
||||
|
||||
/**
|
||||
* Delegated verification. Order-free on purpose: it must match both
|
||||
* "verify X with a subagent" and "use a subagent to verify X".
|
||||
*/
|
||||
const DELEGATE_VERIFY_RE = new RegExp(
|
||||
LB + '(?:subagent|sub-agent|another (?:agent|instance)|task tool)' + RB,
|
||||
'i',
|
||||
);
|
||||
|
||||
/**
|
||||
* Does this text carry a self- or delegate-targeted verification instruction?
|
||||
*
|
||||
* @param {string} text
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function isModelContradictedVerification(text) {
|
||||
return (
|
||||
VERIFY_VERB_RE.test(text) &&
|
||||
(SELF_TARGET_RE.test(text) || DELEGATE_VERIFY_RE.test(text))
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Model names are compared on their alphanumeric skeleton, so "opus-5",
|
||||
* "Opus 5" and "OPUS5" are one model. A typo'd name simply fails to match —
|
||||
* the CLI reports that it did not recognize the name rather than reporting a
|
||||
* silent zero.
|
||||
*
|
||||
* @param {unknown} s
|
||||
* @returns {string}
|
||||
*/
|
||||
const normalizeModel = (s) =>
|
||||
String(s || '')
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]/g, '');
|
||||
|
||||
/**
|
||||
* Annotate a subtraction candidate with a model-scoped register citation.
|
||||
*
|
||||
* @param {string} text candidate block text (already a `BP-SUB-001` candidate)
|
||||
* @param {string|null|undefined} targetModel the model named by `--for-model`
|
||||
* @param {Array<{id: string, claim: string, modelScope?: string[]}>} entries
|
||||
* confirmed register entries with `category: 'prompting-fit'`
|
||||
* @returns {{registerId: string, claim: string, requestedModel: string}|null}
|
||||
*/
|
||||
export function matchModelScope(text, targetModel, entries) {
|
||||
if (!targetModel || !isModelContradictedVerification(text)) return null;
|
||||
const wanted = normalizeModel(targetModel);
|
||||
const entry = (entries || []).find((e) =>
|
||||
(e.modelScope || []).some((m) => normalizeModel(m) === wanted),
|
||||
);
|
||||
if (!entry) return null;
|
||||
return { registerId: entry.id, claim: entry.claim, requestedModel: targetModel };
|
||||
}
|
||||
|
||||
export { normalizeModel };
|
||||
49
scanners/lib/require-target-dir.mjs
Normal file
49
scanners/lib/require-target-dir.mjs
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
/**
|
||||
* Target-path precondition shared by the target-taking CLIs.
|
||||
*
|
||||
* A scan target is a scan ROOT. If it does not exist, or is not a directory,
|
||||
* the scanner cannot do its job — and by the plugin's exit-code contract that
|
||||
* is exit 3, not a verdict. Codes 0/1/2 are PASS/WARNING/FAIL *about a
|
||||
* configuration that was examined*; every command template gates on exactly
|
||||
* that distinction, so returning a verdict for an unreadable target sends a
|
||||
* typo'd path through the whole workflow as a clean result.
|
||||
*
|
||||
* Measured before this guard existed (session #56):
|
||||
* node scanners/posture.mjs /nonexistent/path/xyz → exit 0,
|
||||
* "Health: B (86/100) — Good shape — a few items to address"
|
||||
*
|
||||
* The message and exit code here are not new: `manifest.mjs`,
|
||||
* `token-hotspots-cli.mjs`, `whats-active.mjs` and `optimize-lens-cli.mjs`
|
||||
* already carried this exact block inline. This module is where the CLIs that
|
||||
* lacked it get it from; the four that have their own copies are left alone
|
||||
* (consolidating them is a cleanup, not part of this fix).
|
||||
*/
|
||||
|
||||
import { stat } from 'node:fs/promises';
|
||||
|
||||
/**
|
||||
* Verify that `absPath` is an existing directory.
|
||||
*
|
||||
* Writes the diagnostic to stderr itself, so callers stay a two-line guard:
|
||||
*
|
||||
* if (!(await requireTargetDir(resolvedPath))) { process.exitCode = 3; return; }
|
||||
*
|
||||
* Never throws, and never calls `process.exit()` — an abrupt exit discards
|
||||
* unflushed stdout when the CLI is on a pipe.
|
||||
*
|
||||
* @param {string} absPath - Resolved absolute target path.
|
||||
* @returns {Promise<boolean>} true when the target is usable as a scan root.
|
||||
*/
|
||||
export async function requireTargetDir(absPath) {
|
||||
try {
|
||||
const s = await stat(absPath);
|
||||
if (!s.isDirectory()) {
|
||||
process.stderr.write(`Error: ${absPath} is not a directory\n`);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
} catch {
|
||||
process.stderr.write(`Error: path does not exist: ${absPath}\n`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
|
@ -20,17 +20,17 @@ const GRADE_CONTEXT = {
|
|||
|
||||
// --- Tier weights for utilization calculation ---
|
||||
const TIER_WEIGHTS = { t1: 3, t2: 2, t3: 1, t4: 1 };
|
||||
const TIER_COUNTS = { t1: 5, t2: 7, t3: 8, t4: 5 };
|
||||
const TOTAL_DIMENSIONS = 25;
|
||||
const TIER_COUNTS = { t1: 5, t2: 7, t3: 7, t4: 5 };
|
||||
const TOTAL_DIMENSIONS = 24;
|
||||
const MAX_WEIGHTED = Object.entries(TIER_COUNTS).reduce(
|
||||
(sum, [tier, count]) => sum + count * TIER_WEIGHTS[tier],
|
||||
0,
|
||||
); // 5*3 + 7*2 + 8*1 + 5*1 = 42
|
||||
); // 5*3 + 7*2 + 7*1 + 5*1 = 41
|
||||
|
||||
/**
|
||||
* Calculate weighted utilization from GAP scanner findings.
|
||||
* @param {object[]} gapFindings - Array of GAP scanner findings (each has .category = t1|t2|t3|t4)
|
||||
* @param {number} [totalDimensions=25]
|
||||
* @param {number} [totalDimensions=24]
|
||||
* @returns {{ score: number, overhang: number }}
|
||||
*/
|
||||
export function calculateUtilization(gapFindings, totalDimensions = TOTAL_DIMENSIONS) {
|
||||
|
|
@ -102,7 +102,7 @@ function findGapId(finding) {
|
|||
return TITLE_TO_ID[finding.title] || 'unknown';
|
||||
}
|
||||
|
||||
/** Title→ID mapping for all 25 gap checks */
|
||||
/** Title→ID mapping for all 24 gap checks */
|
||||
const TITLE_TO_ID = {
|
||||
'No CLAUDE.md file': 't1_1',
|
||||
'No permissions configured': 't1_2',
|
||||
|
|
@ -123,7 +123,6 @@ const TITLE_TO_ID = {
|
|||
'No advanced skill frontmatter': 't3_5',
|
||||
'No subagent isolation': 't3_6',
|
||||
'No dynamic skill context': 't3_7',
|
||||
'No autoMode classifier': 't3_8',
|
||||
'No project .mcp.json in git': 't4_1',
|
||||
'No custom plugin': 't4_2',
|
||||
'Agent teams not enabled': 't4_3',
|
||||
|
|
@ -404,4 +403,4 @@ export function generateHealthScorecard(areaScores, opportunityCount, options =
|
|||
return lines.join('\n');
|
||||
}
|
||||
|
||||
export { TITLE_TO_ID, TIER_WEIGHTS, TIER_COUNTS, MAX_WEIGHTED, MATURITY_LEVELS, SEGMENTS };
|
||||
export { TITLE_TO_ID, TIER_WEIGHTS, TIER_COUNTS, TOTAL_DIMENSIONS, MAX_WEIGHTED, MATURITY_LEVELS, SEGMENTS };
|
||||
|
|
|
|||
|
|
@ -36,6 +36,23 @@ export const DESCRIPTION_CAP = 1536;
|
|||
// The 200k/1M window constants live in context-window.mjs (single source of
|
||||
// truth, shared with the CML CLAUDE.md char-budget check); re-exported here so
|
||||
// existing importers of this module keep working.
|
||||
// D2 re-verification (2026-08-09, CC 2.1.226). CC 2.1.226's /doctor reports a
|
||||
// combined skill+command+agent listing and puts the budget near ~1% (~10,000
|
||||
// tok on a 1M window) — half of ours. Checked against the primary source
|
||||
// before touching the number: the CC changelog contains EXACTLY ONE
|
||||
// budget-fraction statement (L3786, under 2.1.32) and no later entry
|
||||
// supersedes it, so 2% stands and CA-SKL-002 is NOT a /doctor duplicate
|
||||
// carrying a stale figure. /doctor's arithmetic could not be reconciled from
|
||||
// the changelog, and /doctor discloses its own numbers as disk estimates
|
||||
// (chars÷4), so its ~1% is recorded, not adopted.
|
||||
//
|
||||
// NOT VERIFIED, deliberately left alone: L3786 says "skill CHARACTER budget
|
||||
// now scales with context window (2% of context)". We express the budget in
|
||||
// TOKENS (0.02 × 200k = 4000 tok). Whether CC's budget is 2% counted in
|
||||
// characters or in tokens is not resolvable from the changelog wording, and no
|
||||
// primary source settles it — a 4× difference rides on the answer. Changing
|
||||
// the constant on that ambiguity would be a guess; it stays until a primary
|
||||
// source decides it.
|
||||
export const BUDGET_FRACTION = 0.02;
|
||||
export const AGGREGATE_BUDGET_TOKENS = Math.round(BUDGET_FRACTION * CONTEXT_WINDOW_ANCHOR); // 4000
|
||||
export const LARGE_CONTEXT_BUDGET_TOKENS = Math.round(BUDGET_FRACTION * LARGE_CONTEXT_WINDOW); // 20000
|
||||
|
|
|
|||
|
|
@ -69,8 +69,8 @@ export const SUBTRACT_DETECTORS = Object.freeze([
|
|||
* Every Norwegian keyword ending in æ/ø/å was silently dead until the dogfood
|
||||
* run surfaced it. Do not reintroduce `\b` around this vocabulary.
|
||||
*/
|
||||
const LB = '(?<![\\wæøåÆØÅ])';
|
||||
const RB = '(?![\\wæøåÆØÅ])';
|
||||
export const LB = '(?<![\\wæøåÆØÅ])';
|
||||
export const RB = '(?![\\wæøåÆØÅ])';
|
||||
|
||||
const ABSOLUTE_RE = new RegExp(
|
||||
LB +
|
||||
|
|
|
|||
242
scanners/lib/subtraction-write.mjs
Normal file
242
scanners/lib/subtraction-write.mjs
Normal file
|
|
@ -0,0 +1,242 @@
|
|||
/**
|
||||
* subtraction-write — the write half of `optimize --subtract` (§C6, chunk #63).
|
||||
*
|
||||
* This is the only path in the plugin that REMOVES configuration, so the split
|
||||
* of labour matters more here than anywhere else: **the judgement is the
|
||||
* agent's, the execution is deterministic.** Everything below is mechanical —
|
||||
* it verifies that the block it was told to remove is still exactly the block
|
||||
* that is there, and refuses otherwise.
|
||||
*
|
||||
* ## Why this is not a `fix-engine` action
|
||||
*
|
||||
* Measured (#63): the subtraction axis appears nowhere in `scan-orchestrator`
|
||||
* or `optimization-lens-scanner` — it is computed inside `optimize-lens-cli`
|
||||
* under `--subtract`. `fix-engine.verifyFixes()` marks a fix `verified` when
|
||||
* the finding is absent from a re-scan, so a subtraction removal would be
|
||||
* verified **whether or not the write happened**: a success-shaped no-op, the
|
||||
* class that made `restoreBackup` silently do nothing before `parseManifest`
|
||||
* learned the second manifest format. And `planFixes` keys on
|
||||
* `finding.autoFixable` + `finding.title` from an envelope, neither of which an
|
||||
* agent prose judgement has.
|
||||
*
|
||||
* Nor is it a `plan`/`implement` step: that pipeline runs on findings, and
|
||||
* `finding-codes.mjs` declares exactly one `OPT` code — for the deterministic
|
||||
* check. Minting a code for a prose judgement breaks that module's invariant
|
||||
* that a code names a deterministic CHECK.
|
||||
*
|
||||
* ## The floor, repeated rather than moved
|
||||
*
|
||||
* §C6 forbids migrating the floor into the write path. That forbids *moving*
|
||||
* the veto, not *repeating* it: `subtraction-prefilter` still consults
|
||||
* `floor-exclusion` before anything is ever proposed, and this module refuses a
|
||||
* load-bearing block again as the last red line before an irreversible-by-
|
||||
* reading delete. A caller that hand-builds an approval therefore cannot route
|
||||
* around the floor.
|
||||
*
|
||||
* ## The archive question
|
||||
*
|
||||
* "`mv` to `_archive/`, never `rm`" is a FILE-level rule; nothing here deletes
|
||||
* a file. The timestamped backup is the recovery artifact — it holds the whole
|
||||
* pre-removal file and `rollback` already restores it. The removed text also
|
||||
* rides back in the payload so the caller can show and log it. A second archive
|
||||
* copy with no restorer behind it would be worse than none.
|
||||
*
|
||||
* Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { readFile, writeFile } from 'node:fs/promises';
|
||||
import { resolve } from 'node:path';
|
||||
|
||||
import { isLoadBearing } from './floor-exclusion.mjs';
|
||||
import { createBackup } from './backup.mjs';
|
||||
import { classifyWriteTarget, strongestGate } from './write-scope.mjs';
|
||||
|
||||
/** Why a removal did not happen. Every refusal carries exactly one of these. */
|
||||
export const REFUSAL_REASONS = Object.freeze({
|
||||
/** The file no longer reads the way the approval says it does. */
|
||||
BLOCK_MISMATCH: 'block-mismatch',
|
||||
/** `floor-exclusion` vetoes the block — never removable, at any layer. */
|
||||
FLOOR: 'floor',
|
||||
/** The target's scope class needs an explicit go-ahead that was not given. */
|
||||
SCOPE_GATE: 'scope-gate',
|
||||
/** The file could not be read, so it can be neither backed up nor excised. */
|
||||
UNREADABLE: 'unreadable',
|
||||
/** The backup does not cover a file the run was about to write. */
|
||||
BACKUP_INCOMPLETE: 'backup-incomplete',
|
||||
});
|
||||
|
||||
/** True for a line that is empty or whitespace only. */
|
||||
const isBlank = (line) => line === undefined || /^\s*$/.test(line);
|
||||
|
||||
/**
|
||||
* Remove approved blocks from one file's content.
|
||||
*
|
||||
* Pure: no filesystem, no clock. Every span is validated against the ORIGINAL
|
||||
* content and the removals are then applied in descending line order, so an
|
||||
* earlier removal cannot shift a later span out from under itself — the shape
|
||||
* that made `fix-engine` apply a file-rename before a fix that still addressed
|
||||
* the old path, failing with ENOENT while the run exited 0.
|
||||
*
|
||||
* @param {string} content - The file as it is on disk right now.
|
||||
* @param {Array<{line:number, endLine:number, text:string}>} removals
|
||||
* @returns {{content: string, applied: object[], refused: object[]}}
|
||||
*/
|
||||
export function exciseBlocks(content, removals) {
|
||||
const lines = content.split('\n');
|
||||
const applied = [];
|
||||
const refused = [];
|
||||
|
||||
for (const removal of removals) {
|
||||
const { line, endLine } = removal;
|
||||
const inRange =
|
||||
Number.isInteger(line) && Number.isInteger(endLine) &&
|
||||
line >= 1 && endLine >= line && endLine <= lines.length;
|
||||
|
||||
if (!inRange) {
|
||||
refused.push({ ...removal, reason: REFUSAL_REASONS.BLOCK_MISMATCH });
|
||||
continue;
|
||||
}
|
||||
|
||||
// The pre-filter reports `text: block.text.trim()`, so compare trimmed —
|
||||
// a raw slice comparison would refuse every genuine approval.
|
||||
const actual = lines.slice(line - 1, endLine).join('\n');
|
||||
if (actual.trim() !== String(removal.text ?? '').trim()) {
|
||||
refused.push({ ...removal, reason: REFUSAL_REASONS.BLOCK_MISMATCH });
|
||||
continue;
|
||||
}
|
||||
|
||||
// Judged on what is actually in the file, not on what the caller claims is.
|
||||
if (isLoadBearing(actual)) {
|
||||
refused.push({ ...removal, reason: REFUSAL_REASONS.FLOOR });
|
||||
continue;
|
||||
}
|
||||
|
||||
applied.push({ ...removal, text: actual.trim() });
|
||||
}
|
||||
|
||||
const descending = [...applied].sort((a, b) => b.line - a.line);
|
||||
for (const { line, endLine } of descending) {
|
||||
lines.splice(line - 1, endLine - line + 1);
|
||||
// A leaf block sits between blank lines; removing it leaves two in a row.
|
||||
if (line >= 2 && isBlank(lines[line - 2]) && isBlank(lines[line - 1])) {
|
||||
lines.splice(line - 1, 1);
|
||||
}
|
||||
}
|
||||
|
||||
return { content: lines.join('\n'), applied, refused };
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply an approved subtraction set to disk, behind the scope gate and a
|
||||
* verified backup.
|
||||
*
|
||||
* The run is all-or-nothing across files: a target that cannot be read aborts
|
||||
* the whole set rather than applying half of one operator decision.
|
||||
*
|
||||
* @param {Array<{file:string, line:number, endLine:number, text:string}>} removals
|
||||
* @param {object} [opts]
|
||||
* @param {string|null} [opts.repoRoot] - Repo root the session stands in.
|
||||
* @param {boolean} [opts.approveScope=false] - Operator's explicit go-ahead for a `require-ok` target.
|
||||
* @param {boolean} [opts.dryRun=false]
|
||||
* @param {string} [opts.home] - Home override, for tests.
|
||||
* @returns {Promise<object>} Verdict payload — never throws for a refused write.
|
||||
*/
|
||||
export async function applySubtraction(removals, opts = {}) {
|
||||
const { repoRoot = null, approveScope = false, dryRun = false, home } = opts;
|
||||
|
||||
const normalized = removals.map((r) => ({ ...r, file: resolve(r.file) }));
|
||||
const files = [...new Set(normalized.map((r) => r.file))];
|
||||
|
||||
const classifyOpts = home ? { home } : {};
|
||||
const targets = files.map((f) => classifyWriteTarget(f, repoRoot, classifyOpts));
|
||||
const gate = strongestGate(targets);
|
||||
const disclosures = [...new Set(targets.map((t) => t.disclosure).filter(Boolean))];
|
||||
|
||||
const base = {
|
||||
gate,
|
||||
requiresApproval: gate === 'require-ok',
|
||||
disclosures,
|
||||
targets,
|
||||
dryRun,
|
||||
backupId: null,
|
||||
applied: [],
|
||||
refused: [],
|
||||
filesWritten: [],
|
||||
};
|
||||
|
||||
// The gate is a verdict about a write, not a tool failure: the caller renders
|
||||
// the disclosure and asks. Nothing is written, and nothing is exit 3 (#62).
|
||||
//
|
||||
// `!dryRun` is load-bearing. The gate guards a WRITE, and a dry run is not
|
||||
// one — refusing it early bought nothing and cost the dry run its whole
|
||||
// purpose on the machine-wide target, which is the mandatory v1 case: the
|
||||
// operator would approve a removal whose spans had never been checked, and
|
||||
// the first run able to discover a stale approval would be the one that
|
||||
// writes. `requiresApproval` is reported either way, so the caller still asks.
|
||||
if (gate === 'require-ok' && !approveScope && !dryRun) {
|
||||
return {
|
||||
...base,
|
||||
refused: normalized.map((r) => ({ ...r, reason: REFUSAL_REASONS.SCOPE_GATE })),
|
||||
};
|
||||
}
|
||||
|
||||
const contents = new Map();
|
||||
const unreadable = [];
|
||||
for (const file of files) {
|
||||
try {
|
||||
contents.set(file, await readFile(file, 'utf-8'));
|
||||
} catch {
|
||||
unreadable.push(file);
|
||||
}
|
||||
}
|
||||
|
||||
if (unreadable.length > 0) {
|
||||
return {
|
||||
...base,
|
||||
refused: normalized.map((r) => ({
|
||||
...r,
|
||||
reason: unreadable.includes(r.file)
|
||||
? REFUSAL_REASONS.UNREADABLE
|
||||
: REFUSAL_REASONS.BACKUP_INCOMPLETE,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
const perFile = new Map();
|
||||
const applied = [];
|
||||
const refused = [];
|
||||
for (const file of files) {
|
||||
const result = exciseBlocks(contents.get(file), normalized.filter((r) => r.file === file));
|
||||
perFile.set(file, result);
|
||||
applied.push(...result.applied.map((a) => ({ ...a, file })));
|
||||
refused.push(...result.refused.map((r) => ({ ...r, file })));
|
||||
}
|
||||
|
||||
const toWrite = files.filter((f) => perFile.get(f).applied.length > 0);
|
||||
if (dryRun || toWrite.length === 0) {
|
||||
return { ...base, applied, refused };
|
||||
}
|
||||
|
||||
// `createBackup` skips a path that does not exist and still returns a
|
||||
// manifest and an id, so "a backup was made" is not evidence that THIS file
|
||||
// is recoverable (M-BUG-31's shape). Assert coverage before writing anything.
|
||||
const backup = createBackup(toWrite);
|
||||
const covered = new Set(backup.manifest.files.map((f) => f.originalPath));
|
||||
const uncovered = toWrite.filter((f) => !covered.has(f));
|
||||
if (uncovered.length > 0) {
|
||||
return {
|
||||
...base,
|
||||
backupId: backup.backupId,
|
||||
applied: [],
|
||||
refused: normalized.map((r) => ({ ...r, reason: REFUSAL_REASONS.BACKUP_INCOMPLETE })),
|
||||
};
|
||||
}
|
||||
|
||||
const filesWritten = [];
|
||||
for (const file of toWrite) {
|
||||
await writeFile(file, perFile.get(file).content, 'utf-8');
|
||||
filesWritten.push(file);
|
||||
}
|
||||
|
||||
return { ...base, backupId: backup.backupId, applied, refused, filesWritten };
|
||||
}
|
||||
|
|
@ -8,6 +8,7 @@
|
|||
import { readFile } from 'node:fs/promises';
|
||||
import { join } from 'node:path';
|
||||
import { homedir } from 'node:os';
|
||||
import { allFindingIds, knownScanners } from './finding-codes.mjs';
|
||||
|
||||
/**
|
||||
* Load suppressions from .config-audit-ignore files.
|
||||
|
|
@ -69,6 +70,39 @@ export function parseIgnoreFile(content) {
|
|||
return suppressions;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find suppression patterns that can never match anything.
|
||||
*
|
||||
* A finding ID names a check (see `finding-codes.mjs`), so an exact pin either
|
||||
* names a declared check or names nothing at all. Silently keeping a dead pin
|
||||
* would reproduce, in the other direction, the very failure the check-code
|
||||
* scheme removed: the user believes a finding is suppressed when it is not.
|
||||
*
|
||||
* Globs are validated only down to the scanner prefix — `CA-GAP-*` stays valid
|
||||
* however GAP's checks change, which is why a glob is the safe way to pin.
|
||||
*
|
||||
* @param {Array<{ pattern: string }>} suppressions
|
||||
* @returns {string[]} patterns that match no declared check
|
||||
*/
|
||||
export function unknownSuppressions(suppressions) {
|
||||
if (!suppressions || suppressions.length === 0) return [];
|
||||
|
||||
const ids = allFindingIds();
|
||||
const scanners = new Set(knownScanners());
|
||||
const unknown = [];
|
||||
|
||||
for (const { pattern } of suppressions) {
|
||||
if (pattern.endsWith('-*')) {
|
||||
const scanner = pattern.slice(3, -2); // "CA-GAP-*" → "GAP"
|
||||
if (!scanners.has(scanner)) unknown.push(pattern);
|
||||
continue;
|
||||
}
|
||||
if (!ids.has(pattern)) unknown.push(pattern);
|
||||
}
|
||||
|
||||
return unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply suppressions to a findings array.
|
||||
* @param {object[]} findings - Array of finding objects with .id
|
||||
|
|
|
|||
39
scanners/lib/write-output.mjs
Normal file
39
scanners/lib/write-output.mjs
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
/**
|
||||
* write-output — the one place a scanner's `--output-file` payload is written.
|
||||
*
|
||||
* Every command in this plugin follows the same contract (`.claude/rules/ux-rules.md`):
|
||||
* run the scanner with `--output-file <path> 2>/dev/null`, check the exit code, then Read
|
||||
* the file. The path the command chooses is frequently one it has never created — e.g.
|
||||
* `commands/campaign.md` writes its report to
|
||||
* `~/.claude/config-audit/sessions/campaign-report.json`, which on a fresh machine does not
|
||||
* exist yet. That is precisely the FIRST run, the case campaign-cli otherwise handles
|
||||
* gracefully by reporting `initialized: false`.
|
||||
*
|
||||
* Before this helper existed, all 13 payload writers called `writeFile` directly and threw
|
||||
* ENOENT there. The exit code was 3, and the command's own exit-code table reads 3 as "the
|
||||
* input is missing or corrupt" — so the user was told the ledger might be corrupt and
|
||||
* warned off the one action that would have fixed anything. `saveLedger` had always created
|
||||
* its parent directory; the payload write simply never did. The asymmetry was accidental.
|
||||
*
|
||||
* Creating the parent is the honest behaviour: the caller asked for a file at a path, and
|
||||
* nothing about a missing intermediate directory is an error the caller can learn from.
|
||||
*/
|
||||
|
||||
import { writeFile, mkdir } from 'node:fs/promises';
|
||||
import { dirname } from 'node:path';
|
||||
|
||||
/**
|
||||
* Write a scanner payload, creating the parent directory if needed.
|
||||
*
|
||||
* Signature-compatible with `writeFile(path, contents, encoding)` so call sites are a pure
|
||||
* rename — the encoding argument is kept rather than defaulted away.
|
||||
*
|
||||
* @param {string} path - destination file
|
||||
* @param {string} contents - serialized payload
|
||||
* @param {string} [encoding='utf-8']
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
export async function writeOutputFile(path, contents, encoding = 'utf-8') {
|
||||
await mkdir(dirname(path), { recursive: true });
|
||||
await writeFile(path, contents, encoding);
|
||||
}
|
||||
242
scanners/lib/write-scope.mjs
Normal file
242
scanners/lib/write-scope.mjs
Normal file
|
|
@ -0,0 +1,242 @@
|
|||
/**
|
||||
* Write-target scope classification (M-BUG-41).
|
||||
*
|
||||
* The workflow observes configuration across repos, but every write it then
|
||||
* proposes was presented as though it landed where the session stands. Five
|
||||
* arms were measured carrying that hole: `implement` (approval prompt names no
|
||||
* path at all — only a count), `rollback` (renders repo-relative-looking paths
|
||||
* while writing to absolute originals), `fix` (`--global` mixes user-scope and
|
||||
* repo rows into one unmarked table), `plan`, and `campaign export`.
|
||||
*
|
||||
* The gate's STRENGTH comes from the target's scope class, never from which
|
||||
* command is asking. Command-owned policy would be five policies to drift apart
|
||||
* — the shape that put the lever table in five copies (#61). Both required
|
||||
* outcomes then fall out of one table without an exception rule: a plan
|
||||
* exported into another repo is *disclosed* (cross-repo is by design there),
|
||||
* while a rewrite of `~/.claude/CLAUDE.md` *requires explicit approval*,
|
||||
* because it costs in every repo on every turn.
|
||||
*
|
||||
* `silent` means "no gate of its own", not "no approval": the existing
|
||||
* confirmation surfaces stand untouched, and this module only adds location to
|
||||
* them.
|
||||
*
|
||||
* Two orderings below are load-bearing, and both were measured rather than
|
||||
* reasoned about:
|
||||
*
|
||||
* `plugin-managed` before `user-scope` — the canonical
|
||||
* `~/.claude/config-audit/` and the legacy `~/.config-audit/` both exist on a
|
||||
* real machine, and every command writes session state into them. Matched the
|
||||
* other way round, the gate fires on every write ever made and gets switched
|
||||
* off, which is worse than having no gate.
|
||||
*
|
||||
* `user-scope` before `cross-repo` — `~/.claude/.git` exists (the operator's
|
||||
* `~/.claude` is a git repo whose `.gitignore` is `*`). A plain
|
||||
* `.git`-upward-walk therefore answers "another repo" for
|
||||
* `~/.claude/CLAUDE.md`, silently downgrading the strongest gate on the
|
||||
* subtraction axis's primary target to disclosure-only.
|
||||
*
|
||||
* This module classifies. It never writes, never prompts, and never decides
|
||||
* whether an approved write is a good idea.
|
||||
*/
|
||||
|
||||
import { existsSync } from 'node:fs';
|
||||
import { homedir } from 'node:os';
|
||||
import { isAbsolute, join, relative, resolve } from 'node:path';
|
||||
|
||||
/**
|
||||
* True when `child` is `parent` itself or lives underneath it.
|
||||
*
|
||||
* Uses `relative()` rather than `startsWith()`: a sibling directory whose name
|
||||
* merely prefixes the parent's (`my-plugin-2` against `my-plugin`) satisfies
|
||||
* `startsWith` and would skip the gate entirely.
|
||||
*
|
||||
* @param {string} parent - Absolute directory path.
|
||||
* @param {string} child - Absolute path to test.
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function isWithin(parent, child) {
|
||||
const rel = relative(parent, child);
|
||||
return rel === '' || (!rel.startsWith('..') && !isAbsolute(rel));
|
||||
}
|
||||
|
||||
/**
|
||||
* Default repo-root test. Kept injectable so classification is testable
|
||||
* without a fixture tree.
|
||||
*
|
||||
* @param {string} dir - Absolute directory path.
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function defaultIsRepoRoot(dir) {
|
||||
return existsSync(join(dir, '.git'));
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk upwards from `absPath` looking for the nearest enclosing repo root.
|
||||
*
|
||||
* @param {string} absPath - Absolute path to start from.
|
||||
* @param {(dir: string) => boolean} isRepoRoot - Repo-root predicate.
|
||||
* @returns {string|null} The nearest repo root, or null if there is none.
|
||||
*/
|
||||
function nearestRepoRoot(absPath, isRepoRoot) {
|
||||
let dir = absPath;
|
||||
for (;;) {
|
||||
if (isRepoRoot(dir)) return dir;
|
||||
const parent = resolve(dir, '..');
|
||||
if (parent === dir) return null;
|
||||
dir = parent;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The scope classes, in match order.
|
||||
*
|
||||
* Declaration order IS match order — this object is the single source for the
|
||||
* class name, its gate, its disclosure wording and its predicate, so no caller
|
||||
* and no test can hold a second copy that drifts.
|
||||
*
|
||||
* @type {Record<string, {gate: 'silent'|'disclose'|'require-ok', disclosure: string|null, matches: Function}>}
|
||||
*/
|
||||
export const SCOPE_CLASSES = {
|
||||
// The plugin's own bookkeeping: session state, backups, ledgers. Not the
|
||||
// user's configuration, and written on essentially every run.
|
||||
'plugin-managed': {
|
||||
gate: 'silent',
|
||||
disclosure: null,
|
||||
matches: (target, ctx) => ctx.pluginRoots.some((root) => isWithin(root, target)),
|
||||
},
|
||||
|
||||
// Where the session stands. The ordinary case.
|
||||
'in-repo': {
|
||||
gate: 'silent',
|
||||
disclosure: null,
|
||||
matches: (target, ctx) => ctx.repoRoot !== null && isWithin(ctx.repoRoot, target),
|
||||
},
|
||||
|
||||
// Machine-wide configuration: loaded in every repo, on every turn, so the
|
||||
// cost of a change here is not confined to the project in front of the user.
|
||||
'user-scope': {
|
||||
gate: 'require-ok',
|
||||
disclosure: 'This writes to your machine-wide Claude configuration, outside this project. '
|
||||
+ 'It affects every project you open, so it needs your explicit go-ahead.',
|
||||
matches: (target, ctx) => isWithin(ctx.userConfigRoot, target),
|
||||
},
|
||||
|
||||
// A different project. Some commands do this by design; the gate is to say
|
||||
// so, not to refuse.
|
||||
'cross-repo': {
|
||||
gate: 'disclose',
|
||||
disclosure: 'This writes into a different project than the one you are working in. '
|
||||
+ 'Any directories it needs there will be created.',
|
||||
matches: (target, ctx) => {
|
||||
const root = nearestRepoRoot(target, ctx.isRepoRoot);
|
||||
return root !== null && root !== ctx.repoRoot;
|
||||
},
|
||||
},
|
||||
|
||||
// Neither this project, nor another project, nor machine-wide config.
|
||||
'outside': {
|
||||
gate: 'require-ok',
|
||||
disclosure: 'This writes to a location outside any project and outside your Claude '
|
||||
+ 'configuration, so it needs your explicit go-ahead.',
|
||||
matches: () => true,
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Gate strengths, weakest first. Lives here rather than in a caller: a second
|
||||
* copy of this ordering would decide, independently, which gate a multi-target
|
||||
* write shows — the drift shape the class table itself exists to prevent.
|
||||
*/
|
||||
export const GATE_RANK = ['silent', 'disclose', 'require-ok'];
|
||||
|
||||
/**
|
||||
* The strongest gate among already-classified targets. One `require-ok` target
|
||||
* in a set drives the whole surface: a run that would write machine-wide config
|
||||
* does not get to be quiet because most of its other targets are ordinary.
|
||||
*
|
||||
* @param {Array<{gate: string}>} targets
|
||||
* @returns {string} The strongest gate, or 'silent' when there are no targets.
|
||||
*/
|
||||
export function strongestGate(targets) {
|
||||
let worst = 'silent';
|
||||
for (const t of targets) {
|
||||
if (GATE_RANK.indexOf(t.gate) > GATE_RANK.indexOf(worst)) worst = t.gate;
|
||||
}
|
||||
return worst;
|
||||
}
|
||||
|
||||
/**
|
||||
* Classify a whole write SET and reduce it to one verdict (Q1).
|
||||
*
|
||||
* Every gated arm needs the same four things — classify each target, take the
|
||||
* strongest gate, de-duplicate the disclosures, report whether approval is owed
|
||||
* — and `lib/subtraction-write.mjs` was the only arm that had them, written
|
||||
* inline. Four more call sites copying those four lines is precisely the shape
|
||||
* `SCOPE_CLASSES` exists to prevent one level down: the copies drift, and the
|
||||
* drift is invisible because each one still looks correct on its own.
|
||||
*
|
||||
* This decides nothing about whether a write is a good idea, and it never
|
||||
* writes. It answers "what does this set of targets oblige you to say?".
|
||||
*
|
||||
* Note the strict default: `sessionRepoRoot` of `null` means the `in-repo`
|
||||
* class can never match, so an omitted repo root fails toward MORE disclosure,
|
||||
* not less. A caller that forgets to pass it gets a noisier gate rather than a
|
||||
* silent one.
|
||||
*
|
||||
* @param {string[]} paths - Paths about to be written. Duplicates are fine.
|
||||
* @param {string|null} sessionRepoRoot - Repo root of the current session.
|
||||
* @param {object} [options] - Forwarded to `classifyWriteTarget`.
|
||||
* @returns {{gate: string, requiresApproval: boolean, disclosures: string[], targets: object[]}}
|
||||
*/
|
||||
export function evaluateWriteTargets(paths, sessionRepoRoot, options = {}) {
|
||||
const unique = [...new Set(paths.map((p) => resolve(p)))];
|
||||
const targets = unique.map((p) => classifyWriteTarget(p, sessionRepoRoot, options));
|
||||
const gate = strongestGate(targets);
|
||||
|
||||
return {
|
||||
gate,
|
||||
requiresApproval: gate === 'require-ok',
|
||||
disclosures: [...new Set(targets.map((t) => t.disclosure).filter(Boolean))],
|
||||
targets,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Classify a write target relative to the repo the session stands in.
|
||||
*
|
||||
* @param {string} targetPath - The path that is about to be written.
|
||||
* @param {string|null} sessionRepoRoot - Repo root of the current session.
|
||||
* @param {object} [options]
|
||||
* @param {(dir: string) => boolean} [options.isRepoRoot] - Repo-root predicate.
|
||||
* @param {string} [options.home] - Override for the home directory.
|
||||
* @returns {{scopeClass: string, gate: string, disclosure: string|null, target: string}}
|
||||
*/
|
||||
export function classifyWriteTarget(targetPath, sessionRepoRoot, options = {}) {
|
||||
const home = options.home ?? homedir();
|
||||
const isRepoRoot = options.isRepoRoot ?? defaultIsRepoRoot;
|
||||
|
||||
const target = resolve(targetPath);
|
||||
const ctx = {
|
||||
repoRoot: sessionRepoRoot === null || sessionRepoRoot === undefined
|
||||
? null
|
||||
: resolve(sessionRepoRoot),
|
||||
userConfigRoot: join(home, '.claude'),
|
||||
// Both roots are live: `backup.mjs` prefers `~/.claude/config-audit/` and
|
||||
// falls back to the legacy `~/.config-audit/`.
|
||||
pluginRoots: [
|
||||
join(home, '.claude', 'config-audit'),
|
||||
join(home, '.config-audit'),
|
||||
],
|
||||
isRepoRoot,
|
||||
};
|
||||
|
||||
for (const [scopeClass, spec] of Object.entries(SCOPE_CLASSES)) {
|
||||
if (spec.matches(target, ctx)) {
|
||||
return { scopeClass, gate: spec.gate, disclosure: spec.disclosure, target };
|
||||
}
|
||||
}
|
||||
|
||||
// Unreachable: `outside` matches unconditionally. Kept so a future edit that
|
||||
// narrows the last predicate fails loudly instead of returning undefined.
|
||||
throw new Error(`write-scope: no class matched ${target}`);
|
||||
}
|
||||
|
|
@ -40,8 +40,13 @@
|
|||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeFile, stat } from 'node:fs/promises';
|
||||
import { stat } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { readActiveConfig, deriveLoadPattern } from './lib/active-config-reader.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
|
||||
/** Flag surface, measured 2026-08-09. Anything else is exit 3. */
|
||||
const ARG_SPEC = { boolean: ['--json', '--raw'], value: ['--output-file'] };
|
||||
|
||||
// CLAUDE.md cascade files are all discovered by walking UP from the repo, so
|
||||
// each one is always-loaded; the scope only changes the derivation confidence.
|
||||
|
|
@ -114,6 +119,10 @@ export function buildManifest(activeConfig) {
|
|||
name: a.name,
|
||||
source: sourceLabel(a, 'project'),
|
||||
estimated_tokens: a.estimatedTokens || 0,
|
||||
// Routing axes (C4) — named explicitly because withLoadPattern copies the
|
||||
// row plus the load-pattern triple, nothing else from the enumeration.
|
||||
model: a.model ?? null,
|
||||
effort: a.effort ?? null,
|
||||
}, a));
|
||||
}
|
||||
|
||||
|
|
@ -240,6 +249,7 @@ function estimateClaudeMdEntryTokens(file, activeConfig) {
|
|||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
let targetPath = '.';
|
||||
let outputFile = null;
|
||||
let jsonMode = false;
|
||||
|
|
@ -259,11 +269,13 @@ async function main() {
|
|||
const s = await stat(absPath);
|
||||
if (!s.isDirectory()) {
|
||||
process.stderr.write(`Error: ${absPath} is not a directory\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
process.stderr.write(`Error: path does not exist: ${absPath}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
const start = Date.now();
|
||||
|
|
@ -285,7 +297,7 @@ async function main() {
|
|||
const json = JSON.stringify(output, null, 2);
|
||||
|
||||
if (outputFile) {
|
||||
await writeFile(outputFile, json, 'utf-8');
|
||||
await writeOutputFile(outputFile, json, 'utf-8');
|
||||
}
|
||||
|
||||
if (jsonMode || rawMode || !outputFile) {
|
||||
|
|
@ -297,6 +309,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -56,6 +56,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!parsed) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Invalid JSON in MCP config',
|
||||
description: `${file.relPath}: Failed to parse as JSON.`,
|
||||
|
|
@ -75,6 +76,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (config.type && !VALID_SERVER_TYPES.has(config.type)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unknown-server-type',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Unknown MCP server type',
|
||||
description: `${file.relPath}: Server "${name}" has unknown type "${config.type}".`,
|
||||
|
|
@ -88,6 +90,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (config.type === 'sse') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'sse-transport',
|
||||
severity: SEVERITY.info,
|
||||
title: 'SSE server type — consider HTTP',
|
||||
description: `${file.relPath}: Server "${name}" uses "sse" type. The "http" type is the current standard.`,
|
||||
|
|
@ -110,6 +113,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!hasEnvBlock) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unreferenced-env-var',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Unreferenced env var in args',
|
||||
description: `${file.relPath}: Server "${name}" references \${${varName}} in args but has no env block defining it.`,
|
||||
|
|
@ -127,6 +131,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!VALID_SERVER_FIELDS.has(key)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unknown-server-field',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Unknown MCP server field',
|
||||
description: `${file.relPath}: Server "${name}" has unknown field "${key}".`,
|
||||
|
|
|
|||
|
|
@ -122,6 +122,7 @@ export async function scan(targetPath, discovery) {
|
|||
findings.push(
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
code: 'procedure-should-be-skill',
|
||||
severity: SEVERITY.low,
|
||||
title: PROCEDURE_TITLE,
|
||||
description: claim,
|
||||
|
|
|
|||
|
|
@ -20,19 +20,36 @@
|
|||
*
|
||||
* Usage:
|
||||
* node optimize-lens-cli.mjs [path] [--output-file <path>] [--global]
|
||||
* [--subtract [--for-model <name>]]
|
||||
*
|
||||
* `--for-model <name>` annotates the subtraction candidates a named model
|
||||
* documents as redundant (BP-PROMPT-001). It never widens the candidate set,
|
||||
* and there is deliberately no auto-detection: a CLAUDE.md has no frontmatter
|
||||
* and no statically-resolvable target model, so the model must be named.
|
||||
*
|
||||
* Exit codes: 0=ok, 3=unrecoverable error. Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { resolve, sep } from 'node:path';
|
||||
import { writeFile, readFile, stat } from 'node:fs/promises';
|
||||
import { readFile, stat } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { discoverConfigFiles } from './lib/file-discovery.mjs';
|
||||
import { resetCounter } from './lib/output.mjs';
|
||||
import { parseFrontmatter } from './lib/yaml-parser.mjs';
|
||||
import { loadRegister, getEntry } from './lib/best-practices-register.mjs';
|
||||
import { prefilterClaudeMd, LENS_DETECTORS } from './lib/lens-prefilter.mjs';
|
||||
import { subtractionCandidates, SUBTRACT_DETECTORS } from './lib/subtraction-prefilter.mjs';
|
||||
import { matchModelScope, normalizeModel } from './lib/prompting-model-scope.mjs';
|
||||
import { scan as optScan } from './optimization-lens-scanner.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
|
||||
/** Flag surface, measured 2026-08-09. Anything else is exit 3. */
|
||||
const ARG_SPEC = {
|
||||
boolean: ['--global', '--subtract'],
|
||||
// `--for-model` is a VALUE flag, so a bare `--for-model` is reported as
|
||||
// "needs a value" rather than "unknown flag" — the two are different failures
|
||||
// and reporting the wrong one hides which mistake the caller made.
|
||||
value: ['--output-file', '--for-model'],
|
||||
};
|
||||
|
||||
// Files under `.claude/plugins/` are shipped by an installed plugin — vendored
|
||||
// CLAUDE.md plus its bundled tests/fixtures and examples. They are not the user's
|
||||
|
|
@ -51,15 +68,18 @@ function confirmedEntry(register, id) {
|
|||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
let targetPath = '.';
|
||||
let outputFile = null;
|
||||
let includeGlobal = false;
|
||||
let subtract = false;
|
||||
let targetModel = null;
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--global') includeGlobal = true;
|
||||
else if (args[i] === '--subtract') subtract = true;
|
||||
else if (args[i] === '--output-file' && args[i + 1]) outputFile = args[++i];
|
||||
else if (args[i] === '--for-model' && args[i + 1]) targetModel = args[++i];
|
||||
else if (!args[i].startsWith('-')) targetPath = args[i];
|
||||
}
|
||||
|
||||
|
|
@ -68,11 +88,13 @@ async function main() {
|
|||
const s = await stat(absPath);
|
||||
if (!s.isDirectory()) {
|
||||
process.stderr.write(`Error: ${absPath} is not a directory\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
process.stderr.write(`Error: path does not exist: ${absPath}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
// Load the register once; tolerate its absence (deterministic half still runs).
|
||||
|
|
@ -83,7 +105,6 @@ async function main() {
|
|||
register = null;
|
||||
}
|
||||
|
||||
resetCounter();
|
||||
const rawDiscovery = await discoverConfigFiles(absPath, { includeGlobal });
|
||||
// Scope the lens to the user's authored config: drop plugin-bundled files for
|
||||
// BOTH halves of the motor (the OPT scanner reads discovery.files directly).
|
||||
|
|
@ -102,6 +123,23 @@ async function main() {
|
|||
// fire on a plain `/config-audit optimize` run (brief §7 q3).
|
||||
const subtractCands = [];
|
||||
const subtractEntry = subtract && register ? confirmedEntry(register, 'BP-SUB-001') : null;
|
||||
// Model-scoped annotation entries (BP-PROMPT-001 and any later sibling). These
|
||||
// never produce candidates of their own — they only tag candidates the
|
||||
// BP-SUB-001 detector above already surfaced.
|
||||
const promptEntries =
|
||||
subtract && register
|
||||
? (register.entries || []).filter(
|
||||
(e) => e.category === 'prompting-fit' && e.confidence === 'confirmed',
|
||||
)
|
||||
: [];
|
||||
// `recognized` is reported separately from the match count so a typo'd model
|
||||
// name is distinguishable from a config that genuinely carries nothing.
|
||||
const modelRecognized =
|
||||
!!targetModel &&
|
||||
promptEntries.some((e) =>
|
||||
(e.modelScope || []).some((m) => normalizeModel(m) === normalizeModel(targetModel)),
|
||||
);
|
||||
let modelMatchedCount = 0;
|
||||
|
||||
for (const file of claudeMdFiles) {
|
||||
let content;
|
||||
|
|
@ -116,6 +154,8 @@ async function main() {
|
|||
|
||||
if (subtractEntry) {
|
||||
for (const cand of subtractionCandidates(body)) {
|
||||
const modelScope = matchModelScope(cand.text, targetModel, promptEntries);
|
||||
if (modelScope) modelMatchedCount++;
|
||||
subtractCands.push({
|
||||
file: file.absPath,
|
||||
line: bodyStartLine - 1 + cand.startLine,
|
||||
|
|
@ -131,6 +171,9 @@ async function main() {
|
|||
severity: subtractEntry.severity || 'low',
|
||||
source: subtractEntry.source,
|
||||
},
|
||||
// Spread only when matched: a run without --for-model must not grow
|
||||
// even a key set to undefined.
|
||||
...(modelScope ? { modelScope } : {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
|
@ -211,12 +254,21 @@ async function main() {
|
|||
: [],
|
||||
detectors: SUBTRACT_DETECTORS.map((d) => ({ ...d })),
|
||||
};
|
||||
// Present ONLY when a model was named — a plain --subtract run stays
|
||||
// byte-identical to the pre-flag payload.
|
||||
if (targetModel) {
|
||||
payload.subtract.forModel = {
|
||||
requested: targetModel,
|
||||
recognized: modelRecognized,
|
||||
matchedCount: modelMatchedCount,
|
||||
};
|
||||
}
|
||||
payload.counts.subtractCandidates = subtractCands.length;
|
||||
}
|
||||
|
||||
const json = JSON.stringify(payload, null, 2);
|
||||
if (outputFile) {
|
||||
await writeFile(outputFile, json, 'utf-8');
|
||||
await writeOutputFile(outputFile, json, 'utf-8');
|
||||
}
|
||||
if (!outputFile) {
|
||||
process.stdout.write(json + '\n');
|
||||
|
|
@ -227,6 +279,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch((err) => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -100,6 +100,7 @@ export async function scan(targetPath, _discovery) {
|
|||
if (kci === true) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'strips-coding-instructions',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Custom output style removes built-in coding instructions',
|
||||
description:
|
||||
|
|
@ -127,6 +128,7 @@ export async function scan(targetPath, _discovery) {
|
|||
if (ffp !== true) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'plugin-forces-style',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Plugin output style overrides your selected output style',
|
||||
description:
|
||||
|
|
@ -155,6 +157,7 @@ export async function scan(targetPath, _discovery) {
|
|||
const customNames = styles.map(s => s.name);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'style-not-found',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Configured output style does not exist',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -9,8 +9,10 @@
|
|||
*/
|
||||
|
||||
import { readdir, stat, readFile } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireTargetDir } from './lib/require-target-dir.mjs';
|
||||
import { join, basename, resolve, sep } from 'node:path';
|
||||
import { finding, scannerResult, resetCounter } from './lib/output.mjs';
|
||||
import { finding, scannerResult } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
import { parseFrontmatter } from './lib/yaml-parser.mjs';
|
||||
import { humanizeFindings } from './lib/humanizer.mjs';
|
||||
|
|
@ -220,6 +222,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-plugin-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Invalid plugin.json',
|
||||
description: `plugin.json is not valid JSON in ${pluginName}`,
|
||||
|
|
@ -236,6 +239,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!parsed[field]) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-required-field',
|
||||
severity: SEVERITY.high,
|
||||
title: `Missing required field in plugin.json: ${field}`,
|
||||
description: `Plugin "${pluginName}" plugin.json is missing required field "${field}"`,
|
||||
|
|
@ -257,6 +261,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!(await dirExists(join(pluginDir, defaultDir)))) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'plugin-json-shadows-default',
|
||||
severity: SEVERITY.medium,
|
||||
title: `plugin.json "${key}" path shadows the default ${defaultDir}/ folder`,
|
||||
description:
|
||||
|
|
@ -296,6 +301,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
const m = SKILLS_ENTRY_MESSAGES[problem];
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'skills-array-entry',
|
||||
severity: SEVERITY.medium,
|
||||
title: m.title(entry),
|
||||
description: `Plugin "${pluginName}": ${m.description}`,
|
||||
|
|
@ -311,6 +317,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-plugin-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Missing plugin.json',
|
||||
description: `No .claude-plugin/plugin.json found in ${pluginName}`,
|
||||
|
|
@ -336,6 +343,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!hasSection) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'claude-md-missing-section',
|
||||
severity: SEVERITY.medium,
|
||||
title: `CLAUDE.md missing ${section} section`,
|
||||
description: `Plugin "${pluginName}" CLAUDE.md should have a ${section} table or section`,
|
||||
|
|
@ -347,6 +355,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-claude-md',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Missing CLAUDE.md',
|
||||
description: `Plugin "${pluginName}" has no CLAUDE.md`,
|
||||
|
|
@ -370,6 +379,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!frontmatter) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'command-missing-frontmatter',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Command missing frontmatter',
|
||||
description: `Command "${file}" in plugin "${pluginName}" has no frontmatter`,
|
||||
|
|
@ -383,6 +393,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!frontmatter[key]) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'command-missing-field',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Command missing frontmatter field: ${display}`,
|
||||
description: `Command "${file}" in plugin "${pluginName}" is missing "${display}" in frontmatter`,
|
||||
|
|
@ -409,6 +420,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!frontmatter) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'agent-missing-frontmatter',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Agent missing frontmatter',
|
||||
description: `Agent "${file}" in plugin "${pluginName}" has no frontmatter`,
|
||||
|
|
@ -422,6 +434,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!frontmatter[key]) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'agent-missing-field',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Agent missing frontmatter field: ${display}`,
|
||||
description: `Agent "${file}" in plugin "${pluginName}" is missing "${display}" in frontmatter`,
|
||||
|
|
@ -437,6 +450,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (frontmatter[key] !== undefined) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'agent-ignored-key',
|
||||
severity,
|
||||
title: `Plugin agent sets "${key}", which Claude Code ignores`,
|
||||
description: `Agent "${file}" in plugin "${pluginName}" sets "${key}" in frontmatter, but Claude Code ignores ${key} for plugin subagents — ${key === 'permissionMode' ? 'the agent runs with default permissions, not the restricted mode this implies' : 'this configuration has no effect'}.`,
|
||||
|
|
@ -459,6 +473,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!parsed.hooks || typeof parsed.hooks !== 'object') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-json-invalid-structure',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Invalid hooks.json structure',
|
||||
description: `hooks.json in "${pluginName}" missing "hooks" object`,
|
||||
|
|
@ -468,6 +483,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} else if (Array.isArray(parsed.hooks)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-json-array',
|
||||
severity: SEVERITY.high,
|
||||
title: 'hooks.json uses array instead of object',
|
||||
description: `hooks.json "hooks" in "${pluginName}" is an array — must be object with event keys`,
|
||||
|
|
@ -478,6 +494,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-json-invalid',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Invalid hooks.json',
|
||||
description: `hooks.json is not valid JSON in "${pluginName}"`,
|
||||
|
|
@ -490,11 +507,18 @@ async function scanSinglePlugin(pluginDir) {
|
|||
const pluginMetaDir = join(pluginDir, '.claude-plugin');
|
||||
try {
|
||||
const entries = await readdir(pluginMetaDir);
|
||||
const known = new Set(['plugin.json']);
|
||||
// `marketplace.json` belongs here: it is the documented, required location
|
||||
// for a marketplace catalog (code.claude.com/docs plugin-marketplaces —
|
||||
// "Create `.claude-plugin/marketplace.json` in your repository root"), and a
|
||||
// marketplace entry with `"source": "./"` makes the repo root its own
|
||||
// plugin. Such a repo legitimately carries both files, so flagging the
|
||||
// catalog as an unknown file was a false positive.
|
||||
const known = new Set(['plugin.json', 'marketplace.json']);
|
||||
for (const entry of entries) {
|
||||
if (!known.has(entry)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unknown-plugin-file',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Unknown file in .claude-plugin/',
|
||||
description: `Unexpected file "${entry}" in .claude-plugin/ of "${pluginName}"`,
|
||||
|
|
@ -508,27 +532,62 @@ async function scanSinglePlugin(pluginDir) {
|
|||
return { name: pluginName, declaredName, findings, commandCount, agentCount };
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-plugin score and grade. Single source for both the terminal report and
|
||||
* the --output-file payload — the grade formula used to live only inside
|
||||
* `formatPluginHealthReport`, which nothing called.
|
||||
* @param {number} issueCount
|
||||
* @returns {{ score: number, grade: string }}
|
||||
*/
|
||||
export function pluginGrade(issueCount) {
|
||||
const score = Math.max(0, 100 - issueCount * 10);
|
||||
const grade = score >= 90 ? 'A' : score >= 75 ? 'B' : score >= 60 ? 'C' : score >= 40 ? 'D' : 'F';
|
||||
return { score, grade };
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan one or more plugins and return aggregated results.
|
||||
*
|
||||
* The envelope is frozen at the v5.0.0 shape (byte-stable `--raw`/`--json`), so
|
||||
* per-plugin rows and the cross-plugin/per-plugin split are NOT in it. Callers
|
||||
* that need those — the `--output-file` payload, and therefore
|
||||
* `/config-audit plugin-health` — use `scanDetailed`.
|
||||
*
|
||||
* @param {string} targetPath - Plugin dir or marketplace root
|
||||
* @returns {Promise<object>} Scanner result
|
||||
*/
|
||||
export async function scan(targetPath) {
|
||||
return (await scanDetailed(targetPath)).result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan, and also return what `scan()`'s frozen envelope cannot carry: one row
|
||||
* per plugin (name, declared namespace, component counts, grade) and the
|
||||
* cross-plugin findings as a distinct set.
|
||||
*
|
||||
* @param {string} targetPath - Plugin dir or marketplace root
|
||||
* @returns {Promise<{ result: object, plugins: object[], crossPluginFindings: object[] }>}
|
||||
*/
|
||||
export async function scanDetailed(targetPath) {
|
||||
const start = Date.now();
|
||||
resetCounter();
|
||||
|
||||
const pluginDirs = await discoverPlugins(resolve(targetPath));
|
||||
|
||||
if (pluginDirs.length === 0) {
|
||||
return scannerResult(SCANNER, 'ok', [
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
title: 'No plugins found',
|
||||
description: `No Claude Code plugins found under ${targetPath}`,
|
||||
recommendation: 'Ensure plugins have .claude-plugin/plugin.json',
|
||||
}),
|
||||
], 0, Date.now() - start);
|
||||
return {
|
||||
result: scannerResult(SCANNER, 'ok', [
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-plugins-found',
|
||||
severity: SEVERITY.info,
|
||||
title: 'No plugins found',
|
||||
description: `No Claude Code plugins found under ${targetPath}`,
|
||||
recommendation: 'Ensure plugins have .claude-plugin/plugin.json',
|
||||
}),
|
||||
], 0, Date.now() - start),
|
||||
plugins: [],
|
||||
crossPluginFindings: [],
|
||||
};
|
||||
}
|
||||
|
||||
const allFindings = [];
|
||||
|
|
@ -540,6 +599,12 @@ export async function scan(targetPath) {
|
|||
allFindings.push(...result.findings);
|
||||
}
|
||||
|
||||
// Everything pushed from here on is a cross-plugin finding — the boundary the
|
||||
// payload uses to split them out (they are flattened into `findings` in the
|
||||
// frozen envelope, where `category: 'plugin-hygiene'` cannot tell them apart
|
||||
// from the per-plugin shadow/skills findings that share it).
|
||||
const crossPluginStart = allFindings.length;
|
||||
|
||||
// Cross-plugin checks: command-name ambiguity across DIFFERENT plugin namespaces.
|
||||
// Commands are namespaced by the plugin's declared name (/name:command), so a
|
||||
// shared command name across DIFFERENT plugins is ambiguity — not a hard
|
||||
|
|
@ -572,6 +637,7 @@ export async function scan(targetPath) {
|
|||
const namespaceList = entries.map(e => e.namespace).join(', ');
|
||||
allFindings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'command-name-collision',
|
||||
severity: SEVERITY.low,
|
||||
title: `Command name "${cmdName}" used by multiple plugins`,
|
||||
description:
|
||||
|
|
@ -607,6 +673,7 @@ export async function scan(targetPath) {
|
|||
if (dirs.length < 2) continue;
|
||||
allFindings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'namespace-collision',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Plugin namespace collision: "${declaredName}"`,
|
||||
description:
|
||||
|
|
@ -632,7 +699,19 @@ export async function scan(targetPath) {
|
|||
}));
|
||||
}
|
||||
|
||||
return scannerResult(SCANNER, 'ok', allFindings, pluginDirs.length, Date.now() - start);
|
||||
return {
|
||||
result: scannerResult(SCANNER, 'ok', allFindings, pluginDirs.length, Date.now() - start),
|
||||
plugins: pluginResults.map((p, idx) => ({
|
||||
name: p.name,
|
||||
declaredName: p.declaredName,
|
||||
path: pluginDirs[idx],
|
||||
commandCount: p.commandCount,
|
||||
agentCount: p.agentCount,
|
||||
findingCount: p.findings.length,
|
||||
...pluginGrade(p.findings.length),
|
||||
})),
|
||||
crossPluginFindings: allFindings.slice(crossPluginStart),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -649,9 +728,7 @@ export function formatPluginHealthReport(pluginResults, crossPluginFindings) {
|
|||
lines.push('');
|
||||
|
||||
for (const p of pluginResults) {
|
||||
const issueCount = p.findings.length;
|
||||
const score = Math.max(0, 100 - issueCount * 10);
|
||||
const grade = score >= 90 ? 'A' : score >= 75 ? 'B' : score >= 60 ? 'C' : score >= 40 ? 'D' : 'F';
|
||||
const { score, grade } = pluginGrade(p.findings.length);
|
||||
const padding = '.'.repeat(Math.max(1, 25 - p.name.length));
|
||||
lines.push(` ${p.name} ${padding} ${grade} (${score}) ${p.commandCount} commands, ${p.agentCount} agents`);
|
||||
}
|
||||
|
|
@ -675,27 +752,55 @@ export function formatPluginHealthReport(pluginResults, crossPluginFindings) {
|
|||
}
|
||||
|
||||
// --- CLI entry point ---
|
||||
const BOOL_FLAGS = ['--json', '--raw'];
|
||||
const VALUE_FLAGS = ['--output-file'];
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
let targetPath = '.';
|
||||
let jsonMode = false;
|
||||
let rawMode = false;
|
||||
let outputFile = null;
|
||||
|
||||
// M-BUG-21, third arm: this loop used to end in
|
||||
// `else if (!args[i].startsWith('-')) targetPath = args[i]` with no
|
||||
// unknown-flag branch. An unrecognised flag was dropped silently and its
|
||||
// VALUE became the scan target, so `--output-file /tmp/x.json` scanned
|
||||
// /tmp/x.json. Unlike drift-cli, the result LOOKS fine: a non-existent path
|
||||
// discovers no plugins, so the scanner reported "No plugins found" (info) and
|
||||
// exit 0 — a green answer to a question nobody asked. Now it fails loudly.
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--json') {
|
||||
jsonMode = true;
|
||||
} else if (args[i] === '--raw') {
|
||||
rawMode = true;
|
||||
} else if (!args[i].startsWith('-')) {
|
||||
targetPath = args[i];
|
||||
const arg = args[i];
|
||||
if (BOOL_FLAGS.includes(arg)) {
|
||||
if (arg === '--json') jsonMode = true;
|
||||
else if (arg === '--raw') rawMode = true;
|
||||
} else if (VALUE_FLAGS.includes(arg)) {
|
||||
const value = args[i + 1];
|
||||
if (value === undefined || value.startsWith('-')) {
|
||||
throw new Error(`Option ${arg} requires a value.`);
|
||||
}
|
||||
outputFile = value;
|
||||
i++;
|
||||
} else if (arg.startsWith('-')) {
|
||||
throw new Error(
|
||||
`Unknown option: ${arg}\n` +
|
||||
`Valid options: ${[...BOOL_FLAGS, ...VALUE_FLAGS].join(' ')}`
|
||||
);
|
||||
} else {
|
||||
targetPath = arg;
|
||||
}
|
||||
}
|
||||
|
||||
if (!(await requireTargetDir(resolve(targetPath)))) {
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
const humanizedProgress = !jsonMode && !rawMode;
|
||||
process.stderr.write(humanizedProgress ? `Plugin Health v2.1.0\n` : `Plugin Health Scanner v2.1.0\n`);
|
||||
process.stderr.write(`Target: ${resolve(targetPath)}\n\n`);
|
||||
|
||||
const result = await scan(targetPath);
|
||||
const { result, plugins, crossPluginFindings } = await scanDetailed(targetPath);
|
||||
|
||||
if (jsonMode || rawMode) {
|
||||
// --json and --raw both write the v5.0.0-shape result (byte-identical).
|
||||
|
|
@ -708,6 +813,24 @@ async function main() {
|
|||
for (const f of findings) {
|
||||
process.stderr.write(` [${f.severity}] ${f.title}\n`);
|
||||
}
|
||||
|
||||
// ux-rules rule 2: the command runs with `2>/dev/null`, so anything it must
|
||||
// ACT on has to ride in the --output-file payload. Everything above this
|
||||
// point is stderr, i.e. invisible to `/config-audit plugin-health`.
|
||||
if (outputFile) {
|
||||
const crossIds = new Set(crossPluginFindings.map(f => f.id));
|
||||
for (const f of findings) {
|
||||
if (crossIds.has(f.id)) f.crossPlugin = true;
|
||||
}
|
||||
const payload = {
|
||||
...result,
|
||||
findings,
|
||||
plugins,
|
||||
cross_plugin_findings: findings.filter(f => crossIds.has(f.id)),
|
||||
};
|
||||
await writeOutputFile(outputFile, JSON.stringify(payload, null, 2), 'utf-8');
|
||||
process.stderr.write(`\nResults written to ${outputFile}\n`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -715,6 +838,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,9 +8,11 @@
|
|||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeFile } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireTargetDir } from './lib/require-target-dir.mjs';
|
||||
import { runAllScanners } from './scan-orchestrator.mjs';
|
||||
import { humanizeEnvelope } from './lib/humanizer.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
import {
|
||||
calculateUtilization,
|
||||
determineMaturityLevel,
|
||||
|
|
@ -21,6 +23,12 @@ import {
|
|||
generateHealthScorecard,
|
||||
} from './lib/scoring.mjs';
|
||||
|
||||
/** Flag surface, measured 2026-08-09. Anything else is exit 3. */
|
||||
const ARG_SPEC = {
|
||||
boolean: ['--json', '--raw', '--global', '--full-machine', '--include-fixtures'],
|
||||
value: ['--output-file', '--context-window'],
|
||||
};
|
||||
|
||||
/**
|
||||
* Run posture assessment and return structured result.
|
||||
* @param {string} targetPath
|
||||
|
|
@ -58,6 +66,7 @@ export async function runPosture(targetPath, opts = {}) {
|
|||
// --- CLI entry point ---
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
let targetPath = '.';
|
||||
let outputFile = null;
|
||||
let jsonMode = false;
|
||||
|
|
@ -86,6 +95,11 @@ async function main() {
|
|||
}
|
||||
}
|
||||
|
||||
if (!(await requireTargetDir(resolve(targetPath)))) {
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
const filterFixtures = !args.includes('--include-fixtures');
|
||||
const humanizedProgress = !jsonMode && !rawMode;
|
||||
const result = await runPosture(targetPath, {
|
||||
|
|
@ -123,7 +137,7 @@ async function main() {
|
|||
? result
|
||||
: { ...result, scannerEnvelope: humanizeEnvelope(result.scannerEnvelope) };
|
||||
const json = JSON.stringify(fileEnv, null, 2);
|
||||
await writeFile(outputFile, json, 'utf-8');
|
||||
await writeOutputFile(outputFile, json, 'utf-8');
|
||||
process.stderr.write(`\nResults written to ${outputFile}\n`);
|
||||
}
|
||||
}
|
||||
|
|
@ -133,6 +147,10 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(1);
|
||||
// 3, not 1. Every command in this plugin is told that 0/1/2 are normal grades
|
||||
// (PASS/WARNING/FAIL) and only 3 is a real error, so exiting 1 here made a crash
|
||||
// indistinguishable from a WARNING — and the command went on to Read a payload file
|
||||
// that was never written.
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
208
scanners/rollback-cli.mjs
Normal file
208
scanners/rollback-cli.mjs
Normal file
|
|
@ -0,0 +1,208 @@
|
|||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Config-Audit Rollback CLI — the runnable entry to the backup/restore engine.
|
||||
*
|
||||
* `rollback-engine.mjs` has verified checksums before AND after each write,
|
||||
* resolved the pre-v2.2.0 backup root and reported `createdNotRemoved` since
|
||||
* M-BUG-22/M-BUG-25. None of it was reachable: measured at the head of this
|
||||
* chunk, 16 files under `scanners/` carried a `process.argv` entry and the
|
||||
* engine was not one of them. `commands/rollback.md` drove the restore as model
|
||||
* prose — an ESM `import` block a template cannot execute, with ad-hoc `cp`
|
||||
* offered underneath as the runnable alternative and "(checksum verified)"
|
||||
* pre-rendered in the success output. `cp` establishes no checksum, so the
|
||||
* verification was a property of the template rather than of the run (R1).
|
||||
*
|
||||
* `--create` lives here for the same reason `--restore` does. The implement
|
||||
* pipeline used to build its backup by hand — `mkdir`, `cp`, a `date`-derived
|
||||
* id and a manifest typed out in the template — while `parseManifest` knew one
|
||||
* frozen sample of that format, pinned by a hand-written fixture rather than by
|
||||
* the template's own text. One side of that contract was maintained by editing
|
||||
* prose (R2). Now both sides are `lib/backup.mjs`.
|
||||
*
|
||||
* Usage:
|
||||
* node rollback-cli.mjs [--list]
|
||||
* node rollback-cli.mjs --restore <backup-id> [--dry-run] [--approve-scope]
|
||||
* node rollback-cli.mjs --delete <backup-id>
|
||||
* node rollback-cli.mjs --create --target <path> [--target <path> ...]
|
||||
* [--created <path> ...] [--backup-id <id>]
|
||||
* ... plus [--repo <session-root>] [--output-file <path>] [--json]
|
||||
*
|
||||
* Exit codes:
|
||||
* 0 done, nothing owed
|
||||
* 1 the run completed but something is outstanding — a restore the scope
|
||||
* gate will not perform without `--approve-scope` (nothing was written),
|
||||
* or a backup that covered fewer targets than it was given
|
||||
* 2 at least one file failed to restore (checksum mismatch or write error)
|
||||
* 3 the CLI could not do its job — malformed argv, or a backup id that
|
||||
* resolves in neither root
|
||||
*
|
||||
* A gated restore is exit 1, not 3: "this write leaves your project" is a
|
||||
* verdict about a write that WAS examined, and it rides in the payload where a
|
||||
* command can act on it. Anything that only reaches stderr is invisible to a
|
||||
* command running under `2>/dev/null` (F3's class).
|
||||
*
|
||||
* Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
import { createBackup, getBackupDir, getLegacyBackupDir } from './lib/backup.mjs';
|
||||
import { listBackups, restoreBackup, deleteBackup } from './rollback-engine.mjs';
|
||||
|
||||
/** Flag surface. Anything else is exit 3. */
|
||||
const ARG_SPEC = {
|
||||
boolean: ['--list', '--create', '--dry-run', '--approve-scope', '--json'],
|
||||
value: ['--restore', '--delete', '--target', '--created', '--backup-id', '--repo', '--output-file'],
|
||||
};
|
||||
|
||||
/** The mode flags, in the order a diagnostic should name them. */
|
||||
const MODES = ['--list', '--create', '--restore', '--delete'];
|
||||
|
||||
function fail(message) {
|
||||
process.stderr.write(`Error: ${message}\n`);
|
||||
process.exitCode = 3;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
|
||||
const targets = [];
|
||||
const created = [];
|
||||
let mode = null;
|
||||
let backupId = null;
|
||||
let dryRun = false;
|
||||
let approveScope = false;
|
||||
let jsonMode = false;
|
||||
let outputFile = null;
|
||||
let overrideId = null;
|
||||
// The session's root, never a path derived from the backup (#63). A restore
|
||||
// writes to the ABSOLUTE originals recorded at backup time, so reading the
|
||||
// root off those paths would call a machine-wide write "in-repo" and silence
|
||||
// the strongest gate exactly where it matters.
|
||||
let repoRoot = process.cwd();
|
||||
|
||||
const setMode = (flag) => {
|
||||
if (mode !== null && mode !== flag) return false;
|
||||
mode = flag;
|
||||
return true;
|
||||
};
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (MODES.includes(a)) {
|
||||
if (!setMode(a)) {
|
||||
// Silently letting the last one win is how a `--delete` rides along
|
||||
// behind a `--list` the caller thought it was running.
|
||||
fail(`only one mode may be given (${MODES.join(', ')}); saw "${mode}" and "${a}"`);
|
||||
return;
|
||||
}
|
||||
if (a === '--restore' || a === '--delete') backupId = args[++i];
|
||||
} else if (a === '--target') targets.push(args[++i]);
|
||||
else if (a === '--created') created.push(args[++i]);
|
||||
else if (a === '--backup-id') overrideId = args[++i];
|
||||
else if (a === '--repo') repoRoot = args[++i];
|
||||
else if (a === '--output-file') outputFile = args[++i];
|
||||
else if (a === '--dry-run') dryRun = true;
|
||||
else if (a === '--approve-scope') approveScope = true;
|
||||
else if (a === '--json') jsonMode = true;
|
||||
}
|
||||
|
||||
if (mode === null) mode = '--list';
|
||||
|
||||
const meta = {
|
||||
mode: mode.slice(2),
|
||||
backupRoot: getBackupDir(),
|
||||
legacyBackupRoot: getLegacyBackupDir(),
|
||||
repo: resolve(repoRoot),
|
||||
};
|
||||
|
||||
let payload;
|
||||
let lines = [];
|
||||
|
||||
if (mode === '--list') {
|
||||
const { backups } = await listBackups();
|
||||
payload = { meta, count: backups.length, backups };
|
||||
lines = backups.map((b) => `${b.id}\t${b.files.length}\t${b.legacy ? 'legacy' : 'current'}`);
|
||||
} else if (mode === '--create') {
|
||||
if (targets.length === 0) {
|
||||
fail('--create needs at least one --target');
|
||||
return;
|
||||
}
|
||||
const result = createBackup(targets, {
|
||||
...(overrideId ? { backupId: overrideId } : {}),
|
||||
created,
|
||||
});
|
||||
payload = {
|
||||
meta,
|
||||
backupId: result.backupId,
|
||||
backupPath: result.backupPath,
|
||||
files: result.manifest.files,
|
||||
created: result.manifest.created,
|
||||
skipped: result.skipped,
|
||||
};
|
||||
lines = [`${result.backupId}\t${result.manifest.files.length}\t${result.skipped.length}`];
|
||||
// A backup that covers fewer files than it was asked for is the state the
|
||||
// caller must not mistake for a clean one: it is about to edit a file it
|
||||
// cannot roll back.
|
||||
if (result.skipped.length > 0) process.exitCode = 1;
|
||||
} else if (mode === '--delete') {
|
||||
const result = await deleteBackup(backupId);
|
||||
if (!result.deleted) {
|
||||
fail(result.error);
|
||||
return;
|
||||
}
|
||||
payload = { meta, backupId, deleted: true, error: null };
|
||||
lines = [`${backupId}\tdeleted`];
|
||||
} else {
|
||||
let result;
|
||||
try {
|
||||
result = await restoreBackup(backupId, { dryRun, approveScope, repoRoot });
|
||||
} catch (err) {
|
||||
// "Backup not found" and "unreadable manifest" are both the CLI failing to
|
||||
// do its job, never a verdict about a restore that happened.
|
||||
fail(err.message);
|
||||
return;
|
||||
}
|
||||
payload = {
|
||||
meta,
|
||||
backupId,
|
||||
dryRun,
|
||||
gate: result.gate,
|
||||
requiresApproval: result.requiresApproval,
|
||||
disclosures: result.disclosures,
|
||||
restored: result.restored,
|
||||
failed: result.failed,
|
||||
refused: result.refused,
|
||||
createdNotRemoved: result.createdNotRemoved ?? [],
|
||||
legacy: result.legacy ?? false,
|
||||
};
|
||||
lines = [
|
||||
...result.restored.map((r) => `${r.status}\t${r.originalPath}`),
|
||||
...result.failed.map((f) => `${f.status}\t${f.originalPath}`),
|
||||
...result.refused.map((r) => `${r.status}\t${r.originalPath}`),
|
||||
];
|
||||
if (result.failed.length > 0) process.exitCode = 2;
|
||||
else if (payload.requiresApproval && !approveScope && !dryRun) process.exitCode = 1;
|
||||
}
|
||||
|
||||
const json = `${JSON.stringify(payload, null, 2)}\n`;
|
||||
if (outputFile) {
|
||||
await writeOutputFile(outputFile, json);
|
||||
// Nothing on stdout when writing to a file — a command rendering this would
|
||||
// otherwise show the user the raw payload (ux-rules rule 1).
|
||||
} else if (jsonMode) {
|
||||
process.stdout.write(json);
|
||||
} else {
|
||||
for (const line of lines) process.stdout.write(`${line}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
await main();
|
||||
} catch (err) {
|
||||
process.stderr.write(`Error: ${err.message}\n`);
|
||||
process.exitCode = 3;
|
||||
}
|
||||
|
|
@ -7,6 +7,7 @@
|
|||
import { readFile, writeFile, readdir, stat, rm } from 'node:fs/promises';
|
||||
import { join } from 'node:path';
|
||||
import { getBackupDir, getLegacyBackupDir, parseManifest, checksum } from './lib/backup.mjs';
|
||||
import { evaluateWriteTargets } from './lib/write-scope.mjs';
|
||||
|
||||
/**
|
||||
* Resolve a backup id to its directory, canonical root first, then the
|
||||
|
|
@ -100,6 +101,35 @@ export async function restoreBackup(backupId, opts = {}) {
|
|||
const restored = [];
|
||||
const failed = [];
|
||||
|
||||
// Q1 — the scope gate, in code. `rollback` is one of the five arms M-BUG-41
|
||||
// measured: it renders repo-relative-looking paths while writing to the
|
||||
// ABSOLUTE originals recorded in the manifest, so what the operator reads and
|
||||
// what the run touches are not the same set. A backup taken under `--global`
|
||||
// restores `~/.claude/…`, which is `user-scope` / `require-ok`.
|
||||
const scope = evaluateWriteTargets(
|
||||
manifest.files.map((f) => f.originalPath),
|
||||
opts.repoRoot ?? null,
|
||||
opts.home ? { home: opts.home } : {},
|
||||
);
|
||||
|
||||
// The gate guards a WRITE; a dry run is not one (#63). `requiresApproval` is
|
||||
// returned either way, so a caller previewing a restore still learns that
|
||||
// approval will be owed.
|
||||
if (scope.requiresApproval && !opts.approveScope && !opts.dryRun) {
|
||||
return {
|
||||
restored: [],
|
||||
failed: [],
|
||||
gate: scope.gate,
|
||||
requiresApproval: true,
|
||||
disclosures: scope.disclosures,
|
||||
refused: manifest.files.map((f) => ({
|
||||
originalPath: f.originalPath,
|
||||
status: 'refused',
|
||||
reason: 'scope-gate',
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
// A manifest with entries that parsed to nothing would restore nothing while
|
||||
// reporting success. Fail loudly instead.
|
||||
if (manifest.files.length === 0 && /^\s+-\s/m.test(manifestContent)) {
|
||||
|
|
@ -167,7 +197,16 @@ export async function restoreBackup(backupId, opts = {}) {
|
|||
// Files implement CREATED are absent from the backup by definition, so they
|
||||
// survive the restore. Report them — a half-restored target is only dangerous
|
||||
// when it is also silent.
|
||||
return { restored, failed, createdNotRemoved: manifest.created, legacy: resolved.legacy };
|
||||
return {
|
||||
restored,
|
||||
failed,
|
||||
createdNotRemoved: manifest.created,
|
||||
legacy: resolved.legacy,
|
||||
gate: scope.gate,
|
||||
requiresApproval: scope.requiresApproval,
|
||||
disclosures: scope.disclosures,
|
||||
refused: [],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -55,6 +55,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (lines > 5) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-frontmatter',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Rule has no frontmatter (always active)',
|
||||
description: `${file.relPath} has no YAML frontmatter. It will be loaded for ALL files. Add paths: frontmatter to scope it.`,
|
||||
|
|
@ -69,6 +70,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (frontmatter.globs && !frontmatter.paths) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'globs-instead-of-paths',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Rule uses "globs" instead of documented "paths"',
|
||||
description: `${file.relPath} uses "globs:" for scoping. Claude Code's documentation specifies "paths:" as the rule-scoping field; "globs:" is not documented. Rename to "paths:" so the rule scopes as intended.`,
|
||||
|
|
@ -99,6 +101,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (matchCount === 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'pattern-matches-nothing',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Rule path pattern matches no files',
|
||||
description: `${file.relPath}: pattern "${pattern}" matches 0 files. This rule will never activate.`,
|
||||
|
|
@ -117,6 +120,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (lines < 2) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'nearly-empty',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Rule file is nearly empty',
|
||||
description: `${file.relPath} has only ${lines} line(s).`,
|
||||
|
|
@ -130,6 +134,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!frontmatter?.paths && !frontmatter?.globs && lines > 50) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'large-unscoped',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Large unscoped rule file',
|
||||
description: `${file.relPath} has ${lines} lines and no path scoping. It loads into context for every file interaction.`,
|
||||
|
|
@ -147,6 +152,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (frontmatter?.paths && lines > 50) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'large-scoped-lost-after-compaction',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Large path-scoped rule is lost after compaction',
|
||||
description: `${file.relPath} is path-scoped (${lines} lines). Path-scoped rules load only when a matching file is read, and after a context compaction they are not re-injected until a matching file is read again — so a large scoped rule carrying must-always-hold instructions can silently drop out mid-session.`,
|
||||
|
|
@ -161,6 +167,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!file.absPath.endsWith('.md')) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'not-markdown',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Rule file is not .md',
|
||||
description: `${file.relPath} is not a .md file. Only .md files are loaded from rules/.`,
|
||||
|
|
|
|||
|
|
@ -9,10 +9,12 @@
|
|||
|
||||
import { resolve, sep } from 'node:path';
|
||||
import { readFile, writeFile } from 'node:fs/promises';
|
||||
import { resetCounter } from './lib/output.mjs';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { evaluateWriteTargets } from './lib/write-scope.mjs';
|
||||
import { requireTargetDir } from './lib/require-target-dir.mjs';
|
||||
import { envelope } from './lib/output.mjs';
|
||||
import { discoverConfigFiles, discoverConfigFilesMulti, discoverFullMachinePaths } from './lib/file-discovery.mjs';
|
||||
import { loadSuppressions, applySuppressions, formatSuppressionSummary } from './lib/suppression.mjs';
|
||||
import { loadSuppressions, applySuppressions, formatSuppressionSummary, unknownSuppressions } from './lib/suppression.mjs';
|
||||
import { humanizeEnvelope } from './lib/humanizer.mjs';
|
||||
import { resolveContextWindow } from './lib/context-window.mjs';
|
||||
import { resolveActiveModel } from './lib/active-model.mjs';
|
||||
|
|
@ -34,6 +36,17 @@ import { scan as scanSkillListing } from './skill-listing-scanner.mjs';
|
|||
import { scan as scanAgentListing } from './agent-listing-scanner.mjs';
|
||||
import { scan as scanOutputStyle } from './output-style-scanner.mjs';
|
||||
import { scan as scanOptimizationLens } from './optimization-lens-scanner.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
|
||||
/** Flag surface, measured 2026-08-09. Anything else is exit 3. */
|
||||
const ARG_SPEC = {
|
||||
boolean: [
|
||||
'--json', '--raw', '--global', '--full-machine', '--no-suppress',
|
||||
'--include-fixtures', '--exclude-cache', '--no-exclude-cache', '--save-baseline',
|
||||
'--approve-scope',
|
||||
],
|
||||
value: ['--output-file', '--context-window', '--baseline'],
|
||||
};
|
||||
|
||||
// Directory names that identify test fixture / example directories
|
||||
const FIXTURE_DIR_NAMES = ['tests', 'examples', '__tests__', 'test-fixtures'];
|
||||
|
|
@ -122,7 +135,6 @@ export async function runAllScanners(targetPath, opts = {}) {
|
|||
const results = [];
|
||||
|
||||
for (const scanner of SCANNERS) {
|
||||
resetCounter();
|
||||
const scanStart = Date.now();
|
||||
try {
|
||||
const result = await scanner.fn(resolvedPath, discovery, { contextWindow });
|
||||
|
|
@ -182,8 +194,14 @@ export async function runAllScanners(targetPath, opts = {}) {
|
|||
const shouldSuppress = opts.suppress !== false;
|
||||
let suppressedFindings = [];
|
||||
|
||||
let deadSuppressions = [];
|
||||
|
||||
if (shouldSuppress) {
|
||||
const { suppressions } = await loadSuppressions(resolvedPath);
|
||||
// A pin that names no declared check can never match. Report it: a silently
|
||||
// dead suppression leaves the user believing a finding is hidden when it is
|
||||
// not (M-BUG-28).
|
||||
deadSuppressions = unknownSuppressions(suppressions);
|
||||
if (suppressions.length > 0) {
|
||||
for (const result of results) {
|
||||
const { active, suppressed } = applySuppressions(result.findings, suppressions);
|
||||
|
|
@ -209,15 +227,23 @@ export async function runAllScanners(targetPath, opts = {}) {
|
|||
if (suppressedFindings.length > 0) {
|
||||
env.suppressed_findings = suppressedFindings;
|
||||
}
|
||||
// ux-rules rule 2: commands run scanners with `2>/dev/null`, so anything they
|
||||
// must ACT on rides in the payload, never in a stderr-only warning. Added only
|
||||
// when a dead pin exists, so a config without one is byte-identical.
|
||||
if (deadSuppressions.length > 0) {
|
||||
env.unknown_suppressions = deadSuppressions;
|
||||
}
|
||||
return env;
|
||||
}
|
||||
|
||||
// --- CLI entry point ---
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
let targetPath = '.';
|
||||
let outputFile = null;
|
||||
let saveBaseline = false;
|
||||
let approveScope = false;
|
||||
let baselinePath = null;
|
||||
let contextWindow = null;
|
||||
|
||||
|
|
@ -226,6 +252,8 @@ async function main() {
|
|||
outputFile = args[++i];
|
||||
} else if (args[i] === '--context-window' && args[i + 1]) {
|
||||
contextWindow = args[++i];
|
||||
} else if (args[i] === '--approve-scope') {
|
||||
approveScope = true;
|
||||
} else if (args[i] === '--save-baseline') {
|
||||
saveBaseline = true;
|
||||
} else if (args[i] === '--baseline' && args[i + 1]) {
|
||||
|
|
@ -257,6 +285,11 @@ async function main() {
|
|||
const jsonMode = args.includes('--json');
|
||||
const rawMode = args.includes('--raw');
|
||||
|
||||
if (!(await requireTargetDir(resolve(targetPath)))) {
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
const humanizedProgress = !jsonMode && !rawMode;
|
||||
process.stderr.write(humanizedProgress ? `Config-Audit v2.2.0\n` : `Config-Audit Scanner v2.2.0\n`);
|
||||
process.stderr.write(`Target: ${resolve(targetPath)}\n`);
|
||||
|
|
@ -278,7 +311,7 @@ async function main() {
|
|||
const json = JSON.stringify(output, null, 2);
|
||||
|
||||
if (outputFile) {
|
||||
await writeFile(outputFile, json, 'utf-8');
|
||||
await writeOutputFile(outputFile, json, 'utf-8');
|
||||
process.stderr.write(`\nResults written to ${outputFile}\n`);
|
||||
} else {
|
||||
process.stdout.write(json + '\n');
|
||||
|
|
@ -286,10 +319,26 @@ async function main() {
|
|||
|
||||
if (saveBaseline) {
|
||||
const bPath = baselinePath || resolve(targetPath, '.config-audit-baseline.json');
|
||||
// Always save baselines as raw v5.0.0-shape envelope so future humanizer
|
||||
// changes don't trigger false-positive drift findings.
|
||||
await writeFile(bPath, JSON.stringify(result, null, 2), 'utf-8');
|
||||
process.stderr.write(`Baseline saved to ${bPath}\n`);
|
||||
|
||||
// Q1 — the scope gate, in code. This write was carried in the plan text as
|
||||
// one of the plugin's own artifacts, legitimately exempt — measured false: the
|
||||
// default path is derived from the SCAN TARGET, not from a plugin root, so
|
||||
// `--global --save-baseline` lands `~/.claude/.config-audit-baseline.json`,
|
||||
// which is `user-scope` / `require-ok`. `lib/baseline.mjs` is the genuinely
|
||||
// exempt one — it writes only under `~/.config-audit/baselines`.
|
||||
const scope = evaluateWriteTargets([bPath], process.cwd());
|
||||
|
||||
if (scope.requiresApproval && !approveScope) {
|
||||
for (const line of scope.disclosures) process.stderr.write(`\n${line}\n`);
|
||||
process.stderr.write(
|
||||
`Baseline NOT saved to ${bPath} — re-run with --approve-scope to write it.\n`,
|
||||
);
|
||||
} else {
|
||||
// Always save baselines as raw v5.0.0-shape envelope so future humanizer
|
||||
// changes don't trigger false-positive drift findings.
|
||||
await writeFile(bPath, JSON.stringify(result, null, 2), 'utf-8');
|
||||
process.stderr.write(`Baseline saved to ${bPath}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
// Summary
|
||||
|
|
@ -299,10 +348,12 @@ async function main() {
|
|||
process.stderr.write(`Risk: ${agg.risk_score}/100 (${agg.risk_band})\n`);
|
||||
process.stderr.write(`Verdict: ${agg.verdict}\n`);
|
||||
|
||||
// Exit code
|
||||
if (agg.verdict === 'FAIL') process.exit(2);
|
||||
if (agg.verdict === 'WARNING') process.exit(1);
|
||||
process.exit(0);
|
||||
// Exit code. Set, never process.exit(): stdout is written ASYNCHRONOUSLY when
|
||||
// it is a pipe, and process.exit() discards whatever is still buffered. Piping
|
||||
// this envelope used to yield truncated, unparseable JSON (246 854 bytes to a
|
||||
// file vs 65 536 to a pipe) — a corruption that looks like a bad file, not a
|
||||
// cut-off. Letting Node exit naturally drains stdout first.
|
||||
process.exitCode = agg.verdict === 'FAIL' ? 2 : agg.verdict === 'WARNING' ? 1 : 0;
|
||||
}
|
||||
|
||||
// Only run CLI if invoked directly
|
||||
|
|
@ -310,6 +361,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -20,6 +20,10 @@ import { gradeFromPassRate } from './lib/severity.mjs';
|
|||
import { loadSuppressions, applySuppressions } from './lib/suppression.mjs';
|
||||
import { parseJson } from './lib/yaml-parser.mjs';
|
||||
import { humanizeEnvelope, humanizeFindings } from './lib/humanizer.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
|
||||
/** Flag surface, measured 2026-08-09. Anything else is exit 3. */
|
||||
const ARG_SPEC = { boolean: ['--json', '--fix', '--check-readme'] };
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
|
|
@ -330,6 +334,7 @@ export function formatSelfAudit(result) {
|
|||
// --- CLI entry point ---
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
const jsonMode = args.includes('--json');
|
||||
const fixMode = args.includes('--fix');
|
||||
const checkReadmeMode = args.includes('--check-readme');
|
||||
|
|
@ -350,6 +355,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(file
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -71,8 +71,11 @@ const TYPE_CHECKS = new Map([
|
|||
['wheelScrollAccelerationEnabled', 'boolean'],
|
||||
]);
|
||||
|
||||
/** Valid effortLevel values (CC 2.1.154 added 'xhigh' as the Opus-4.8 top tier) */
|
||||
const VALID_EFFORT_LEVELS = new Set(['low', 'medium', 'high', 'xhigh', 'max']);
|
||||
/** Valid effortLevel values (CC 2.1.154 added 'xhigh' as the Opus-4.8 top tier).
|
||||
* Exported because the fix engine's nearest-match needs the SAME list: a second
|
||||
* copy there had gone stale on `xhigh` and quietly corrected near-misses on the
|
||||
* top tier down to `high` (C2). One table, no drift. */
|
||||
export const VALID_EFFORT_LEVELS = new Set(['low', 'medium', 'high', 'xhigh', 'max']);
|
||||
|
||||
/** v5 M6: warn when additionalDirectories grows beyond this — each entry adds
|
||||
* a project root to walks/discovery, inflating per-turn cost and confusing scope. */
|
||||
|
|
@ -118,6 +121,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed === null) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Invalid JSON in settings file',
|
||||
description: `${file.relPath} contains invalid JSON and will be ignored by Claude Code.`,
|
||||
|
|
@ -148,6 +152,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (nearest) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'key-typo',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Possible typo in settings key',
|
||||
description: `${file.relPath}: "${key}" is not a recognized settings.json key, but it closely matches "${nearest}". Claude Code forwards unrecognized keys unchanged (it does not reject them), so if "${key}" is a typo of "${nearest}" the intended setting silently has no effect.`,
|
||||
|
|
@ -164,6 +169,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed[key] !== undefined) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deprecated-key',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Deprecated settings key',
|
||||
description: `${file.relPath}: "${key}" is deprecated. ${migration}`,
|
||||
|
|
@ -180,6 +186,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed[key] !== undefined && typeof parsed[key] !== expectedType) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'type-mismatch',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Type mismatch in settings',
|
||||
description: `${file.relPath}: "${key}" should be ${expectedType}, got ${typeof parsed[key]}.`,
|
||||
|
|
@ -195,6 +202,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed.effortLevel && !VALID_EFFORT_LEVELS.has(parsed.effortLevel)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-effort-level',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Invalid effortLevel value',
|
||||
description: `${file.relPath}: effortLevel "${parsed.effortLevel}" is not valid.`,
|
||||
|
|
@ -209,6 +217,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!parsed.$schema) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-schema',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Missing $schema reference',
|
||||
description: `${file.relPath} lacks a $schema reference. Adding one enables autocomplete in VS Code/Cursor.`,
|
||||
|
|
@ -225,6 +234,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!perms.deny || (Array.isArray(perms.deny) && perms.deny.length === 0)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-deny-rules',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'No deny rules configured',
|
||||
description: `${file.relPath}: No permission deny rules. Claude can access all files including .env and secrets.`,
|
||||
|
|
@ -237,6 +247,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!perms.allow || (Array.isArray(perms.allow) && perms.allow.length === 0)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-allow-rules',
|
||||
severity: SEVERITY.low,
|
||||
title: 'No allow rules configured',
|
||||
description: `${file.relPath}: No permission allow rules. This means frequent permission prompts for common operations.`,
|
||||
|
|
@ -252,6 +263,7 @@ export async function scan(targetPath, discovery) {
|
|||
parsed.additionalDirectories.length > ADDITIONAL_DIRS_THRESHOLD) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'many-additional-dirs',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Many additionalDirectories entries',
|
||||
description:
|
||||
|
|
@ -278,6 +290,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (typeof am !== 'object' || am === null || Array.isArray(am)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'automode-not-object',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'autoMode must be an object',
|
||||
description: `${file.relPath}: "autoMode" must be an object with environment/allow/soft_deny/hard_deny arrays, got ${Array.isArray(am) ? 'array' : typeof am}.`,
|
||||
|
|
@ -292,6 +305,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!AUTO_MODE_SUBKEYS.has(subKey)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'automode-unknown-subkey',
|
||||
severity: SEVERITY.medium,
|
||||
title: `autoMode has an unknown sub-key: ${subKey}`,
|
||||
description: `${file.relPath}: "autoMode.${subKey}" is not a recognized sub-key. Valid keys are environment, allow, soft_deny, hard_deny. It is silently ignored — a typo of a real key (e.g. "hard_denies") means those rules never apply.`,
|
||||
|
|
@ -308,6 +322,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!isStringArray) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'automode-subkey-not-string-array',
|
||||
severity: SEVERITY.medium,
|
||||
title: `autoMode.${subKey} must be an array of strings`,
|
||||
description: `${file.relPath}: "autoMode.${subKey}" must be an array of prose-rule strings (the literal "$defaults" is allowed), got ${Array.isArray(val) ? 'an array with a non-string entry' : typeof val}.`,
|
||||
|
|
@ -324,6 +339,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (file.scope === 'project') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'automode-in-shared-settings',
|
||||
severity: SEVERITY.low,
|
||||
title: 'autoMode in shared project settings is ignored by Claude Code',
|
||||
description: `${file.relPath}: Claude Code does not read "autoMode" from shared project settings (.claude/settings.json), so a checked-in repo cannot inject its own rules. This autoMode block has no effect where it is.`,
|
||||
|
|
@ -341,6 +357,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (Array.isArray(parsed.hooks)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-as-array',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Hooks configured as array instead of object',
|
||||
description: `${file.relPath}: "hooks" must be an object with event keys, not an array. All hooks will be ignored.`,
|
||||
|
|
|
|||
|
|
@ -87,6 +87,7 @@ export async function scan(_targetPath, _discovery, opts = {}) {
|
|||
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'description-over-cap',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Skill description exceeds the listing cap (Claude Code truncates it)',
|
||||
description:
|
||||
|
|
@ -116,6 +117,7 @@ export async function scan(_targetPath, _discovery, opts = {}) {
|
|||
// Conservative 200k anchor — byte-identical to the pre-B8 finding.
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'aggregate-listing-budget',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Aggregate skill descriptions may exceed the listing budget',
|
||||
description:
|
||||
|
|
@ -138,6 +140,7 @@ export async function scan(_targetPath, _discovery, opts = {}) {
|
|||
const winLabel = withCommas(window);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'aggregate-listing-budget',
|
||||
severity: advisory ? SEVERITY.info : SEVERITY.low,
|
||||
title: 'Aggregate skill descriptions may exceed the listing budget',
|
||||
description:
|
||||
|
|
@ -173,6 +176,7 @@ export async function scan(_targetPath, _discovery, opts = {}) {
|
|||
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'oversized-body',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Skill body is large (loads on demand when the skill runs)',
|
||||
description:
|
||||
|
|
|
|||
147
scanners/subtraction-write-cli.mjs
Normal file
147
scanners/subtraction-write-cli.mjs
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* subtraction-write CLI — execute an APPROVED subtraction set (§C6, chunk #63).
|
||||
*
|
||||
* The one path in the plugin that removes configuration. It takes no judgement
|
||||
* of its own: it is handed a set of blocks a human approved, and its whole job
|
||||
* is to refuse anything that no longer matches, is load-bearing, or leaves the
|
||||
* repo without an explicit go-ahead.
|
||||
*
|
||||
* Usage:
|
||||
* node subtraction-write-cli.mjs --approved <path.json>
|
||||
* [--repo <session-repo-root>]
|
||||
* [--approve-scope] [--dry-run]
|
||||
* [--output-file <path>] [--json]
|
||||
*
|
||||
* The approval file is written by MAIN CONTEXT, not by the lens agent —
|
||||
* `optimize.md` renders the candidates, the operator picks, and the command
|
||||
* materializes the choice. That is where the decision actually happens, and it
|
||||
* keeps this path off the unverified agent write surface
|
||||
* ([[subagent-harness-blocks-report-writes]] lists optimize as open).
|
||||
*
|
||||
* { "sessionId": "...",
|
||||
* "removals": [ { "file": "...", "line": 12, "endLine": 15, "text": "..." } ] }
|
||||
*
|
||||
* Exit codes: 0 = verdict, 3 = the CLI could not do its job (bad argv,
|
||||
* unreadable or malformed approval file).
|
||||
*
|
||||
* A gated or refused removal is NOT exit 3. "This write leaves the repo" and
|
||||
* "that block no longer looks like that" are verdicts about a write, and they
|
||||
* ride in the payload — a command cannot act on something that only ever
|
||||
* reached stderr (#62, F3's class).
|
||||
*
|
||||
* Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { resolve } from 'node:path';
|
||||
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
import { applySubtraction } from './lib/subtraction-write.mjs';
|
||||
|
||||
/** Flag surface. Anything else is exit 3. */
|
||||
const ARG_SPEC = {
|
||||
boolean: ['--json', '--dry-run', '--approve-scope'],
|
||||
value: ['--approved', '--repo', '--output-file'],
|
||||
};
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
|
||||
let approvedPath = null;
|
||||
let repo = process.cwd();
|
||||
let outputFile = null;
|
||||
let jsonMode = false;
|
||||
let dryRun = false;
|
||||
let approveScope = false;
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--json') jsonMode = true;
|
||||
else if (args[i] === '--dry-run') dryRun = true;
|
||||
else if (args[i] === '--approve-scope') approveScope = true;
|
||||
else if (args[i] === '--approved') approvedPath = args[++i];
|
||||
else if (args[i] === '--repo') repo = args[++i];
|
||||
else if (args[i] === '--output-file') outputFile = args[++i];
|
||||
}
|
||||
|
||||
if (!approvedPath) {
|
||||
process.stderr.write('Error: --approved <path> is required\n');
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
let approval;
|
||||
try {
|
||||
approval = JSON.parse(await readFile(resolve(approvedPath), 'utf-8'));
|
||||
} catch (err) {
|
||||
process.stderr.write(`Error: could not read approval file: ${err.message}\n`);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
// Malformed input is a tool error, not a verdict: an empty or wrong-shaped
|
||||
// approval must not read as "nothing to remove, all done".
|
||||
if (!Array.isArray(approval.removals) || approval.removals.length === 0) {
|
||||
process.stderr.write('Error: approval file has no `removals` array\n');
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
for (const r of approval.removals) {
|
||||
if (!r || typeof r.file !== 'string' || typeof r.text !== 'string') {
|
||||
process.stderr.write('Error: every removal needs `file`, `line`, `endLine` and `text`\n');
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
const result = await applySubtraction(approval.removals, { repoRoot: repo, dryRun, approveScope });
|
||||
|
||||
const payload = {
|
||||
meta: {
|
||||
repo: resolve(repo),
|
||||
sessionId: approval.sessionId || null,
|
||||
approvedCount: approval.removals.length,
|
||||
dryRun,
|
||||
},
|
||||
gate: result.gate,
|
||||
requiresApproval: result.requiresApproval,
|
||||
disclosures: result.disclosures,
|
||||
targets: result.targets,
|
||||
backupId: result.backupId,
|
||||
filesWritten: result.filesWritten,
|
||||
// The removed text travels back so the caller can show and log exactly what
|
||||
// left the file. The backup is the recovery artifact; this is the receipt.
|
||||
applied: result.applied,
|
||||
refused: result.refused,
|
||||
counts: {
|
||||
applied: result.applied.length,
|
||||
refused: result.refused.length,
|
||||
filesWritten: result.filesWritten.length,
|
||||
},
|
||||
};
|
||||
|
||||
const json = `${JSON.stringify(payload, null, 2)}\n`;
|
||||
if (outputFile) {
|
||||
await writeOutputFile(outputFile, json);
|
||||
// Nothing on stdout when writing to a file (ux-rules rule 1).
|
||||
} else if (jsonMode) {
|
||||
process.stdout.write(json);
|
||||
} else {
|
||||
for (const a of payload.applied) {
|
||||
process.stdout.write(`${payload.meta.dryRun ? 'would-remove' : 'removed'}\t${a.file}:${a.line}-${a.endLine}\n`);
|
||||
}
|
||||
for (const r of payload.refused) {
|
||||
process.stdout.write(`refused:${r.reason}\t${r.file}:${r.line}-${r.endLine}\n`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
await main();
|
||||
} catch (err) {
|
||||
process.stderr.write(`Error: ${err.message}\n`);
|
||||
process.exitCode = 3;
|
||||
}
|
||||
|
|
@ -14,12 +14,22 @@
|
|||
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { writeFile, readFile, stat } from 'node:fs/promises';
|
||||
import { readFile, stat } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { discoverConfigFiles } from './lib/file-discovery.mjs';
|
||||
import { resetCounter } from './lib/output.mjs';
|
||||
import { scan } from './token-hotspots.mjs';
|
||||
import * as tokenizerApi from './lib/tokenizer-api.mjs';
|
||||
import { humanizeFindings } from './lib/humanizer.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
|
||||
/** Flag surface, measured 2026-08-09. Anything else is exit 3. */
|
||||
const ARG_SPEC = {
|
||||
boolean: [
|
||||
'--json', '--raw', '--global', '--with-telemetry-recipe',
|
||||
'--accurate-tokens', '--exclude-cache', '--no-exclude-cache',
|
||||
],
|
||||
value: ['--output-file'],
|
||||
};
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const TELEMETRY_RECIPE_PATH = resolve(__dirname, '..', 'knowledge', 'cache-telemetry-recipe.md');
|
||||
|
|
@ -49,6 +59,7 @@ async function calibrateAgainstApi(hotspots, apiKey) {
|
|||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
let targetPath = '.';
|
||||
let outputFile = null;
|
||||
let jsonMode = false;
|
||||
|
|
@ -78,14 +89,15 @@ async function main() {
|
|||
const s = await stat(absPath);
|
||||
if (!s.isDirectory()) {
|
||||
process.stderr.write(`Error: ${absPath} is not a directory\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
process.stderr.write(`Error: path does not exist: ${absPath}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(absPath, { includeGlobal, excludeCache });
|
||||
const result = await scan(absPath, discovery);
|
||||
|
||||
|
|
@ -129,7 +141,7 @@ async function main() {
|
|||
const json = JSON.stringify(payload, null, 2);
|
||||
|
||||
if (outputFile) {
|
||||
await writeFile(outputFile, json, 'utf-8');
|
||||
await writeOutputFile(outputFile, json, 'utf-8');
|
||||
}
|
||||
|
||||
if (jsonMode || rawMode || !outputFile) {
|
||||
|
|
@ -141,6 +153,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -400,6 +400,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (detectVolatileTop(content)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'volatile-top',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Cache-breaking volatile content at top of CLAUDE.md',
|
||||
description:
|
||||
|
|
@ -428,6 +429,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (issues.length === 0) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'redundant-permissions',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Redundant permission declarations',
|
||||
description:
|
||||
|
|
@ -452,6 +454,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (depth > MAX_IMPORT_DEPTH) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deep-import-chain',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Deep @import chain defeats prompt-cache reuse',
|
||||
description:
|
||||
|
|
@ -484,6 +487,7 @@ export async function scan(targetPath, discovery) {
|
|||
const skillName = (fm && fm.name) || f.absPath.split('/').slice(-2, -1)[0] || f.absPath;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'bloated-skill-description',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Bloated skill description (loads on every turn)',
|
||||
description:
|
||||
|
|
@ -535,6 +539,7 @@ export async function scan(targetPath, discovery) {
|
|||
'and user-scopes so per-project budget stays tight.';
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'mcp-schema-budget',
|
||||
severity,
|
||||
title: `High MCP tool-schema budget on server "${m.name}"`,
|
||||
description,
|
||||
|
|
@ -552,6 +557,7 @@ export async function scan(targetPath, discovery) {
|
|||
const fileCount = activeConfig.claudeMd.files?.length ?? 0;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'cascade-over-budget',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md cascade exceeds 10k tokens per turn',
|
||||
description:
|
||||
|
|
@ -584,6 +590,7 @@ export async function scan(targetPath, discovery) {
|
|||
const keys = stale.map(v => v.key);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'stale-plugin-cache',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Stale plugin-cache versions (disk cleanup, zero live-context impact)',
|
||||
description:
|
||||
|
|
@ -654,6 +661,7 @@ export async function scan(targetPath, discovery) {
|
|||
'(gh / aws / gcloud) over MCP for common operations.';
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'mcp-schema-deferral',
|
||||
severity,
|
||||
title: 'MCP tool schemas forced into the always-loaded prefix',
|
||||
file: null,
|
||||
|
|
|
|||
|
|
@ -13,11 +13,17 @@
|
|||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeFile, stat } from 'node:fs/promises';
|
||||
import { stat } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { readActiveConfig } from './lib/active-config-reader.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
|
||||
/** Flag surface, measured 2026-08-09. Anything else is exit 3. */
|
||||
const ARG_SPEC = { boolean: ['--json', '--raw', '--verbose', '--suggest-disables'], value: ['--output-file'] };
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
let targetPath = '.';
|
||||
let outputFile = null;
|
||||
let jsonMode = false;
|
||||
|
|
@ -41,18 +47,20 @@ async function main() {
|
|||
const s = await stat(absPath);
|
||||
if (!s.isDirectory()) {
|
||||
process.stderr.write(`Error: ${absPath} is not a directory\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
process.stderr.write(`Error: path does not exist: ${absPath}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
const result = await readActiveConfig(absPath, { verbose, suggestDisables });
|
||||
const json = JSON.stringify(result, null, 2);
|
||||
|
||||
if (outputFile) {
|
||||
await writeFile(outputFile, json, 'utf-8');
|
||||
await writeOutputFile(outputFile, json, 'utf-8');
|
||||
}
|
||||
|
||||
if (jsonMode || rawMode || !outputFile) {
|
||||
|
|
@ -64,6 +72,6 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
|
|||
if (isDirectRun) {
|
||||
main().catch(err => {
|
||||
process.stderr.write(`Fatal: ${err.message}\n`);
|
||||
process.exit(3);
|
||||
process.exitCode = 3;
|
||||
});
|
||||
}
|
||||
|
|
|
|||
97
scanners/write-scope-cli.mjs
Normal file
97
scanners/write-scope-cli.mjs
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* write-scope CLI — classify the write targets a command is about to touch,
|
||||
* relative to the repo the session stands in (M-BUG-41).
|
||||
*
|
||||
* Exists so the gate has ONE implementation. Five command templates need the
|
||||
* same answer before their approval surface; five prose paraphrases of the
|
||||
* class table would be five policies drifting apart — the shape that put the
|
||||
* lever table in five copies (#61). The templates call this and render what
|
||||
* comes back.
|
||||
*
|
||||
* Usage:
|
||||
* node write-scope-cli.mjs --target <path> [--target <path> ...]
|
||||
* [--repo <session-repo-root>]
|
||||
* [--output-file <path>] [--json]
|
||||
*
|
||||
* Exit codes: 0 = classified, 3 = argument or tool error.
|
||||
*
|
||||
* A gated target is NOT an error exit. The exit-code contract reserves 3 for
|
||||
* "the scanner could not do its job"; "this write leaves the repo" is a verdict
|
||||
* about a write, and it rides in the payload — a command cannot act on
|
||||
* something that only ever reached stderr (F3's class).
|
||||
*
|
||||
* Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireValidArgs } from './lib/cli-args.mjs';
|
||||
import { SCOPE_CLASSES, classifyWriteTarget, strongestGate } from './lib/write-scope.mjs';
|
||||
|
||||
/** Flag surface. Anything else is exit 3. */
|
||||
const ARG_SPEC = { boolean: ['--json'], value: ['--target', '--repo', '--output-file'] };
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
if (!requireValidArgs(args, ARG_SPEC)) return;
|
||||
|
||||
const targets = [];
|
||||
let repo = process.cwd();
|
||||
let outputFile = null;
|
||||
let jsonMode = false;
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--json') jsonMode = true;
|
||||
else if (args[i] === '--target') targets.push(args[++i]);
|
||||
else if (args[i] === '--repo') repo = args[++i];
|
||||
else if (args[i] === '--output-file') outputFile = args[++i];
|
||||
}
|
||||
|
||||
if (targets.length === 0) {
|
||||
process.stderr.write('Error: at least one --target is required\n');
|
||||
process.exitCode = 3;
|
||||
return;
|
||||
}
|
||||
|
||||
const classified = targets.map((t) => classifyWriteTarget(t, repo));
|
||||
const gate = strongestGate(classified);
|
||||
|
||||
const payload = {
|
||||
meta: {
|
||||
repo: resolve(repo),
|
||||
targetCount: classified.length,
|
||||
// The class table travels with the answer so a template never has to
|
||||
// restate what a class means.
|
||||
classes: Object.fromEntries(
|
||||
Object.entries(SCOPE_CLASSES).map(([name, spec]) => [name, { gate: spec.gate }]),
|
||||
),
|
||||
},
|
||||
gate,
|
||||
requiresApproval: gate === 'require-ok',
|
||||
// Distinct disclosure lines, in class order, ready to render verbatim.
|
||||
disclosures: [...new Set(classified.map((t) => t.disclosure).filter(Boolean))],
|
||||
targets: classified,
|
||||
};
|
||||
|
||||
const json = `${JSON.stringify(payload, null, 2)}\n`;
|
||||
if (outputFile) {
|
||||
await writeOutputFile(outputFile, json);
|
||||
// Nothing on stdout when writing to a file: a command that also renders
|
||||
// this would otherwise show the user the raw payload (ux-rules rule 1).
|
||||
} else if (jsonMode) {
|
||||
process.stdout.write(json);
|
||||
} else {
|
||||
for (const t of payload.targets) {
|
||||
process.stdout.write(`${t.scopeClass}\t${t.gate}\t${t.target}\n`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
await main();
|
||||
} catch (err) {
|
||||
process.stderr.write(`Error: ${err.message}\n`);
|
||||
process.exitCode = 3;
|
||||
}
|
||||
109
tests/agents/agent-write-contract.test.mjs
Normal file
109
tests/agents/agent-write-contract.test.mjs
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
/**
|
||||
* R4 — an agent file must not carry a contract its `tools:` cannot keep.
|
||||
*
|
||||
* `agents/verifier-agent.md` said both things at once: §Output Format
|
||||
* ("Append to: implementation-log.md") and §Read-Only Guarantee ("never
|
||||
* modifies any files"), while its frontmatter granted only Read/Glob/Grep.
|
||||
* Which instruction wins is nondeterministic, and the failure mode is not
|
||||
* "the write fails" — it is the agent improvising a full-file Write on the
|
||||
* SHARED implementation log, clobbering parallel implementer entries. That is
|
||||
* exactly the defect `implement-log-append.test.mjs` exists to prevent,
|
||||
* entering through the one file that test does not read.
|
||||
*
|
||||
* The guard is the blanket invariant over the whole agents/ catalogue, not a
|
||||
* statement about verifier-agent: any agent whose tools grant no write
|
||||
* capability must (a) instruct no file write, and (b) say positively that it
|
||||
* returns its findings inline. Both sides are read from the file — the tools
|
||||
* list AND the body — so removing a write tool from any agent whose body still
|
||||
* instructs a write turns this red.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFile, readdir } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = resolve(__dirname, '..', '..');
|
||||
const AGENTS_DIR = resolve(ROOT, 'agents');
|
||||
|
||||
/** Tools that can put bytes on disk. Bash counts: `>>` is a write. */
|
||||
const WRITE_TOOLS = new Set(['Write', 'Edit', 'NotebookEdit', 'Bash']);
|
||||
|
||||
/**
|
||||
* Directive lines that tell the agent to put its output in a file.
|
||||
* Anchored to line start so prose ABOUT writing ("do not write it to a file")
|
||||
* is not caught — the defect is an instruction, not a mention.
|
||||
*/
|
||||
const WRITE_DIRECTIVE_RE =
|
||||
/^(?:\*\*)?(?:Append|Write|Save|Output|Persist)\b(?![^\n]*\bnot\b)[^\n]*?(?:\bto\b|`[^`\n]+\.(?:md|ya?ml|json)`)/mi;
|
||||
|
||||
/** The positive half: the file must say the findings come back inline. */
|
||||
const RETURN_INLINE_RE =
|
||||
/\breturn\b[^.]{0,120}?\b(?:as\s+your\s+final\s+message|inline)\b/i;
|
||||
|
||||
function frontmatterOf(content) {
|
||||
const m = content.match(/^---\r?\n([\s\S]*?)\r?\n---\r?\n/);
|
||||
return m ? m[1] : '';
|
||||
}
|
||||
|
||||
function bodyOf(content) {
|
||||
const m = content.match(/^---\r?\n[\s\S]*?\r?\n---\r?\n/);
|
||||
return m ? content.slice(m[0].length) : content;
|
||||
}
|
||||
|
||||
function toolsOf(frontmatter) {
|
||||
const m = frontmatter.match(/^tools:\s*(.+)$/m);
|
||||
if (!m) return [];
|
||||
return (m[1].match(/[A-Za-z_][A-Za-z0-9_]*/g) || []);
|
||||
}
|
||||
|
||||
async function loadAgents() {
|
||||
const names = (await readdir(AGENTS_DIR)).filter((n) => n.endsWith('.md')).sort();
|
||||
return Promise.all(
|
||||
names.map(async (name) => {
|
||||
const content = await readFile(resolve(AGENTS_DIR, name), 'utf-8');
|
||||
const frontmatter = frontmatterOf(content);
|
||||
const tools = toolsOf(frontmatter);
|
||||
return {
|
||||
name,
|
||||
body: bodyOf(content),
|
||||
tools,
|
||||
canWrite: tools.some((t) => WRITE_TOOLS.has(t)),
|
||||
};
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
test('R4 sweep is not vacuous: the agents catalogue is read and at least one agent has no write tool', async () => {
|
||||
const agents = await loadAgents();
|
||||
assert.ok(agents.length >= 7,
|
||||
`expected the agents/ catalogue to be swept, got ${agents.length} files`);
|
||||
assert.ok(agents.every((a) => a.tools.length > 0),
|
||||
`every agent must declare tools:, missing in ${agents.filter((a) => !a.tools.length).map((a) => a.name).join(', ')}`);
|
||||
const writeless = agents.filter((a) => !a.canWrite);
|
||||
assert.ok(writeless.length >= 1,
|
||||
'no write-tool-less agent found — the invariant below would be vacuously green');
|
||||
});
|
||||
|
||||
test('R4: no agent instructs a file write its tools cannot perform', async () => {
|
||||
const agents = await loadAgents();
|
||||
for (const agent of agents.filter((a) => !a.canWrite)) {
|
||||
const offending = agent.body.match(WRITE_DIRECTIVE_RE);
|
||||
assert.ok(
|
||||
offending === null,
|
||||
`${agent.name} grants no write tool (tools: ${agent.tools.join(', ')}) but instructs a write: ${JSON.stringify(offending && offending[0])}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('R4: a write-tool-less agent states positively that it returns findings inline', async () => {
|
||||
const agents = await loadAgents();
|
||||
for (const agent of agents.filter((a) => !a.canWrite)) {
|
||||
assert.ok(
|
||||
RETURN_INLINE_RE.test(agent.body),
|
||||
`${agent.name} has no write tool, so it must say its findings are returned as its final message`,
|
||||
);
|
||||
}
|
||||
});
|
||||
138
tests/agents/frontmatter-contract.test.mjs
Normal file
138
tests/agents/frontmatter-contract.test.mjs
Normal file
|
|
@ -0,0 +1,138 @@
|
|||
/**
|
||||
* R6 — the two "Required Frontmatter" contracts are enforced by something.
|
||||
*
|
||||
* `.claude/rules/agent-development.md` and `.claude/rules/command-development.md`
|
||||
* write down which frontmatter keys an agent/command MUST carry, and that agent
|
||||
* colors must be unique within the plugin. Nothing enforced any of it: the only
|
||||
* test that looked at agent frontmatter (`agent-prompt-shape`) asserted `name:`
|
||||
* on a HAND-WRITTEN list of 3 of the 7 agents. Everything was compliant and
|
||||
* unwatched — the state in which a rule becomes fiction one file at a time.
|
||||
*
|
||||
* Nothing here is hand-maintained, because a hand-kept list of what to sweep is
|
||||
* a premise, not a measurement (the #57 shape):
|
||||
* - the required KEYS are parsed out of each rule's own ```yaml block;
|
||||
* - the files swept are resolved from each rule's own `paths:` frontmatter;
|
||||
* - the plugin name is read from `.claude-plugin/plugin.json`.
|
||||
* Add a key to a rule and it is enforced on the next run, with no test edit.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFile, readdir } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = resolve(__dirname, '..', '..');
|
||||
const RULES_DIR = resolve(ROOT, '.claude', 'rules');
|
||||
|
||||
function frontmatterOf(content) {
|
||||
const m = content.match(/^---\r?\n([\s\S]*?)\r?\n---\r?\n/);
|
||||
return m ? m[1] : '';
|
||||
}
|
||||
|
||||
/** Top-level keys of a frontmatter block, in source order. */
|
||||
function topLevelKeys(frontmatter) {
|
||||
return frontmatter
|
||||
.split('\n')
|
||||
.map((line) => line.match(/^([A-Za-z][A-Za-z0-9_-]*):/))
|
||||
.filter(Boolean)
|
||||
.map((m) => m[1]);
|
||||
}
|
||||
|
||||
function valueOf(frontmatter, key) {
|
||||
const m = frontmatter.match(new RegExp(`^${key}:[ \\t]*(.*)$`, 'm'));
|
||||
return m ? m[1].trim() : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* A rule file is the contract. Read the required keys out of the ```yaml block
|
||||
* under "## Required Frontmatter", and the swept directory out of `paths:`.
|
||||
*/
|
||||
async function loadRule(fileName) {
|
||||
const content = await readFile(resolve(RULES_DIR, fileName), 'utf-8');
|
||||
const paths = valueOf(frontmatterOf(content), 'paths');
|
||||
const fence = content.match(/##\s+Required Frontmatter[\s\S]*?```ya?ml\r?\n([\s\S]*?)```/);
|
||||
const requiredKeys = fence ? topLevelKeys(fence[1]).filter((k) => k !== '---') : [];
|
||||
const dir = paths ? paths.split('/')[0] : null;
|
||||
return { fileName, paths, dir, requiredKeys };
|
||||
}
|
||||
|
||||
async function loadTargets(rule) {
|
||||
const dirAbs = resolve(ROOT, rule.dir);
|
||||
const names = (await readdir(dirAbs)).filter((n) => n.endsWith('.md')).sort();
|
||||
return Promise.all(
|
||||
names.map(async (name) => {
|
||||
const frontmatter = frontmatterOf(await readFile(resolve(dirAbs, name), 'utf-8'));
|
||||
return { name, frontmatter, keys: topLevelKeys(frontmatter) };
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
const AGENT_RULE = 'agent-development.md';
|
||||
const COMMAND_RULE = 'command-development.md';
|
||||
|
||||
test('R6 derivation is not vacuous: both rules yield required keys and a non-empty file set', async () => {
|
||||
for (const fileName of [AGENT_RULE, COMMAND_RULE]) {
|
||||
const rule = await loadRule(fileName);
|
||||
assert.ok(rule.requiredKeys.length > 0,
|
||||
`${fileName}: no required keys parsed from its "Required Frontmatter" yaml block — every assertion below would be vacuously green`);
|
||||
assert.ok(rule.dir, `${fileName}: no paths: frontmatter to resolve a file set from`);
|
||||
const targets = await loadTargets(rule);
|
||||
assert.ok(targets.length > 0,
|
||||
`${fileName}: paths: ${rule.paths} resolved to 0 files — the sweep would prove nothing`);
|
||||
}
|
||||
});
|
||||
|
||||
test('R6: every agent and command carries the keys its rule requires, non-empty', async () => {
|
||||
for (const fileName of [AGENT_RULE, COMMAND_RULE]) {
|
||||
const rule = await loadRule(fileName);
|
||||
const targets = await loadTargets(rule);
|
||||
for (const target of targets) {
|
||||
for (const key of rule.requiredKeys) {
|
||||
assert.ok(target.keys.includes(key),
|
||||
`${rule.dir}/${target.name} is missing required frontmatter key "${key}" (required by .claude/rules/${fileName}; ${targets.length} files swept)`);
|
||||
const value = valueOf(target.frontmatter, key);
|
||||
assert.ok(value !== null && value !== '',
|
||||
`${rule.dir}/${target.name} has an empty "${key}" — a present-but-empty key satisfies no contract`);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('R6: agent colors are unique within the plugin', async () => {
|
||||
const rule = await loadRule(AGENT_RULE);
|
||||
const targets = await loadTargets(rule);
|
||||
const seen = new Map();
|
||||
for (const target of targets) {
|
||||
const color = valueOf(target.frontmatter, 'color');
|
||||
if (seen.has(color)) {
|
||||
assert.fail(`duplicate agent color "${color}": ${seen.get(color)} and ${target.name} (.claude/rules/${AGENT_RULE}: "Color must be unique within the plugin")`);
|
||||
}
|
||||
seen.set(color, target.name);
|
||||
}
|
||||
assert.equal(seen.size, targets.length, `expected ${targets.length} distinct colors, got ${seen.size}`);
|
||||
});
|
||||
|
||||
test('R6: agent names are kebab-case with the -agent suffix', async () => {
|
||||
const rule = await loadRule(AGENT_RULE);
|
||||
for (const target of await loadTargets(rule)) {
|
||||
const name = valueOf(target.frontmatter, 'name');
|
||||
assert.match(name, /^[a-z0-9]+(?:-[a-z0-9]+)*-agent$/,
|
||||
`agents/${target.name}: name "${name}" must be kebab-case with an -agent suffix`);
|
||||
}
|
||||
});
|
||||
|
||||
test('R6: command names are plugin:action, or the bare plugin name for the router', async () => {
|
||||
const plugin = JSON.parse(await readFile(resolve(ROOT, '.claude-plugin', 'plugin.json'), 'utf-8')).name;
|
||||
const rule = await loadRule(COMMAND_RULE);
|
||||
const targets = await loadTargets(rule);
|
||||
const routers = [];
|
||||
for (const target of targets) {
|
||||
const name = valueOf(target.frontmatter, 'name');
|
||||
if (name === plugin) { routers.push(target.name); continue; }
|
||||
assert.match(name, new RegExp(`^${plugin}:[a-z0-9]+(?:-[a-z0-9]+)*$`),
|
||||
`commands/${target.name}: name "${name}" must be "${plugin}:action" (or the bare "${plugin}" router)`);
|
||||
}
|
||||
assert.equal(routers.length, 1, `expected exactly one bare-"${plugin}" router command, got ${routers.length}: ${routers.join(', ')}`);
|
||||
});
|
||||
|
|
@ -41,13 +41,33 @@ async function readCommand(name) {
|
|||
return await readFile(resolve(COMMANDS_DIR, name), 'utf-8');
|
||||
}
|
||||
|
||||
test('Action: every file contains a Bash invocation block', async () => {
|
||||
// plan.md invokes no scanner — it spawns the planner agent. Its only bash
|
||||
// block used to be a `RAW_FLAG=` assignment referenced from the agent prompt
|
||||
// below it; a prompt is not a shell, so the agent received the literal string
|
||||
// `$RAW_FLAG` (confirmed at runtime by the planner agent, session #49).
|
||||
// Removing that block is the fix, so this assertion skips it.
|
||||
const AGENT_DRIVEN = new Set(['plan.md']);
|
||||
|
||||
test('Action: every scanner-invoking file contains a Bash invocation block', async () => {
|
||||
for (const name of ACTION_FILES) {
|
||||
if (AGENT_DRIVEN.has(name)) continue;
|
||||
const content = await readCommand(name);
|
||||
assert.match(content, BASH_BLOCK_REGEX, `${name} missing bash block`);
|
||||
}
|
||||
});
|
||||
|
||||
test('Action: agent-driven files spawn an Agent instead of a scanner', async () => {
|
||||
for (const name of AGENT_DRIVEN) {
|
||||
const content = await readCommand(name);
|
||||
assert.match(content, /Agent\(subagent_type:/, `${name} should spawn an Agent`);
|
||||
assert.doesNotMatch(
|
||||
content,
|
||||
/RAW_FLAG=/,
|
||||
`${name} must not assign a shell variable it then references from an agent prompt`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('Action: every file references the Read tool', async () => {
|
||||
for (const name of ACTION_FILES) {
|
||||
const content = await readCommand(name);
|
||||
|
|
|
|||
227
tests/commands/backup-restore-contract.test.mjs
Normal file
227
tests/commands/backup-restore-contract.test.mjs
Normal file
|
|
@ -0,0 +1,227 @@
|
|||
/**
|
||||
* R1 + R2 — the backup/restore data contract belongs to the code.
|
||||
*
|
||||
* Two templates used to own it in prose, and each half made the other's failure
|
||||
* silent:
|
||||
*
|
||||
* R1 `commands/rollback.md` drove the restore itself. Its "Implementation"
|
||||
* section showed an ESM `import` block a command template cannot execute and
|
||||
* offered ad-hoc `cp` as the runnable alternative, then pre-rendered
|
||||
* "`(checksum verified)`" in the success output. `cp` verifies nothing, so the
|
||||
* claim was a property of the template, not of the run.
|
||||
*
|
||||
* R2 `commands/implement.md` Step 3 hand-built the backup: `mkdir`, `cp`, a
|
||||
* `date +%Y%m%d_%H%M%S` id, and a manifest typed out in the template. The
|
||||
* parser on the other side (`parseManifest`) knew ONE frozen sample of that
|
||||
* format, pinned by a HAND-WRITTEN fixture rather than by the template's own
|
||||
* text — the #63 shape on the data side. Rename a key in the template and
|
||||
* `parseManifest` returns zero files while `rollback` reports success.
|
||||
*
|
||||
* The fix is one chunk because half of it does not hold: a CLI over a prose
|
||||
* format still parses prose, and a clean format with no runnable entry still
|
||||
* cannot restore. What this file asserts is that neither half came back.
|
||||
*
|
||||
* The last test is the one that replaces "(checksum verified)": every field the
|
||||
* template renders is checked against a payload produced by RUNNING the CLI, so
|
||||
* the output prose can only claim what the code actually reports.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { readFile, writeFile, mkdtemp, mkdir, rm } from 'node:fs/promises';
|
||||
import { resolve, dirname, join } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = resolve(__dirname, '..', '..');
|
||||
const COMMANDS_DIR = join(ROOT, 'commands');
|
||||
const CLI = join(ROOT, 'scanners', 'rollback-cli.mjs');
|
||||
|
||||
const read = (name) => readFile(join(COMMANDS_DIR, name), 'utf-8');
|
||||
|
||||
/** Fenced blocks as `[{ lang, body }]`. */
|
||||
function fences(content) {
|
||||
return [...content.matchAll(/```(\w*)\n([\s\S]*?)```/g)].map((m) => ({ lang: m[1], body: m[2] }));
|
||||
}
|
||||
|
||||
test('rollback.md drives the engine through the CLI, like every other command', async () => {
|
||||
const content = await read('rollback.md');
|
||||
|
||||
assert.match(
|
||||
content,
|
||||
/node \$\{CLAUDE_PLUGIN_ROOT\}\/scanners\/rollback-cli\.mjs/,
|
||||
'rollback.md must call the rollback CLI at an anchored path. A relative path resolves\n' +
|
||||
"against the user's working directory, not the plugin ([[plugin-root-is-the-cache]]).",
|
||||
);
|
||||
assert.match(
|
||||
content,
|
||||
/rollback-cli\.mjs[^\n]*--output-file[^\n]*2>\/dev\/null/,
|
||||
'The CLI must be invoked as `--output-file <path> 2>/dev/null` (ux-rules rule 2). A payload\n' +
|
||||
'the command has to act on cannot ride on stdout or stderr.',
|
||||
);
|
||||
});
|
||||
|
||||
test('rollback.md no longer pre-renders a verification it did not perform', async () => {
|
||||
const content = await read('rollback.md');
|
||||
|
||||
assert.doesNotMatch(
|
||||
content,
|
||||
/\(checksum verified\)/,
|
||||
'The success block hard-codes "(checksum verified)" for every file. The runnable path in\n' +
|
||||
'this template (`cp`) establishes no checksum at all, so the words came from the template\n' +
|
||||
'rather than from the run. Render the per-file `status` the CLI returns instead.',
|
||||
);
|
||||
});
|
||||
|
||||
test('rollback.md offers no hand-rolled restore beside the engine', async () => {
|
||||
const content = await read('rollback.md');
|
||||
|
||||
for (const { lang, body } of fences(content)) {
|
||||
if (lang !== 'bash') continue;
|
||||
assert.doesNotMatch(
|
||||
body,
|
||||
/^\s*cp\s+/m,
|
||||
'A `cp` restore skips the checksum verification before and after each write that the\n' +
|
||||
'engine performs. Two restore paths means the safe one is optional.',
|
||||
);
|
||||
}
|
||||
|
||||
assert.doesNotMatch(
|
||||
content,
|
||||
/import\s*\{[^}]*\}\s*from\s*['"][^'"]*rollback-engine\.mjs['"]/,
|
||||
'A command template is not an ES module; an `import` block here is instructions the\n' +
|
||||
'runtime never executes, which is what left `cp` as the only runnable path.',
|
||||
);
|
||||
});
|
||||
|
||||
test('implement.md Step 3 creates its backup through the code, not by hand', async () => {
|
||||
const content = await read('implement.md');
|
||||
|
||||
// `--create(?![a-z])`, not `--create`: the same line carries `--created`, so a
|
||||
// prefix match is satisfied by the very flag whose presence proves nothing
|
||||
// about whether a backup is being MADE. Measured by mutating `--create` away
|
||||
// and watching this assertion stay green.
|
||||
assert.match(
|
||||
content,
|
||||
/node \$\{CLAUDE_PLUGIN_ROOT\}\/scanners\/rollback-cli\.mjs[^\n]*--create(?![a-z])/,
|
||||
'Step 3 must call `rollback-cli.mjs --create`. Backing up through the same code path the\n' +
|
||||
'fix pipeline uses is what deletes the second copy of the backup policy.',
|
||||
);
|
||||
assert.match(
|
||||
content,
|
||||
/--created\b/,
|
||||
'Files this run will CREATE have no backup and must be recorded so rollback can say what\n' +
|
||||
'it is leaving behind. The list moved from hand-written YAML to a flag; it must still exist.',
|
||||
);
|
||||
});
|
||||
|
||||
test('implement.md hand-builds no manifest and invents no backup id', async () => {
|
||||
const content = await read('implement.md');
|
||||
|
||||
for (const { body } of fences(content)) {
|
||||
assert.doesNotMatch(
|
||||
body,
|
||||
/^\s*sha256:/m,
|
||||
'Step 3 types out a manifest whose only reader is `parseManifest`. Every key here is a\n' +
|
||||
'contract with code, maintained by hand on one side — the seam that already failed once\n' +
|
||||
'(M-BUG-25) and whose fixture was hand-written rather than derived from this template.',
|
||||
);
|
||||
assert.doesNotMatch(
|
||||
body,
|
||||
/mkdir\s+-p\s+~\/\.claude\/config-audit\/backups/,
|
||||
'The template builds the backup directory layout itself. The layout is `createBackup`\'s\n' +
|
||||
'to define; a second definition in prose drifts away from it silently.',
|
||||
);
|
||||
assert.doesNotMatch(
|
||||
body,
|
||||
/BACKUP_ID=\$\(date/,
|
||||
'A backup id derived by the template is a second id generator. `createBackup` returns the\n' +
|
||||
'id it actually used; anything else can name a directory that does not exist.',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('every field rollback.md renders is a field the CLI really emits', async () => {
|
||||
// The replacement for "(checksum verified)": the output prose is checked
|
||||
// against a payload produced by RUNNING the CLI. A renamed payload key turns
|
||||
// the template's claim into a violation here rather than into a confident
|
||||
// sentence in front of a user who is already in trouble.
|
||||
//
|
||||
// `K` is the only derived reference — a count the model computes from the
|
||||
// classified targets, not a field any scanner emits.
|
||||
const DERIVED = new Set(['K']);
|
||||
|
||||
const root = await mkdtemp(join(tmpdir(), 'ca-rbrender-'));
|
||||
try {
|
||||
const home = join(root, 'home');
|
||||
const work = join(root, 'work');
|
||||
const cwd = join(root, 'cwd');
|
||||
await mkdir(join(home, '.claude'), { recursive: true });
|
||||
await mkdir(work, { recursive: true });
|
||||
await mkdir(cwd, { recursive: true });
|
||||
const backupRoot = join(home, '.claude', 'config-audit', 'backups');
|
||||
const env = {
|
||||
...process.env,
|
||||
HOME: home,
|
||||
USERPROFILE: home,
|
||||
CONFIG_AUDIT_BACKUP_ROOT: backupRoot,
|
||||
CONFIG_AUDIT_LEGACY_BACKUP_ROOT: join(home, '.config-audit', 'backups'),
|
||||
};
|
||||
const runCli = (argv) => new Promise((res) => {
|
||||
const child = spawn(process.execPath, [CLI, ...argv], { cwd, env });
|
||||
child.stdout.on('data', () => {});
|
||||
child.stderr.on('data', () => {});
|
||||
child.on('close', (code) => res(code));
|
||||
});
|
||||
|
||||
const target = join(work, 'CLAUDE.md');
|
||||
await writeFile(target, '# original\n');
|
||||
const createOut = join(root, 'create.json');
|
||||
await runCli([
|
||||
'--create', '--target', target,
|
||||
'--created', join(work, 'made-by-implement.md'),
|
||||
'--repo', work, '--output-file', createOut,
|
||||
]);
|
||||
const created = JSON.parse(await readFile(createOut, 'utf-8'));
|
||||
|
||||
const listOut = join(root, 'list.json');
|
||||
await runCli(['--list', '--output-file', listOut]);
|
||||
const listed = JSON.parse(await readFile(listOut, 'utf-8'));
|
||||
|
||||
const restoreOut = join(root, 'restore.json');
|
||||
await runCli([
|
||||
'--restore', created.backupId, '--repo', work, '--dry-run', '--output-file', restoreOut,
|
||||
]);
|
||||
const restored = JSON.parse(await readFile(restoreOut, 'utf-8'));
|
||||
|
||||
const keys = new Set();
|
||||
const walk = (node) => {
|
||||
if (Array.isArray(node)) node.slice(0, 5).forEach(walk);
|
||||
else if (node && typeof node === 'object') {
|
||||
for (const k of Object.keys(node)) { keys.add(k); walk(node[k]); }
|
||||
}
|
||||
};
|
||||
[created, listed, restored].forEach(walk);
|
||||
|
||||
const content = await read('rollback.md');
|
||||
// Render fences only. A `bash` fence is a command, not a render contract,
|
||||
// and `${CLAUDE_PLUGIN_ROOT}` would otherwise read as a payload field.
|
||||
const refs = new Set();
|
||||
for (const { lang, body } of fences(content)) {
|
||||
if (lang === 'bash') continue;
|
||||
for (const m of body.matchAll(/\{([a-zA-Z][a-zA-Z0-9_.]*)\}/g)) refs.add(m[1]);
|
||||
}
|
||||
assert.ok(refs.size > 0, 'no render reference found in rollback.md — the sweep certifies nothing');
|
||||
|
||||
const missing = [...refs].filter((r) => !DERIVED.has(r) && !keys.has(r.split('.').pop()));
|
||||
assert.deepEqual(
|
||||
missing,
|
||||
[],
|
||||
'rollback.md renders fields rollback-cli.mjs never emits:\n ' + missing.join('\n '),
|
||||
);
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
208
tests/commands/command-cli-contract.test.mjs
Normal file
208
tests/commands/command-cli-contract.test.mjs
Normal file
|
|
@ -0,0 +1,208 @@
|
|||
/**
|
||||
* Q2 — the command layer's contract with the CLI layer.
|
||||
*
|
||||
* The templates tell an agent to run a CLI with a particular argv. Until now
|
||||
* nothing checked that the CLI on the other end still *accepts* that argv. The
|
||||
* measured cost of that gap is M-BUG-45: `--stale-after` reached
|
||||
* `knowledge-refresh-cli` malformed, the CLI ignored it, and the command
|
||||
* reported "✓ all 14 entries re-verified within the last 90 days" about a
|
||||
* threshold the user had just overridden. A flag that quietly stops existing
|
||||
* produces the same sentence.
|
||||
*
|
||||
* Three properties are load-bearing here, each learned from a guard that was
|
||||
* green on its own defect:
|
||||
*
|
||||
* 1. **The argv is built from the template's own text**, never hand-typed
|
||||
* ([[dogfood-the-command-not-the-cli]], #63 — a hand-written call is a path
|
||||
* no user takes). `tests/helpers/command-invocations.mjs` reads all three
|
||||
* forms a flag appears in, including the comment-only form
|
||||
* (`GLOBAL_FLAG="" # --global`), which is the one that can die unobserved
|
||||
* because the default path leaves the variable empty.
|
||||
*
|
||||
* 2. **The probe proves itself per CLI before it is trusted.** A CLI that
|
||||
* exits on a required-arg check before reaching flag parsing would report
|
||||
* nothing about any flag, and every pair for it would pass vacuously. So each
|
||||
* CLI must first be seen rejecting a flag that certainly does not exist. An
|
||||
* empty sweep certifies nothing (#63, #64). Measured when this was written:
|
||||
* 15 of 15 report the unknown flag first, so no prefix argv is needed — a
|
||||
* fact worth re-deriving rather than assuming, which is why it is asserted.
|
||||
*
|
||||
* 3. **"Unknown" is distinguished from "needs a value" by the CLI's own
|
||||
* words**, so a value-taking flag passed last is not mistaken for a dead one.
|
||||
* That distinction is only sound because every CLI classifies the two
|
||||
* correctly — measured 14/15 when this was written; the fifteenth
|
||||
* (`campaign-export-cli`, the last hand-rolled parser) called its own
|
||||
* required `--repo` an unknown flag and was moved onto the shared
|
||||
* `requireValidArgs` gate in the same chunk. A guard that special-cased it
|
||||
* instead would have rebuilt, in test code, the prose exception Q1 deleted.
|
||||
*
|
||||
* Not asserted here: that a template calling a gated writer also calls
|
||||
* `write-scope-cli`. That arm belongs to write-scope-gate-shape.test.mjs, and
|
||||
* deriving it from the writer set would be false-red — `discover` and
|
||||
* `config-audit` invoke `scan-orchestrator` without ever reaching its
|
||||
* `--save-baseline` write.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { access, mkdtemp, readdir, rm } from 'node:fs/promises';
|
||||
import { resolve, dirname, join } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { allInvocations, commandTemplates, extractInvocations } from '../helpers/command-invocations.mjs';
|
||||
import { hermeticEnv } from '../helpers/hermetic-home.mjs';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const SCANNERS_DIR = resolve(__dirname, '..', '..', 'scanners');
|
||||
|
||||
/** A flag no CLI can plausibly define — the probe's own control. */
|
||||
const IMPOSSIBLE_FLAG = '--zzz-not-a-real-flag';
|
||||
|
||||
/**
|
||||
* Probing a flag means RUNNING the CLI, and some of these flags are writers.
|
||||
* Measured the first time this file ran, against the real environment:
|
||||
* `drift-cli --save` takes its target from the working directory and its name
|
||||
* from a default, so the probe scanned the temp dir and overwrote the
|
||||
* operator's `~/.config-audit/baselines/default.json` — an ungated write
|
||||
* outside the repo, produced by the guard whose whole subject is ungated writes
|
||||
* outside the repo. `fix-cli --apply` is the same shape one step worse.
|
||||
*
|
||||
* So every probe runs with HOME redirected into an empty temp dir AND its own
|
||||
* empty working directory, and the working directory is asserted to have stayed
|
||||
* empty. Isolation that is only a convention is not isolation.
|
||||
*/
|
||||
async function run(cli, argv) {
|
||||
const sandbox = await mkdtemp(join(tmpdir(), 'ca-contract-'));
|
||||
try {
|
||||
const { code, stderr } = await new Promise((res) => {
|
||||
const child = spawn(process.execPath, [resolve(SCANNERS_DIR, cli), ...argv], {
|
||||
cwd: sandbox,
|
||||
env: hermeticEnv(),
|
||||
});
|
||||
let err = '';
|
||||
child.stderr.on('data', (d) => { err += d; });
|
||||
child.stdout.on('data', () => {});
|
||||
child.on('close', (c) => res({ code: c, stderr: err }));
|
||||
});
|
||||
return { code, stderr, wrote: await readdir(sandbox) };
|
||||
} finally {
|
||||
await rm(sandbox, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Did the CLI call THIS flag unknown?
|
||||
*
|
||||
* Both wordings in the tree name the flag on the same line — `unknown flag
|
||||
* "--x"` (shared `cli-args`) and `Unknown option: --x` (the BOOL_FLAGS
|
||||
* parsers). "needs a value" / "requires a value" are deliberately NOT matched:
|
||||
* a value-taking flag passed last is well-formed as far as this contract goes.
|
||||
*/
|
||||
function reportedUnknown(stderr, flag) {
|
||||
const escaped = flag.replace(/[.*+?^${}()|[\]\\-]/g, '\\$&');
|
||||
return new RegExp(`unknown (flag|option)[^\\n]*${escaped}`, 'i').test(stderr);
|
||||
}
|
||||
|
||||
const invocations = await allInvocations();
|
||||
const clis = [...new Set(invocations.map((i) => i.cli))].sort();
|
||||
|
||||
/** `Map<cli, Map<flag, site[]>>` — every flag a template hands each CLI. */
|
||||
const surface = new Map();
|
||||
for (const inv of invocations) {
|
||||
if (!surface.has(inv.cli)) surface.set(inv.cli, new Map());
|
||||
const flags = surface.get(inv.cli);
|
||||
for (const flag of inv.flags) {
|
||||
if (!flags.has(flag)) flags.set(flag, []);
|
||||
flags.get(flag).push(`${inv.file}:${inv.line} (${inv.source.get(flag)})`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The extractor is the floor everything else stands on. A regex that stops
|
||||
* matching would make every assertion below vacuously green (#63, #64), and the
|
||||
* failure would look exactly like success. Coverage is derived from the tree
|
||||
* rather than pinned to a literal count, which would only be a drift point
|
||||
* (#60, #61).
|
||||
*/
|
||||
test('the extractor finds the invocations that are actually in the templates', async () => {
|
||||
assert.ok(invocations.length > 0, 'No CLI invocation was extracted from any command template.');
|
||||
assert.ok(clis.length > 0, 'Invocations were found but named no CLI.');
|
||||
|
||||
const missed = [];
|
||||
for (const { file, content } of await commandTemplates()) {
|
||||
const namesACli = /node\s+\$\{CLAUDE_PLUGIN_ROOT\}\/scanners\//.test(content);
|
||||
if (namesACli && extractInvocations(content).length === 0) missed.push(file);
|
||||
}
|
||||
|
||||
assert.deepEqual(
|
||||
missed,
|
||||
[],
|
||||
'A template writes an anchored `node ${CLAUDE_PLUGIN_ROOT}/scanners/…` call that the\n' +
|
||||
'extractor did not see. Every assertion in this file is silent about whatever it cannot\n' +
|
||||
'parse, so a shrinking sweep reads as a passing one.',
|
||||
);
|
||||
});
|
||||
|
||||
test('every scanner a template names exists on disk', async () => {
|
||||
const dead = [];
|
||||
for (const cli of clis) {
|
||||
try {
|
||||
await access(resolve(SCANNERS_DIR, cli));
|
||||
} catch {
|
||||
const sites = invocations.filter((i) => i.cli === cli).map((i) => `${i.file}:${i.line}`);
|
||||
dead.push(`${cli} <- ${sites.join(', ')}`);
|
||||
}
|
||||
}
|
||||
|
||||
assert.deepEqual(
|
||||
dead,
|
||||
[],
|
||||
'A command template tells the agent to run a scanner that is not there. The user gets a\n' +
|
||||
"node stack trace the ux-rules promise they never see, because the command's own\n" +
|
||||
'`2>/dev/null` hides the one line that would explain it.',
|
||||
);
|
||||
});
|
||||
|
||||
for (const cli of clis) {
|
||||
const flags = surface.get(cli);
|
||||
|
||||
test(`${cli} accepts every flag the command templates hand it`, async () => {
|
||||
// Property 2 — the probe proves itself against this CLI before its silence
|
||||
// is allowed to mean anything.
|
||||
const control = await run(cli, [IMPOSSIBLE_FLAG]);
|
||||
assert.ok(
|
||||
reportedUnknown(control.stderr, IMPOSSIBLE_FLAG),
|
||||
`${cli} did not report ${IMPOSSIBLE_FLAG} as unknown (exit ${control.code}), so this test\n` +
|
||||
'cannot tell an accepted flag from an unreachable parser — every flag below would pass\n' +
|
||||
'for the wrong reason. Give this CLI the argv it needs to reach its flag parsing, or\n' +
|
||||
'fix the CLI. stderr was: ' + JSON.stringify(control.stderr),
|
||||
);
|
||||
|
||||
const dead = [];
|
||||
const spilled = [];
|
||||
for (const [flag, sites] of [...flags].sort()) {
|
||||
const { stderr, wrote } = await run(cli, [flag]);
|
||||
if (reportedUnknown(stderr, flag)) dead.push(`${flag} <- ${sites.join(', ')}`);
|
||||
if (wrote.length) spilled.push(`${flag} left ${JSON.stringify(wrote)}`);
|
||||
}
|
||||
|
||||
assert.deepEqual(
|
||||
spilled,
|
||||
[],
|
||||
`${cli} wrote into the working directory while being probed with a single flag and no\n` +
|
||||
'`--output-file`. Either the CLI writes somewhere it should not, or this probe is no\n' +
|
||||
'longer contained — and an uncontained probe of a writer is how this test overwrote a\n' +
|
||||
'real baseline the first time it ran.',
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
dead,
|
||||
[],
|
||||
`${cli} rejects a flag a command template still passes it. The template keeps working\n` +
|
||||
'right up to the moment a user turns that knob on, and then the CLI exits 3 behind the\n' +
|
||||
"command's `2>/dev/null` — or, before the unknown-flag guard existed, answered a\n" +
|
||||
'question nobody asked (M-BUG-45). Flags:\n ' + dead.join('\n '),
|
||||
);
|
||||
});
|
||||
}
|
||||
91
tests/commands/command-flag-value-portability.test.mjs
Normal file
91
tests/commands/command-flag-value-portability.test.mjs
Normal file
|
|
@ -0,0 +1,91 @@
|
|||
/**
|
||||
* Session #51 — command-template flag-value portability.
|
||||
*
|
||||
* Dogfooding `knowledge-refresh` surfaced a defect that only exists at the
|
||||
* seam between the command template and the shell that runs it:
|
||||
*
|
||||
* STALE_AFTER="--stale-after 30"
|
||||
* node …-cli.mjs --reference-date "$TODAY" $STALE_AFTER --output-file …
|
||||
*
|
||||
* The unquoted `$STALE_AFTER` is meant to split into TWO argv entries. Under
|
||||
* **bash** it does. Under **zsh** — the macOS default since Catalina, and the
|
||||
* shell the Bash tool actually runs on this machine — unquoted parameter
|
||||
* expansions are NOT word-split, so the CLI receives ONE argv entry with the
|
||||
* literal text `--stale-after 30`, matches no known flag, and (because the CLI
|
||||
* silently ignored unknown flags — see cli-unknown-flag-rejection.test.mjs)
|
||||
* falls back to the 90-day default while reporting success. Measured:
|
||||
*
|
||||
* $ STALE_AFTER="--stale-after 30"; set -- $STALE_AFTER; echo $#
|
||||
* 1 # zsh (bash prints 2)
|
||||
* → payload staleAfterDays: 90, exit 0, "✓ All 14 entries fresh"
|
||||
*
|
||||
* The user-facing knob was silently dead. Note the asymmetry that makes this
|
||||
* survivable elsewhere: an EMPTY unquoted expansion yields ZERO argv entries in
|
||||
* both shells, so the `FLAG=""` idiom used by ~25 other sites is portable. Only
|
||||
* a variable that can hold a flag AND its value is affected.
|
||||
*
|
||||
* The invariant asserted here is therefore about VALUE-carrying flags, not
|
||||
* about quoting in general: a command template must never depend on the shell
|
||||
* splitting one variable into a flag plus its argument. Pass the value through
|
||||
* its own quoted variable instead.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFile, readdir } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const COMMANDS_DIR = resolve(__dirname, '..', '..', 'commands');
|
||||
|
||||
async function commandFiles() {
|
||||
const entries = await readdir(COMMANDS_DIR);
|
||||
return entries.filter((e) => e.endsWith('.md')).sort();
|
||||
}
|
||||
|
||||
/**
|
||||
* Assignments whose right-hand side contains a flag followed by a value —
|
||||
* i.e. the value only reaches argv if the shell word-splits. Matches both
|
||||
* `X="--flag value"` and `X="--flag $(cmd)"`.
|
||||
*/
|
||||
const MULTIWORD_FLAG_ASSIGN = /^\s*([A-Z_][A-Z0-9_]*)=(["'])(--[a-z0-9-]+)[ \t]+\S.*\2\s*$/;
|
||||
|
||||
test('no command template builds a flag AND its value into one shell variable', async () => {
|
||||
const offenders = [];
|
||||
|
||||
for (const file of await commandFiles()) {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, file), 'utf-8');
|
||||
content.split('\n').forEach((line, i) => {
|
||||
const m = line.match(MULTIWORD_FLAG_ASSIGN);
|
||||
if (m) offenders.push(`${file}:${i + 1} ${m[1]}=${m[2]}${m[3]} …${m[2]}`);
|
||||
});
|
||||
}
|
||||
|
||||
assert.deepEqual(
|
||||
offenders,
|
||||
[],
|
||||
'A variable holding "--flag value" only reaches argv correctly if the shell\n' +
|
||||
'word-splits an unquoted expansion. zsh does not. Pass the value in its own\n' +
|
||||
'quoted variable instead:\n' +
|
||||
' N=$(… extract …); [ -n "$N" ] && node cli.mjs --flag "$N"\n' +
|
||||
'Offending assignments:\n ' + offenders.join('\n '),
|
||||
);
|
||||
});
|
||||
|
||||
test('knowledge-refresh.md passes --stale-after with a quoted value', async () => {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, 'knowledge-refresh.md'), 'utf-8');
|
||||
|
||||
assert.ok(
|
||||
!/\$STALE_AFTER\b(?!")/.test(content.replace(/"\$STALE_AFTER"/g, '')),
|
||||
'knowledge-refresh.md still expands a flag-carrying variable unquoted; under zsh the\n' +
|
||||
'threshold silently reverts to the 90-day default while the command reports success.',
|
||||
);
|
||||
|
||||
assert.match(
|
||||
content,
|
||||
/--stale-after "\$[A-Z_]+"/,
|
||||
'knowledge-refresh.md must pass the extracted threshold as its own quoted argument\n' +
|
||||
'(`--stale-after "$STALE_AFTER_DAYS"`), so no word-splitting is required.',
|
||||
);
|
||||
});
|
||||
222
tests/commands/command-output-discipline.test.mjs
Normal file
222
tests/commands/command-output-discipline.test.mjs
Normal file
|
|
@ -0,0 +1,222 @@
|
|||
/**
|
||||
* Session #50 — command-template output-discipline tests.
|
||||
*
|
||||
* Dogfooding the four read commands (`posture`, `tokens`, `manifest`,
|
||||
* `whats-active`) surfaced three defect classes that all live in the same
|
||||
* seam: what a command template PROMISES versus what the scanner behind it
|
||||
* actually does.
|
||||
*
|
||||
* 1. M-BUG-43 — a scanner invoked with `--raw`/`--json` writes the payload
|
||||
* to stdout *even when `--output-file` is set*
|
||||
* (`token-hotspots-cli.mjs:137`, `manifest.mjs:293`,
|
||||
* `whats-active.mjs:60`, `posture.mjs:101`, `drift-cli.mjs`,
|
||||
* `plugin-health-scanner.mjs`). The command templates redirect only
|
||||
* stderr, so the payload lands in the transcript. Measured on this repo:
|
||||
* posture 255 182 B, whats-active 35 922 B, drift 28 316 B,
|
||||
* manifest 23 825 B, tokens 8 768 B. `.claude/rules/ux-rules.md` rule 1
|
||||
* says NEVER show raw JSON — and a plugin that exists to cut token cost
|
||||
* must not dump a quarter-megabyte to report one grade.
|
||||
*
|
||||
* 2. Flags documented in a command's prose that never reach any shell
|
||||
* (`tokens.md`: `--json`, `--with-telemetry-recipe`). The scanner
|
||||
* supports them; the template silently swallows them. Same "green is
|
||||
* worse than an error" shape as the arg-sink class.
|
||||
*
|
||||
* 3. Render contracts that name a field the scanner never emits
|
||||
* (`manifest.md` asked for `{load}`; the payload carries `loadPattern`),
|
||||
* so the column the command's own prose calls the whole point renders
|
||||
* blank for every row.
|
||||
*
|
||||
* Test 3 runs the real scanners against a fixture rather than asserting
|
||||
* against a hardcoded key list — a hardcoded list drifts, a live payload
|
||||
* cannot.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFile, readdir, mkdtemp, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { execFile } from 'node:child_process';
|
||||
import { promisify } from 'node:util';
|
||||
import { resolve, dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = resolve(__dirname, '..', '..');
|
||||
const COMMANDS_DIR = join(ROOT, 'commands');
|
||||
const FIXTURE = join(ROOT, 'tests', 'fixtures', 'marketplace-medium');
|
||||
|
||||
async function commandFiles() {
|
||||
const entries = await readdir(COMMANDS_DIR);
|
||||
return entries.filter((e) => e.endsWith('.md')).sort();
|
||||
}
|
||||
|
||||
/** Strip `#` comments so the tests never match their own explanatory prose. */
|
||||
function stripComment(line) {
|
||||
const h = line.indexOf('#');
|
||||
return h === -1 ? line : line.slice(0, h);
|
||||
}
|
||||
|
||||
/** Yield [lineNumber, line] for lines inside ```bash fences only. */
|
||||
function bashLines(content) {
|
||||
const out = [];
|
||||
let open = null;
|
||||
content.split('\n').forEach((line, i) => {
|
||||
const m = line.match(/^\s*```(\w*)/);
|
||||
if (m) {
|
||||
open = open === null ? m[1] : null;
|
||||
return;
|
||||
}
|
||||
if (open === 'bash') out.push([i + 1, line]);
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Yield lines inside ```markdown fences — the render contracts. */
|
||||
function markdownFenceLines(content) {
|
||||
const out = [];
|
||||
let open = null;
|
||||
content.split('\n').forEach((line) => {
|
||||
const m = line.match(/^\s*```(\w*)/);
|
||||
if (m) {
|
||||
open = open === null ? m[1] : null;
|
||||
return;
|
||||
}
|
||||
if (open === 'markdown') out.push(line);
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
test('Output discipline: a scanner asked for --output-file never also prints to stdout', async () => {
|
||||
// The rule is blanket and needs no per-scanner allowlist: if the command
|
||||
// asked for the payload in a FILE, the same invocation must not let it reach
|
||||
// the transcript. Invocations with no --output-file are out of scope — there
|
||||
// the payload has nowhere else to go (e.g. `drift --save`, 112 B).
|
||||
const violations = [];
|
||||
for (const name of await commandFiles()) {
|
||||
const content = await readFile(join(COMMANDS_DIR, name), 'utf-8');
|
||||
for (const [lineNo, raw] of bashLines(content)) {
|
||||
const line = stripComment(raw);
|
||||
if (!/node \$\{CLAUDE_PLUGIN_ROOT\}\/scanners\//.test(line)) continue;
|
||||
if (!line.includes('--output-file')) continue;
|
||||
const rawish = /--json\b|--raw\b|\$RAW_FLAG/.test(line);
|
||||
if (!rawish) continue;
|
||||
// Mask `2>` so a stderr redirect is never mistaken for a stdout one.
|
||||
const masked = line.replace(/2>/g, '2@');
|
||||
if (/(^|\s)>\s*\S+/.test(masked)) continue;
|
||||
violations.push(
|
||||
`${name}:${lineNo} runs a scanner in raw/json mode with --output-file but never redirects stdout — the payload lands in the transcript`,
|
||||
);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(violations, [], `Unredirected scanner payloads:\n${violations.join('\n')}`);
|
||||
});
|
||||
|
||||
test('Flag threading: every flag documented in prose reaches a shell', async () => {
|
||||
// A flag the user is told to pass must arrive somewhere. Two legitimate
|
||||
// destinations exist: a bash fence (threaded to the scanner) or a documented
|
||||
// control-flow branch handled by the model. The allowlist below carries only
|
||||
// the second kind, each verified by reading the code:
|
||||
// posture --drift / --plugin-health : step 5 runs *different* scanners
|
||||
// fix --dry-run : dry-run is fix-cli's DEFAULT and the
|
||||
// flag is an explicit no-op alias
|
||||
// (`scanners/fix-cli.mjs:18-21`)
|
||||
// knowledge-refresh --no-candidates : skips a web-poll step; never a CLI flag
|
||||
// manifest/whats-active --json : served by `cat` of the payload, whose
|
||||
// content is byte-identical to --raw
|
||||
// (both scanners document --raw as a
|
||||
// no-op; verified by diffing payloads)
|
||||
const PROSE_HANDLED = new Set([
|
||||
'posture.md:--drift',
|
||||
'posture.md:--plugin-health',
|
||||
'fix.md:--dry-run',
|
||||
'knowledge-refresh.md:--no-candidates',
|
||||
'manifest.md:--json',
|
||||
'whats-active.md:--json',
|
||||
]);
|
||||
|
||||
const violations = [];
|
||||
for (const name of await commandFiles()) {
|
||||
const content = await readFile(join(COMMANDS_DIR, name), 'utf-8');
|
||||
const bash = bashLines(content).map(([, l]) => l).join('\n');
|
||||
const documented = new Set();
|
||||
for (const line of content.split('\n')) {
|
||||
const m = line.match(/^\s*[-*]\s+`(--[a-z][a-z0-9-]*)`/);
|
||||
if (m) documented.add(m[1]);
|
||||
}
|
||||
for (const flag of documented) {
|
||||
if (bash.includes(flag)) continue;
|
||||
if (PROSE_HANDLED.has(`${name}:${flag}`)) continue;
|
||||
violations.push(`${name} documents \`${flag}\` but no bash fence ever passes it`);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(violations, [], `Flags dropped between docs and shell:\n${violations.join('\n')}`);
|
||||
});
|
||||
|
||||
test('Render contract: every {field} in a render fence exists in the real payload', async () => {
|
||||
// Derived values the model computes rather than reads. Kept deliberately
|
||||
// short — every entry here is a field the render fence does NOT get from the
|
||||
// scanner, so a long list would hollow out the test.
|
||||
const DERIVED = new Set(['rank']);
|
||||
const CASES = [
|
||||
{ command: 'manifest.md', scanner: 'manifest.mjs', args: [] },
|
||||
{ command: 'tokens.md', scanner: 'token-hotspots-cli.mjs', args: [] },
|
||||
{
|
||||
command: 'whats-active.md',
|
||||
scanner: 'whats-active.mjs',
|
||||
args: ['--verbose', '--suggest-disables'],
|
||||
},
|
||||
];
|
||||
|
||||
const dir = await mkdtemp(join(tmpdir(), 'ca-render-'));
|
||||
try {
|
||||
const violations = [];
|
||||
for (const { command, scanner, args } of CASES) {
|
||||
const outFile = join(dir, `${scanner}.json`);
|
||||
await execFileAsync('node', [
|
||||
join(ROOT, 'scanners', scanner), FIXTURE, '--output-file', outFile, ...args,
|
||||
]);
|
||||
const payload = JSON.parse(await readFile(outFile, 'utf-8'));
|
||||
|
||||
const keys = new Set();
|
||||
const walk = (node) => {
|
||||
if (Array.isArray(node)) node.slice(0, 5).forEach(walk);
|
||||
else if (node && typeof node === 'object') {
|
||||
for (const k of Object.keys(node)) {
|
||||
keys.add(k);
|
||||
walk(node[k]);
|
||||
}
|
||||
}
|
||||
};
|
||||
walk(payload);
|
||||
|
||||
const content = await readFile(join(COMMANDS_DIR, command), 'utf-8');
|
||||
const refs = new Set();
|
||||
for (const line of markdownFenceLines(content)) {
|
||||
const re = /\{([a-zA-Z][a-zA-Z0-9_.]*)\}/g;
|
||||
let m;
|
||||
while ((m = re.exec(line)) !== null) refs.add(m[1]);
|
||||
}
|
||||
for (const ref of refs) {
|
||||
if (DERIVED.has(ref)) continue;
|
||||
if (ref.endsWith('.length')) {
|
||||
// `{foo.length}` is a count of a collection — assert the collection
|
||||
// itself exists, since that is the part the scanner owns.
|
||||
const base = ref.slice(0, -'.length'.length).split('.').pop();
|
||||
if (!keys.has(base)) {
|
||||
violations.push(`${command}: {${ref}} — no \`${base}\` collection in the payload`);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
const leaf = ref.split('.').pop();
|
||||
if (!keys.has(leaf) && !keys.has(ref)) {
|
||||
violations.push(`${command}: {${ref}} is never emitted by ${scanner}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
assert.deepEqual(violations, [], `Render fields the scanner never emits:\n${violations.join('\n')}`);
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
102
tests/commands/command-placeholder-shell-safety.test.mjs
Normal file
102
tests/commands/command-placeholder-shell-safety.test.mjs
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
/**
|
||||
* Session #56 — placeholders inside runnable bash fences must not be shell-active.
|
||||
*
|
||||
* Dogfooding the ROUTER (`commands/config-audit.md`) surfaced a defect one layer
|
||||
* below the CLIs: the fence on the router's step 3 carries the placeholder
|
||||
* `<target-path>` **bare** — unquoted, preceded by a space. `<` and `>` are
|
||||
* redirection operators. Measured under the shell the Bash tool actually runs
|
||||
* (zsh), in a scratch directory:
|
||||
*
|
||||
* $ node …/scan-orchestrator.mjs <target-path> --output-file scan.json \
|
||||
* >/dev/null 2>/dev/null; node …/posture.mjs <target-path> \
|
||||
* --output-file posture.json 2>/dev/null; echo $?
|
||||
* zsh:1: no such file or directory: target-path
|
||||
* zsh:1: no such file or directory: target-path
|
||||
* 1
|
||||
* → neither file written, neither CLI ever started
|
||||
*
|
||||
* Three properties make this worse than a plain typo:
|
||||
*
|
||||
* 1. **The CLI never runs.** Redirection is resolved by the shell before the
|
||||
* command is executed, so the CLI's own argument validation — the layer that
|
||||
* `cli-unknown-flag-rejection.test.mjs` hardened — never sees it.
|
||||
* 2. **The failure is quiet where it counts.** The echoed status is `1`, and
|
||||
* `1` is inside the band the router's own step 3 classifies as
|
||||
* "continue normally" (0/1/2 = PASS/WARNING/FAIL; only 3 is a real error).
|
||||
* A total non-execution is indistinguishable from a healthy WARNING run.
|
||||
* 3. **The file already knew about the neighbouring hazard.** Two lines above
|
||||
* the offending call sits a comment warning that a *square-bracket*
|
||||
* placeholder "does not start with a dash, so both CLIs' arg loops would
|
||||
* take it as the TARGET PATH instead of a flag" — awareness of the
|
||||
* placeholder class, while carrying a strictly worse member of it.
|
||||
*
|
||||
* The invariant is not "substitute your placeholders" (a template cannot enforce
|
||||
* that). It is that an UNSUBSTITUTED placeholder must fail **loudly, in the
|
||||
* CLI**, not silently in the shell. Quoting achieves exactly that: `"<path>"`
|
||||
* reaches argv as a literal, the CLI reports an unreadable target, and the exit
|
||||
* code means what the router thinks it means.
|
||||
*
|
||||
* Measured breadth at the time of writing: 13 of 21 command files, 33 sites.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFile, readdir } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const COMMANDS_DIR = resolve(__dirname, '..', '..', 'commands');
|
||||
|
||||
async function commandFiles() {
|
||||
const entries = await readdir(COMMANDS_DIR);
|
||||
return entries.filter((e) => e.endsWith('.md')).sort();
|
||||
}
|
||||
|
||||
/**
|
||||
* Yield the lines that live inside ```bash fences, with their 1-based file line
|
||||
* numbers. Only bash-tagged fences count: an untagged or json fence is prose.
|
||||
*/
|
||||
function bashFenceLines(content) {
|
||||
const out = [];
|
||||
let inFence = false;
|
||||
content.split('\n').forEach((line, i) => {
|
||||
if (/^\s*```/.test(line)) {
|
||||
inFence = /^\s*```bash\s*$/.test(line);
|
||||
return;
|
||||
}
|
||||
if (inFence) out.push({ line, n: i + 1 });
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* A placeholder that the shell would read as a redirection: `<` at the start of
|
||||
* a word (start of line, or after whitespace / `;` / `|` / `&`), a lowercase
|
||||
* placeholder name, then `>`. A quoted placeholder (`"<path>"`) is excluded by
|
||||
* construction — the `<` is preceded by a quote, not a word boundary.
|
||||
*/
|
||||
const BARE_PLACEHOLDER = /(?:^|[\s;|&(])(<[a-z][a-z0-9._-]*>)/;
|
||||
|
||||
test('no runnable bash fence carries a bare (shell-active) angle-bracket placeholder', async () => {
|
||||
const offenders = [];
|
||||
|
||||
for (const file of await commandFiles()) {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, file), 'utf-8');
|
||||
for (const { line, n } of bashFenceLines(content)) {
|
||||
const m = line.match(BARE_PLACEHOLDER);
|
||||
if (m) offenders.push(`${file}:${n} ${m[1]} in: ${line.trim().slice(0, 90)}`);
|
||||
}
|
||||
}
|
||||
|
||||
assert.deepEqual(
|
||||
offenders,
|
||||
[],
|
||||
'A bare `<name>` inside a bash fence is a REDIRECTION, not an argument. Left\n' +
|
||||
'unsubstituted it fails in the shell before the CLI starts — no output file,\n' +
|
||||
'no CLI diagnostics, and an exit code (1) that the router reads as a normal\n' +
|
||||
'WARNING run. Quote the placeholder (`"<name>"`) so an unsubstituted template\n' +
|
||||
'fails loudly in the CLI instead.\n' +
|
||||
'Offending sites:\n ' + offenders.join('\n '),
|
||||
);
|
||||
});
|
||||
258
tests/commands/command-shell-state-shape.test.mjs
Normal file
258
tests/commands/command-shell-state-shape.test.mjs
Normal file
|
|
@ -0,0 +1,258 @@
|
|||
/**
|
||||
* Session #49 — command-template shell-state shape tests.
|
||||
*
|
||||
* Dogfooding the `plan` + `implement` chunk surfaced one root defect with
|
||||
* several arms: **command templates assume shell state survives between
|
||||
* fenced blocks.** It does not. Every ```bash fence is executed as its own
|
||||
* Bash tool call, in its own process:
|
||||
*
|
||||
* - A variable assigned in block N is empty in block N+1.
|
||||
* - `$$` (the PID) differs between calls, so a `/tmp/foo-$$.json` path
|
||||
* created in one block can never be reconstructed in a later one.
|
||||
* - The Read tool expands neither shell variables nor `$$` nor globs; it
|
||||
* takes one literal path.
|
||||
*
|
||||
* Measured arms at the time of writing (all fixed by the accompanying commit):
|
||||
* - `$RAW_FLAG` referenced inside a (non-bash) agent-prompt fence in
|
||||
* analyze.md, plan.md, implement.md — the agent received the literal
|
||||
* string `$RAW_FLAG`, confirmed at runtime by the planner-agent itself.
|
||||
* - `$TMPFILE` referenced across blocks in manifest.md, tokens.md,
|
||||
* whats-active.md, plugin-health.md.
|
||||
* - `$GLOBAL_FLAG` across blocks in fix.md.
|
||||
* - `$TODAY` across blocks in campaign.md (6 sites).
|
||||
* - `$$` temp paths referenced outside their creating fence in fix.md.
|
||||
* - plan.md asked the Read tool to expand
|
||||
* `~/.claude/config-audit/sessions/*_/state.yaml`.
|
||||
*
|
||||
* The hardened pattern already present in drift.md is the target shape: a
|
||||
* fixed literal temp path, repeated literally in every block that needs it.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFile, readdir } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const COMMANDS_DIR = resolve(__dirname, '..', '..', 'commands');
|
||||
|
||||
/** Shell variables supplied by the environment, not by a prior block. */
|
||||
const AMBIENT = new Set([
|
||||
'CLAUDE_PLUGIN_ROOT', 'ARGUMENTS', 'HOME', 'PATH', 'PWD', 'USER', 'TMPDIR',
|
||||
]);
|
||||
|
||||
async function commandFiles() {
|
||||
const entries = await readdir(COMMANDS_DIR);
|
||||
return entries.filter((e) => e.endsWith('.md')).sort();
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse fenced blocks. Returns { lines, blockIndexOf(lineIdx) } where
|
||||
* blockIndexOf returns -1 for prose outside any fence.
|
||||
*/
|
||||
function parseFences(content) {
|
||||
const lines = content.split('\n');
|
||||
const blocks = [];
|
||||
let open = null;
|
||||
lines.forEach((line, i) => {
|
||||
const m = line.match(/^\s*```(\w*)/);
|
||||
if (!m) return;
|
||||
if (open === null) open = { lang: m[1], start: i };
|
||||
else {
|
||||
blocks.push({ lang: open.lang, start: open.start, end: i });
|
||||
open = null;
|
||||
}
|
||||
});
|
||||
const blockIndexOf = (i) => blocks.findIndex((b) => i > b.start && i < b.end);
|
||||
return { lines, blocks, blockIndexOf };
|
||||
}
|
||||
|
||||
/** Strip `#` comments from a bash line so the test never matches its own prose. */
|
||||
function stripComment(line) {
|
||||
const h = line.indexOf('#');
|
||||
return h === -1 ? line : line.slice(0, h);
|
||||
}
|
||||
|
||||
test('Shell state: no variable is referenced outside the block that assigned it', async () => {
|
||||
const violations = [];
|
||||
for (const name of await commandFiles()) {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, name), 'utf-8');
|
||||
const { lines, blocks, blockIndexOf } = parseFences(content);
|
||||
|
||||
const assignedIn = new Map();
|
||||
lines.forEach((raw, i) => {
|
||||
const bi = blockIndexOf(i);
|
||||
if (bi < 0) return;
|
||||
// An assignment starts a command, and a command starts at the beginning of
|
||||
// the line OR after a separator. Capturing only the line-initial form would
|
||||
// miss the idiomatic status capture `node …; STATUS=$?` — the one form that
|
||||
// MUST trail a command — and report the variable as never assigned.
|
||||
const re = /(?:^|[;&|]|&&|\|\|)\s*([A-Z_][A-Z0-9_]*)=/g;
|
||||
let m;
|
||||
while ((m = re.exec(stripComment(raw))) !== null) {
|
||||
if (!assignedIn.has(m[1])) assignedIn.set(m[1], new Set());
|
||||
assignedIn.get(m[1]).add(bi);
|
||||
}
|
||||
});
|
||||
|
||||
lines.forEach((raw, i) => {
|
||||
const line = stripComment(raw);
|
||||
const bi = blockIndexOf(i);
|
||||
const re = /\$\{?([A-Z_][A-Z0-9_]*)\}?/g;
|
||||
let m;
|
||||
while ((m = re.exec(line)) !== null) {
|
||||
const v = m[1];
|
||||
if (AMBIENT.has(v)) continue;
|
||||
// Skip the assignment site itself (`FOO=$FOO...` right-hand side is fine).
|
||||
const eq = line.indexOf('=');
|
||||
if (/^\s*[A-Z_][A-Z0-9_]*=/.test(line) && line.indexOf(m[0]) < eq) continue;
|
||||
const where = assignedIn.get(v);
|
||||
if (!where) {
|
||||
violations.push(`${name}:${i + 1} $${v} is never assigned in any block`);
|
||||
} else if (bi < 0) {
|
||||
violations.push(
|
||||
`${name}:${i + 1} $${v} referenced in prose/agent-prompt — no shell expands it there`,
|
||||
);
|
||||
} else if (!where.has(bi)) {
|
||||
const lang = blocks[bi].lang || 'none';
|
||||
violations.push(
|
||||
`${name}:${i + 1} $${v} referenced in block ${bi} (lang=${lang}) but assigned only in block(s) ${[...where].join(', ')} — separate Bash calls, separate processes`,
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
assert.deepEqual(violations, [], `Cross-block shell-variable references:\n${violations.join('\n')}`);
|
||||
});
|
||||
|
||||
test('Shell state: no $$ temp path is referenced outside the block that created it', async () => {
|
||||
const violations = [];
|
||||
for (const name of await commandFiles()) {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, name), 'utf-8');
|
||||
const { lines, blockIndexOf } = parseFences(content);
|
||||
|
||||
const firstSeen = new Map();
|
||||
const scan = (raw, i, cb) => {
|
||||
const re = /(\/tmp\/[A-Za-z0-9._-]*\$\$[A-Za-z0-9._-]*)/g;
|
||||
let m;
|
||||
while ((m = re.exec(stripComment(raw))) !== null) cb(m[1], i);
|
||||
};
|
||||
lines.forEach((raw, i) => scan(raw, i, (p) => {
|
||||
if (!firstSeen.has(p)) firstSeen.set(p, { blk: blockIndexOf(i), line: i + 1 });
|
||||
}));
|
||||
lines.forEach((raw, i) => scan(raw, i, (p) => {
|
||||
const origin = firstSeen.get(p);
|
||||
if (origin.line === i + 1) return;
|
||||
const bi = blockIndexOf(i);
|
||||
if (bi < 0) {
|
||||
violations.push(`${name}:${i + 1} ${p} referenced in prose — the Read tool cannot expand $$`);
|
||||
} else if (bi !== origin.blk) {
|
||||
violations.push(
|
||||
`${name}:${i + 1} ${p} referenced in block ${bi} but created in block ${origin.blk} — $$ is a different PID there`,
|
||||
);
|
||||
}
|
||||
}));
|
||||
}
|
||||
assert.deepEqual(violations, [], `Unresolvable $$ temp paths:\n${violations.join('\n')}`);
|
||||
});
|
||||
|
||||
test('Shell state: no $$ appears in any temp path at all', async () => {
|
||||
// Session #50 closed a blind spot in the test above: it only flags a `$$`
|
||||
// path that is *referenced twice*, because it compares each occurrence to
|
||||
// the block that created it. A path written once and then read via prose
|
||||
// ("Read the JSON output file using the Read tool") has no second
|
||||
// occurrence — so posture.md sat green through #49 while being unreadable
|
||||
// by construction: the PID is never printed, so no later step can name the
|
||||
// file. Measured live: written by PID 21614, read attempted from PID 23772.
|
||||
//
|
||||
// The invariant is therefore blanket, not relational: a command template
|
||||
// must not put `$$` in a temp path at all. The hardened pattern from
|
||||
// drift.md — one fixed literal path, repeated literally — is the only
|
||||
// shape that survives the fence boundary.
|
||||
const violations = [];
|
||||
for (const name of await commandFiles()) {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, name), 'utf-8');
|
||||
content.split('\n').forEach((raw, i) => {
|
||||
const re = /(\/tmp\/[A-Za-z0-9._-]*\$\$[A-Za-z0-9._-]*)/g;
|
||||
let m;
|
||||
while ((m = re.exec(stripComment(raw))) !== null) {
|
||||
violations.push(
|
||||
`${name}:${i + 1} ${m[1]} — $$ differs per Bash call; no later step can name this file`,
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
assert.deepEqual(violations, [], `Unresolvable $$ temp paths:\n${violations.join('\n')}`);
|
||||
});
|
||||
|
||||
test('Read tool: never asked to expand a glob', async () => {
|
||||
const violations = [];
|
||||
for (const name of await commandFiles()) {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, name), 'utf-8');
|
||||
content.split('\n').forEach((raw, i) => {
|
||||
// Only flag when the glob is the *object* of a Read instruction, i.e.
|
||||
// "Read tool on `<glob>`" / "Read `<glob>`". Prose that merely explains
|
||||
// that Read cannot expand a glob is the fix, not the defect — matching
|
||||
// any co-occurrence on the line would flag this repo's own warning text.
|
||||
const m = raw.match(/\bRead(?:\s+the)?(?:\s+tool)?\s+(?:tool\s+)?on\s+`([~/][^`]*)`|\bRead\s+`([~/][^`]*)`/);
|
||||
if (!m) return;
|
||||
const path = m[1] ?? m[2];
|
||||
if (!path.includes('*')) return;
|
||||
violations.push(`${name}:${i + 1} Read tool pointed at a glob \`${path}\` — use Glob`);
|
||||
});
|
||||
}
|
||||
assert.deepEqual(violations, [], `Read-tool glob misuse:\n${violations.join('\n')}`);
|
||||
});
|
||||
|
||||
test('state.yaml: phase commands name all four fields the rule requires', async () => {
|
||||
// .claude/rules/state-management.md mandates current_phase, completed_phases,
|
||||
// next_phase, updated_at after EVERY phase. A command that writes the file
|
||||
// while naming only two fields silently drops the other two.
|
||||
const REQUIRED = ['current_phase', 'completed_phases', 'next_phase', 'updated_at'];
|
||||
const violations = [];
|
||||
for (const name of await commandFiles()) {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, name), 'utf-8');
|
||||
if (!/Update\s+`?state\.yaml`?|Update state/i.test(content)) continue;
|
||||
const missing = REQUIRED.filter((f) => !content.includes(f));
|
||||
if (missing.length) {
|
||||
violations.push(`${name} updates state.yaml but never names: ${missing.join(', ')}`);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(violations, [], `Incomplete state.yaml contracts:\n${violations.join('\n')}`);
|
||||
});
|
||||
|
||||
/**
|
||||
* Session #51 — the instruction that CAUSED the $TODAY defect must not outlive its fix.
|
||||
*
|
||||
* #49 fixed campaign.md by re-deriving `TODAY=$(date +%F)` inside each of the six write
|
||||
* blocks. But step 1 still carried the original prose — "Set a shared date stamp for any
|
||||
* write: `TODAY=$(date +%F)`" — which is not in a fence, cannot set anything, and directly
|
||||
* contradicts the six comments added below it. A template that argues with itself is not a
|
||||
* contract, and the next edit is the one that believes the wrong half.
|
||||
*
|
||||
* The guard above asserts fences; this one asserts that no PROSE line instructs the reader
|
||||
* to establish shell state for later blocks.
|
||||
*/
|
||||
test('no command template instructs shell state to be set outside a fence', async () => {
|
||||
const offenders = [];
|
||||
|
||||
for (const file of await commandFiles()) {
|
||||
const content = await readFile(resolve(COMMANDS_DIR, file), 'utf-8');
|
||||
const { lines, blockIndexOf } = parseFences(content);
|
||||
lines.forEach((line, i) => {
|
||||
if (blockIndexOf(i) !== -1) return; // inside a fence: that is where state belongs
|
||||
if (/`[A-Z_][A-Z0-9_]*=\$\(/.test(line) || /^\s*[A-Z_][A-Z0-9_]*=\$\(/.test(line)) {
|
||||
offenders.push(`${file}:${i + 1} ${line.trim()}`);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
assert.deepEqual(
|
||||
offenders,
|
||||
[],
|
||||
'Prose that tells the reader to set a shell variable implies it survives to a later\n' +
|
||||
'block. It does not — every fence is its own process. Delete the instruction; the\n' +
|
||||
'blocks that need the value derive it themselves:\n ' + offenders.join('\n '),
|
||||
);
|
||||
});
|
||||
|
|
@ -68,13 +68,33 @@ async function readCommand(name) {
|
|||
return await readFile(resolve(COMMANDS_DIR, name), 'utf-8');
|
||||
}
|
||||
|
||||
test('Group B: every file contains a Bash invocation block', async () => {
|
||||
// Agent-driven commands invoke no scanner, so they have no bash block to
|
||||
// assert. analyze.md's only bash block used to be a `RAW_FLAG=` assignment
|
||||
// referenced from the agent prompt below it — a prompt is not a shell, so the
|
||||
// agent received the literal string `$RAW_FLAG` (session #49). Removing that
|
||||
// block is the fix; requiring one here would re-assert the defect.
|
||||
const AGENT_DRIVEN = new Set(['analyze.md']);
|
||||
|
||||
test('Group B: every scanner-invoking file contains a Bash invocation block', async () => {
|
||||
for (const name of GROUP_B_FILES) {
|
||||
if (AGENT_DRIVEN.has(name)) continue;
|
||||
const content = await readCommand(name);
|
||||
assert.match(content, BASH_BLOCK_REGEX, `${name} missing bash block`);
|
||||
}
|
||||
});
|
||||
|
||||
test('Group B: agent-driven files spawn an Agent instead of a scanner', async () => {
|
||||
for (const name of AGENT_DRIVEN) {
|
||||
const content = await readCommand(name);
|
||||
assert.match(content, /Agent\(subagent_type:/, `${name} should spawn an Agent`);
|
||||
assert.doesNotMatch(
|
||||
content,
|
||||
/RAW_FLAG=/,
|
||||
`${name} must not assign a shell variable it then references from an agent prompt`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('Group B: every file references the Read tool', async () => {
|
||||
for (const name of GROUP_B_FILES) {
|
||||
const content = await readCommand(name);
|
||||
|
|
@ -132,3 +152,29 @@ test('status.md: preserves current_phase machine field and adds humanized phase
|
|||
`status.md must include at least 3 humanized phase labels; found ${present.length}: ${present.join(', ')}`,
|
||||
);
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Økt #46 — ux-rules rule 2 for the plugin-health scanner.
|
||||
//
|
||||
// plugin-health.md passed the humanized-field assertion above while the data it
|
||||
// names was unreachable: the scanner had no --output-file, and its default-mode
|
||||
// report went to stderr, which the command discards with `2>/dev/null`. A .md
|
||||
// contract test that only greps for prose cannot catch that — these assert the
|
||||
// plumbing that makes the prose true.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('plugin-health.md invokes the scanner with --output-file (ux-rules rule 2)', async () => {
|
||||
const content = await readCommand('plugin-health.md');
|
||||
const call = content.split('\n').find(l => l.includes('plugin-health-scanner.mjs'));
|
||||
assert.ok(call, 'plugin-health.md must invoke plugin-health-scanner.mjs');
|
||||
assert.match(call, /--output-file/, 'scanner call must write to a file, not stdout/stderr');
|
||||
});
|
||||
|
||||
test('posture.md invokes the plugin-health and drift scanners with --output-file', async () => {
|
||||
const content = await readCommand('posture.md');
|
||||
for (const scanner of ['plugin-health-scanner.mjs', 'drift-cli.mjs']) {
|
||||
const call = content.split('\n').find(l => l.includes(`scanners/${scanner}`));
|
||||
assert.ok(call, `posture.md must invoke ${scanner}`);
|
||||
assert.match(call, /--output-file/, `${scanner} call in posture.md discards its output`);
|
||||
}
|
||||
});
|
||||
|
|
|
|||
151
tests/commands/interview-chunk-shape.test.mjs
Normal file
151
tests/commands/interview-chunk-shape.test.mjs
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
/**
|
||||
* DEL B chunk `interview` (session #47) — command-contract shape tests.
|
||||
*
|
||||
* Every assertion here encodes a defect that was MEASURED against the shipped
|
||||
* command templates before the fix, per the fasit in
|
||||
* `docs/interview-fasit.local.md`:
|
||||
*
|
||||
* - interview.md / analyze.md referenced `{session-id}` with no rule for
|
||||
* resolving WHICH session — every other session-aware command has one.
|
||||
* - interview.md could push a finished session backwards to `interview`
|
||||
* with no guard (state-management.md mandates the write, nothing bounded it).
|
||||
* - status.md advertised `/config-audit resume {session-id}`; no resume
|
||||
* command exists. ux-rules: "Never reference commands that don't exist."
|
||||
* - status.md documented `/config-audit status all` but its flag-parse step
|
||||
* knew only `--raw`.
|
||||
* - cleanup.md ran `rm -rf .../sessions/{session-id}/` with no guard against
|
||||
* an empty/malformed id — an empty expansion deletes every session.
|
||||
* - discover.md carried literal `[--full-machine] [--global]` inside an
|
||||
* executable bash block; `[` is not `-`, so scan-orchestrator's arg loop
|
||||
* turns it into the SCAN TARGET (M-BUG-21 class).
|
||||
* - feature-gap.md called `fix-cli.mjs --json` under the heading
|
||||
* "Create backup". fix-cli is dry-run by default: no backup was created,
|
||||
* `backupId` came back null, and the command then edited config believing
|
||||
* it could roll back (M-BUG-31 class).
|
||||
* - fix-cli.mjs told the user to recover with `node scanners/rollback-cli.mjs`
|
||||
* — a file that does not exist.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFileSync, existsSync, readdirSync } from 'node:fs';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = resolve(__dirname, '..', '..');
|
||||
const COMMANDS_DIR = resolve(ROOT, 'commands');
|
||||
|
||||
const read = (name) => readFileSync(resolve(COMMANDS_DIR, name), 'utf-8');
|
||||
|
||||
/** Bash fenced blocks only — prose may legitimately use bracket notation. */
|
||||
function bashBlocks(text) {
|
||||
return [...text.matchAll(/```bash\n([\s\S]*?)```/g)].map((m) => m[1]);
|
||||
}
|
||||
|
||||
test('interview.md and analyze.md resolve WHICH session before using {session-id}', () => {
|
||||
for (const file of ['interview.md', 'analyze.md']) {
|
||||
const text = read(file);
|
||||
assert.match(
|
||||
text,
|
||||
/sessions\/\*\/state\.yaml/,
|
||||
`${file} references {session-id} but never says how to find the session`
|
||||
);
|
||||
assert.match(
|
||||
text,
|
||||
/most recent|most recently modified/i,
|
||||
`${file} must state which session wins when several exist`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('interview.md guards against pushing a finished session backwards', () => {
|
||||
const text = read('interview.md');
|
||||
assert.match(
|
||||
text,
|
||||
/completed_phases/,
|
||||
'interview.md must reason about completed_phases before rewriting current_phase'
|
||||
);
|
||||
assert.match(
|
||||
text,
|
||||
/do not (re-?add|add)|already contains|leave .*completed_phases/i,
|
||||
'interview.md must forbid re-adding a phase already in completed_phases'
|
||||
);
|
||||
});
|
||||
|
||||
test('no command references a /config-audit subcommand that does not exist', () => {
|
||||
// Scope words are arguments to the bare router command, not subcommands.
|
||||
const SCOPE_WORDS = new Set(['current', 'repo', 'home', 'full']);
|
||||
const dead = [];
|
||||
for (const file of readdirSync(COMMANDS_DIR).filter((f) => f.endsWith('.md'))) {
|
||||
const text = readFileSync(resolve(COMMANDS_DIR, file), 'utf-8');
|
||||
for (const m of text.matchAll(/\/config-audit ([a-z][a-z-]+)/g)) {
|
||||
const word = m[1];
|
||||
if (SCOPE_WORDS.has(word)) continue;
|
||||
if (!existsSync(resolve(COMMANDS_DIR, `${word}.md`))) dead.push(`${file}: /config-audit ${word}`);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(dead, [], `dead command references: ${dead.join(', ')}`);
|
||||
});
|
||||
|
||||
test('status.md documents how the `all` argument is parsed', () => {
|
||||
const text = read('status.md');
|
||||
assert.match(
|
||||
text,
|
||||
/ALL_FLAG|contains? `all`|\ball\b[^\n]*\$ARGUMENTS|\$ARGUMENTS[^\n]*\ball\b/i,
|
||||
'status.md advertises `/config-audit status all` but never parses `all`'
|
||||
);
|
||||
});
|
||||
|
||||
test('cleanup.md guards its rm -rf against an empty or malformed session id', () => {
|
||||
const text = read('cleanup.md');
|
||||
// The destructive line lives in prose, not a bash block, so scan the whole
|
||||
// file. An earlier version of this test matched the example session ids in
|
||||
// the display table and went green without any guard existing.
|
||||
assert.ok(/rm -rf/.test(text), 'cleanup.md is expected to document a deletion step');
|
||||
assert.match(
|
||||
text,
|
||||
/\^\[0-9\]\{8\}_\[0-9\]\{6\}\$|\^\\d\{8\}_\\d\{6\}\$/,
|
||||
'cleanup.md must state the exact session-id pattern it validates against before deleting'
|
||||
);
|
||||
assert.match(
|
||||
text,
|
||||
/(refuse|abort|skip)[^\n]*(delet|remov)|never[^\n]*empty[^\n]*(id|path)/i,
|
||||
'cleanup.md must say what happens when the id fails validation'
|
||||
);
|
||||
});
|
||||
|
||||
test('no executable bash block passes a bracketed placeholder flag', () => {
|
||||
// `[--full-machine]` does not start with `-`, so the scanner arg loops take
|
||||
// it as the scan target and silently scan a path that does not exist.
|
||||
const offenders = [];
|
||||
for (const file of readdirSync(COMMANDS_DIR).filter((f) => f.endsWith('.md'))) {
|
||||
for (const block of bashBlocks(readFileSync(resolve(COMMANDS_DIR, file), 'utf-8'))) {
|
||||
if (/\[--[a-z-]+\]/.test(block)) offenders.push(file);
|
||||
}
|
||||
}
|
||||
assert.deepEqual([...new Set(offenders)], [], `bracketed flags inside bash blocks: ${offenders.join(', ')}`);
|
||||
});
|
||||
|
||||
test('feature-gap.md does not present a dry-run fix-cli call as a backup', () => {
|
||||
const text = read('feature-gap.md');
|
||||
for (const block of bashBlocks(text)) {
|
||||
if (!/fix-cli\.mjs/.test(block)) continue;
|
||||
assert.match(
|
||||
block,
|
||||
/--apply/,
|
||||
'feature-gap.md calls fix-cli without --apply; dry-run creates no backup'
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('fix-cli.mjs recovery hint points at a command that exists', () => {
|
||||
const text = readFileSync(resolve(ROOT, 'scanners', 'fix-cli.mjs'), 'utf-8');
|
||||
const hints = [...text.matchAll(/scanners\/([a-z-]+\.mjs)/g)].map((m) => m[1]);
|
||||
for (const hint of hints) {
|
||||
assert.ok(
|
||||
existsSync(resolve(ROOT, 'scanners', hint)),
|
||||
`fix-cli.mjs tells the user to run scanners/${hint}, which does not exist`
|
||||
);
|
||||
}
|
||||
});
|
||||
91
tests/commands/knowledge-refresh-write-target.test.mjs
Normal file
91
tests/commands/knowledge-refresh-write-target.test.mjs
Normal file
|
|
@ -0,0 +1,91 @@
|
|||
/**
|
||||
* Session #51 — knowledge-refresh must write the register it actually read.
|
||||
*
|
||||
* The command reads the register through `knowledge-refresh-cli.mjs`, whose `REGISTER_PATH`
|
||||
* is anchored to the CLI module itself — i.e. `${CLAUDE_PLUGIN_ROOT}/knowledge/
|
||||
* best-practices.json`. For any installed plugin that is the marketplace cache; measured
|
||||
* live during this chunk:
|
||||
*
|
||||
* registerPath: …/.claude/plugins/cache/ktg-plugin-marketplace/config-audit/5.13.0/
|
||||
* knowledge/best-practices.json
|
||||
*
|
||||
* Step 6 then said: Edit `knowledge/best-practices.json` — an UNANCHORED relative path,
|
||||
* which resolves against whatever repo the user happens to be sitting in. Three consequences,
|
||||
* none of them visible at the time:
|
||||
*
|
||||
* 1. For a normal user, that path does not exist in their repo at all, so the approved
|
||||
* change either fails or drops a stray file into their project.
|
||||
* 2. In the plugin's own checkout it resolves to the working tree, so the command reads
|
||||
* one file and writes a different one — the refresh appears to do nothing, because the
|
||||
* CLI keeps reporting the cached copy's dates.
|
||||
* 3. Step 6.3's validation gate runs the plugin's own test file, which loads the register
|
||||
* via the same anchored `REGISTER_PATH`. So the gate validates the copy that was NOT
|
||||
* edited and passes no matter what was written — a guard that cannot see the file it
|
||||
* guards.
|
||||
*
|
||||
* The two register copies were byte-identical on the day this was found, which is exactly
|
||||
* why the defect was invisible to a casual dogfood run.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const FILE = resolve(__dirname, '..', '..', 'commands', 'knowledge-refresh.md');
|
||||
|
||||
/**
|
||||
* Scoped to lines that tell the model to MODIFY the register. Descriptive prose ("this
|
||||
* command keeps knowledge/best-practices.json current") and the closing `git commit`
|
||||
* suggestion name the file without acting on it, and a git path is correctly repo-relative.
|
||||
* The defect was specifically an unanchored path in an imperative write step.
|
||||
*/
|
||||
const WRITE_VERB = /\b(edit|write|append|overwrite|save)\b/i;
|
||||
|
||||
function unanchoredWriteTargets(content) {
|
||||
const out = [];
|
||||
content.split('\n').forEach((line, i) => {
|
||||
if (!WRITE_VERB.test(line)) return;
|
||||
for (const m of line.matchAll(/(\S*)knowledge\/best-practices\.json/g)) {
|
||||
if (!m[1].endsWith('${CLAUDE_PLUGIN_ROOT}/')) out.push(`${i + 1}: ${line.trim()}`);
|
||||
}
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
test('the guard still catches the original defect', () => {
|
||||
const original = '1. Edit `knowledge/best-practices.json` — bump `source.verified`, update the `claim`/';
|
||||
assert.equal(
|
||||
unanchoredWriteTargets(original).length,
|
||||
1,
|
||||
'A narrowed guard must still fail on the text it was written for.',
|
||||
);
|
||||
});
|
||||
|
||||
test('every register path in knowledge-refresh.md is anchored to the plugin root', async () => {
|
||||
const content = await readFile(FILE, 'utf-8');
|
||||
const unanchored = unanchoredWriteTargets(content);
|
||||
|
||||
assert.deepEqual(
|
||||
unanchored,
|
||||
[],
|
||||
'A bare `knowledge/best-practices.json` resolves against the user\'s current repo, not\n' +
|
||||
'against the register the CLI actually read. Anchor every reference to\n' +
|
||||
'${CLAUDE_PLUGIN_ROOT}/ so the file that is read, written, and validated is one file:\n ' +
|
||||
unanchored.join('\n '),
|
||||
);
|
||||
});
|
||||
|
||||
test('the write step says where the register really lives', async () => {
|
||||
const content = await readFile(FILE, 'utf-8');
|
||||
assert.match(
|
||||
content,
|
||||
/marketplace|plugin cache|replaced on upgrade|plugin's own checkout/i,
|
||||
'Step 6 must state that the register is part of the installed plugin, so an approved\n' +
|
||||
'edit to a marketplace-installed copy is discarded by the next plugin upgrade and the\n' +
|
||||
'durable change belongs in the plugin\'s own checkout. Silently editing a cache\n' +
|
||||
'directory is the kind of write that looks successful and evaporates.',
|
||||
);
|
||||
});
|
||||
161
tests/commands/router-shape.test.mjs
Normal file
161
tests/commands/router-shape.test.mjs
Normal file
|
|
@ -0,0 +1,161 @@
|
|||
/**
|
||||
* Session #56 — shape invariants for the ROUTER (`commands/config-audit.md`).
|
||||
*
|
||||
* `/config-audit` with no arguments is the plugin's front door: it auto-detects
|
||||
* scope, runs `scan-orchestrator` + `posture`, and renders the result. Dogfooding
|
||||
* it surfaced four defects that a scanner test could not see, because each lives
|
||||
* at the seam between the command template and something it does not own — the
|
||||
* shell, the scanner registry, the area registry, or the stderr stream.
|
||||
*
|
||||
* 1. **The orchestrator's exit code is discarded.** Step 3 runs both CLIs on one
|
||||
* line and appends a single `echo $?`, which reports only the LAST command.
|
||||
* Measured:
|
||||
*
|
||||
* $ node -e "process.exitCode=3" >/dev/null 2>/dev/null; \
|
||||
* node -e "process.exitCode=0" 2>/dev/null; echo $?
|
||||
* 0
|
||||
*
|
||||
* An orchestrator hard-error (exit 3) is therefore invisible to the very gate
|
||||
* step 3 defines for it ("3 → tell user … and stop").
|
||||
*
|
||||
* 2. **The narrated scanner count is stale.** Step 3 tells the user "Running 12
|
||||
* configuration scanners"; the orchestrator registers and runs 16. Asserted
|
||||
* against the registry rather than a literal, so the next scanner addition
|
||||
* cannot re-stale it silently.
|
||||
*
|
||||
* 3. **The Area Breakdown table drops areas.** Posture emits 10 areas (9 quality
|
||||
* areas plus Feature Coverage, which the template excludes by design). The
|
||||
* table hardcodes 7 rows, so `Token Efficiency` and `Plugin Hygiene` — both
|
||||
* real, both graded, one of them a B on this very repo — never reach the
|
||||
* user. Asserted against the area registry in `lib/scoring.mjs`.
|
||||
*
|
||||
* 4. **The template consumes a stream the same file mandates be thrown away.**
|
||||
* Step 6 instructs: "Use the headline line from the humanized stderr
|
||||
* scorecard … Avoid hardcoding a separate per-grade prose ladder." That
|
||||
* headline (`Health: A (93/100) — Healthy setup, only minor polish needed`)
|
||||
* exists ONLY on posture's stderr — measured absent from the JSON payload —
|
||||
* and step 3's fence sends posture's stderr to /dev/null, as UX rule 2
|
||||
* requires. So the router is told to render something it cannot obtain and
|
||||
* forbidden from deriving a replacement; the slot can only be improvised.
|
||||
* `commands/posture.md` already shows the fix in-repo: redirect stderr to a
|
||||
* FILE (`2>/tmp/…-stderr.txt`), which satisfies "the user never sees it"
|
||||
* while keeping the text readable.
|
||||
*
|
||||
* This is the mirror image of [[stderr-only-warnings-invisible-to-commands]]:
|
||||
* there a WARNING was lost to /dev/null, here a REQUIRED INPUT is.
|
||||
*/
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = resolve(__dirname, '..', '..');
|
||||
const ROUTER = resolve(ROOT, 'commands', 'config-audit.md');
|
||||
|
||||
const router = await readFile(ROUTER, 'utf-8');
|
||||
|
||||
/** Scanner ids the orchestrator actually registers, read from its registry. */
|
||||
async function registeredScanners() {
|
||||
const src = await readFile(resolve(ROOT, 'scanners', 'scan-orchestrator.mjs'), 'utf-8');
|
||||
const block = src.slice(src.indexOf('const SCANNERS = ['));
|
||||
const arr = block.slice(0, block.indexOf('\n];'));
|
||||
return [...arr.matchAll(/\{\s*name:\s*'([A-Z]+)'/g)].map((m) => m[1]);
|
||||
}
|
||||
|
||||
/** Distinct posture area names, read from the scanner→area registry. */
|
||||
async function registeredAreas() {
|
||||
const src = await readFile(resolve(ROOT, 'scanners', 'lib', 'scoring.mjs'), 'utf-8');
|
||||
const block = src.slice(src.indexOf('const SCANNER_AREA_MAP = {'));
|
||||
const arr = block.slice(0, block.indexOf('\n};'));
|
||||
return [...new Set([...arr.matchAll(/:\s*'([^']+)'/g)].map((m) => m[1]))];
|
||||
}
|
||||
|
||||
test('the router surfaces the exit code of BOTH scanner invocations', async () => {
|
||||
const fence = router.slice(router.indexOf('### Step 3'), router.indexOf('### Step 4'));
|
||||
const chained = fence
|
||||
.split('\n')
|
||||
.filter((l) => (l.match(/\bnode \$\{CLAUDE_PLUGIN_ROOT\}/g) || []).length > 1);
|
||||
|
||||
assert.deepEqual(
|
||||
chained,
|
||||
[],
|
||||
'Step 3 runs scan-orchestrator and posture on ONE line with a single trailing\n' +
|
||||
'`echo $?`, which reports only the LAST command. An orchestrator exit 3 echoes\n' +
|
||||
'as posture\'s 0 and the "3 → stop" gate never fires. Capture each status into\n' +
|
||||
'its own variable and echo both.\n' +
|
||||
'Offending line(s):\n ' + chained.map((l) => l.trim().slice(0, 120)).join('\n '),
|
||||
);
|
||||
|
||||
assert.match(
|
||||
fence,
|
||||
/echo "\$[A-Z_]+ \$[A-Z_]+"/,
|
||||
'Step 3 must echo both captured exit codes (e.g. `echo "$ORCH_STATUS $POSTURE_STATUS"`)\n' +
|
||||
'so the gate can act on either scanner failing.',
|
||||
);
|
||||
});
|
||||
|
||||
test('the scanner count the router narrates matches the orchestrator registry', async () => {
|
||||
const scanners = await registeredScanners();
|
||||
const narrated = [...router.matchAll(/Running (\d+) configuration scanners/g)].map((m) =>
|
||||
Number(m[1]),
|
||||
);
|
||||
|
||||
assert.ok(narrated.length > 0, 'The router should still narrate how many scanners are running.');
|
||||
assert.deepEqual(
|
||||
narrated,
|
||||
narrated.map(() => scanners.length),
|
||||
`The router tells the user it is running ${narrated.join('/')} scanners; the ` +
|
||||
`orchestrator registers ${scanners.length} (${scanners.join(', ')}). A user-facing ` +
|
||||
'count that no test binds to the registry goes stale on the next scanner added.',
|
||||
);
|
||||
});
|
||||
|
||||
test('the Area Breakdown table has a row for every quality area posture emits', async () => {
|
||||
const areas = (await registeredAreas()).filter((a) => a !== 'Feature Coverage');
|
||||
const table = router.slice(router.indexOf('### Area Breakdown'), router.indexOf('{For the status column'));
|
||||
|
||||
const missing = areas.filter((a) => !table.includes(`| ${a} |`));
|
||||
|
||||
assert.deepEqual(
|
||||
missing,
|
||||
[],
|
||||
'Posture grades these areas but the router\'s table has no row for them, so they\n' +
|
||||
'are silently dropped from the user\'s results. (Feature Coverage is excluded by\n' +
|
||||
'design — it is reported as opportunities, not as a quality grade.)\n' +
|
||||
`Areas emitted: ${areas.join(', ')}\n` +
|
||||
`Missing rows: ${missing.join(', ')}`,
|
||||
);
|
||||
});
|
||||
|
||||
test('the router does not consume a stderr scorecard it discards', async () => {
|
||||
const fence = router.slice(router.indexOf('### Step 3'), router.indexOf('### Step 4'));
|
||||
const dependsOnStderr = /stderr scorecard/i.test(router);
|
||||
const postureDiscardsStderr = /posture\.mjs[^\n]*2>\/dev\/null/.test(fence);
|
||||
|
||||
assert.ok(
|
||||
!(dependsOnStderr && postureDiscardsStderr),
|
||||
'Step 6 renders "the headline line from the humanized stderr scorecard" and forbids\n' +
|
||||
'deriving a grade-prose ladder instead — but step 3 sends posture\'s stderr to\n' +
|
||||
'/dev/null, and the prose is absent from the JSON payload (measured). The slot can\n' +
|
||||
'only be improvised. Capture posture stderr to a FILE and read the headline from\n' +
|
||||
'it, as commands/posture.md already does.',
|
||||
);
|
||||
});
|
||||
|
||||
test('the --raw detection is not substring-based', async () => {
|
||||
const substringMatch = /grep -q -- "--raw"/.test(router);
|
||||
|
||||
assert.ok(
|
||||
!substringMatch,
|
||||
'Measured: `echo "$ARGUMENTS" | grep -q -- "--raw"` turns raw mode ON for `--rawdog`\n' +
|
||||
'and for any path containing `--raw`. Anchor the match to whole arguments.',
|
||||
);
|
||||
assert.match(
|
||||
router,
|
||||
/grep -qE -- '\(\^\| \)--raw\( \|\$\)'/,
|
||||
'The --raw check must match a whole argument, not a substring.',
|
||||
);
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue