open/config-audit
1
0
Fork 0
Code Issues Pull requests Releases Activity

fix(commands): stop answering questions the caller did not ask

Browse source 
Dogfooding `campaign` + `knowledge-refresh` against a throwaway ledger. Seven
defects, all found by running the commands as written and measuring, not by
reading them.

The headline pair only existed together. `knowledge-refresh` built
`STALE_AFTER="--stale-after 30"` and expanded it unquoted, trusting the shell to
split it in two. bash does; zsh — the macOS default, and what the Bash tool runs
here — does not. The CLI got one argv entry, matched no flag, and because it had
no unknown-flag branch, silently kept the 90-day default and reported "✓ All 14
register entries were re-verified within the last 90 days": a true-sounding
sentence about a threshold the user had just overridden. Fixing either half alone
leaves a silent wrong answer or a loud one; both are fixed, and a guard now
rejects any template that packs a flag and its value into one variable.

`knowledge-refresh` also read one register and wrote another: step 6 named an
unanchored `knowledge/best-practices.json` while the CLI reads
`${CLAUDE_PLUGIN_ROOT}/…`, which for an installed plugin is the cache. The
validation gate then ran the cached test against the cached register — green no
matter what was written. The two copies were byte-identical that day, which is
exactly why it was invisible.

`campaign` vouched for repos it could not read. `add /finnes/ikke` returned
`added` + exit 0; `refresh-tokens` then put the phantom in `swept[]` with a
0-token delta and left `skipped[]` empty, so the machine-wide bill claimed
coverage of three repos on a machine with two. Paths stay tracked — an unmounted
volume is a legitimate absence — but are reported as `addedUnverified`, and the
command names them.

Two class sweeps, both measured rather than assumed. `posture` was the single
scanner (1 of 14) whose fatal catch exited 1, which ux-rules defines as a normal
WARNING grade — a crash indistinguishable from a result. And all 13 payload
writers failed on a `--output-file` whose parent did not exist, which on a fresh
machine turned `campaign`'s first run into "the ledger may be corrupt"; they now
share `scanners/lib/write-output.mjs`.

Predicted breadth was too wide for the first time in five sessions: 6 of 8 CLIs
predicted to lack unknown-flag rejection, 4 measured. `drift` and `fix` already
reject them, via a construct the grep did not recognise — a grep matches an
implementation, the invariant is a behaviour. The sweep was rewritten to run each
CLI with a bogus flag and read the exit code.

Suite 1453 → 1469/0. Frozen snapshots untouched. `optimize-lens-cli` and
`token-hotspots-cli` share the unknown-flag defect and are deferred to the v5.14
argument-handling chunk with their positional-swallow arm; the count is recorded
in the guard rather than rounded down to zero.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012NHWjN8EnoxSqRvMTLK2NE
This commit is contained in:
Kjell Tore Guttormsen 2026-08-01 21:26:39 +02:00
commit caea8aca23
23 changed files with 742 additions and 48 deletions
Download patch file Download diff file Expand all files Collapse all files

47
CHANGELOG.md
View file

@ -8,6 +8,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- **`M-BUG-45` — `/config-audit knowledge-refresh --stale-after N` was silently dead under zsh.** The
command built `STALE_AFTER="--stale-after 30"` and expanded it unquoted, relying on the shell to
split it into two argv entries. bash does; **zsh — the macOS default since Catalina — does not**.
The CLI received one argv entry reading `--stale-after 30`, matched no flag, and fell back to the
90-day default while reporting success: "✓ All 14 register entries were re-verified within the last
90 days" — a true-sounding sentence about a threshold the user had just overridden. Measured:
`set -- $STALE_AFTER; echo $#` prints 1 under zsh, 2 under bash. The threshold is now passed as its
own quoted argument, and a guard rejects any command template that packs a flag and its value into
one variable.
- **`M-BUG-46` — four CLIs accepted unknown flags in silence.** No `else` branch at all in the parse
loop, so an unrecognised flag vanished without a trace: a typo'd `--ledger-file` made
`campaign-cli` report confidently on the *default* ledger instead of the one the caller named, and
a mistyped `--stale-after` reverted to 90 days. This is what made `M-BUG-45` silent rather than
loud. `campaign-cli` and `knowledge-refresh-cli` now fail with exit 3 and name the offending flag;
`optimize-lens-cli` and `token-hotspots-cli` share the defect and are closed together with their
positional-swallow arm in the v5.14 argument-handling work (tracked in the guard's `KNOWN_OPEN`).
- **`M-BUG-47` — the machine-wide token bill counted repos it could not read.** `refresh-tokens`
routed a repo to `skipped[]` only when `readActiveConfig` *threw*, but that function resolves any
path and its sub-readers all tolerate ENOENT, so a repo that does not exist yields an empty config
instead of an error. Measured: a phantom path landed in `swept[]` with a 0-token delta,
`skipped[]` was empty, and the roll-up claimed `reposWithTokens: 3` for a machine with two real
repos — so the command's own honesty clause ("name those repos plainly so the user knows the bill
omits them") could never fire. Readability is now checked before the sweep.
- **`M-BUG-48` — `campaign add` vouched for paths that do not exist.** `add /finnes/ikke` returned
`added: [...]` and exit 0, and the phantom row then sat in the backlog permanently. Paths are still
tracked (an unmounted volume is a legitimate reason for a repo to be absent today) but are now
reported separately as `addedUnverified`, and `campaign.md` names them instead of glossing over them.
- **`M-BUG-49` — `posture` reported a crash as a passing grade.** Its top-level catch set
`process.exitCode = 1`, while every command in this plugin is told that "codes 0, 1, 2 are normal
(PASS/WARNING/FAIL). Only 3 is a real error". A fatal error was therefore indistinguishable from a
WARNING, and the command went on to Read a payload file that was never written. Measured as the
single outlier: 1 of 14 scanners. Now exits 3.
- **`M-BUG-50` — `knowledge-refresh` read one register and wrote another, and the gate saw neither.**
Step 6 said `Edit knowledge/best-practices.json` — an unanchored relative path — while the CLI reads
`${CLAUDE_PLUGIN_ROOT}/knowledge/best-practices.json`, which for a marketplace install is the plugin
cache. So a normal user has no such file in their repo at all; in the plugin's own checkout the
command read the cache and wrote the working tree; and step 6.3's validation gate ran the cached
test against the cached register — **validating the copy that was not edited, and passing no matter
what was written.** Every write step is now anchored, and the command states where the register
actually lives (a marketplace copy is discarded on the next plugin upgrade).
- **No scanner created its `--output-file` parent directory.** `saveLedger` always did; the payload
write never did — an accidental asymmetry across all 13 writers. `commands/campaign.md` writes its
report under `~/.claude/config-audit/sessions/`, so on a fresh machine — precisely the first run
that `campaign-cli` otherwise handles gracefully with `initialized: false` — the write threw ENOENT
and the command's exit-code table reported it as a possibly-corrupt ledger, steering the user away
from the one action that would have helped. All payload writes now go through
`scanners/lib/write-output.mjs`.
- **`M-BUG-40`, fifth arm — `posture` wrote four temp files it could never read back.** #49 closed the
`$$`/cross-block class in four commands, but `posture.md` survived it, and so did the guard written
to prevent exactly this. The guard compared each `$$` path against the block that created it, so a

12
commands/campaign.md
View file

@ -62,7 +62,8 @@ From `$ARGUMENTS`, pick the mode:
- `export <path>` → export a planned repo's action plan into that repo's own `docs/`.
- `help` → show this surface and stop.
Set a shared date stamp for any write: `TODAY=$(date +%F)`.
Every write step derives its own date stamp inside its own block — there is no shared one to
set here, because each fenced block runs as a separate process.
### Step 2: Always report current state first
@ -182,7 +183,14 @@ node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs add <path1> <path2> .
```
(For a single repo with a custom display name, add `--name "<name>"`.) Read the result file and
report what was `added` vs `skipped`, then re-show the repo table.
report what was `added`, `addedUnverified`, and `skipped`, then re-show the repo table.
`addedUnverified[]` holds paths that were tracked but could **not** be read right now (they do
not exist, or are not directories). They are tracked deliberately — an unmounted volume is a
legitimate reason for a repo to be missing today — but they must be named, not glossed over:
"Tracked, but I couldn't read `<path>` — check for a typo, or mount it before the next token
sweep." An unreported phantom row stays in the backlog forever and quietly widens every
machine-wide total.
### Step 5 (mode `set-status`): Transition a repo — propose, approve, write

40
commands/knowledge-refresh.md
View file

@ -44,14 +44,21 @@ re-verification) and polling for new Claude Code practices...
### Step 2: Run the stale-check CLI
```bash
# Pass the threshold as its OWN quoted argument. Building "--stale-after 30" into
# one variable and expanding it unquoted only works if the shell word-splits —
# bash does, zsh (the macOS default) does not, and there the flag silently
# reverted to the 90-day default while the command reported success.
TODAY=$(date +%F)
STALE_AFTER=""
if echo "$ARGUMENTS" | grep -qE -- '--stale-after'; then
STALE_AFTER="--stale-after $(echo "$ARGUMENTS" | sed -nE 's/.*--stale-after[ =]+([0-9]+).*/\1/p')"
STALE_AFTER_DAYS=$(echo "$ARGUMENTS" | sed -nE 's/.*--stale-after[ =]+([0-9]+).*/\1/p')
if [ -n "$STALE_AFTER_DAYS" ]; then
node ${CLAUDE_PLUGIN_ROOT}/scanners/knowledge-refresh-cli.mjs \
--reference-date "$TODAY" --stale-after "$STALE_AFTER_DAYS" \
--output-file ~/.claude/config-audit/sessions/knowledge-refresh.json 2>/dev/null; echo $?
else
node ${CLAUDE_PLUGIN_ROOT}/scanners/knowledge-refresh-cli.mjs \
--reference-date "$TODAY" \
--output-file ~/.claude/config-audit/sessions/knowledge-refresh.json 2>/dev/null; echo $?
fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/knowledge-refresh-cli.mjs \
--reference-date "$TODAY" $STALE_AFTER \
--output-file ~/.claude/config-audit/sessions/knowledge-refresh.json 2>/dev/null; echo $?
```
Exit code **0** = all fresh, **1** = some stale (advisory, normal), **3** = real error →
@ -100,16 +107,27 @@ Be explicit: **"I will not change any file until you approve specific items."**
### Step 6: Apply approved writes (only the approved ones)
**Where the register lives — say this before writing.** The register is part of the plugin,
and the stale check above read it from `${CLAUDE_PLUGIN_ROOT}/knowledge/best-practices.json`
(the `registerPath` field in the payload names the exact file). For a marketplace install that
is the plugin cache, so **an edit there is discarded by the next plugin upgrade** — the durable
home for an approved change is the plugin's own checkout. Tell the user which of the two they
are about to write to, using the `registerPath` they can see, before asking for approval.
For each approved item:
1. Edit `knowledge/best-practices.json` — bump `source.verified`, update the `claim`/
`recommendation`, or append the new entry. Keep the file's 2-space JSON formatting.
2. If a `knowledge/*.md` mirror states the same fact, update it too so the human-readable
mirror doesn't drift from the register.
1. Edit `${CLAUDE_PLUGIN_ROOT}/knowledge/best-practices.json` — bump `source.verified`, update
the `claim`/`recommendation`, or append the new entry. Keep the file's 2-space JSON
formatting. Use the anchored path, never a bare `knowledge/…` — a relative path resolves
against the user's current repo, which is not the file the CLI read.
2. If a `${CLAUDE_PLUGIN_ROOT}/knowledge/*.md` mirror states the same fact, update it too so
the human-readable mirror doesn't drift from the register.
3. **Validate before declaring done** — re-run the register schema check and confirm zero errors:
```bash
node --test ${CLAUDE_PLUGIN_ROOT}/tests/lib/best-practices-register.test.mjs 2>&1 | tail -5
```
If validation fails, revert that edit and report it — never leave the register invalid.
This test loads the register through the same anchored path, so it validates the file you
just edited — that only holds while step 1 uses the anchored path too. If validation fails,
revert that edit and report it — never leave the register invalid.
Report exactly what changed (ids + fields), and what was deferred to manual review.

9
scanners/campaign-cli.mjs
View file

@ -23,7 +23,7 @@
*/
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import {
loadLedger,
validateLedger,
@ -52,6 +52,11 @@ async function main() {
const a = args[i];
if (a === '--ledger-file' && args[i + 1]) ledgerFile = args[++i];
else if (a === '--output-file' && args[i + 1]) outputFile = args[++i];
// A flag we do not understand must fail loudly. Silently dropping it lets a
// caller-side mistake — a typo'd `--ledger-file`, or a shell that did not
// word-split "--flag value" into two argv entries — produce a confident
// report about the wrong ledger.
else if (a.startsWith('--')) fail(`unknown flag "${a}"`);
}
const ledgerPath = resolve(ledgerFile || defaultLedgerPath());
@ -102,7 +107,7 @@ async function main() {
}
const json = JSON.stringify(payload, null, 2);
if (outputFile) await writeFile(outputFile, json, 'utf-8');
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
else process.stdout.write(json + '\n');
process.exitCode = exitCode;

3
scanners/campaign-export-cli.mjs
View file

@ -35,6 +35,7 @@
import { resolve, join, dirname } from 'node:path';
import { homedir } from 'node:os';
import { readFile, writeFile, mkdir } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import {
loadLedger,
validateLedger,
@ -78,7 +79,7 @@ function parseArgs(argv) {
async function emit(payload, outputFile, exitCode) {
const json = JSON.stringify(payload, null, 2);
if (outputFile) await writeFile(outputFile, json, 'utf-8');
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
else process.stdout.write(json + '\n');
process.exitCode = exitCode;
}

38
scanners/campaign-write-cli.mjs
View file

@ -43,7 +43,8 @@
*/
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { stat } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import {
createLedger,
addRepo,
@ -89,6 +90,23 @@ function parseArgs(argv) {
return { subcommand: positionals[0], rest: positionals.slice(1), flags };
}
/**
* Can this path actually be read as a repo right now?
*
* `readActiveConfig` resolves any string and its sub-readers all tolerate ENOENT, so a repo
* that does not exist yields an EMPTY config instead of an error. Without this check the
* sweep records a phantom repo as successfully swept with 0 tokens, and the machine-wide
* bill claims coverage it does not have. A missing path is reported, never rejected — an
* unmounted volume is a legitimate reason for a tracked repo to be absent today.
*/
async function isReadableRepoDir(path) {
try {
return (await stat(path)).isDirectory();
} catch {
return false;
}
}
/** Load an existing ledger, treating a parse error as a hard failure (never clobber corrupt data). */
async function loadOrFail(path) {
try {
@ -100,7 +118,7 @@ async function loadOrFail(path) {
async function emit(payload, outputFile, exitCode) {
const json = JSON.stringify(payload, null, 2);
if (outputFile) await writeFile(outputFile, json, 'utf-8');
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
else process.stdout.write(json + '\n');
process.exitCode = exitCode;
}
@ -146,6 +164,7 @@ async function main() {
let ledger = loaded === null ? createLedger({ now }) : loaded;
const added = [];
const addedUnverified = [];
const skipped = [];
for (const p of paths) {
const resolved = resolve(p);
@ -153,13 +172,17 @@ async function main() {
// --name applies only to a lone path; multi-add lets the lib derive each basename.
const name = paths.length === 1 ? flags.name || undefined : undefined;
ledger = addRepo(ledger, { path: p, name }, { now });
(present ? skipped : added).push(resolved);
if (present) skipped.push(resolved);
else if (await isReadableRepoDir(resolved)) added.push(resolved);
// Tracked either way, but never silently vouched for: the command reports these
// separately so a typo does not become a permanent phantom row in the backlog.
else addedUnverified.push(resolved);
}
await saveLedger(ledgerPath, ledger);
return emit(
{
status: 'ok', action: 'add', written: true, autoInitialized, ledgerPath,
added, skipped, repos: ledger.repos, rollUp: rollUp(ledger),
added, addedUnverified, skipped, repos: ledger.repos, rollUp: rollUp(ledger),
},
flags.outputFile,
0,
@ -216,6 +239,13 @@ async function main() {
let sharedSummary = null;
for (const repo of ledger0.repos) {
// Check readability FIRST: readActiveConfig returns an empty config for a path that
// does not exist rather than throwing, so the catch below would never see it and the
// repo would be recorded as swept with a 0-token delta — a bill that looks complete.
if (!(await isReadableRepoDir(repo.path))) {
skipped.push({ path: repo.path, reason: 'repo path is not readable (does not exist or is not a directory)' });
continue;
}
let split;
try {
const activeConfig = await readActiveConfig(repo.path, { verbose: false });

6
scanners/drift-cli.mjs
View file

@ -12,7 +12,7 @@
*/
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { runAllScanners } from './scan-orchestrator.mjs';
import { diffEnvelopes, formatDiffReport } from './lib/diff-engine.mjs';
import { saveBaseline, loadBaseline, listBaselines } from './lib/baseline.mjs';
@ -73,7 +73,7 @@ async function main() {
// human listing below goes to stderr, so without --output-file the command
// received 0 bytes and could render nothing. The flag was already accepted
// by the arg parser; only list mode ignored it.
if (outputFile) await writeFile(outputFile, JSON.stringify(result, null, 2) + '\n', 'utf-8');
if (outputFile) await writeOutputFile(outputFile, JSON.stringify(result, null, 2) + '\n', 'utf-8');
if (jsonMode || rawMode) {
process.stdout.write(JSON.stringify(result, null, 2) + '\n');
} else {
@ -190,7 +190,7 @@ async function main() {
// otherwise; stdout is unaffected.
if (outputFile) {
const fileDiff = (jsonMode || rawMode) ? diff : humanizedDiff;
await writeFile(outputFile, JSON.stringify(fileDiff, null, 2), 'utf-8');
await writeOutputFile(outputFile, JSON.stringify(fileDiff, null, 2), 'utf-8');
process.stderr.write(`\nResults written to ${outputFile}\n`);
}

6
scanners/fix-cli.mjs
View file

@ -9,7 +9,7 @@
*/
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { runAllScanners } from './scan-orchestrator.mjs';
import { planFixes, applyFixes, verifyFixes } from './fix-engine.mjs';
import { createBackup } from './lib/backup.mjs';
@ -134,7 +134,7 @@ async function main() {
if (machineMode) {
process.stdout.write(JSON.stringify(output, null, 2) + '\n');
}
if (outputFile) await writeFile(outputFile, JSON.stringify(output, null, 2) + '\n', 'utf-8');
if (outputFile) await writeOutputFile(outputFile, JSON.stringify(output, null, 2) + '\n', 'utf-8');
return;
}
@ -235,7 +235,7 @@ async function main() {
// --output-file carries the same payload to disk. ux-rules rule 2 requires
// it: commands run scanners with `2>/dev/null`, so anything the command has
// to act on must ride in a file, not in stdout or stderr.
if (outputFile) await writeFile(outputFile, serialized, 'utf-8');
if (outputFile) await writeOutputFile(outputFile, serialized, 'utf-8');
// Exit code follows the convention the other scanners use: 0 PASS,
// 2 FAIL, 3 tool error. A failed fix used to exit 0, so a caller could not

9
scanners/knowledge-refresh-cli.mjs
View file

@ -24,7 +24,7 @@
*/
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { loadRegister, REGISTER_PATH } from './lib/best-practices-register.mjs';
import { assessFreshness, STALE_AFTER_DAYS_DEFAULT } from './lib/knowledge-refresh.mjs';
@ -60,6 +60,11 @@ async function main() {
referenceDate = args[++i];
if (!DATE_RE.test(referenceDate)) fail('--reference-date must be YYYY-MM-DD');
}
// A flag we do not understand must fail loudly. Silently dropping it is how
// `--stale-after 30` — arriving as ONE argv entry from a shell that does not
// word-split — became "all 14 entries are fresh within 90 days": a confident
// answer to a question the caller did not ask.
else if (a.startsWith('--')) fail(`unknown flag "${a}"`);
}
// The clock is read here ONLY — the core takes an injected date and stays pure.
@ -93,7 +98,7 @@ async function main() {
};
const json = JSON.stringify(payload, null, 2);
if (outputFile) await writeFile(outputFile, json, 'utf-8');
if (outputFile) await writeOutputFile(outputFile, json, 'utf-8');
else process.stdout.write(json + '\n');
process.exitCode = assessment.counts.stale > 0 ? 1 : 0;

39
scanners/lib/write-output.mjs Normal file
View file

@ -0,0 +1,39 @@
/**
* write-output — the one place a scanner's `--output-file` payload is written.
*
* Every command in this plugin follows the same contract (`.claude/rules/ux-rules.md`):
* run the scanner with `--output-file <path> 2>/dev/null`, check the exit code, then Read
* the file. The path the command chooses is frequently one it has never created — e.g.
* `commands/campaign.md` writes its report to
* `~/.claude/config-audit/sessions/campaign-report.json`, which on a fresh machine does not
* exist yet. That is precisely the FIRST run, the case campaign-cli otherwise handles
* gracefully by reporting `initialized: false`.
*
* Before this helper existed, all 13 payload writers called `writeFile` directly and threw
* ENOENT there. The exit code was 3, and the command's own exit-code table reads 3 as "the
* input is missing or corrupt" — so the user was told the ledger might be corrupt and
* warned off the one action that would have fixed anything. `saveLedger` had always created
* its parent directory; the payload write simply never did. The asymmetry was accidental.
*
* Creating the parent is the honest behaviour: the caller asked for a file at a path, and
* nothing about a missing intermediate directory is an error the caller can learn from.
*/
import { writeFile, mkdir } from 'node:fs/promises';
import { dirname } from 'node:path';
/**
* Write a scanner payload, creating the parent directory if needed.
*
* Signature-compatible with `writeFile(path, contents, encoding)` so call sites are a pure
* rename — the encoding argument is kept rather than defaulted away.
*
* @param {string} path - destination file
* @param {string} contents - serialized payload
* @param {string} [encoding='utf-8']
* @returns {Promise<void>}
*/
export async function writeOutputFile(path, contents, encoding = 'utf-8') {
await mkdir(dirname(path), { recursive: true });
await writeFile(path, contents, encoding);
}

5
scanners/manifest.mjs
View file

@ -40,7 +40,8 @@
*/
import { resolve } from 'node:path';
import { writeFile, stat } from 'node:fs/promises';
import { stat } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { readActiveConfig, deriveLoadPattern } from './lib/active-config-reader.mjs';
// CLAUDE.md cascade files are all discovered by walking UP from the repo, so
@ -287,7 +288,7 @@ async function main() {
const json = JSON.stringify(output, null, 2);
if (outputFile) {
await writeFile(outputFile, json, 'utf-8');
await writeOutputFile(outputFile, json, 'utf-8');
}
if (jsonMode || rawMode || !outputFile) {

5
scanners/optimize-lens-cli.mjs
View file

@ -25,7 +25,8 @@
*/
import { resolve, sep } from 'node:path';
import { writeFile, readFile, stat } from 'node:fs/promises';
import { readFile, stat } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { discoverConfigFiles } from './lib/file-discovery.mjs';
import { resetCounter } from './lib/output.mjs';
import { parseFrontmatter } from './lib/yaml-parser.mjs';
@ -218,7 +219,7 @@ async function main() {
const json = JSON.stringify(payload, null, 2);
if (outputFile) {
await writeFile(outputFile, json, 'utf-8');
await writeOutputFile(outputFile, json, 'utf-8');
}
if (!outputFile) {
process.stdout.write(json + '\n');

5
scanners/plugin-health-scanner.mjs
View file

@ -8,7 +8,8 @@
* Zero external dependencies.
*/
import { readdir, stat, readFile, writeFile } from 'node:fs/promises';
import { readdir, stat, readFile } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { join, basename, resolve, sep } from 'node:path';
import { finding, scannerResult, resetCounter } from './lib/output.mjs';
import { SEVERITY } from './lib/severity.mjs';
@ -803,7 +804,7 @@ async function main() {
plugins,
cross_plugin_findings: findings.filter(f => crossIds.has(f.id)),
};
await writeFile(outputFile, JSON.stringify(payload, null, 2), 'utf-8');
await writeOutputFile(outputFile, JSON.stringify(payload, null, 2), 'utf-8');
process.stderr.write(`\nResults written to ${outputFile}\n`);
}
}

10
scanners/posture.mjs
View file

@ -8,7 +8,7 @@
*/
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { runAllScanners } from './scan-orchestrator.mjs';
import { humanizeEnvelope } from './lib/humanizer.mjs';
import {
@ -123,7 +123,7 @@ async function main() {
? result
: { ...result, scannerEnvelope: humanizeEnvelope(result.scannerEnvelope) };
const json = JSON.stringify(fileEnv, null, 2);
await writeFile(outputFile, json, 'utf-8');
await writeOutputFile(outputFile, json, 'utf-8');
process.stderr.write(`\nResults written to ${outputFile}\n`);
}
}
@ -133,6 +133,10 @@ const isDirectRun = process.argv[1] && resolve(process.argv[1]) === resolve(new
if (isDirectRun) {
main().catch(err => {
process.stderr.write(`Fatal: ${err.message}\n`);
process.exitCode = 1;
// 3, not 1. Every command in this plugin is told that 0/1/2 are normal grades
// (PASS/WARNING/FAIL) and only 3 is a real error, so exiting 1 here made a crash
// indistinguishable from a WARNING — and the command went on to Read a payload file
// that was never written.
process.exitCode = 3;
});
}

3
scanners/scan-orchestrator.mjs
View file

@ -9,6 +9,7 @@
import { resolve, sep } from 'node:path';
import { readFile, writeFile } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { resetCounter } from './lib/output.mjs';
import { envelope } from './lib/output.mjs';
import { discoverConfigFiles, discoverConfigFilesMulti, discoverFullMachinePaths } from './lib/file-discovery.mjs';
@ -278,7 +279,7 @@ async function main() {
const json = JSON.stringify(output, null, 2);
if (outputFile) {
await writeFile(outputFile, json, 'utf-8');
await writeOutputFile(outputFile, json, 'utf-8');
process.stderr.write(`\nResults written to ${outputFile}\n`);
} else {
process.stdout.write(json + '\n');

5
scanners/token-hotspots-cli.mjs
View file

@ -14,7 +14,8 @@
import { resolve, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { writeFile, readFile, stat } from 'node:fs/promises';
import { readFile, stat } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { discoverConfigFiles } from './lib/file-discovery.mjs';
import { resetCounter } from './lib/output.mjs';
import { scan } from './token-hotspots.mjs';
@ -131,7 +132,7 @@ async function main() {
const json = JSON.stringify(payload, null, 2);
if (outputFile) {
await writeFile(outputFile, json, 'utf-8');
await writeOutputFile(outputFile, json, 'utf-8');
}
if (jsonMode || rawMode || !outputFile) {

5
scanners/whats-active.mjs
View file

@ -13,7 +13,8 @@
*/
import { resolve } from 'node:path';
import { writeFile, stat } from 'node:fs/promises';
import { stat } from 'node:fs/promises';
import { writeOutputFile } from './lib/write-output.mjs';
import { readActiveConfig } from './lib/active-config-reader.mjs';
async function main() {
@ -54,7 +55,7 @@ async function main() {
const json = JSON.stringify(result, null, 2);
if (outputFile) {
await writeFile(outputFile, json, 'utf-8');
await writeOutputFile(outputFile, json, 'utf-8');
}
if (jsonMode || rawMode || !outputFile) {

91
tests/commands/command-flag-value-portability.test.mjs Normal file
View file

@ -0,0 +1,91 @@
/**
* Session #51 — command-template flag-value portability.
*
* Dogfooding `knowledge-refresh` surfaced a defect that only exists at the
* seam between the command template and the shell that runs it:
*
* STALE_AFTER="--stale-after 30"
* node …-cli.mjs --reference-date "$TODAY" $STALE_AFTER --output-file …
*
* The unquoted `$STALE_AFTER` is meant to split into TWO argv entries. Under
* **bash** it does. Under **zsh** — the macOS default since Catalina, and the
* shell the Bash tool actually runs on this machine — unquoted parameter
* expansions are NOT word-split, so the CLI receives ONE argv entry with the
* literal text `--stale-after 30`, matches no known flag, and (because the CLI
* silently ignored unknown flags — see cli-unknown-flag-rejection.test.mjs)
* falls back to the 90-day default while reporting success. Measured:
*
* $ STALE_AFTER="--stale-after 30"; set -- $STALE_AFTER; echo $#
* 1 # zsh (bash prints 2)
* → payload staleAfterDays: 90, exit 0, "✓ All 14 entries fresh"
*
* The user-facing knob was silently dead. Note the asymmetry that makes this
* survivable elsewhere: an EMPTY unquoted expansion yields ZERO argv entries in
* both shells, so the `FLAG=""` idiom used by ~25 other sites is portable. Only
* a variable that can hold a flag AND its value is affected.
*
* The invariant asserted here is therefore about VALUE-carrying flags, not
* about quoting in general: a command template must never depend on the shell
* splitting one variable into a flag plus its argument. Pass the value through
* its own quoted variable instead.
*/
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFile, readdir } from 'node:fs/promises';
import { resolve, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const COMMANDS_DIR = resolve(__dirname, '..', '..', 'commands');
async function commandFiles() {
const entries = await readdir(COMMANDS_DIR);
return entries.filter((e) => e.endsWith('.md')).sort();
}
/**
* Assignments whose right-hand side contains a flag followed by a value —
* i.e. the value only reaches argv if the shell word-splits. Matches both
* `X="--flag value"` and `X="--flag $(cmd)"`.
*/
const MULTIWORD_FLAG_ASSIGN = /^\s*([A-Z_][A-Z0-9_]*)=(["'])(--[a-z0-9-]+)[ \t]+\S.*\2\s*$/;
test('no command template builds a flag AND its value into one shell variable', async () => {
const offenders = [];
for (const file of await commandFiles()) {
const content = await readFile(resolve(COMMANDS_DIR, file), 'utf-8');
content.split('\n').forEach((line, i) => {
const m = line.match(MULTIWORD_FLAG_ASSIGN);
if (m) offenders.push(`${file}:${i + 1} ${m[1]}=${m[2]}${m[3]} …${m[2]}`);
});
}
assert.deepEqual(
offenders,
[],
'A variable holding "--flag value" only reaches argv correctly if the shell\n' +
'word-splits an unquoted expansion. zsh does not. Pass the value in its own\n' +
'quoted variable instead:\n' +
' N=$(… extract …); [ -n "$N" ] && node cli.mjs --flag "$N"\n' +
'Offending assignments:\n ' + offenders.join('\n '),
);
});
test('knowledge-refresh.md passes --stale-after with a quoted value', async () => {
const content = await readFile(resolve(COMMANDS_DIR, 'knowledge-refresh.md'), 'utf-8');
assert.ok(
!/\$STALE_AFTER\b(?!")/.test(content.replace(/"\$STALE_AFTER"/g, '')),
'knowledge-refresh.md still expands a flag-carrying variable unquoted; under zsh the\n' +
'threshold silently reverts to the 90-day default while the command reports success.',
);
assert.match(
content,
/--stale-after "\$[A-Z_]+"/,
'knowledge-refresh.md must pass the extracted threshold as its own quoted argument\n' +
'(`--stale-after "$STALE_AFTER_DAYS"`), so no word-splitting is required.',
);
});

35
tests/commands/command-shell-state-shape.test.mjs
View file

@ -215,3 +215,38 @@ test('state.yaml: phase commands name all four fields the rule requires', async
}
assert.deepEqual(violations, [], `Incomplete state.yaml contracts:\n${violations.join('\n')}`);
});
/**
* Session #51 — the instruction that CAUSED the $TODAY defect must not outlive its fix.
*
* #49 fixed campaign.md by re-deriving `TODAY=$(date +%F)` inside each of the six write
* blocks. But step 1 still carried the original prose — "Set a shared date stamp for any
* write: `TODAY=$(date +%F)`" — which is not in a fence, cannot set anything, and directly
* contradicts the six comments added below it. A template that argues with itself is not a
* contract, and the next edit is the one that believes the wrong half.
*
* The guard above asserts fences; this one asserts that no PROSE line instructs the reader
* to establish shell state for later blocks.
*/
test('no command template instructs shell state to be set outside a fence', async () => {
const offenders = [];
for (const file of await commandFiles()) {
const content = await readFile(resolve(COMMANDS_DIR, file), 'utf-8');
const { lines, blockIndexOf } = parseFences(content);
lines.forEach((line, i) => {
if (blockIndexOf(i) !== -1) return; // inside a fence: that is where state belongs
if (/`[A-Z_][A-Z0-9_]*=\$\(/.test(line) || /^\s*[A-Z_][A-Z0-9_]*=\$\(/.test(line)) {
offenders.push(`${file}:${i + 1} ${line.trim()}`);
}
});
}
assert.deepEqual(
offenders,
[],
'Prose that tells the reader to set a shell variable implies it survives to a later\n' +
'block. It does not — every fence is its own process. Delete the instruction; the\n' +
'blocks that need the value derive it themselves:\n ' + offenders.join('\n '),
);
});

91
tests/commands/knowledge-refresh-write-target.test.mjs Normal file
View file

@ -0,0 +1,91 @@
/**
* Session #51 — knowledge-refresh must write the register it actually read.
*
* The command reads the register through `knowledge-refresh-cli.mjs`, whose `REGISTER_PATH`
* is anchored to the CLI module itself — i.e. `${CLAUDE_PLUGIN_ROOT}/knowledge/
* best-practices.json`. For any installed plugin that is the marketplace cache; measured
* live during this chunk:
*
* registerPath: …/.claude/plugins/cache/ktg-plugin-marketplace/config-audit/5.13.0/
* knowledge/best-practices.json
*
* Step 6 then said: Edit `knowledge/best-practices.json` — an UNANCHORED relative path,
* which resolves against whatever repo the user happens to be sitting in. Three consequences,
* none of them visible at the time:
*
* 1. For a normal user, that path does not exist in their repo at all, so the approved
* change either fails or drops a stray file into their project.
* 2. In the plugin's own checkout it resolves to the working tree, so the command reads
* one file and writes a different one — the refresh appears to do nothing, because the
* CLI keeps reporting the cached copy's dates.
* 3. Step 6.3's validation gate runs the plugin's own test file, which loads the register
* via the same anchored `REGISTER_PATH`. So the gate validates the copy that was NOT
* edited and passes no matter what was written — a guard that cannot see the file it
* guards.
*
* The two register copies were byte-identical on the day this was found, which is exactly
* why the defect was invisible to a casual dogfood run.
*/
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFile } from 'node:fs/promises';
import { resolve, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const FILE = resolve(__dirname, '..', '..', 'commands', 'knowledge-refresh.md');
/**
* Scoped to lines that tell the model to MODIFY the register. Descriptive prose ("this
* command keeps knowledge/best-practices.json current") and the closing `git commit`
* suggestion name the file without acting on it, and a git path is correctly repo-relative.
* The defect was specifically an unanchored path in an imperative write step.
*/
const WRITE_VERB = /\b(edit|write|append|overwrite|save)\b/i;
function unanchoredWriteTargets(content) {
const out = [];
content.split('\n').forEach((line, i) => {
if (!WRITE_VERB.test(line)) return;
for (const m of line.matchAll(/(\S*)knowledge\/best-practices\.json/g)) {
if (!m[1].endsWith('${CLAUDE_PLUGIN_ROOT}/')) out.push(`${i + 1}: ${line.trim()}`);
}
});
return out;
}
test('the guard still catches the original defect', () => {
const original = '1. Edit `knowledge/best-practices.json` — bump `source.verified`, update the `claim`/';
assert.equal(
unanchoredWriteTargets(original).length,
1,
'A narrowed guard must still fail on the text it was written for.',
);
});
test('every register path in knowledge-refresh.md is anchored to the plugin root', async () => {
const content = await readFile(FILE, 'utf-8');
const unanchored = unanchoredWriteTargets(content);
assert.deepEqual(
unanchored,
[],
'A bare `knowledge/best-practices.json` resolves against the user\'s current repo, not\n' +
'against the register the CLI actually read. Anchor every reference to\n' +
'${CLAUDE_PLUGIN_ROOT}/ so the file that is read, written, and validated is one file:\n ' +
unanchored.join('\n '),
);
});
test('the write step says where the register really lives', async () => {
const content = await readFile(FILE, 'utf-8');
assert.match(
content,
/marketplace|plugin cache|replaced on upgrade|plugin's own checkout/i,
'Step 6 must state that the register is part of the installed plugin, so an approved\n' +
'edit to a marketplace-installed copy is discarded by the next plugin upgrade and the\n' +
'durable change belongs in the plugin\'s own checkout. Silently editing a cache\n' +
'directory is the kind of write that looks successful and evaporates.',
);
});

87
tests/scanners/campaign-write-cli.test.mjs
View file

@ -85,15 +85,21 @@ describe('campaign-write-cli — add', () => {
it('auto-initializes when no ledger exists, then adds repos (exit 0)', async () => {
const dir = newDir();
const file = join(dir, 'ledger.json');
// Real directories: since #51 `add` reports a path it cannot read under
// `addedUnverified` instead of vouching for it. This test is about auto-init and
// tracking, so its fixtures must be repos that actually exist.
const a = newDir();
const b = newDir();
assert.ok(!existsSync(file));
const { status, stdout } = runWrite([
'add', '/r/a', '/r/b', '--ledger-file', file, '--reference-date', NOW,
'add', a, b, '--ledger-file', file, '--reference-date', NOW,
]);
assert.equal(status, 0);
const out = JSON.parse(stdout);
assert.equal(out.action, 'add');
assert.equal(out.autoInitialized, true);
assert.deepEqual(out.added.sort(), [resolve('/r/a'), resolve('/r/b')].sort());
assert.deepEqual(out.added.sort(), [resolve(a), resolve(b)].sort());
assert.deepEqual(out.addedUnverified, []);
const { ledger, validation } = await readLedger(file);
assert.ok(validation.valid, validation.errors.join('; '));
@ -104,16 +110,19 @@ describe('campaign-write-cli — add', () => {
it('appends to an existing ledger and is idempotent on re-add (added vs skipped)', async () => {
const dir = newDir();
const file = join(dir, 'ledger.json');
runWrite(['add', '/r/a', '/r/b', '--ledger-file', file, '--reference-date', NOW]);
const a = newDir();
const b = newDir();
const c = newDir();
runWrite(['add', a, b, '--ledger-file', file, '--reference-date', NOW]);
const { status, stdout } = runWrite([
'add', '/r/a', '/r/c', '--ledger-file', file, '--reference-date', NOW,
'add', a, c, '--ledger-file', file, '--reference-date', NOW,
]);
assert.equal(status, 0);
const out = JSON.parse(stdout);
assert.equal(out.autoInitialized, false);
assert.deepEqual(out.added, [resolve('/r/c')]);
assert.deepEqual(out.skipped, [resolve('/r/a')]);
assert.deepEqual(out.added, [resolve(c)]);
assert.deepEqual(out.skipped, [resolve(a)]);
const { ledger } = await readLedger(file);
assert.equal(ledger.repos.length, 3);
@ -316,3 +325,69 @@ describe('campaign-write-cli — determinism + --output-file', () => {
assert.equal(status, 3);
});
});
// ── Session #51: honest coverage — the ledger must not vouch for repos it cannot read ──
//
// Dogfooding the campaign lifecycle put a path that does not exist into the ledger and
// swept it. Both halves lied, quietly:
//
// add → added: ["/finnes/absolutt/ikke/noe-repo"], exit 0, no warning.
// sweep → swept: [… , "/finnes/absolutt/ikke/noe-repo"], skipped: [], byRepo entry with 0
// tokens, reposWithTokens: 3 for a machine with 2 real repos.
//
// The root cause is that `readActiveConfig` resolves a path and every sub-reader tolerates
// ENOENT, so a missing repo yields an EMPTY config rather than an error — and the sweep's
// try/catch only routes THROWN errors to `skipped`. The command's own honesty clause ("If
// anything was skipped, name those repos plainly so the user knows the bill omits them")
// could therefore never fire. A token bill that silently counts phantom repos as 0 is worse
// than one that refuses to answer: it looks complete.
describe('campaign-write-cli — honest coverage for unreadable repos', () => {
it('add flags a path that does not exist instead of vouching for it', async () => {
const dir = newDir();
const file = join(dir, 'ledger.json');
const real = newDir();
const phantom = join(dir, 'no-such-repo');
const { status, stdout } = runWrite([
'add', real, phantom, '--ledger-file', file, '--reference-date', NOW,
]);
assert.equal(status, 0, 'a missing path is reported, not rejected — it may be unmounted');
const out = JSON.parse(stdout);
assert.deepEqual(out.added, [resolve(real)], 'only the readable repo is vouched for');
assert.deepEqual(
out.addedUnverified,
[resolve(phantom)],
'a path that does not exist must be reported separately so the command can say so',
);
// Still tracked — the campaign is a work register, and an unmounted volume is a
// legitimate reason for a path to be absent today and present tomorrow.
const { ledger } = await readLedger(file);
assert.equal(ledger.repos.length, 2);
});
it('refresh-tokens skips an unreadable repo instead of counting it as 0 tokens', async () => {
const dir = newDir();
const file = join(dir, 'ledger.json');
const phantom = join(dir, 'no-such-repo');
runWrite(['add', phantom, '--ledger-file', file, '--reference-date', NOW]);
const { status, stdout } = runWrite([
'refresh-tokens', '--ledger-file', file, '--reference-date', NOW,
]);
assert.equal(status, 0);
const out = JSON.parse(stdout);
assert.deepEqual(out.swept, [], 'a repo that cannot be read was never actually swept');
assert.equal(out.skipped.length, 1, 'it belongs in skipped, with a reason the user can act on');
assert.equal(out.skipped[0].path, resolve(phantom));
assert.match(out.skipped[0].reason, /not readable|does not exist|ENOENT/i);
assert.equal(
out.rollUp.tokens.reposWithTokens,
0,
'the machine-wide bill must not claim coverage of a repo it could not read',
);
});
});

112
tests/scanners/cli-unknown-flag-rejection.test.mjs Normal file
View file

@ -0,0 +1,112 @@
/**
* Session #51 — CLIs must reject an unknown flag, never ignore it.
*
* Third arm of the argument-handling class first measured in #44/#47/#50. The
* earlier arms were about a flag's VALUE being swallowed as the scan target
* (`else if (!args[i].startsWith('-')) targetPath = args[i]`). This arm is
* quieter and worse: several CLIs have no `else` branch at all, so an
* unrecognised flag falls out of the parse loop leaving no trace — exit 0, a
* full payload, and an answer to a question the caller did not ask.
*
* Measured cost, live, in the same session that wrote this test: the
* `knowledge-refresh` command's only user-facing knob (`--stale-after N`)
* reached the CLI as one malformed argv entry (see
* command-flag-value-portability.test.mjs). Because the CLI ignored it, the
* command reported "✓ All 14 register entries were re-verified within the last
* 90 days" — a true-sounding sentence about a threshold the user had just
* overridden. Had the CLI failed loudly, the shell bug would have been a
* one-line exit-3 message instead of a silent wrong answer.
*
* Scope of this guard: the CLIs whose commands were dogfooded in this chunk.
* `optimize-lens-cli.mjs` and `token-hotspots-cli.mjs` share the defect but
* also carry the still-open positional-swallow arm; both are fixed together in
* the v5.14 arg-handling chunk, where every call site's flags can be audited at
* once. They are listed in KNOWN_OPEN so the number stays visible rather than
* being quietly rounded down to zero.
*/
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawn } from 'node:child_process';
import { readFile } from 'node:fs/promises';
import { resolve, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const SCANNERS_DIR = resolve(__dirname, '..', '..', 'scanners');
/** CLIs this guard holds to the invariant, with the argv they need to get past required-arg checks. */
const GUARDED = [
{ cli: 'campaign-cli.mjs', argv: [] },
{ cli: 'knowledge-refresh-cli.mjs', argv: [] },
{ cli: 'campaign-write-cli.mjs', argv: ['init'] },
{ cli: 'campaign-export-cli.mjs', argv: ['--repo', '.'] },
];
/** Same defect, deferred to the v5.14 arg-handling chunk together with their positional-swallow arm. */
const KNOWN_OPEN = ['optimize-lens-cli.mjs', 'token-hotspots-cli.mjs'];
function run(cli, argv) {
return new Promise((res) => {
const child = spawn(process.execPath, [resolve(SCANNERS_DIR, cli), ...argv], {
cwd: resolve(__dirname, '..', '..'),
});
let stderr = '';
child.stderr.on('data', (d) => { stderr += d; });
child.stdout.on('data', () => {});
child.on('close', (code) => res({ code, stderr }));
});
}
for (const { cli, argv } of GUARDED) {
test(`${cli} rejects an unknown flag with exit 3`, async () => {
const { code, stderr } = await run(cli, [...argv, '--zzz-not-a-real-flag']);
assert.equal(
code,
3,
`${cli} accepted an unknown flag (exit ${code}). A flag the CLI does not understand\n` +
'must fail loudly — silently ignoring it turns a caller-side bug into a confident\n' +
'wrong answer. stderr was: ' + JSON.stringify(stderr),
);
assert.match(
stderr,
/--zzz-not-a-real-flag/,
`${cli} must name the offending flag so the caller can find it.`,
);
});
}
test('the deferred CLIs are still deferred, and still counted', () => {
assert.equal(
KNOWN_OPEN.length,
2,
'When the v5.14 arg-handling chunk closes optimize-lens-cli and token-hotspots-cli,\n' +
'move them from KNOWN_OPEN into GUARDED rather than deleting them — the count is\n' +
'the record of how wide the class was.',
);
});
/**
* The caller arm. #45/#46/#47 all taught the same lesson: fixing a CLI does not fix the
* command that reads its payload. `addedUnverified` and the `skipped` reasons only reach the
* user if the command template is told to report them — otherwise the CLI is honest into a
* void, and the phantom repo is just as invisible as before.
*/
const CAMPAIGN_MD = resolve(__dirname, '..', '..', 'commands', 'campaign.md');
test('campaign.md reports the fields the write-CLI added for honest coverage', async () => {
const content = await readFile(CAMPAIGN_MD, 'utf-8');
assert.match(
content,
/addedUnverified/,
'campaign.md must report `addedUnverified` after an add — a tracked path the CLI could\n' +
'not read is exactly the row that silently pollutes the backlog and the token bill.',
);
assert.match(
content,
/skipped/,
'campaign.md must report `skipped[]` after a token sweep so the bill\'s coverage is honest.',
);
});

127
tests/scanners/output-file-robustness.test.mjs Normal file
View file

@ -0,0 +1,127 @@
/**
* Session #51 — `--output-file` must be usable, and a crash must never look normal.
*
* Two defects found while sweeping the campaign/knowledge-refresh chunk, both about the
* seam every command depends on: the command runs a scanner with `--output-file <path>
* 2>/dev/null`, checks the exit code, and Reads the file (ux-rules 2-4).
*
* 1. NO SCANNER CREATES THE PARENT DIRECTORY. `saveLedger` does; the payload write does
* not. `commands/campaign.md` step 2 writes to
* `~/.claude/config-audit/sessions/campaign-report.json` — on a machine where that
* directory does not exist yet (the FIRST run, exactly the case campaign-cli otherwise
* handles gracefully with `initialized:false`) the write throws ENOENT, and the command's
* own exit-code table then tells the user "the campaign ledger couldn't be read — it may
* be corrupt. Stop; do not attempt a write over a corrupt ledger." The ledger is not
* corrupt; it does not exist. The user is steered away from the one action that helps.
*
* 2. `posture.mjs` REPORTED A CRASH AS A PASSING GRADE. Its top-level catch set
* `process.exitCode = 1`, and every command in this plugin is instructed that "codes 0,
* 1, 2 are normal (PASS/WARNING/FAIL). Only 3 is a real error." So a fatal error was
* indistinguishable from a WARNING grade — measured live: posture exited 1, wrote no
* file, and the command would have gone on to Read a file that was never created.
* Every other scanner used 3; posture was the single outlier (1 of 14, measured).
*/
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawn } from 'node:child_process';
import { readFile, mkdtemp, mkdir, writeFile, readdir } from 'node:fs/promises';
import { existsSync } from 'node:fs';
import { resolve, dirname, join } from 'node:path';
import { tmpdir } from 'node:os';
import { fileURLToPath } from 'node:url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const ROOT = resolve(__dirname, '..', '..');
const SCANNERS_DIR = resolve(ROOT, 'scanners');
/**
* Every scanner that writes a `--output-file` payload, with argv that reaches the write.
* Determined empirically (each one was confirmed to produce the file when the parent
* directory already exists); `self-audit.mjs` is absent because it has no such flag.
*/
const WRITERS = [
{ cli: 'campaign-cli.mjs', argv: () => ['--ledger-file', LEDGER] },
{ cli: 'campaign-write-cli.mjs', argv: (d) => ['init', '--ledger-file', join(d, 'l.json')] },
{ cli: 'campaign-export-cli.mjs', argv: () => ['--repo', REPO, '--ledger-file', LEDGER] },
{ cli: 'knowledge-refresh-cli.mjs', argv: () => [] },
{ cli: 'optimize-lens-cli.mjs', argv: () => [ROOT] },
{ cli: 'token-hotspots-cli.mjs', argv: () => [ROOT] },
{ cli: 'drift-cli.mjs', argv: () => [ROOT] },
{ cli: 'fix-cli.mjs', argv: () => [ROOT] },
{ cli: 'manifest.mjs', argv: () => [ROOT] },
{ cli: 'posture.mjs', argv: () => [ROOT] },
{ cli: 'whats-active.mjs', argv: () => [ROOT] },
{ cli: 'plugin-health-scanner.mjs', argv: () => [ROOT] },
{ cli: 'scan-orchestrator.mjs', argv: () => [ROOT] },
];
let LEDGER;
let REPO;
function run(cli, argv) {
return new Promise((res) => {
const child = spawn(process.execPath, [resolve(SCANNERS_DIR, cli), ...argv], { cwd: ROOT });
child.stdout.on('data', () => {});
child.stderr.on('data', () => {});
child.on('close', (code) => res(code));
});
}
test('every --output-file writer creates its parent directory', async (t) => {
const base = await mkdtemp(join(tmpdir(), 'ca-outfile-'));
// A tracked repo + ledger so the campaign CLIs get past their own gates and reach the write.
REPO = join(base, 'repo');
await mkdir(REPO, { recursive: true });
LEDGER = join(base, 'ledger.json');
await writeFile(
LEDGER,
JSON.stringify({
schemaVersion: 1, createdDate: '2026-06-22', updatedDate: '2026-06-22',
repos: [{ path: REPO, name: 'repo', status: 'pending', sessionId: null, findingsBySeverity: null, tokens: null, updatedDate: '2026-06-22' }],
}, null, 2),
'utf-8',
);
const failures = [];
for (const { cli, argv } of WRITERS) {
const dir = join(base, `work-${cli}`);
await mkdir(dir, { recursive: true });
// The parent of the output file deliberately does not exist.
const out = join(dir, 'not', 'created', 'yet', 'payload.json');
const code = await run(cli, [...argv(dir), '--output-file', out]);
if (!existsSync(out)) failures.push(`${cli} (exit ${code})`);
}
assert.deepEqual(
failures,
[],
'These scanners crash instead of creating the output path. A command that writes its\n' +
'payload under ~/.claude/config-audit/sessions/ on a fresh machine gets ENOENT and\n' +
'reports it as a corrupt/unreadable input. Add mkdir(dirname(outputFile),\n' +
'{recursive:true}) before the write:\n ' + failures.join('\n '),
);
});
test('a fatal error exits 3 — never a code the commands treat as a normal grade', async () => {
const entries = (await readdir(SCANNERS_DIR)).filter((f) => f.endsWith('.mjs')).sort();
const offenders = [];
for (const file of entries) {
const src = await readFile(resolve(SCANNERS_DIR, file), 'utf-8');
const idx = src.indexOf('main().catch');
if (idx === -1) continue;
const block = src.slice(idx, idx + 400);
const m = block.match(/process\.exitCode\s*=\s*(\d+)/);
if (m && m[1] !== '3') offenders.push(`${file}: exitCode = ${m[1]}`);
}
assert.deepEqual(
offenders,
[],
'ux-rules tells every command that exit 0/1/2 are normal results (PASS/WARNING/FAIL)\n' +
'and only 3 is a real error. A fatal catch that sets anything else makes a crash\n' +
'indistinguishable from a grade, and the command goes on to Read a file that was\n' +
'never written:\n ' + offenders.join('\n '),
);
});