fix(commands): stop answering questions the caller did not ask

Dogfooding `campaign` + `knowledge-refresh` against a throwaway ledger. Seven
defects, all found by running the commands as written and measuring, not by
reading them.

The headline pair only existed together. `knowledge-refresh` built
`STALE_AFTER="--stale-after 30"` and expanded it unquoted, trusting the shell to
split it in two. bash does; zsh — the macOS default, and what the Bash tool runs
here — does not. The CLI got one argv entry, matched no flag, and because it had
no unknown-flag branch, silently kept the 90-day default and reported "✓ All 14
register entries were re-verified within the last 90 days": a true-sounding
sentence about a threshold the user had just overridden. Fixing either half alone
leaves a silent wrong answer or a loud one; both are fixed, and a guard now
rejects any template that packs a flag and its value into one variable.

`knowledge-refresh` also read one register and wrote another: step 6 named an
unanchored `knowledge/best-practices.json` while the CLI reads
`${CLAUDE_PLUGIN_ROOT}/…`, which for an installed plugin is the cache. The
validation gate then ran the cached test against the cached register — green no
matter what was written. The two copies were byte-identical that day, which is
exactly why it was invisible.

`campaign` vouched for repos it could not read. `add /finnes/ikke` returned
`added` + exit 0; `refresh-tokens` then put the phantom in `swept[]` with a
0-token delta and left `skipped[]` empty, so the machine-wide bill claimed
coverage of three repos on a machine with two. Paths stay tracked — an unmounted
volume is a legitimate absence — but are reported as `addedUnverified`, and the
command names them.

Two class sweeps, both measured rather than assumed. `posture` was the single
scanner (1 of 14) whose fatal catch exited 1, which ux-rules defines as a normal
WARNING grade — a crash indistinguishable from a result. And all 13 payload
writers failed on a `--output-file` whose parent did not exist, which on a fresh
machine turned `campaign`'s first run into "the ledger may be corrupt"; they now
share `scanners/lib/write-output.mjs`.

Predicted breadth was too wide for the first time in five sessions: 6 of 8 CLIs
predicted to lack unknown-flag rejection, 4 measured. `drift` and `fix` already
reject them, via a construct the grep did not recognise — a grep matches an
implementation, the invariant is a behaviour. The sweep was rewritten to run each
CLI with a bogus flag and read the exit code.

Suite 1453 → 1469/0. Frozen snapshots untouched. `optimize-lens-cli` and
`token-hotspots-cli` share the unknown-flag defect and are deferred to the v5.14
argument-handling chunk with their positional-swallow arm; the count is recorded
in the guard rather than rounded down to zero.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012NHWjN8EnoxSqRvMTLK2NE
This commit is contained in:
Kjell Tore Guttormsen 2026-08-01 21:26:39 +02:00
commit caea8aca23
23 changed files with 742 additions and 48 deletions

View file

@ -85,15 +85,21 @@ describe('campaign-write-cli — add', () => {
it('auto-initializes when no ledger exists, then adds repos (exit 0)', async () => {
const dir = newDir();
const file = join(dir, 'ledger.json');
// Real directories: since #51 `add` reports a path it cannot read under
// `addedUnverified` instead of vouching for it. This test is about auto-init and
// tracking, so its fixtures must be repos that actually exist.
const a = newDir();
const b = newDir();
assert.ok(!existsSync(file));
const { status, stdout } = runWrite([
'add', '/r/a', '/r/b', '--ledger-file', file, '--reference-date', NOW,
'add', a, b, '--ledger-file', file, '--reference-date', NOW,
]);
assert.equal(status, 0);
const out = JSON.parse(stdout);
assert.equal(out.action, 'add');
assert.equal(out.autoInitialized, true);
assert.deepEqual(out.added.sort(), [resolve('/r/a'), resolve('/r/b')].sort());
assert.deepEqual(out.added.sort(), [resolve(a), resolve(b)].sort());
assert.deepEqual(out.addedUnverified, []);
const { ledger, validation } = await readLedger(file);
assert.ok(validation.valid, validation.errors.join('; '));
@ -104,16 +110,19 @@ describe('campaign-write-cli — add', () => {
it('appends to an existing ledger and is idempotent on re-add (added vs skipped)', async () => {
const dir = newDir();
const file = join(dir, 'ledger.json');
runWrite(['add', '/r/a', '/r/b', '--ledger-file', file, '--reference-date', NOW]);
const a = newDir();
const b = newDir();
const c = newDir();
runWrite(['add', a, b, '--ledger-file', file, '--reference-date', NOW]);
const { status, stdout } = runWrite([
'add', '/r/a', '/r/c', '--ledger-file', file, '--reference-date', NOW,
'add', a, c, '--ledger-file', file, '--reference-date', NOW,
]);
assert.equal(status, 0);
const out = JSON.parse(stdout);
assert.equal(out.autoInitialized, false);
assert.deepEqual(out.added, [resolve('/r/c')]);
assert.deepEqual(out.skipped, [resolve('/r/a')]);
assert.deepEqual(out.added, [resolve(c)]);
assert.deepEqual(out.skipped, [resolve(a)]);
const { ledger } = await readLedger(file);
assert.equal(ledger.repos.length, 3);
@ -316,3 +325,69 @@ describe('campaign-write-cli — determinism + --output-file', () => {
assert.equal(status, 3);
});
});
// ── Session #51: honest coverage — the ledger must not vouch for repos it cannot read ──
//
// Dogfooding the campaign lifecycle put a path that does not exist into the ledger and
// swept it. Both halves lied, quietly:
//
// add → added: ["/finnes/absolutt/ikke/noe-repo"], exit 0, no warning.
// sweep → swept: [… , "/finnes/absolutt/ikke/noe-repo"], skipped: [], byRepo entry with 0
// tokens, reposWithTokens: 3 for a machine with 2 real repos.
//
// The root cause is that `readActiveConfig` resolves a path and every sub-reader tolerates
// ENOENT, so a missing repo yields an EMPTY config rather than an error — and the sweep's
// try/catch only routes THROWN errors to `skipped`. The command's own honesty clause ("If
// anything was skipped, name those repos plainly so the user knows the bill omits them")
// could therefore never fire. A token bill that silently counts phantom repos as 0 is worse
// than one that refuses to answer: it looks complete.
describe('campaign-write-cli — honest coverage for unreadable repos', () => {
it('add flags a path that does not exist instead of vouching for it', async () => {
const dir = newDir();
const file = join(dir, 'ledger.json');
const real = newDir();
const phantom = join(dir, 'no-such-repo');
const { status, stdout } = runWrite([
'add', real, phantom, '--ledger-file', file, '--reference-date', NOW,
]);
assert.equal(status, 0, 'a missing path is reported, not rejected — it may be unmounted');
const out = JSON.parse(stdout);
assert.deepEqual(out.added, [resolve(real)], 'only the readable repo is vouched for');
assert.deepEqual(
out.addedUnverified,
[resolve(phantom)],
'a path that does not exist must be reported separately so the command can say so',
);
// Still tracked — the campaign is a work register, and an unmounted volume is a
// legitimate reason for a path to be absent today and present tomorrow.
const { ledger } = await readLedger(file);
assert.equal(ledger.repos.length, 2);
});
it('refresh-tokens skips an unreadable repo instead of counting it as 0 tokens', async () => {
const dir = newDir();
const file = join(dir, 'ledger.json');
const phantom = join(dir, 'no-such-repo');
runWrite(['add', phantom, '--ledger-file', file, '--reference-date', NOW]);
const { status, stdout } = runWrite([
'refresh-tokens', '--ledger-file', file, '--reference-date', NOW,
]);
assert.equal(status, 0);
const out = JSON.parse(stdout);
assert.deepEqual(out.swept, [], 'a repo that cannot be read was never actually swept');
assert.equal(out.skipped.length, 1, 'it belongs in skipped, with a reason the user can act on');
assert.equal(out.skipped[0].path, resolve(phantom));
assert.match(out.skipped[0].reason, /not readable|does not exist|ENOENT/i);
assert.equal(
out.rollUp.tokens.reposWithTokens,
0,
'the machine-wide bill must not claim coverage of a repo it could not read',
);
});
});