config-audit/tests/helpers
Kjell Tore Guttormsen 30c78aeda0 fix(scanners): the command layer's argv is now checked against the CLI that receives it
A command template is a caller with no compiler behind it. It names a scanner and
an argv; nothing checked that the scanner still accepts them. M-BUG-45 measured
what that costs: `--stale-after` arrived malformed, was ignored, and the command
reported "all 14 entries re-verified within the last 90 days" about a threshold
the user had just overridden.

The new guard builds the argv from each template's OWN text (#63 — a hand-typed
call is a path no user takes), reading all three forms a flag appears in,
including the comment-only `GLOBAL_FLAG=""  # --global`; that third form is the
one that dies unobserved, since the default path leaves the variable empty.
Measured: 38 invocations, 54 (CLI, flag) pairs, 15 CLIs, 0 dead scanner paths.

Two premises in the plan text were falsified by measuring:

  - "the flag exists in the CLI's BOOL_FLAGS/VALUE_FLAGS" — only 3 of 34 scanner
    files declare such a surface. The contract is checked on BEHAVIOUR instead:
    run the CLI, ask whether it calls the flag unknown.
  - `--full-machine` was predicted dead on `posture`. It is live. The fasit was
    wrong, not the code.

What the measurement found instead: `campaign-export-cli` was the only one of the
fifteen without the shared `requireValidArgs` gate. Its hand-rolled chain guards
every value branch with `argv[i + 1] !== undefined`, so a trailing `--repo` fell
past all of them to the `startsWith('--')` catch-all and was reported as an
unknown flag — about the flag the CLI itself requires. Classification of "value
flag, no value" across all fifteen: 14 correct, 1 wrong. It now uses ARG_SPEC +
requireValidArgs like the other twelve; valid argv reaches the existing loop
byte-for-byte unchanged. Special-casing it in the test would have rebuilt, in
test code, the prose exception Q1 deleted.

And what the guard itself got wrong, which is worse than what it was looking for:
probing a flag means RUNNING the CLI, and some flags are writers. Its first run
let `drift-cli --save` default its target to the working directory and overwrite
the operator's real ~/.config-audit/baselines/default.json — an ungated write
outside the repo, produced by the guard whose whole subject is ungated writes
outside the repo. Every probe now runs under hermeticEnv() with its own empty
cwd, and the cwd is asserted empty afterwards. Isolation that is only a
convention is not isolation. Side effect: 65s -> 13s, because a hermetic HOME
stops every probe from enumerating ~/.claude.

All six arms seen RED against their own defect, twice — including the ORIGINAL
class (remove --approve-scope from fix-cli) and the plan's own verification
(delete the write-scope-cli line from a template). The non-emptiness arm is
derived from the tree, not pinned to a count that would only be a drift point.

Suite 1707 -> 1724, frozen v5.0.0 + default-output snapshots 0 changed files.

Not fixed here, found while verifying and pre-existing at 749b710: the suite was
NOT green on HEAD. output-file-robustness fails on drift-cli, root cause
diff-engine.mjs:194 — `m.from.severity` where `m.from` is undefined in the moved
section of the drift report. It crashes after the scan, in formatting, so the
CLI exits 3 with no output file. Its own chunk, not this one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pj6UoTi6iPsAB2B2j6EZ1k
2026-08-12 21:58:38 +02:00
..
command-invocations.mjs fix(scanners): the command layer's argv is now checked against the CLI that receives it 2026-08-12 21:58:38 +02:00
hermetic-home.mjs test(snapshots): make byte/snapshot tests hermetic + re-seed baseline 2026-06-18 12:26:00 +02:00
mask-finding-ids.mjs fix(scanners)!: a finding ID names the check, not the emission (M-BUG-28) 2026-08-09 23:26:36 +02:00
scanner-run.mjs fix(scanners)!: a finding ID names the check, not the emission (M-BUG-28) 2026-08-09 23:26:36 +02:00
strip-added-scanner.mjs feat(scanner): add AGT agent-listing always-loaded budget finding (v5.9 B1) 2026-06-23 15:02:59 +02:00
strip-hotspot-load-pattern.mjs feat(tok,acr): v5.6 B2 — load-pattern column in token-hotspots 2026-06-20 20:21:01 +02:00
strip-retired-gap.mjs fix(scanners)!: a finding ID names the check, not the emission (M-BUG-28) 2026-08-09 23:26:36 +02:00