Dogfooding `/config-audit` (the router) against the repo, fasit written before
any run (docs/router-fasit.local.md, untouched). Every claim below is measured
behaviour, not a reading of the source.
1. Bare `<target-path>` inside the step-3 fence is a shell REDIRECTION, not an
argument. Measured in zsh: both CLIs failed before starting, no output file
was written, and the echoed status was 1 — inside the band the router's own
gate calls "continue normally". Quoting makes an unsubstituted placeholder
reach argv, so it fails in the CLI where the exit code means something.
Swept the whole class: 30 sites across 12 further command files, since a
defect in one file is a class until the opposite is measured. New guard:
command-placeholder-shell-safety.test.mjs.
2. The orchestrator's exit code was discarded. Two commands on one line share a
single trailing `echo $?`, which reports only the last: measured, an
orchestrator exit 3 echoed as posture's 0, so the "3 -> stop" gate could
never fire. Both statuses are now captured and echoed.
3. "Running 12 configuration scanners" — the orchestrator registers 16. The new
test binds the narrated count to the registry so the next scanner added
cannot re-stale it silently.
4. The Area Breakdown table hardcoded 7 rows; posture emits 9 quality areas.
Token Efficiency (a B on this repo) and Plugin Hygiene never reached the
user. Rows added, and the row set is now asserted against lib/scoring.mjs.
Label aligned: "MCP Servers" -> "MCP", as posture emits it.
5. Step 6 rendered "the headline line from the humanized stderr scorecard" and
forbade deriving a replacement — while step 3 sent posture's stderr to
/dev/null, as UX rule 2 requires, and the prose is absent from the JSON
payload (measured). The slot could only be improvised. posture's stderr now
goes to a file in the session dir, as commands/posture.md already did; the
user still never sees raw scanner output.
Also: `grep -q -- "--raw"` matched any argument CONTAINING --raw (measured on
`--rawdog` and on a path with --raw in it) — anchored to whole arguments.
SCOPE_FLAGS renamed SCOPE_FLAG, since zsh does not word-split and the plural
invited the M-BUG-45 shape.
command-shell-state-shape.test.mjs only recognised line-initial assignments, so
it reported the idiomatic `node …; STATUS=$?` capture as never assigned. Widened
to assignments after a separator; verified it still fails on a real cross-block
reference before trusting it.
Suite 1477 -> 1483, frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDAwy1ZXRpZxht1wyCeSbF
Dogfooding `campaign` + `knowledge-refresh` against a throwaway ledger. Seven
defects, all found by running the commands as written and measuring, not by
reading them.
The headline pair only existed together. `knowledge-refresh` built
`STALE_AFTER="--stale-after 30"` and expanded it unquoted, trusting the shell to
split it in two. bash does; zsh — the macOS default, and what the Bash tool runs
here — does not. The CLI got one argv entry, matched no flag, and because it had
no unknown-flag branch, silently kept the 90-day default and reported "✓ All 14
register entries were re-verified within the last 90 days": a true-sounding
sentence about a threshold the user had just overridden. Fixing either half alone
leaves a silent wrong answer or a loud one; both are fixed, and a guard now
rejects any template that packs a flag and its value into one variable.
`knowledge-refresh` also read one register and wrote another: step 6 named an
unanchored `knowledge/best-practices.json` while the CLI reads
`${CLAUDE_PLUGIN_ROOT}/…`, which for an installed plugin is the cache. The
validation gate then ran the cached test against the cached register — green no
matter what was written. The two copies were byte-identical that day, which is
exactly why it was invisible.
`campaign` vouched for repos it could not read. `add /finnes/ikke` returned
`added` + exit 0; `refresh-tokens` then put the phantom in `swept[]` with a
0-token delta and left `skipped[]` empty, so the machine-wide bill claimed
coverage of three repos on a machine with two. Paths stay tracked — an unmounted
volume is a legitimate absence — but are reported as `addedUnverified`, and the
command names them.
Two class sweeps, both measured rather than assumed. `posture` was the single
scanner (1 of 14) whose fatal catch exited 1, which ux-rules defines as a normal
WARNING grade — a crash indistinguishable from a result. And all 13 payload
writers failed on a `--output-file` whose parent did not exist, which on a fresh
machine turned `campaign`'s first run into "the ledger may be corrupt"; they now
share `scanners/lib/write-output.mjs`.
Predicted breadth was too wide for the first time in five sessions: 6 of 8 CLIs
predicted to lack unknown-flag rejection, 4 measured. `drift` and `fix` already
reject them, via a construct the grep did not recognise — a grep matches an
implementation, the invariant is a behaviour. The sweep was rewritten to run each
CLI with a bogus flag and read the exit code.
Suite 1453 → 1469/0. Frozen snapshots untouched. `optimize-lens-cli` and
`token-hotspots-cli` share the unknown-flag defect and are deferred to the v5.14
argument-handling chunk with their positional-swallow arm; the count is recorded
in the guard rather than rounded down to zero.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012NHWjN8EnoxSqRvMTLK2NE
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).
The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:
- `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
- `$TMPFILE` read across blocks (tokens, manifest, whats-active,
plugin-health) — each command could not read the file it had just written
- `$GLOBAL_FLAG` across blocks (fix)
- `$TODAY` never assigned in any block (campaign), passing
`--reference-date ""` to a write CLI in six places
- three `$$` temp paths handed to the Read tool (fix), which expands neither
All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.
Also fixed, all confirmed against ground truth rather than inferred:
- `implement` printed a rollback ID it never captured (the timestamp lived only
inside a command substitution) — the one message a user reads after a bad run
- `plan` reported "No analysis results found" for valid sessions, because Read
was pointed at a glob it cannot expand; now uses Glob and verifies the
analysis report exists before spawning the agent
- five phase commands wrote state.yaml with two of four required fields; since
the agent writes all four, a follow-up write silently deleted the rest
- `implement` promised rollback deletes created files; rollback deliberately
leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
- `implement` claimed a score delta with no pre-change measurement
- `verifier-agent` was told to write a report it has no tool to write
- dead `Task` tool name in always-loaded rule context; planner-agent template
demonstrated the inline file content its own line 110 forbids
The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.
Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
Wires the user-facing surface of the machine-wide token roll-up (executable CLI
shipped in B2b-2, d664b70). The campaign report (Step 2) now renders rollUp.tokens:
the headline machine-wide always-loaded total split into the shared global layer
(paid once, every repo) + per-repo deltas, plus a ranked 'most expensive repos'
table. New mode 'refresh-tokens' (Step 6) documents the human-approved live sweep —
idempotent, skips unreadable repos, names any skipped so the bill's coverage stays
honest (Verifiseringsplikt).
Shapes documented carefully: rollUp.tokens.{sharedGlobal,perRepoDelta,machineWide}
are flat number maps; byRepo[] is the ranked {name,path,always,...} list. campaign-cli
already emitted rollUp.tokens (B2a) — no reader change; this is rendering only.
commands/campaign.md (73 lines) is the primary doc; README + CLAUDE.md campaign rows
get the matching one-line summary. Markdown-only → suite unchanged at 1198 green;
campaign command stays judgment-driven (not byte-stable).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes Block 4 (4b backlog + 4c export/execution). Asymmetric: plan
export is new testable code; execution is pure reuse of the existing
per-repo implement/rollback (no new execution machinery), per the plan's
"reuse existing backup/rollback".
Plan export ("planer følger arbeidsstedet"):
- scanners/lib/campaign-export.mjs (pure, now injected, 8 tests):
planExportPath(repo,sessionId) -> <repo>/docs/config-audit-plan-<sessionId>.md
(sessionId-keyed so same-day re-audits never collide);
buildPlanExportDocument({...,now}) -> provenance header + verbatim plan.
- scanners/campaign-export-cli.mjs (-cli, read-only by default, 10 tests):
--repo resolves the repo's linked session, reads its action-plan.md,
assembles the doc, emits {exportable,problems,targetPath,document}. Two
gates -> exit 1 advisory: no-session-linked / no-action-plan. Writes the
file ONLY under opt-in --write (byte-faithful copy; the LLM never re-types
a 200-line plan). --sessions-dir override for hermetic tests; exit 0/1/3.
Command: commands/campaign.md gains an `export <path>` mode (Step 6:
preview -> approve -> --write), then routes the user to the existing
/config-audit implement (backup + verify) + rollback + set-status
implemented. Nothing auto-written (Verifiseringsplikt).
Byte-stable: lib + -cli + command-doc only -> scanner count stays 15,
agents 7, commands 21 (export is a mode, not a new command), SC-5 +
backcompat suite untouched. suite 1150->1168. Block 4a (migrateLedger)
still deferred to the first breaking schema change.
Docs: CLAUDE.md section + badge 1150->1168/65->67 files; README badge +
campaign row + Testing prose (fixed stale 1055/59 -> true 1168/67).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the THIN machine-wide campaign surface (ledger + roll-up + status).
The write half of the campaign motor, mirroring how knowledge-refresh gates
register writes:
- scanners/campaign-write-cli.mjs (-cli → NOT a scanner; 11 tests): init/add/
set-status, each a thin wrapper over the invariant-enforcing lib transforms
(createLedger/addRepo/setRepoStatus) + saveLedger — path-normalization/dedup,
idempotent add, status-lifecycle guard and updatedDate bump never hand-rolled.
init refuses to clobber an existing/corrupt ledger (exit 1, file untouched);
add auto-inits + reports added vs skipped; set-status takes --findings/--session.
Deterministic: --reference-date is the only clock read, injected as `now`.
Exit 0=write, 1=advisory no-op, 3=error.
- commands/campaign.md (opus, no Web): thin orchestrator — always reports first
(read-only campaign-cli), then proposes init/add/set-status and invokes ONE
write-CLI subcommand only on explicit human approval (Verifiseringsplikt; never
hand-edits the JSON). add --discover finds git repos under a root to pick from.
Not byte-stable (own command, outside snapshot suite) like /config-audit optimize
+ knowledge-refresh. Both CLIs are -cli → scanner count stays 15, snapshot suite
untouched. commands 20→21, suite 1127→1138 (+11), test files 64→65.
Docs: CLAUDE.md §campaign-write-cli + command table + Testing badge; README
commands badge 20→21 + table row; help.md + router wiring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
