Dogfooding `campaign` + `knowledge-refresh` against a throwaway ledger. Seven
defects, all found by running the commands as written and measuring, not by
reading them.
The headline pair only existed together. `knowledge-refresh` built
`STALE_AFTER="--stale-after 30"` and expanded it unquoted, trusting the shell to
split it in two. bash does; zsh — the macOS default, and what the Bash tool runs
here — does not. The CLI got one argv entry, matched no flag, and because it had
no unknown-flag branch, silently kept the 90-day default and reported "✓ All 14
register entries were re-verified within the last 90 days": a true-sounding
sentence about a threshold the user had just overridden. Fixing either half alone
leaves a silent wrong answer or a loud one; both are fixed, and a guard now
rejects any template that packs a flag and its value into one variable.
`knowledge-refresh` also read one register and wrote another: step 6 named an
unanchored `knowledge/best-practices.json` while the CLI reads
`${CLAUDE_PLUGIN_ROOT}/…`, which for an installed plugin is the cache. The
validation gate then ran the cached test against the cached register — green no
matter what was written. The two copies were byte-identical that day, which is
exactly why it was invisible.
`campaign` vouched for repos it could not read. `add /finnes/ikke` returned
`added` + exit 0; `refresh-tokens` then put the phantom in `swept[]` with a
0-token delta and left `skipped[]` empty, so the machine-wide bill claimed
coverage of three repos on a machine with two. Paths stay tracked — an unmounted
volume is a legitimate absence — but are reported as `addedUnverified`, and the
command names them.
Two class sweeps, both measured rather than assumed. `posture` was the single
scanner (1 of 14) whose fatal catch exited 1, which ux-rules defines as a normal
WARNING grade — a crash indistinguishable from a result. And all 13 payload
writers failed on a `--output-file` whose parent did not exist, which on a fresh
machine turned `campaign`'s first run into "the ledger may be corrupt"; they now
share `scanners/lib/write-output.mjs`.
Predicted breadth was too wide for the first time in five sessions: 6 of 8 CLIs
predicted to lack unknown-flag rejection, 4 measured. `drift` and `fix` already
reject them, via a construct the grep did not recognise — a grep matches an
implementation, the invariant is a behaviour. The sweep was rewritten to run each
CLI with a bogus flag and read the exit code.
Suite 1453 → 1469/0. Frozen snapshots untouched. `optimize-lens-cli` and
`token-hotspots-cli` share the unknown-flag defect and are deferred to the v5.14
argument-handling chunk with their positional-swallow arm; the count is recorded
in the guard rather than rounded down to zero.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012NHWjN8EnoxSqRvMTLK2NE
process.exit() terminates immediately, but Node writes stdout asynchronously
when stdout is a pipe — everything still buffered is dropped. scan-orchestrator
measured 246 854 bytes to a file against 65 536 to a pipe (131 072 on another
run; the cut point is a flush race), so every machine consumer that pipes the
envelope got truncated, unparseable JSON. The failure reads like a corrupt file,
not like a cut-off, which is what made it survive this long. Reported by
org-ops, whose census pipes our output.
Closes the class rather than the one CLI where it was visible. campaign-cli,
campaign-export-cli, campaign-write-cli, knowledge-refresh-cli, drift-cli and
fix-cli all exited the same way on their success paths and were green only
because their payloads fit the pipe buffer today; size is not correctness. All
38 sites across 14 files now set process.exitCode and return, which is the
pattern self-audit.mjs already used.
Two contracts needed care rather than substitution: fail() is a never-returns
guard at ~25 call sites, so it throws a CliUsageError the top-level catch
renders with the identical "Error: " prefix and exit code 3; the path guards
needed an explicit return so main() stops instead of running on. Exit codes and
stderr text are unchanged, and the frozen v5.0.0 snapshots are untouched.
The class sweep is landed as a test, not as fourteen edits — it caught one site
this commit had missed. Suite 1443/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8sS1DuDV6bUJcyumLwbvj
Every command so far asked an addition question — what to add, what to move,
what it costs. Nothing asked what is no longer earning its always-loaded rent.
This adds that axis as a fourth lensCheck on the existing hybrid motor rather
than a new scanner or a 22nd command: the measured payoff (~18% of one file)
justifies a mode, not machinery.
It is the only lens that proposes REMOVING config, so it carries a guarantee
the others don't need: a load-bearing block is never a candidate. Precision is
asymmetric — a missed dead line costs a few tokens per turn, a deleted one
costs a wrong remote or a broken script — so the floor is decided in code
(lib/floor-exclusion.mjs) before the opus judge sees anything, never in prose.
Granularity is the leaf block, with two structural exceptions: a paragraph
ending in ':' merges with the list it introduces, and an ordered list is a
contract whose steps inherit floor from any sibling. Unordered lists
deliberately do not inherit — a load-bearing bullet and a disposable one
routinely share a list, and container-reasoning is the error the hand-built
ground truth exists to catch.
Verified against that ground truth (built before any classifier existed), with
the comparison machine-checked rather than read by eye: zero load-bearing
blocks proposed, 11/18 deletable groups surfaced, ~756 tok ~ 18% of a ~4300
token file — inside the pre-registered band. The first run found five floor
violations the synthesized fixture missed; each got a structural rule and a
fixture shape so it cannot regress.
Three real bugs the dogfood run exposed, all now covered:
- JS \b is ASCII-only, so /\bunngå\b/ never matches — every Norwegian keyword
ending in æ/ø/å was silently dead.
- A bare word/word is not a path; "pros/cons" vetoed the largest deletable
block until PATH_RE was tightened to rooted paths and globs.
- "Mid-sentence" must key on a preceding lowercase letter; the loose version
read **bold labels:** and quoted openers as entities, costing 4 of 11 groups.
BP-SUB-001 is grounded entirely in the Anthropic steering blog already cited by
BP-MECH-001..004 and asserts nothing from the talk that motivated the feature —
no "80%", no ablation figure.
Suite 1365 -> 1382/0. Frozen v5.0.0 snapshots untouched; plain optimize output
byte-identical on identical input (--subtract adds keys only when passed).
knowledge-refresh-cli's reference date moved to 2026-08-01: its premise that
every seed entry was verified 2026-06-20 expired when BP-SUB-001 got a genuine
verification date, and backdating the entry to fit the test would have been a
lie about when its source was checked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RW2haJXbxZpKivKHseSXNh
The optimize lens CLI fed its precision-gate agent every CLAUDE.md that discovery
returned, including the 256 files under ~/.claude/plugins/ — vendored plugin
CLAUDE.md across every cached version (7 config-audit, 6 ms-ai-architect, 5 okr,
...) plus their bundled tests/fixtures and examples. Running `optimize --global`
on this machine produced 454 candidates across 92 "files", ~250 of them sourced
from plugin-internal files a user cannot act on (the plugin overwrites them on
update). Same class as M-BUG-2: plugin-bundled config is not the user's cascade.
Second defect: candidates were keyed by `relPath || absPath`, but relPath
collides across scopes — a repo-root `CLAUDE.md` and the user-global
`~/.claude/CLAUDE.md` both relPath to `CLAUDE.md`. The two files that actually
matter were merged into one indistinguishable bucket (21 candidates), the agent's
Read(file) would resolve the wrong one, and cache-file relPaths were not readable
relative to cwd at all.
Fix (lens-CLI-local, surgical):
- Filter isPluginBundled (absPath under `.claude/plugins/`) from discovery for
BOTH halves of the motor (candidate loop + the OPT scanner, which reads
discovery.files directly). Drops vendored files regardless of active/stale
version, so excludeCache is unnecessary here.
- Key each candidate by absPath: unique + readable.
No change to file-discovery.mjs or the OPT scanner, so their byte-stable
snapshots are untouched.
Suite 1348/0 (+4: candidate scoping, real-config survives, deterministic scoping,
absolute-path identity). Frozen v5.0.0 + SC-5 snapshots untouched (the lens CLI
has no snapshot; the command is agent-driven, not byte-stable). Dogfood ~/.claude
`optimize --global`: candidates 454->45, deterministic 2->0 (both were stale
plugin-cache copies), distinct files 92->11, repo vs user-global now distinct
(12 + 9 = 21). Residual 45 includes config-audit's own tests/fixtures CLAUDE.md
(repo-specific dogfooding artifact, not a general bug — left alone).
The recall+precision halves of the CA-OPT hybrid motor for the three
mechanism-fit cases the deterministic OPT scanner (2a) deliberately skips:
lifecycle→hook (BP-MECH-001), unscoped path-specific→rule (BP-MECH-002),
absolute "never"→permission (BP-MECH-004).
New:
- scanners/lib/lens-prefilter.mjs — cheap, recall-oriented line scan of the
CLAUDE.md body; detector names mirror the register lensCheck fields; skips
fenced code, gates the path class on an instruction verb. Pure + 13 tests.
- scanners/optimize-lens-cli.mjs — discovery + OPT scanner + pre-filter; attaches
only the CONFIRMED register entry to each candidate (unverifiable → dropped,
Verifiseringsplikt); emits {deterministic, candidates, register, counts}.
- agents/optimization-lens-agent.md — opus precision gate (7th agent, orange):
reads the real CLAUDE.md, drops low-confidence candidates, keeps only genuine
opportunities, cites register id + source.
- commands/optimize.md — /config-audit optimize orchestrates pre-filter→agent→report.
Agent-driven → deliberately NOT byte-stable (own command, outside the snapshot
suite). No new orchestrated scanner → scanner count stays 15. Counts: agents
6→7, commands 18→19, suite 1055→1068. Self-audit A/A unchanged, readmeCheck
passed (clean HOME). Plan: docs/v5.7-optimization-lens-plan.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
