Dogfooding `/config-audit` (the router) against the repo, fasit written before
any run (docs/router-fasit.local.md, untouched). Every claim below is measured
behaviour, not a reading of the source.
1. Bare `<target-path>` inside the step-3 fence is a shell REDIRECTION, not an
argument. Measured in zsh: both CLIs failed before starting, no output file
was written, and the echoed status was 1 — inside the band the router's own
gate calls "continue normally". Quoting makes an unsubstituted placeholder
reach argv, so it fails in the CLI where the exit code means something.
Swept the whole class: 30 sites across 12 further command files, since a
defect in one file is a class until the opposite is measured. New guard:
command-placeholder-shell-safety.test.mjs.
2. The orchestrator's exit code was discarded. Two commands on one line share a
single trailing `echo $?`, which reports only the last: measured, an
orchestrator exit 3 echoed as posture's 0, so the "3 -> stop" gate could
never fire. Both statuses are now captured and echoed.
3. "Running 12 configuration scanners" — the orchestrator registers 16. The new
test binds the narrated count to the registry so the next scanner added
cannot re-stale it silently.
4. The Area Breakdown table hardcoded 7 rows; posture emits 9 quality areas.
Token Efficiency (a B on this repo) and Plugin Hygiene never reached the
user. Rows added, and the row set is now asserted against lib/scoring.mjs.
Label aligned: "MCP Servers" -> "MCP", as posture emits it.
5. Step 6 rendered "the headline line from the humanized stderr scorecard" and
forbade deriving a replacement — while step 3 sent posture's stderr to
/dev/null, as UX rule 2 requires, and the prose is absent from the JSON
payload (measured). The slot could only be improvised. posture's stderr now
goes to a file in the session dir, as commands/posture.md already did; the
user still never sees raw scanner output.
Also: `grep -q -- "--raw"` matched any argument CONTAINING --raw (measured on
`--rawdog` and on a path with --raw in it) — anchored to whole arguments.
SCOPE_FLAGS renamed SCOPE_FLAG, since zsh does not word-split and the plural
invited the M-BUG-45 shape.
command-shell-state-shape.test.mjs only recognised line-initial assignments, so
it reported the idiomatic `node …; STATUS=$?` capture as never assigned. Widened
to assignments after a separator; verified it still fails on a real cross-block
reference before trusting it.
Suite 1477 -> 1483, frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDAwy1ZXRpZxht1wyCeSbF
Dogfooding the four read commands (posture, tokens, manifest, whats-active)
surfaced four defect classes, all in the seam between what a command template
promises and what the scanner behind it actually does.
M-BUG-40, fifth arm: posture wrote four temp files it could never read back.
#49 closed the $$/cross-block class in four commands, but posture survived it —
and so did the guard written to prevent exactly this. The guard compared each
$$ path to the block that created it, so a path written once and then read via
prose had no second occurrence to flag. Measured live: written from PID 21614,
read attempted from PID 23772. The invariant is now blanket (no $$ in any temp
path), which also caught fix.md and feature-gap.md.
M-BUG-43: 6 of 7 scanners write their payload to stdout when --raw/--json is
set even when --output-file was given, and the templates redirected only
stderr. Measured: posture 255 182 B, whats-active 35 922 B, drift 28 316 B,
manifest 23 825 B, tokens 8 768 B. fix and feature-gap never read the file they
wrote, so both recovered one letter grade from a quarter-megabyte dump.
tokens swallowed --json and --with-telemetry-recipe: documented, never
threaded, so --json returned the humanized payload where the docs promise
byte-stable v5.0.0 output.
M-BUG-42: manifest's render contract asked for {load}; the payload carries
loadPattern, so the Load column rendered blank for all 96 rows.
Four new tests (1449 -> 1453), each verified red before the fix. The
render-contract test checks {field} names against a live payload from a
fixture, since a hardcoded key list would drift. Frozen v5.0.0 snapshots
untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGCk9o27eWo9uXLjkZTXEq
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).
The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:
- `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
- `$TMPFILE` read across blocks (tokens, manifest, whats-active,
plugin-health) — each command could not read the file it had just written
- `$GLOBAL_FLAG` across blocks (fix)
- `$TODAY` never assigned in any block (campaign), passing
`--reference-date ""` to a write CLI in six places
- three `$$` temp paths handed to the Read tool (fix), which expands neither
All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.
Also fixed, all confirmed against ground truth rather than inferred:
- `implement` printed a rollback ID it never captured (the timestamp lived only
inside a command substitution) — the one message a user reads after a bad run
- `plan` reported "No analysis results found" for valid sessions, because Read
was pointed at a glob it cannot expand; now uses Glob and verifies the
analysis report exists before spawning the agent
- five phase commands wrote state.yaml with two of four required fields; since
the agent writes all four, a follow-up write silently deleted the rest
- `implement` promised rollback deletes created files; rollback deliberately
leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
- `implement` claimed a score delta with no pre-change measurement
- `verifier-agent` was told to write a report it has no tool to write
- dead `Task` tool name in always-loaded rule context; planner-agent template
demonstrated the inline file content its own line 110 forbids
The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.
Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
DEL B chunk `interview` (+ discover/status/cleanup/help). Fasit written before
the run predicted 8 defects and refuted 4 candidates; all 8 confirmed, all 4
refutations held, and three predictions turned out too narrow.
- M-BUG-36: `drift --list` reached the command as 0 bytes. drift-cli accepted
--output-file but list mode ignored it, and the listing goes to stderr, which
the command discards per ux-rules rule 2. Fixing the caller alone would not
have helped.
- M-BUG-37: feature-gap's "Create backup" step ran fix-cli without --apply.
Dry-run is the default, so no backup existed (backupId: null) while the
command went on to edit config believing it could roll back.
- M-BUG-38: fix-cli told users to recover with scanners/rollback-cli.mjs, which
does not exist. Dead reference in the one message read after a bad fix.
- M-BUG-21 fourth arm: five templates carried literal [--global]/[--full-machine]
inside executable bash blocks. A bracketed placeholder does not start with a
dash, so every scanner's arg loop takes it as the scan target.
- interview and analyze never said which session they act on; interview could
rewind a finished session; cleanup interpolated an unvalidated id into rm -rf
(an empty id deletes every session); status advertised a `resume` command that
does not exist and documented an `all` argument it never parsed.
TDD: 9 red tests first, including a machine sweep for dead /config-audit
references and for bracketed flags in bash blocks. Suite 1432 -> 1441/0.
Frozen v5.0.0 snapshots untouched; --raw/--json contracts unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGvA1uUQn2hPBPMaCKK6x3
The B3a cache filter already exposes discovery.staleCacheVersions; surface them
as a finding so the user knows the superseded plugin versions are safe to delete.
Honesty (Verifiseringsplikt): the finding loads on ZERO turns, so it must NOT
read as a per-turn token cost. TOK normally humanizes to "Wasted tokens"; a new
per-finding category override ('plugin-cache-hygiene' -> "Dead config") plus a
dedicated humanizer translation ("Old plugin versions are sitting on disk (safe
to delete) ... cost zero tokens per turn ... housekeeping, not a performance
problem") keep the prose accurate instead of the generic "using more space"
default. evidence carries the explicit "zero live-context impact" note.
- token-hotspots: Pattern H emits CA-TOK (low) from discovery.staleCacheVersions
when stale versions exist (`--global`); silent otherwise.
- humanizer: CATEGORY_TO_IMPACT lets a finding's category override the
scanner-default impact label (raw `category` field unchanged -> --json/--raw
byte-stable). humanizer-data: honest static translation for the finding.
- Tests: finding fires/severity/category, lists stale keys + zero-impact note,
silent when none; humanizer override -> Dead config; honest translation locked.
- Docs: tokens command render note (disk-cleanup, not a token problem) +
--no-exclude-cache flag; README + CLAUDE.md rows (7 patterns, cache-aware).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Annotate every ranked TOK hotspot with the load-pattern triple
(loadPattern/survivesCompaction/derivationConfidence):
- hotspotLoadPattern() maps each discovery `type` → a deriveLoadPattern kind.
Rules reuse activeConfig.rules for precise `scoped` handling; claude-md maps
by scope. Two new deriveLoadPattern kinds back the rest: `command`
(on-demand — body loads on /invoke) and `harness-config` (external —
settings/keybindings/.mcp.json/hooks.json/plugin.json configure the CLI, not
the model context, so they cost no per-turn context tokens). Honest split:
the .mcp.json FILE is external; the MCP server's tool schemas are a separate
`always` hotspot.
Byte-stability — the opposite of B1's manifest. token-hotspots IS a byte-equal
SC-6/SC-7 CLI, and its hotspots ride inside scan-orchestrator + posture, so the
change touched SIX frozen-v5.0.0 comparisons across five test files. Resolved by
preserving the frozen baselines: a shared tests/helpers/strip-hotspot-load-pattern.mjs
strips the additive triple before each byte-equal compare (proves the original
schema is byte-identical). SC-5 default-output snapshots (scan-orchestrator +
token-hotspots) regenerated — diff reviewed as additive-only.
Tests 1008→1012. Self-audit A/A, scanner count unchanged at 13 (C bumps to 14).
Completes v5.6 B (B1 manifest + B2 token-hotspots).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fase 4 token-opt, Item 1 of gap-review NEXT STEP #2. The prompt-cache pattern corpus + TOK scanner were frozen at an "Opus 4.7" framing after CC shipped Opus 4.8 (default, 2.1.154) and Fable 5 (2.1.170). Model-era facts re-verified against the official changelog cache before editing.
The patterns are properties of prompt-caching, not of any model, so mechanic text is now model-neutral with a single "current default: Opus 4.8" anchor — preventing a re-freeze at the next model bump.
- rename knowledge/opus-4.7-patterns.md -> prompt-cache-patterns.md (git mv, history preserved); 6 reference sites updated
- TOK scanner: line-318 finding text (human-facing) made model-neutral; header + cache-prefix-scanner + CLI comments refreshed
- configuration-best-practices.md body + footnote 4.7 -> 4.8
- human-facing docs: commands/{tokens,help,manifest}.md, project CLAUDE.md, README, docs/scanner-internals.md
- gap-matrix row marked DONE; future Items 2/3 retargeted to new filename
Failing-test-first (Iron Law): +2 knowledge staleness guards (era-anchor + no-refreeze) +1 scanner assertion (no stale model anchor in finding text). Suite 853 -> 856 green; zero snapshot drift; self-audit A(97) PASS. CHANGELOG / v5 plan / ratified gap-plan keep historical opus-4.7-patterns refs (correct record of past state).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
Wave 5 Step 13. Threads the humanizer vocabulary through five audit/
analysis command templates and adds a shape test that locks the
structure in place.
- commands/posture.md, tokens.md, feature-gap.md (findings-renderers):
reference userImpactCategory/userActionLanguage/relevanceContext;
remove hardcoded A/B/C/D/F-to-prose tables (humanizer owns the
grade-context vocabulary now via the stderr scorecard headline).
- commands/manifest.md, whats-active.md (inventory CLIs): add --raw
pass-through for CLI-surface consistency. --raw is a no-op in these
CLIs, but the flag is threaded through so users get uniform behaviour.
- All five files: --raw flag parsed from $ARGUMENTS and passed verbatim
to the underlying scanner CLI when present.
tests/commands/group-a-shape.test.mjs (new, +5 tests, 767 → 772):
- structural: every file has a bash invocation block, Read tool
reference, and --raw/$ARGUMENTS plumbing
- findings-renderers only: at least one humanized field referenced;
no hardcoded "[grade] grade is..." prose tables
