Dogfooding the four read commands (posture, tokens, manifest, whats-active)
surfaced four defect classes, all in the seam between what a command template
promises and what the scanner behind it actually does.
M-BUG-40, fifth arm: posture wrote four temp files it could never read back.
#49 closed the $$/cross-block class in four commands, but posture survived it —
and so did the guard written to prevent exactly this. The guard compared each
$$ path to the block that created it, so a path written once and then read via
prose had no second occurrence to flag. Measured live: written from PID 21614,
read attempted from PID 23772. The invariant is now blanket (no $$ in any temp
path), which also caught fix.md and feature-gap.md.
M-BUG-43: 6 of 7 scanners write their payload to stdout when --raw/--json is
set even when --output-file was given, and the templates redirected only
stderr. Measured: posture 255 182 B, whats-active 35 922 B, drift 28 316 B,
manifest 23 825 B, tokens 8 768 B. fix and feature-gap never read the file they
wrote, so both recovered one letter grade from a quarter-megabyte dump.
tokens swallowed --json and --with-telemetry-recipe: documented, never
threaded, so --json returned the humanized payload where the docs promise
byte-stable v5.0.0 output.
M-BUG-42: manifest's render contract asked for {load}; the payload carries
loadPattern, so the Load column rendered blank for all 96 rows.
Four new tests (1449 -> 1453), each verified red before the fix. The
render-contract test checks {field} names against a live payload from a
fixture, since a hardcoded key list would drift. Frozen v5.0.0 snapshots
untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGCk9o27eWo9uXLjkZTXEq
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).
The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:
- `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
- `$TMPFILE` read across blocks (tokens, manifest, whats-active,
plugin-health) — each command could not read the file it had just written
- `$GLOBAL_FLAG` across blocks (fix)
- `$TODAY` never assigned in any block (campaign), passing
`--reference-date ""` to a write CLI in six places
- three `$$` temp paths handed to the Read tool (fix), which expands neither
All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.
Also fixed, all confirmed against ground truth rather than inferred:
- `implement` printed a rollback ID it never captured (the timestamp lived only
inside a command substitution) — the one message a user reads after a bad run
- `plan` reported "No analysis results found" for valid sessions, because Read
was pointed at a glob it cannot expand; now uses Glob and verifies the
analysis report exists before spawning the agent
- five phase commands wrote state.yaml with two of four required fields; since
the agent writes all four, a follow-up write silently deleted the rest
- `implement` promised rollback deletes created files; rollback deliberately
leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
- `implement` claimed a score delta with no pre-change measurement
- `verifier-agent` was told to write a report it has no tool to write
- dead `Task` tool name in always-loaded rule context; planner-agent template
demonstrated the inline file content its own line 110 forbids
The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.
Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
DEL B chunk `interview` (+ discover/status/cleanup/help). Fasit written before
the run predicted 8 defects and refuted 4 candidates; all 8 confirmed, all 4
refutations held, and three predictions turned out too narrow.
- M-BUG-36: `drift --list` reached the command as 0 bytes. drift-cli accepted
--output-file but list mode ignored it, and the listing goes to stderr, which
the command discards per ux-rules rule 2. Fixing the caller alone would not
have helped.
- M-BUG-37: feature-gap's "Create backup" step ran fix-cli without --apply.
Dry-run is the default, so no backup existed (backupId: null) while the
command went on to edit config believing it could roll back.
- M-BUG-38: fix-cli told users to recover with scanners/rollback-cli.mjs, which
does not exist. Dead reference in the one message read after a bad fix.
- M-BUG-21 fourth arm: five templates carried literal [--global]/[--full-machine]
inside executable bash blocks. A bracketed placeholder does not start with a
dash, so every scanner's arg loop takes it as the scan target.
- interview and analyze never said which session they act on; interview could
rewind a finished session; cleanup interpolated an unvalidated id into rm -rf
(an empty id deletes every session); status advertised a `resume` command that
does not exist and documented an `all` argument it never parsed.
TDD: 9 red tests first, including a machine sweep for dead /config-audit
references and for bracketed flags in bash blocks. Suite 1432 -> 1441/0.
Frozen v5.0.0 snapshots untouched; --raw/--json contracts unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGvA1uUQn2hPBPMaCKK6x3
The B3a cache filter already exposes discovery.staleCacheVersions; surface them
as a finding so the user knows the superseded plugin versions are safe to delete.
Honesty (Verifiseringsplikt): the finding loads on ZERO turns, so it must NOT
read as a per-turn token cost. TOK normally humanizes to "Wasted tokens"; a new
per-finding category override ('plugin-cache-hygiene' -> "Dead config") plus a
dedicated humanizer translation ("Old plugin versions are sitting on disk (safe
to delete) ... cost zero tokens per turn ... housekeeping, not a performance
problem") keep the prose accurate instead of the generic "using more space"
default. evidence carries the explicit "zero live-context impact" note.
- token-hotspots: Pattern H emits CA-TOK (low) from discovery.staleCacheVersions
when stale versions exist (`--global`); silent otherwise.
- humanizer: CATEGORY_TO_IMPACT lets a finding's category override the
scanner-default impact label (raw `category` field unchanged -> --json/--raw
byte-stable). humanizer-data: honest static translation for the finding.
- Tests: finding fires/severity/category, lists stale keys + zero-impact note,
silent when none; humanizer override -> Dead config; honest translation locked.
- Docs: tokens command render note (disk-cleanup, not a token problem) +
--no-exclude-cache flag; README + CLAUDE.md rows (7 patterns, cache-aware).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Annotate every ranked TOK hotspot with the load-pattern triple
(loadPattern/survivesCompaction/derivationConfidence):
- hotspotLoadPattern() maps each discovery `type` → a deriveLoadPattern kind.
Rules reuse activeConfig.rules for precise `scoped` handling; claude-md maps
by scope. Two new deriveLoadPattern kinds back the rest: `command`
(on-demand — body loads on /invoke) and `harness-config` (external —
settings/keybindings/.mcp.json/hooks.json/plugin.json configure the CLI, not
the model context, so they cost no per-turn context tokens). Honest split:
the .mcp.json FILE is external; the MCP server's tool schemas are a separate
`always` hotspot.
Byte-stability — the opposite of B1's manifest. token-hotspots IS a byte-equal
SC-6/SC-7 CLI, and its hotspots ride inside scan-orchestrator + posture, so the
change touched SIX frozen-v5.0.0 comparisons across five test files. Resolved by
preserving the frozen baselines: a shared tests/helpers/strip-hotspot-load-pattern.mjs
strips the additive triple before each byte-equal compare (proves the original
schema is byte-identical). SC-5 default-output snapshots (scan-orchestrator +
token-hotspots) regenerated — diff reviewed as additive-only.
Tests 1008→1012. Self-audit A/A, scanner count unchanged at 13 (C bumps to 14).
Completes v5.6 B (B1 manifest + B2 token-hotspots).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fase 4 token-opt, Item 1 of gap-review NEXT STEP #2. The prompt-cache pattern corpus + TOK scanner were frozen at an "Opus 4.7" framing after CC shipped Opus 4.8 (default, 2.1.154) and Fable 5 (2.1.170). Model-era facts re-verified against the official changelog cache before editing.
The patterns are properties of prompt-caching, not of any model, so mechanic text is now model-neutral with a single "current default: Opus 4.8" anchor — preventing a re-freeze at the next model bump.
- rename knowledge/opus-4.7-patterns.md -> prompt-cache-patterns.md (git mv, history preserved); 6 reference sites updated
- TOK scanner: line-318 finding text (human-facing) made model-neutral; header + cache-prefix-scanner + CLI comments refreshed
- configuration-best-practices.md body + footnote 4.7 -> 4.8
- human-facing docs: commands/{tokens,help,manifest}.md, project CLAUDE.md, README, docs/scanner-internals.md
- gap-matrix row marked DONE; future Items 2/3 retargeted to new filename
Failing-test-first (Iron Law): +2 knowledge staleness guards (era-anchor + no-refreeze) +1 scanner assertion (no stale model anchor in finding text). Suite 853 -> 856 green; zero snapshot drift; self-audit A(97) PASS. CHANGELOG / v5 plan / ratified gap-plan keep historical opus-4.7-patterns refs (correct record of past state).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
Wave 5 Step 13. Threads the humanizer vocabulary through five audit/
analysis command templates and adds a shape test that locks the
structure in place.
- commands/posture.md, tokens.md, feature-gap.md (findings-renderers):
reference userImpactCategory/userActionLanguage/relevanceContext;
remove hardcoded A/B/C/D/F-to-prose tables (humanizer owns the
grade-context vocabulary now via the stderr scorecard headline).
- commands/manifest.md, whats-active.md (inventory CLIs): add --raw
pass-through for CLI-surface consistency. --raw is a no-op in these
CLIs, but the flag is threaded through so users get uniform behaviour.
- All five files: --raw flag parsed from $ARGUMENTS and passed verbatim
to the underlying scanner CLI when present.
tests/commands/group-a-shape.test.mjs (new, +5 tests, 767 → 772):
- structural: every file has a bash invocation block, Read tool
reference, and --raw/$ARGUMENTS plumbing
- findings-renderers only: at least one humanized field referenced;
no hardcoded "[grade] grade is..." prose tables
