Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).
The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:
- `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
- `$TMPFILE` read across blocks (tokens, manifest, whats-active,
plugin-health) — each command could not read the file it had just written
- `$GLOBAL_FLAG` across blocks (fix)
- `$TODAY` never assigned in any block (campaign), passing
`--reference-date ""` to a write CLI in six places
- three `$$` temp paths handed to the Read tool (fix), which expands neither
All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.
Also fixed, all confirmed against ground truth rather than inferred:
- `implement` printed a rollback ID it never captured (the timestamp lived only
inside a command substitution) — the one message a user reads after a bad run
- `plan` reported "No analysis results found" for valid sessions, because Read
was pointed at a glob it cannot expand; now uses Glob and verifies the
analysis report exists before spawning the agent
- five phase commands wrote state.yaml with two of four required fields; since
the agent writes all four, a follow-up write silently deleted the rest
- `implement` promised rollback deletes created files; rollback deliberately
leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
- `implement` claimed a score delta with no pre-change measurement
- `verifier-agent` was told to write a report it has no tool to write
- dead `Task` tool name in always-loaded rule context; planner-agent template
demonstrated the inline file content its own line 110 forbids
The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.
Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
5.1 KiB
| name | description | argument-hint | allowed-tools | model |
|---|---|---|---|---|
| config-audit:manifest | Show ranked token-source manifest — every CLAUDE.md, rule, agent, skill, output style, MCP server, and hook ordered DESC by estimated tokens, each tagged with its load pattern (always-loaded vs on-demand vs external), plus an always-loaded subtotal | [path] [--json] | Read, Bash | sonnet |
Config-Audit: Manifest
Produce a ranked, single-table view of every token source loaded for a given repo path. Where whats-active shows separate tables per category, manifest collapses everything into one ordered list — making it easy to see what's costing the most regardless of category.
Every source is tagged with its load pattern, derived from the published Claude Code loading model:
- always — enters context every turn before you type (project/user CLAUDE.md, unscoped rules, agents, output styles, MCP tool schemas). This is the cost that matters most: it is paid on every request.
- on-demand — loaded only when needed (skill bodies on invoke, path-scoped rules on a matching file read).
- external — runs outside the context window entirely (hooks).
The always-loaded subtotal is the headline number. Sources are component-level (a plugin contributes via its skills/rules/agents/output styles/hooks/MCP, each listed once — there is no coarse "plugin" roll-up, which would double-count).
UX Rules (MANDATORY — from .claude/rules/ux-rules.md)
- Never show raw JSON or stderr output. Always use
--output-file+2>/dev/null. - Narrate before acting. Tell the user what you're about to do.
- Read, don't dump. Read the JSON file and render a formatted table.
- End with context-sensitive next steps.
Implementation
Step 1: Parse $ARGUMENTS
First non-flag argument is the path (default .). Recognized flags:
--json— emit raw JSON instead of the rendered table.--raw— pass-through to the scanner; accepted for CLI surface consistency with the other config-audit commands. The manifest CLI is data-table only (no findings prose), so--rawis a no-op here, but the flag is still threaded through so users get uniform behaviour across--raw.
Step 2: Run the CLI silently
Tell the user: "Building token-source manifest for <path>..."
RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/manifest.mjs <path> --output-file /tmp/config-audit-manifest.json $RAW_FLAG 2>/dev/null; echo $?
Exit code handling:
0→ continue3→ tell user: "Couldn't read configuration. Check that the path exists and is a directory." Stop.
Step 3: If --json was requested, cat the file and stop
cat /tmp/config-audit-manifest.json
Do NOT render the table in JSON mode.
Step 4: Read JSON and render
Use the Read tool on /tmp/config-audit-manifest.json. Extract meta.repoPath, total, summary, and sources[]. Lead with the always-loaded subtotal (the headline), then render the top 20 sources (or fewer if the manifest is shorter):
**Token-source manifest for `<repoPath>`** — ~{total} tokens total
- 🔴 **~{summary.always.tokens} tokens enter context every turn** before you type ({summary.always.count} always-loaded sources)
- 🟡 ~{summary.onDemand.tokens} tokens on-demand ({summary.onDemand.count} sources — loaded only when invoked / matched)
- ⚪ ~{summary.external.tokens} tokens external ({summary.external.count} sources — hooks, run outside context)
| Rank | Kind | Name | Source | Tokens | Load |
|------|------|------|--------|--------|------|
| 1 | {kind} | `<name>` | {source} | ~{estimated_tokens} | {load} |
| ... | ... | ... | ... | ... | ... |
_Load column: **always** / **on-demand** / **external**. Append `°` when `derivationConfidence` is `inferred` (no primary-doc row pins it exactly)._
_Estimates assume ~4 chars/token (Claude ballpark). Real token count varies ±15%._
If sources.length > 20, follow the table with: "Showing top 20 of {N} sources. Run with --json to see the full list."
When narrating, prioritize the always-loaded subtotal: a large always source is worse than an equally large on-demand one, because it is paid on every request. Call out any single always-loaded source that dwarfs the rest.
Step 5: Suggest next steps
**Next steps:**
- `/config-audit tokens` — prompt-cache token-hotspot patterns (cache-breaking, redundant perms, deep imports, MCP budget)
- `/config-audit whats-active` — same data grouped by category, with disable suggestions
- `/config-audit feature-gap` — what *could* improve here, grouped by impact
Tone (key on the always-loaded subtotal — the every-turn cost — not the grand total):
- High always-loaded (>40k): empathetic — "That's a heavy per-turn cost; it taxes every request before you've typed a word. Look at the largest always-loaded sources first."
- Moderate (10–40k): neutral — "Reasonable. Skim the top always-loaded sources to see if anything is unexpectedly large."
- Low (<10k): encouraging — "Tight setup. The model has plenty of room for the actual work each turn."