config-audit/commands/manifest.md
Kjell Tore Guttormsen 09f817977c fix(commands): stop assuming shell state survives between blocks
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).

The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:

  - `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
  - `$TMPFILE` read across blocks (tokens, manifest, whats-active,
    plugin-health) — each command could not read the file it had just written
  - `$GLOBAL_FLAG` across blocks (fix)
  - `$TODAY` never assigned in any block (campaign), passing
    `--reference-date ""` to a write CLI in six places
  - three `$$` temp paths handed to the Read tool (fix), which expands neither

All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.

Also fixed, all confirmed against ground truth rather than inferred:

  - `implement` printed a rollback ID it never captured (the timestamp lived only
    inside a command substitution) — the one message a user reads after a bad run
  - `plan` reported "No analysis results found" for valid sessions, because Read
    was pointed at a glob it cannot expand; now uses Glob and verifies the
    analysis report exists before spawning the agent
  - five phase commands wrote state.yaml with two of four required fields; since
    the agent writes all four, a follow-up write silently deleted the rest
  - `implement` promised rollback deletes created files; rollback deliberately
    leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
  - `implement` claimed a score delta with no pre-change measurement
  - `verifier-agent` was told to write a report it has no tool to write
  - dead `Task` tool name in always-loaded rule context; planner-agent template
    demonstrated the inline file content its own line 110 forbids

The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.

Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
2026-08-01 20:12:17 +02:00

95 lines
5.1 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
name: config-audit:manifest
description: Show ranked token-source manifest — every CLAUDE.md, rule, agent, skill, output style, MCP server, and hook ordered DESC by estimated tokens, each tagged with its load pattern (always-loaded vs on-demand vs external), plus an always-loaded subtotal
argument-hint: "[path] [--json]"
allowed-tools: Read, Bash
model: sonnet
---
# Config-Audit: Manifest
Produce a ranked, single-table view of every token source loaded for a given repo path. Where `whats-active` shows separate tables per category, `manifest` collapses everything into one ordered list — making it easy to see what's costing the most regardless of category.
Every source is tagged with its **load pattern**, derived from the published Claude Code loading model:
- **always** — enters context every turn before you type (project/user CLAUDE.md, unscoped rules, agents, output styles, MCP tool schemas). This is the cost that matters most: it is paid on *every* request.
- **on-demand** — loaded only when needed (skill bodies on invoke, path-scoped rules on a matching file read).
- **external** — runs outside the context window entirely (hooks).
The **always-loaded subtotal** is the headline number. Sources are component-level (a plugin contributes via its skills/rules/agents/output styles/hooks/MCP, each listed once — there is no coarse "plugin" roll-up, which would double-count).
## UX Rules (MANDATORY — from `.claude/rules/ux-rules.md`)
1. **Never show raw JSON or stderr output.** Always use `--output-file` + `2>/dev/null`.
2. **Narrate before acting.** Tell the user what you're about to do.
3. **Read, don't dump.** Read the JSON file and render a formatted table.
4. **End with context-sensitive next steps.**
## Implementation
### Step 1: Parse `$ARGUMENTS`
First non-flag argument is the path (default `.`). Recognized flags:
- `--json` — emit raw JSON instead of the rendered table.
- `--raw` — pass-through to the scanner; accepted for CLI surface consistency with the other config-audit commands. The manifest CLI is data-table only (no findings prose), so `--raw` is a no-op here, but the flag is still threaded through so users get uniform behaviour across `--raw`.
### Step 2: Run the CLI silently
Tell the user: **"Building token-source manifest for `<path>`..."**
```bash
RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/manifest.mjs <path> --output-file /tmp/config-audit-manifest.json $RAW_FLAG 2>/dev/null; echo $?
```
**Exit code handling:**
- `0` → continue
- `3` → tell user: "Couldn't read configuration. Check that the path exists and is a directory." Stop.
### Step 3: If `--json` was requested, cat the file and stop
```bash
cat /tmp/config-audit-manifest.json
```
Do NOT render the table in JSON mode.
### Step 4: Read JSON and render
Use the Read tool on `/tmp/config-audit-manifest.json`. Extract `meta.repoPath`, `total`, `summary`, and `sources[]`. Lead with the **always-loaded subtotal** (the headline), then render the top 20 sources (or fewer if the manifest is shorter):
```markdown
**Token-source manifest for `<repoPath>`** — ~{total} tokens total
- 🔴 **~{summary.always.tokens} tokens enter context every turn** before you type ({summary.always.count} always-loaded sources)
- 🟡 ~{summary.onDemand.tokens} tokens on-demand ({summary.onDemand.count} sources — loaded only when invoked / matched)
- ⚪ ~{summary.external.tokens} tokens external ({summary.external.count} sources — hooks, run outside context)
| Rank | Kind | Name | Source | Tokens | Load |
|------|------|------|--------|--------|------|
| 1 | {kind} | `<name>` | {source} | ~{estimated_tokens} | {load} |
| ... | ... | ... | ... | ... | ... |
_Load column: **always** / **on-demand** / **external**. Append `°` when `derivationConfidence` is `inferred` (no primary-doc row pins it exactly)._
_Estimates assume ~4 chars/token (Claude ballpark). Real token count varies ±15%._
```
If `sources.length > 20`, follow the table with: _"Showing top 20 of {N} sources. Run with `--json` to see the full list."_
When narrating, prioritize the always-loaded subtotal: a large **always** source is worse than an equally large **on-demand** one, because it is paid on every request. Call out any single always-loaded source that dwarfs the rest.
### Step 5: Suggest next steps
```markdown
**Next steps:**
- `/config-audit tokens` — prompt-cache token-hotspot patterns (cache-breaking, redundant perms, deep imports, MCP budget)
- `/config-audit whats-active` — same data grouped by category, with disable suggestions
- `/config-audit feature-gap` — what *could* improve here, grouped by impact
```
Tone (key on the **always-loaded subtotal** — the every-turn cost — not the grand total):
- High always-loaded (>40k): empathetic — "That's a heavy per-turn cost; it taxes every request before you've typed a word. Look at the largest always-loaded sources first."
- Moderate (1040k): neutral — "Reasonable. Skim the top always-loaded sources to see if anything is unexpectedly large."
- Low (<10k): encouraging — "Tight setup. The model has plenty of room for the actual work each turn."