config-audit/commands/tokens.md
Kjell Tore Guttormsen 8376dab83f fix(tokens): refresh stale "Opus 4.7" framing to model-neutral + Opus 4.8 anchor
Fase 4 token-opt, Item 1 of gap-review NEXT STEP #2. The prompt-cache pattern corpus + TOK scanner were frozen at an "Opus 4.7" framing after CC shipped Opus 4.8 (default, 2.1.154) and Fable 5 (2.1.170). Model-era facts re-verified against the official changelog cache before editing.

The patterns are properties of prompt-caching, not of any model, so mechanic text is now model-neutral with a single "current default: Opus 4.8" anchor — preventing a re-freeze at the next model bump.

- rename knowledge/opus-4.7-patterns.md -> prompt-cache-patterns.md (git mv, history preserved); 6 reference sites updated
- TOK scanner: line-318 finding text (human-facing) made model-neutral; header + cache-prefix-scanner + CLI comments refreshed
- configuration-best-practices.md body + footnote 4.7 -> 4.8
- human-facing docs: commands/{tokens,help,manifest}.md, project CLAUDE.md, README, docs/scanner-internals.md
- gap-matrix row marked DONE; future Items 2/3 retargeted to new filename

Failing-test-first (Iron Law): +2 knowledge staleness guards (era-anchor + no-refreeze) +1 scanner assertion (no stale model anchor in finding text). Suite 853 -> 856 green; zero snapshot drift; self-audit A(97) PASS. CHANGELOG / v5 plan / ratified gap-plan keep historical opus-4.7-patterns refs (correct record of past state).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
2026-06-18 15:27:36 +02:00

131 lines
5.9 KiB
Markdown

---
name: config-audit:tokens
description: Show ranked token hotspots and prompt-cache pattern findings — what's costing the most per turn and how to reduce it
argument-hint: "[path] [--global]"
allowed-tools: Read, Bash
model: sonnet
---
# Config-Audit: Token Hotspots
Show the configuration sources that contribute the most tokens per turn, ranked by estimated tokens, with prompt-cache-aware recommendations for reducing cache misses, schema bloat, and deep import chains.
Complementary to `/config-audit whats-active`:
- **`whats-active`** = inventory view (what loads).
- **`tokens`** = action view (what to trim and why).
## UX Rules (MANDATORY — from `.claude/rules/ux-rules.md`)
1. **Never show raw JSON or stderr output.** Always use `--output-file` + `2>/dev/null`.
2. **Narrate before acting.** Tell the user what you're about to do.
3. **Read, don't dump.** Read the JSON file and render formatted tables.
4. **End with context-sensitive next steps.**
## Implementation
### Step 1: Parse `$ARGUMENTS`
Split `$ARGUMENTS` into a path and flags. Path is the first non-flag argument. Default to `.` (current working directory). Recognized flags:
- `--global` — also include the user-level `~/.claude/` cascade
- `--json` — emit raw JSON instead of rendered tables (power-user mode; bypasses the humanizer for byte-stable v5.0.0 output)
- `--raw` — pass-through to the scanner; produces v5.0.0 verbatim JSON (bypasses the humanizer). Use when piping into v5.0.0-baseline diff tooling.
- `--with-telemetry-recipe` — include `telemetry_recipe_path` in the JSON output, pointing to `knowledge/cache-telemetry-recipe.md`. Use this when you want to verify a structural fix actually improved cache hit rate (manual jq recipe, opt-in)
### Step 2: Run the CLI silently
Tell the user: **"Analysing token hotspots for `<path>`..."**
Default mode (no `--json`, no `--raw`) emits a humanized JSON envelope: each finding carries `userImpactCategory`, `userActionLanguage`, and `relevanceContext` in addition to the v5.0.0 fields. Pass `--raw` through verbatim if the user requested it.
```bash
TMPFILE="/tmp/config-audit-tokens-$$.json"
RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/token-hotspots-cli.mjs <path> --output-file "$TMPFILE" [--global] $RAW_FLAG 2>/dev/null; echo $?
```
**Exit code handling:**
- `0` → continue
- `3` → tell user: "Couldn't analyse tokens. Check that the path exists and is a directory." Stop.
### Step 3: If `--json` was requested, cat the file and stop
```bash
cat "$TMPFILE"
```
Do NOT render tables in JSON mode.
### Step 4: Read JSON and render
Use the Read tool on `$TMPFILE`. Extract:
- `total_estimated_tokens` — top-line number
- `hotspots[]` — top 10 ranked sources
- `findings[]` — prompt-cache pattern findings (CA-TOK-001..003); each finding in default mode carries humanizer fields (`userImpactCategory`, `userActionLanguage`, `relevanceContext`) alongside the v5.0.0 fields
- `counts` — severity breakdown
Render as markdown. Group findings by `userImpactCategory` (e.g., "Wasted tokens" vs "Configuration mistake") rather than re-deriving severity prose; lead each line with `userActionLanguage` ("Fix this now", "Fix soon", "Optional cleanup", etc.) so the urgency phrasing stays consistent with the rest of the toolchain. The humanizer already replaced jargon-heavy `title`/`description`/`recommendation` strings with plain-language equivalents — render them verbatim.
```markdown
**Token hotspots for `<path>`** — ~{total_estimated_tokens} estimated tokens loaded per turn
### Top hotspots (ranked by estimated tokens)
| Rank | Source | Tokens | Recommendations |
|------|--------|--------|-----------------|
| {rank} | `{source}` | ~{estimated_tokens} | {recommendations joined as `· ` bullets} |
### Findings, grouped by impact
{Group findings[] by their userImpactCategory. Within each group, sort by userActionLanguage urgency (Fix this now → Fix soon → Fix when convenient → Optional cleanup → FYI), then render:}
- **{userActionLanguage}** — {title} ({id})
- {description}
- **Fix:** {recommendation}
- _{relevanceContext}_ when not "affects-everyone" (mention the scope so the user knows whether a fix touches shared config or just their machine)
### Severity summary
| Severity | Count |
|----------|-------|
| critical | {counts.critical} |
| high | {counts.high} |
| medium | {counts.medium} |
| low | {counts.low} |
| info | {counts.info} |
_Estimates assume ~4 chars/token (Claude ballpark). Real token count varies ±20%._
```
### Step 5: Cleanup and next steps
```bash
rm -f "$TMPFILE"
```
```markdown
### What's next
- **`/config-audit whats-active`** — full inventory of what loads (plugins, skills, MCP, hooks)
- **`/config-audit posture`** — overall health scorecard (Token Efficiency is the 8th area)
- **`/config-audit fix`** — auto-fix deterministic issues (where applicable)
- See `knowledge/prompt-cache-patterns.md` for the full pattern catalogue (CA-TOK-001 … 003)
- **Verify cache hit rate after a fix:** rerun with `--with-telemetry-recipe` to surface the path to `knowledge/cache-telemetry-recipe.md` — a copy-paste `jq` recipe that reads cache hit rate from your session transcripts. Opt-in. The TOK scanner is structural; this recipe is the runtime escape hatch.
```
## Scope and limits
- **Read-only.** Inspects config files; never writes.
- **Single repo.** Scans one path per invocation.
- **Structural only.** Hotspots are deterministic byte→token estimates from disk; runtime cache hit-rate is out of scope.
- **Heuristic estimates.** ~4 chars/token for markdown, ~3.5 for JSON. Real counts vary ±20%.
## Error handling
| Condition | Action |
|-----------|--------|
| Exit code 3 | Tell user path is invalid, suggest checking path exists |
| JSON parse fails | Tell user to re-run, mention as a bug to report |
| Empty hotspots | Suggest adding a CLAUDE.md or running `/config-audit feature-gap` first |