The B3a cache filter already exposes discovery.staleCacheVersions; surface them
as a finding so the user knows the superseded plugin versions are safe to delete.
Honesty (Verifiseringsplikt): the finding loads on ZERO turns, so it must NOT
read as a per-turn token cost. TOK normally humanizes to "Wasted tokens"; a new
per-finding category override ('plugin-cache-hygiene' -> "Dead config") plus a
dedicated humanizer translation ("Old plugin versions are sitting on disk (safe
to delete) ... cost zero tokens per turn ... housekeeping, not a performance
problem") keep the prose accurate instead of the generic "using more space"
default. evidence carries the explicit "zero live-context impact" note.
- token-hotspots: Pattern H emits CA-TOK (low) from discovery.staleCacheVersions
when stale versions exist (`--global`); silent otherwise.
- humanizer: CATEGORY_TO_IMPACT lets a finding's category override the
scanner-default impact label (raw `category` field unchanged -> --json/--raw
byte-stable). humanizer-data: honest static translation for the finding.
- Tests: finding fires/severity/category, lists stale keys + zero-impact note,
silent when none; humanizer override -> Dead config; honest translation locked.
- Docs: tokens command render note (disk-cleanup, not a token problem) +
--no-exclude-cache flag; README + CLAUDE.md rows (7 patterns, cache-aware).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
138 lines
7.5 KiB
Markdown
138 lines
7.5 KiB
Markdown
---
|
|
name: config-audit:tokens
|
|
description: Show ranked token hotspots and prompt-cache pattern findings — what's costing the most per turn and how to reduce it
|
|
argument-hint: "[path] [--global]"
|
|
allowed-tools: Read, Bash
|
|
model: sonnet
|
|
---
|
|
|
|
# Config-Audit: Token Hotspots
|
|
|
|
Show the configuration sources that contribute the most tokens per turn, ranked by estimated tokens, with prompt-cache-aware recommendations for reducing cache misses, schema bloat, and deep import chains.
|
|
|
|
Complementary to `/config-audit whats-active`:
|
|
- **`whats-active`** = inventory view (what loads).
|
|
- **`tokens`** = action view (what to trim and why).
|
|
|
|
## UX Rules (MANDATORY — from `.claude/rules/ux-rules.md`)
|
|
|
|
1. **Never show raw JSON or stderr output.** Always use `--output-file` + `2>/dev/null`.
|
|
2. **Narrate before acting.** Tell the user what you're about to do.
|
|
3. **Read, don't dump.** Read the JSON file and render formatted tables.
|
|
4. **End with context-sensitive next steps.**
|
|
|
|
## Implementation
|
|
|
|
### Step 1: Parse `$ARGUMENTS`
|
|
|
|
Split `$ARGUMENTS` into a path and flags. Path is the first non-flag argument. Default to `.` (current working directory). Recognized flags:
|
|
|
|
- `--global` — also include the user-level `~/.claude/` cascade
|
|
- `--no-exclude-cache` — include stale `~/.claude/plugins/cache` versions in the ranking. **By default they are excluded** (cache-aware filtering, default ON): the cache holds superseded plugin versions that load on *zero* turns, and counting them used to crowd the top-10 with dead config. The active version of each plugin (per `installed_plugins.json`) is always kept — only stale versions are filtered. Use `--no-exclude-cache` to see the full on-disk walk.
|
|
- `--json` — emit raw JSON instead of rendered tables (power-user mode; bypasses the humanizer for byte-stable v5.0.0 output)
|
|
- `--raw` — pass-through to the scanner; produces v5.0.0 verbatim JSON (bypasses the humanizer). Use when piping into v5.0.0-baseline diff tooling.
|
|
- `--with-telemetry-recipe` — include `telemetry_recipe_path` in the JSON output, pointing to `knowledge/cache-telemetry-recipe.md`. Use this when you want to verify a structural fix actually improved cache hit rate (manual jq recipe, opt-in)
|
|
|
|
### Step 2: Run the CLI silently
|
|
|
|
Tell the user: **"Analysing token hotspots for `<path>`..."**
|
|
|
|
Default mode (no `--json`, no `--raw`) emits a humanized JSON envelope: each finding carries `userImpactCategory`, `userActionLanguage`, and `relevanceContext` in addition to the v5.0.0 fields. Pass `--raw` through verbatim if the user requested it.
|
|
|
|
```bash
|
|
TMPFILE="/tmp/config-audit-tokens-$$.json"
|
|
RAW_FLAG=""
|
|
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
|
node ${CLAUDE_PLUGIN_ROOT}/scanners/token-hotspots-cli.mjs <path> --output-file "$TMPFILE" [--global] [--no-exclude-cache] $RAW_FLAG 2>/dev/null; echo $?
|
|
```
|
|
|
|
**Exit code handling:**
|
|
- `0` → continue
|
|
- `3` → tell user: "Couldn't analyse tokens. Check that the path exists and is a directory." Stop.
|
|
|
|
### Step 3: If `--json` was requested, cat the file and stop
|
|
|
|
```bash
|
|
cat "$TMPFILE"
|
|
```
|
|
|
|
Do NOT render tables in JSON mode.
|
|
|
|
### Step 4: Read JSON and render
|
|
|
|
Use the Read tool on `$TMPFILE`. Extract:
|
|
|
|
- `total_estimated_tokens` — top-line number
|
|
- `hotspots[]` — top 10 ranked sources; each carries a **load pattern** (`loadPattern` ∈ always / on-demand / external, plus `survivesCompaction` / `derivationConfidence`)
|
|
- `findings[]` — prompt-cache pattern findings; each finding in default mode carries humanizer fields (`userImpactCategory`, `userActionLanguage`, `relevanceContext`) alongside the v5.0.0 fields
|
|
- `counts` — severity breakdown
|
|
|
|
A hotspot's **load pattern** matters as much as its size: an **always**-loaded source (CLAUDE.md, MCP tool schemas) is paid on *every* turn, an **on-demand** one (skill body, path-scoped rule) only when invoked/matched, and an **external** one (hooks, harness-config files like settings.json/.mcp.json) costs no per-turn context tokens at all. A big always-loaded hotspot is the most worth trimming.
|
|
|
|
**The stale plugin-cache finding is different from the rest.** All other TOK findings are about per-turn token cost. The *"Old plugin versions are sitting on disk"* finding (category `plugin-cache-hygiene`, impact **Dead config**, `--global` only) is a pure **disk-cleanup** item with **zero live-context impact** — the listed versions are never loaded. Render it as housekeeping, not a token problem: don't conflate its disk bytes with the per-turn token numbers above it.
|
|
|
|
Render as markdown. Group findings by `userImpactCategory` (e.g., "Wasted tokens" vs "Configuration mistake") rather than re-deriving severity prose; lead each line with `userActionLanguage` ("Fix this now", "Fix soon", "Optional cleanup", etc.) so the urgency phrasing stays consistent with the rest of the toolchain. The humanizer already replaced jargon-heavy `title`/`description`/`recommendation` strings with plain-language equivalents — render them verbatim.
|
|
|
|
```markdown
|
|
**Token hotspots for `<path>`** — ~{total_estimated_tokens} estimated tokens loaded per turn
|
|
|
|
### Top hotspots (ranked by estimated tokens)
|
|
|
|
| Rank | Source | Tokens | Load | Recommendations |
|
|
|------|--------|--------|------|-----------------|
|
|
| {rank} | `{source}` | ~{estimated_tokens} | {loadPattern} | {recommendations joined as `· ` bullets} |
|
|
|
|
_Load column: **always** (every turn) / **on-demand** (on invoke/match) / **external** (out-of-context). Append `°` when `derivationConfidence` is `inferred`._
|
|
|
|
### Findings, grouped by impact
|
|
|
|
{Group findings[] by their userImpactCategory. Within each group, sort by userActionLanguage urgency (Fix this now → Fix soon → Fix when convenient → Optional cleanup → FYI), then render:}
|
|
|
|
- **{userActionLanguage}** — {title} ({id})
|
|
- {description}
|
|
- **Fix:** {recommendation}
|
|
- _{relevanceContext}_ when not "affects-everyone" (mention the scope so the user knows whether a fix touches shared config or just their machine)
|
|
|
|
### Severity summary
|
|
|
|
| Severity | Count |
|
|
|----------|-------|
|
|
| critical | {counts.critical} |
|
|
| high | {counts.high} |
|
|
| medium | {counts.medium} |
|
|
| low | {counts.low} |
|
|
| info | {counts.info} |
|
|
|
|
_Estimates assume ~4 chars/token (Claude ballpark). Real token count varies ±20%._
|
|
```
|
|
|
|
### Step 5: Cleanup and next steps
|
|
|
|
```bash
|
|
rm -f "$TMPFILE"
|
|
```
|
|
|
|
```markdown
|
|
### What's next
|
|
|
|
- **`/config-audit whats-active`** — full inventory of what loads (plugins, skills, MCP, hooks)
|
|
- **`/config-audit posture`** — overall health scorecard (Token Efficiency is the 8th area)
|
|
- **`/config-audit fix`** — auto-fix deterministic issues (where applicable)
|
|
- See `knowledge/prompt-cache-patterns.md` for the full pattern catalogue (CA-TOK-001 … 003)
|
|
- **Verify cache hit rate after a fix:** rerun with `--with-telemetry-recipe` to surface the path to `knowledge/cache-telemetry-recipe.md` — a copy-paste `jq` recipe that reads cache hit rate from your session transcripts. Opt-in. The TOK scanner is structural; this recipe is the runtime escape hatch.
|
|
```
|
|
|
|
## Scope and limits
|
|
|
|
- **Read-only.** Inspects config files; never writes.
|
|
- **Single repo.** Scans one path per invocation.
|
|
- **Structural only.** Hotspots are deterministic byte→token estimates from disk; runtime cache hit-rate is out of scope.
|
|
- **Heuristic estimates.** ~4 chars/token for markdown, ~3.5 for JSON. Real counts vary ±20%.
|
|
|
|
## Error handling
|
|
|
|
| Condition | Action |
|
|
|-----------|--------|
|
|
| Exit code 3 | Tell user path is invalid, suggest checking path exists |
|
|
| JSON parse fails | Tell user to re-run, mention as a bug to report |
|
|
| Empty hotspots | Suggest adding a CLAUDE.md or running `/config-audit feature-gap` first |
|