diff --git a/CLAUDE.md b/CLAUDE.md index dbbd249..0dde5c5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -18,7 +18,7 @@ Analyzes and optimizes Claude Code configuration across three pillars: | `/config-audit` | Full audit with auto-scope detection (no setup needed) | | `/config-audit posture` | Quick health scorecard (A-F grades, 10 quality areas incl. Token Efficiency, Plugin Hygiene) | | `/config-audit tokens` | prompt-cache-aware token hotspots (6 patterns: cache-breaking, redundant perms, deep imports, oversized cascade, bloated SKILL.md desc, MCP tool-schema budget) — optional `--accurate-tokens` API calibration, `--with-telemetry-recipe` cache-hit recipe pointer | -| `/config-audit manifest` | Ranked table of every system-prompt token source (CLAUDE.md, plugins, skills, MCP, hooks) sorted by estimated tokens | +| `/config-audit manifest` | Ranked table of every token source (CLAUDE.md, rules, agents, skills, output styles, MCP, hooks) sorted by estimated tokens, each tagged with its load pattern (always-loaded / on-demand / external) + an always-loaded subtotal ("tokens that enter context every turn") | | `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact (incl. a conditional `disableBundledSkills` lever when the active skill listing is over budget — remediation companion to SKL `CA-SKL-002`) | | `/config-audit fix` | Auto-fix deterministic issues with backup + verification | | `/config-audit rollback` | Restore configuration from backup | @@ -109,7 +109,7 @@ Default: auto-detects scope from git context. Override with `/config-audit full| node --test 'tests/**/*.test.mjs' ``` -996 tests across 56 test files (17 lib + 29 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Top-level humanizer tests: `json-backcompat.test.mjs`, `raw-backcompat.test.mjs`, `scenario-read-test.test.mjs`, `snapshot-default-output.test.mjs`. +1008 tests across 56 test files (17 lib + 29 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Top-level humanizer tests: `json-backcompat.test.mjs`, `raw-backcompat.test.mjs`, `scenario-read-test.test.mjs`, `snapshot-default-output.test.mjs`. ### active-config-reader — load-pattern model + rule/agent/output-style enumeration (v5.6 Foundation) @@ -131,6 +131,32 @@ false-positive (a block-sequence-scoped rule was misread as unscoped). An empty- no following `- ` items still resolves to `null` (backwards-compatible); only a real `- ` item list becomes an array. +### manifest — load-pattern accounting (v5.6 B) + +`buildManifest` (`scanners/manifest.mjs`) now consumes the Foundation enumeration. Two changes: + +1. **Component-level sources (plugin roll-up dropped).** The coarse `kind:'plugin'` aggregate is + gone. A plugin contributes via its skills/rules/agents/output-styles/hooks/MCP — each already + enumerated **once** by `readActiveConfig` — so the old roll-up double-counted them (the plugin + aggregate's `estimatedTokens` already summed its components). Source kinds are now + `claude-md`/`skill`/`rule`/`agent`/`output-style`/`mcp-server`/`hook`. +2. **Load-pattern triple on every record + a `summary`.** Each source carries + `loadPattern`/`survivesCompaction`/`derivationConfidence`. Rules/agents/output-styles + **propagate** the foundation-derived values (rules vary by `scoped`); CLAUDE.md maps `scope`→ + kind via `CLAUDE_MD_SCOPE_KIND` (all cascade files walk **up**, so all are always-loaded); + skills are tagged **on-demand** via `deriveLoadPattern('skill-body')` — the measured tokens are + the skill **body** (paid on invoke), not the tiny always-loaded name+desc listing (tracked by + `skill-listing-budget`/posture), so tagging the body always would inflate the headline. The new + `summary` buckets sources into `always`/`onDemand`/`external`/`unknown` `{tokens,count}`; the + **always-loaded subtotal** ("≈X tokens enter context every turn before you type") is the headline. + +**Byte-stability.** manifest is an **environment-aware CLI** → SC-6/SC-7 verify it by +**mode-equivalence** (`--json == --raw`), not byte-equal against a frozen snapshot, and it is not in +SC-5 default-output. Adding fields in place therefore keeps all snapshots green with **no regen** +(verified). `total` changes (de-duped, component-level) — that is the intended correctness fix. +**TOK's load-pattern column is deferred to the next chunk** (it *is* a byte-equal SC-6 CLI, so it +needs snapshot handling); this chunk is manifest-only. + ### CML scanner — context-window-scaled char budget Beyond the line-count checks (200/500 lines, both MEDIUM), the CML scanner mirrors diff --git a/README.md b/README.md index c305321..f031240 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ ![Commands](https://img.shields.io/badge/commands-18-green) ![Agents](https://img.shields.io/badge/agents-6-orange) ![Hooks](https://img.shields.io/badge/hooks-4-red) -![Tests](https://img.shields.io/badge/tests-996+-brightgreen) +![Tests](https://img.shields.io/badge/tests-1008+-brightgreen) ![License](https://img.shields.io/badge/license-MIT-lightgrey) A Claude Code plugin that checks configuration health, suggests context-aware improvements, and auto-fixes issues — `CLAUDE.md`, `settings.json`, hooks, rules, MCP servers, `@imports`, and plugins. 13 deterministic scanners across 10 quality areas, context-aware feature recommendations, auto-fix with backup/rollback, a prompt-cache-aware Token Hotspots scanner with optional API-calibrated `--accurate-tokens` mode, plus cache-prefix stability, dead-tool, and cross-plugin collision detection. Zero external dependencies. @@ -276,7 +276,7 @@ Your team configuration changes over time. Track it: | `/config-audit` | Full audit with auto-scope detection (no setup needed) | | `/config-audit posture` | Quick health scorecard: A-F grades across 10 quality areas (incl. Token Efficiency, Plugin Hygiene) | | `/config-audit tokens` | prompt-cache-aware token hotspots — ranked by estimated waste; 6 patterns + optional `--accurate-tokens` API calibration | -| `/config-audit manifest` | Ranked table of every system-prompt token source (CLAUDE.md, plugins, skills, MCP, hooks) sorted by estimated tokens | +| `/config-audit manifest` | Ranked table of every token source (CLAUDE.md, rules, agents, skills, output styles, MCP, hooks) sorted by estimated tokens — each tagged with its **load pattern** (always-loaded / on-demand / external) plus an **always-loaded subtotal** ("≈X tokens enter context every turn before you type"). Component-level: no coarse plugin roll-up (it would double-count) | | `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact | | `/config-audit fix` | Auto-fix deterministic issues with backup + verification | | `/config-audit rollback` | Restore configuration from a previous backup | @@ -411,7 +411,7 @@ All tools work standalone — no Claude Code session needed: | **Fix** | `node scanners/fix-cli.mjs [--apply] [--json] [--global]` | | **Drift** | `node scanners/drift-cli.mjs [--save] [--baseline name] [--json]` | | **Tokens** | `node scanners/token-hotspots-cli.mjs [--json] [--global] [--output-file path] [--accurate-tokens] [--with-telemetry-recipe]` | -| **Manifest** | `node scanners/manifest.mjs [--json]` — ranked system-prompt source table | +| **Manifest** | `node scanners/manifest.mjs [--json]` — ranked component-level source table with per-source load pattern + always-loaded subtotal | | **What's active** | `node scanners/whats-active.mjs [--json] [--verbose] [--suggest-disables]` | | **Self-audit** | `node scanners/self-audit.mjs [--json] [--fix] [--check-readme]` | | **Full scan** | `node scanners/scan-orchestrator.mjs [--global] [--full-machine] [--no-suppress]` | @@ -559,7 +559,7 @@ Shared modules used by all scanners — useful if you're reading the source or e | `rollback-engine.mjs` | `listBackups()`, `restoreBackup()`, `deleteBackup()` | | `fix-cli.mjs` | CLI entry point for auto-fix | | `drift-cli.mjs` | CLI entry point for drift detection | -| `manifest.mjs` | CLI: ranked system-prompt source table (v5 N2) | +| `manifest.mjs` | CLI: ranked component-level source table w/ load-pattern accounting (v5 N2; v5.6 B) | | `whats-active.mjs` | CLI: read-only active-config inventory (v3.1.0+) | | `token-hotspots-cli.mjs` | CLI: token hotspots ranking with optional `--accurate-tokens` | @@ -588,7 +588,7 @@ Reference documents that inform the feature-gap agent and context-aware recommen node --test 'tests/**/*.test.mjs' ``` -996 tests across 56 test files (17 lib + 29 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Requires Node.js 18+ (`node:test`). +1008 tests across 56 test files (17 lib + 29 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Requires Node.js 18+ (`node:test`). --- diff --git a/STATE.md b/STATE.md index d609543..6cefe42 100644 --- a/STATE.md +++ b/STATE.md @@ -2,32 +2,32 @@ _Current state-of-play for DENNE pluginen. Auto-injisert ved sesjonsstart. Overskrives (ikke append) ved sesjonsslutt. Historikk → git (STATE.md er tracked). Stier relative til plugin-mappa._ -## Aktiv kontekst (v5.6 Foundation LANDET på main — B neste) -v5.5.0 released (tag `v5.5.0`, catalog v5.5.0). Siden da: **v5.6 Foundation implementert + committet på main** (TDD). Suite **996** (961→996), self-audit A/A (config 93, plugin 100), count **13** (Foundation legger ingen scanner), SC-5/6/7 byte-stabil UTEN regen. v5.6 er IKKE release-cuttet (krever B + C + eget release-GO). +## Aktiv kontekst (v5.6 B1 = manifest load-pattern accounting LANDET på main — B2/TOK + C neste) +v5.5.0 released. v5.6 Foundation landet (`62d910e`). Nå: **v5.6 B delt i to** — **B1 (manifest) implementert + committet på main** (TDD). TOK-kolonnen ble bevisst utsatt (B2) fordi TOK er en byte-equal SC-6-CLI (krever snapshot-håndtering); manifest er env-aware → mode-equivalence → byte-stabil UTEN regen. Suite **1008** (996→1008), self-audit **A/A** (config A, plugin A), count **13** (ingen ny scanner). v5.6 IKKE release-cuttet (krever B2 + C + eget GO). -## Gjort denne økten (2026-06-20, lør — alt pushet) -- **Parser-fiks (`yaml-parser.mjs`):** `parseSimpleYaml` leser nå YAML block-sequences (`paths:\n - a`), ikke bare inline `paths: "a, b"`. Empty-valued key uten `- `-items → fortsatt `null` (backcompat). Løser pre-eksisterende RUL false-positive (block-seq-scoped rule sett som unscoped) — fiks flyter gjennom uendret RUL-kode. -- **Foundation (`active-config-reader.mjs`):** ny `deriveLoadPattern(kind,{scoped})` (loadPattern/survivesCompaction/derivationConfidence, V-row-forankret) + `enumerateRules`/`enumerateAgents`/`enumerateOutputStyles` (mirror `enumerateSkills`). `readActiveConfig` eksponerer `rules`/`agents`/`outputStyles` + totals/subtotals (i grandTotal). Output-style-discovery gjøres direkte (IKKE ny `file-discovery`-type) → discovery-surface stabil. -- **Tester:** +35 (yaml-parser block-seq, RUL block-seq regresjon, deriveLoadPattern table, 3 enumeratorer pos+neg). Amendet 2 eksisterende ACR-asserts (top-level key shape + grandTotal sum). -- **Docs (docs-gate krever non-trivial README+CLAUDE for feat):** README testing-prosa 635/36→996/56 (var v5.0.0-stale), tests-badge 961+→996+, CLAUDE.md test-count 936→996 + ny Foundation-seksjon. +## Gjort denne økten (2026-06-20, lør — pushet hvis i vindu) +- **`buildManifest` (`scanners/manifest.mjs`):** komponent-nivå — **plugin roll-up FJERNET** (dobbelttalte komponenter; plugin-bidrag enumereres allerede én gang som skills/rules/agents/output-styles/hooks/mcp). Source-kinds nå claude-md/skill/rule/agent/output-style/mcp-server/hook. Hver record får `loadPattern`/`survivesCompaction`/`derivationConfidence`. Rules/agents/output-styles **propagerer** Foundation-verdiene; CLAUDE.md scope→kind via `CLAUDE_MD_SCOPE_KIND` (alle cascade-filer walker opp → always); skills tagget **on-demand** (`skill-body` — målt token = body, ikke always-listing). Ny `summary` (always/onDemand/external/unknown {tokens,count}) i CLI-output. Ny `summarizeByLoadPattern` (eksportert). +- **Headline:** always-loaded subtotal. På plugin-rota: 24,3k always / 107 kilder (101 agenter dominerer), vs 125k on-demand (skill-bodies), 855 external (hooks). `total` endret (de-dup) — tilsiktet korrekthetsfiks. +- **Tester:** +12 (deterministisk enhetstest på `buildManifest` med syntetisk activeConfig: loadPattern per kind, summary-reconcil, plugin/disabled-mcp ekskludert fra total; + CLI loadPattern/summary-presence). Endret brutt `kind==='plugin'`-test → komponent-nivå. +- **Docs (docs-gate):** README (badge 996→1008, prosa + 3 manifest-beskrivelser), CLAUDE.md (test 1008 + ny «manifest — load-pattern accounting (v5.6 B)»-seksjon + tabellrad), scanner-internals manifest-rad. ## Åpne tråder / neste steg (prioritert, alle KREVER GO) -1. **v5.6 B (load-pattern accounting):** render `loadPattern`-kolonne + «always-loaded subtotal» i `manifest` + `tokens`. FORMAT-endring → **regen SC-5 + SC-6/7 token-hotspots-snapshots**; beslutt `--json` back-compat (add-field-in-place vs schema-versjon). Konsumerer Foundation (deriveLoadPattern + enumerasjoner ligger klare). -2. **v5.6 C (output-style scanner):** ny `CA-OST` (count 13→**14**): `keep-coding-instructions`/`force-for-plugin`/dead `outputStyle`. Badge + lore-oppdatering, humanizer/scoring/discovery-wiring. -3. **v5.6 release-cut:** etter B+C lander på main (eget GO, som v5.5.0). +1. **v5.6 B2 (TOK load-pattern-kolonne):** legg loadPattern på token-hotspots source-records. TOK ER byte-equal SC-6 → **bevar v5.0.0-baseline via normalizer-strip** (additive felt ignoreres, frozen-kontrakt intakt) + **regen SC-5 default-output/token-hotspots.json** (UPDATE_SNAPSHOT=1). Foundation + manifest-mønsteret ligger klart. +2. **v5.6 C (output-style scanner):** ny `CA-OST` (count 13→**14**): keep-coding-instructions/force-for-plugin/dead outputStyle. Badge + lore + humanizer/scoring/discovery-wiring. +3. **v5.6 release-cut:** etter B2+C (eget GO, som v5.5.0). 4. **v5.7:** D (mechanism-fit, heuristisk, precision-gated). -5. **llm-security KRITISK** (annet repo, F-1 RCE, disclosure-hold) + 3 aktive plugins + STATE-sveip — uendret. +5. **llm-security KRITISK** (annet repo) + 3 aktive plugins + STATE-sveip — uendret. ## Gotchas (UFRAVIKELIG) -- **Parser-begrensning RESOLVED denne økten** — block-sequences leses nå. (Hvis du ser gammel STATE/plan som sier «fiks i v5.6 Foundation»: gjort.) -- **fix-engine + humanizer-data kobler på finding-TITTEL.** Endrer/legger du til tittel: oppdater begge (+ evt. SC-5 snapshot). Hver finding: positiv OG negativ fixture (TDD). -- Versjon KUN i plugin.json + README version-badge. **tests-badge synces til EKSAKT suite-tall ved hver chunk-landing** (961→996 nå); driver IKKE exit-kode — verifiser `readmeCheck.passed`/`mismatches:[]` via `--json`. Catalog = SEPARAT repo; bump ref ETTER tag (egen GO). -- **docs-gate (non-marketplace gren) krever non-trivial README OG CLAUDE.md (≥3 ikke-ws-linjer hver) for `feat:`**-commits. Foundation oppfylte ærlig (stale-fiks + arch-note). Alt.: `[skip-docs]` (logges). -- Scanner-count 13. C gir 14 (v5.6). -- Path-guard blokkerer Write på `.claude-plugin/` + settings/hooks → hermetisk `mkdtemp` i tester. Push-vindu: hverdag 20–23, helg fritt. +- **manifest er env-aware → SC-6/SC-7 = mode-equivalence (--json==--raw), IKKE byte-equal.** Derfor byte-stabil ved add-field-in-place (verifisert). **TOK er motsatt** (byte-equal) — B2 må håndtere snapshot. manifest IKKE i SC-5. +- **manifest har INGEN findings** → ingen humanizer/fix-engine-tittelkobling (gjelder kun finding-scannere). +- **Skill on-demand, IKKE always:** målt token = body (invoke). Always-listing (navn+desc) er liten, spores av skill-listing-budget/posture. Ikke tagg skill-body always — det blåser opp headline. +- Versjon KUN i plugin.json + README version-badge. **tests-badge = EKSAKT `ℹ tests N` fra `node --test`** (self-audit `countTestCases`); verifiser `readmeCheck.passed:true`/`mismatches:[]` via `--json`. 1008 nå. +- **docs-gate (non-marketplace gren) krever ≥3 ikke-ws-linjer i README OG CLAUDE.md for `feat:`.** Oppfylt ærlig denne økten. Alt.: `[skip-docs]`. +- Scanner-count 13. C gir 14. Path-guard blokkerer Write på `.claude-plugin/`+settings/hooks → hermetisk `mkdtemp` i tester. Push-vindu: hverdag 20–23, helg fritt. ## Scope-gjerde -v5.6 Foundation godkjent (GO) og landet. B + C + v5.6 release-cut + v5.7 + andre repos = egne GO. Ikke start B uten GO. Byte-stabilitet var by-construction (Foundation rendrer ingenting nytt) — B BRYTER den bevisst (snapshot-regen). +v5.6 B1 (manifest) godkjent (GO) og landet. B2 (TOK) + C + release-cut + v5.7 + andre repos = egne GO. Ikke start B2 uten GO. ## Kontinuitet Tre lag: STATE.md (tracked) + auto-memory + CLAUDE.md. Oppdater DENNE som siste handling. diff --git a/commands/manifest.md b/commands/manifest.md index f2e8bee..15d63cb 100644 --- a/commands/manifest.md +++ b/commands/manifest.md @@ -1,6 +1,6 @@ --- name: config-audit:manifest -description: Show ranked token-source manifest — every CLAUDE.md, plugin, skill, MCP server, and hook ordered DESC by estimated tokens +description: Show ranked token-source manifest — every CLAUDE.md, rule, agent, skill, output style, MCP server, and hook ordered DESC by estimated tokens, each tagged with its load pattern (always-loaded vs on-demand vs external), plus an always-loaded subtotal argument-hint: "[path] [--json]" allowed-tools: Read, Bash model: sonnet @@ -10,6 +10,14 @@ model: sonnet Produce a ranked, single-table view of every token source loaded for a given repo path. Where `whats-active` shows separate tables per category, `manifest` collapses everything into one ordered list — making it easy to see what's costing the most regardless of category. +Every source is tagged with its **load pattern**, derived from the published Claude Code loading model: + +- **always** — enters context every turn before you type (project/user CLAUDE.md, unscoped rules, agents, output styles, MCP tool schemas). This is the cost that matters most: it is paid on *every* request. +- **on-demand** — loaded only when needed (skill bodies on invoke, path-scoped rules on a matching file read). +- **external** — runs outside the context window entirely (hooks). + +The **always-loaded subtotal** is the headline number. Sources are component-level (a plugin contributes via its skills/rules/agents/output styles/hooks/MCP, each listed once — there is no coarse "plugin" roll-up, which would double-count). + ## UX Rules (MANDATORY — from `.claude/rules/ux-rules.md`) 1. **Never show raw JSON or stderr output.** Always use `--output-file` + `2>/dev/null`. @@ -51,21 +59,28 @@ Do NOT render the table in JSON mode. ### Step 4: Read JSON and render -Use the Read tool on `$TMPFILE`. Extract `meta.repoPath`, `total`, and `sources[]`. Render the top 20 sources (or fewer if the manifest is shorter): +Use the Read tool on `$TMPFILE`. Extract `meta.repoPath`, `total`, `summary`, and `sources[]`. Lead with the **always-loaded subtotal** (the headline), then render the top 20 sources (or fewer if the manifest is shorter): ```markdown -**Token-source manifest for ``** — ~{total} tokens at startup +**Token-source manifest for ``** — ~{total} tokens total -| Rank | Kind | Name | Source | Tokens | -|------|------|------|--------|--------| -| 1 | {kind} | `` | {source} | ~{estimated_tokens} | -| ... | ... | ... | ... | ... | +- 🔴 **~{summary.always.tokens} tokens enter context every turn** before you type ({summary.always.count} always-loaded sources) +- 🟡 ~{summary.onDemand.tokens} tokens on-demand ({summary.onDemand.count} sources — loaded only when invoked / matched) +- ⚪ ~{summary.external.tokens} tokens external ({summary.external.count} sources — hooks, run outside context) +| Rank | Kind | Name | Source | Tokens | Load | +|------|------|------|--------|--------|------| +| 1 | {kind} | `` | {source} | ~{estimated_tokens} | {load} | +| ... | ... | ... | ... | ... | ... | + +_Load column: **always** / **on-demand** / **external**. Append `°` when `derivationConfidence` is `inferred` (no primary-doc row pins it exactly)._ _Estimates assume ~4 chars/token (Claude ballpark). Real token count varies ±15%._ ``` If `sources.length > 20`, follow the table with: _"Showing top 20 of {N} sources. Run with `--json` to see the full list."_ +When narrating, prioritize the always-loaded subtotal: a large **always** source is worse than an equally large **on-demand** one, because it is paid on every request. Call out any single always-loaded source that dwarfs the rest. + ### Step 5: Suggest next steps ```markdown @@ -75,7 +90,7 @@ If `sources.length > 20`, follow the table with: _"Showing top 20 of {N} sources - `/config-audit feature-gap` — what *could* improve here, grouped by impact ``` -Tone: -- High total (>50k): empathetic — "That's a heavy startup cost; tokens bullet anything you'd otherwise spend on the actual conversation." -- Moderate (10–50k): neutral — "Reasonable. Skim the top 5 to see if anything is unexpectedly large." -- Low (<10k): encouraging — "Tight setup. The model has plenty of room for the actual work." +Tone (key on the **always-loaded subtotal** — the every-turn cost — not the grand total): +- High always-loaded (>40k): empathetic — "That's a heavy per-turn cost; it taxes every request before you've typed a word. Look at the largest always-loaded sources first." +- Moderate (10–40k): neutral — "Reasonable. Skim the top always-loaded sources to see if anything is unexpectedly large." +- Low (<10k): encouraging — "Tight setup. The model has plenty of room for the actual work each turn." diff --git a/docs/scanner-internals.md b/docs/scanner-internals.md index 8c20575..05de705 100644 --- a/docs/scanner-internals.md +++ b/docs/scanner-internals.md @@ -54,7 +54,7 @@ Scanner CLI: `node scanners/scan-orchestrator.mjs [--global] [--full-mach | `drift-cli.mjs` | CLI: `node drift-cli.mjs [--save] [--baseline name] [--json]` | | `whats-active.mjs` | CLI: `node whats-active.mjs [--json] [--verbose] [--suggest-disables]` — read-only active-config inventory | | `token-hotspots-cli.mjs` | CLI: `node token-hotspots-cli.mjs [--json] [--global] [--output-file path] [--accurate-tokens] [--with-telemetry-recipe]` — prompt-cache token hotspots ranking with optional API calibration | -| `manifest.mjs` | CLI: `node manifest.mjs [--json]` — ranked system-prompt token-source table (v5 N2) | +| `manifest.mjs` | CLI: `node manifest.mjs [--json]` — ranked component-level token-source table, each source tagged with its load pattern + an always-loaded subtotal (v5 N2; load-pattern accounting v5.6 B) | ## Standalone Scanner diff --git a/scanners/manifest.mjs b/scanners/manifest.mjs index 2480a2b..da9c32b 100644 --- a/scanners/manifest.mjs +++ b/scanners/manifest.mjs @@ -9,13 +9,29 @@ * { * meta: { repoPath, generatedAt, durationMs }, * sources: [ - * { kind: 'claude-md'|'plugin'|'skill'|'mcp-server'|'hook', - * name: string, source: string, estimated_tokens: number }, + * { kind: 'claude-md'|'skill'|'rule'|'agent'|'output-style'|'mcp-server'|'hook', + * name: string, source: string, estimated_tokens: number, + * loadPattern: 'always'|'on-demand'|'external'|'unknown', + * survivesCompaction: 'yes'|'no'|'n/a', + * derivationConfidence: 'confirmed'|'inferred' }, * ... * ], + * summary: { + * always: { tokens, count }, // enter context every turn before you type + * onDemand: { tokens, count }, // loaded on invoke / on file read + * external: { tokens, count }, // run outside the context window (hooks) + * unknown: { tokens, count }, + * }, * total: * } * + * v5.6 B — load-pattern accounting. Sources are component-level: the coarse + * "plugin" roll-up was dropped because a plugin's contributions (skills, rules, + * agents, output styles, hooks, MCP) are each enumerated once on their own — + * keeping the roll-up double-counted them and corrupted the always-loaded + * subtotal. Every record now carries the load pattern derived from the + * published Claude Code loading model (deriveLoadPattern). + * * Usage: * node manifest.mjs [path] [--json] [--output-file ] * @@ -25,64 +41,131 @@ import { resolve } from 'node:path'; import { writeFile, stat } from 'node:fs/promises'; -import { readActiveConfig } from './lib/active-config-reader.mjs'; +import { readActiveConfig, deriveLoadPattern } from './lib/active-config-reader.mjs'; + +// CLAUDE.md cascade files are all discovered by walking UP from the repo, so +// each one is always-loaded; the scope only changes the derivation confidence. +const CLAUDE_MD_SCOPE_KIND = { + project: 'claude-md-root', + local: 'claude-md-root', + user: 'claude-md-user', + managed: 'claude-md-managed', + import: 'claude-md-import', +}; + +/** Spread the three load-pattern fields onto a source record. */ +function withLoadPattern(record, lp) { + return { + ...record, + loadPattern: lp.loadPattern, + survivesCompaction: lp.survivesCompaction, + derivationConfidence: lp.derivationConfidence, + }; +} + +const sourceLabel = (item, fallback) => + item.pluginName ? `plugin:${item.pluginName}` : item.source || fallback; /** - * Flatten an activeConfig snapshot into a single ranked array of sources. + * Flatten an activeConfig snapshot into a single ranked array of sources, each + * tagged with its load pattern, plus a load-pattern summary. */ export function buildManifest(activeConfig) { const sources = []; for (const f of activeConfig.claudeMd?.files || []) { const tokens = estimateClaudeMdEntryTokens(f, activeConfig); - sources.push({ + const kind = CLAUDE_MD_SCOPE_KIND[f.scope] || 'claude-md-root'; + sources.push(withLoadPattern({ kind: 'claude-md', name: f.path, source: f.scope, estimated_tokens: tokens, - }); - } - - for (const p of activeConfig.plugins || []) { - sources.push({ - kind: 'plugin', - name: p.name, - source: p.path, - estimated_tokens: p.estimatedTokens || 0, - }); + }, deriveLoadPattern(kind))); } + // Skills: the measured tokens are the skill BODY (full file), paid on invoke. + // The always-loaded part (name+description listing) is small and tracked + // separately (skill-listing-budget / posture), so the body is tagged + // on-demand here rather than inflating the always-loaded subtotal. for (const s of activeConfig.skills || []) { - sources.push({ + sources.push(withLoadPattern({ kind: 'skill', name: s.name, - source: s.pluginName ? `plugin:${s.pluginName}` : s.source || 'user', + source: sourceLabel(s, 'user'), estimated_tokens: s.estimatedTokens || 0, - }); + }, deriveLoadPattern('skill-body'))); + } + + // Rules / agents / output styles — the foundation enumeration already derived + // the load pattern (rules vary by `scoped`), so propagate it verbatim. + for (const r of activeConfig.rules || []) { + sources.push(withLoadPattern({ + kind: 'rule', + name: r.name, + source: sourceLabel(r, 'project'), + estimated_tokens: r.estimatedTokens || 0, + }, r)); + } + + for (const a of activeConfig.agents || []) { + sources.push(withLoadPattern({ + kind: 'agent', + name: a.name, + source: sourceLabel(a, 'project'), + estimated_tokens: a.estimatedTokens || 0, + }, a)); + } + + for (const o of activeConfig.outputStyles || []) { + sources.push(withLoadPattern({ + kind: 'output-style', + name: o.name, + source: sourceLabel(o, 'project'), + estimated_tokens: o.estimatedTokens || 0, + }, o)); } for (const m of activeConfig.mcpServers || []) { if (m && m.enabled === false) continue; - sources.push({ + sources.push(withLoadPattern({ kind: 'mcp-server', name: m.name, source: m.source || 'unknown', estimated_tokens: m.estimatedTokens || 0, - }); + }, deriveLoadPattern('mcp'))); } for (const h of activeConfig.hooks || []) { - sources.push({ + sources.push(withLoadPattern({ kind: 'hook', name: `${h.event}${h.matcher ? `:${h.matcher}` : ''}`, source: h.source || h.sourcePath || 'unknown', estimated_tokens: h.estimatedTokens || 0, - }); + }, deriveLoadPattern('hook'))); } sources.sort((a, b) => b.estimated_tokens - a.estimated_tokens); const total = sources.reduce((s, x) => s + (x.estimated_tokens || 0), 0); - return { sources, total }; + const summary = summarizeByLoadPattern(sources); + return { sources, total, summary }; +} + +/** + * Bucket sources by load pattern into {tokens, count} subtotals. The `always` + * bucket is the headline: tokens that enter context every turn before the user + * types anything. + */ +export function summarizeByLoadPattern(sources) { + const mk = () => ({ tokens: 0, count: 0 }); + const summary = { always: mk(), onDemand: mk(), external: mk(), unknown: mk() }; + const BUCKET = { always: 'always', 'on-demand': 'onDemand', external: 'external' }; + for (const s of sources) { + const key = BUCKET[s.loadPattern] || 'unknown'; + summary[key].tokens += s.estimated_tokens || 0; + summary[key].count += 1; + } + return summary; } /** @@ -138,6 +221,7 @@ async function main() { durationMs: Date.now() - start, }, sources: manifest.sources, + summary: manifest.summary, total: manifest.total, }; diff --git a/tests/scanners/manifest.test.mjs b/tests/scanners/manifest.test.mjs index 1164eed..519f075 100644 --- a/tests/scanners/manifest.test.mjs +++ b/tests/scanners/manifest.test.mjs @@ -5,6 +5,8 @@ import { resolve, join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; import { mkdir, writeFile, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; +import { buildManifest } from '../../scanners/manifest.mjs'; +import { deriveLoadPattern } from '../../scanners/lib/active-config-reader.mjs'; const __dirname = fileURLToPath(new URL('.', import.meta.url)); const PLUGIN_ROOT = resolve(__dirname, '../..'); @@ -64,6 +66,131 @@ describe('manifest CLI — real-config path (plugin root)', () => { it('meta.repoPath matches the requested path', () => { assert.equal(output.meta.repoPath, PLUGIN_ROOT); }); + + it('every source carries a load-pattern triple', () => { + const LOAD_PATTERNS = new Set(['always', 'on-demand', 'external', 'unknown']); + const SURVIVES = new Set(['yes', 'no', 'n/a']); + for (const s of output.sources) { + assert.ok(LOAD_PATTERNS.has(s.loadPattern), + `${s.kind}:${s.name} has invalid loadPattern: ${s.loadPattern}`); + assert.ok(SURVIVES.has(s.survivesCompaction), + `${s.kind}:${s.name} has invalid survivesCompaction: ${s.survivesCompaction}`); + assert.ok(typeof s.derivationConfidence === 'string' && s.derivationConfidence.length > 0, + `${s.kind}:${s.name} missing derivationConfidence`); + } + }); + + it('emits a load-pattern summary with always/on-demand/external buckets', () => { + assert.ok(output.summary, 'expected output.summary'); + for (const bucket of ['always', 'onDemand', 'external']) { + assert.ok(output.summary[bucket], `expected summary.${bucket}`); + assert.equal(typeof output.summary[bucket].tokens, 'number'); + assert.equal(typeof output.summary[bucket].count, 'number'); + } + // The bucket token sums must reconcile with the per-source loadPattern tags. + const alwaysSum = output.sources + .filter(s => s.loadPattern === 'always') + .reduce((a, s) => a + s.estimated_tokens, 0); + assert.equal(output.summary.always.tokens, alwaysSum, + 'summary.always.tokens must equal sum of always-loaded sources'); + }); +}); + +describe('buildManifest — load-pattern accounting (unit)', () => { + // Hand-built activeConfig covering every source kind, with a disabled MCP + // server and a plugin roll-up entry that must both be excluded from the + // ranked sources + totals. + const activeConfig = { + claudeMd: { + files: [ + { path: '/root/CLAUDE.md', scope: 'project', bytes: 100 }, + { path: '/home/.claude/CLAUDE.md', scope: 'user', bytes: 100 }, + ], + totalBytes: 200, + estimatedTokens: 50, // → 25 tokens each + }, + // Must be ignored entirely (roll-up double-counts components). + plugins: [{ name: 'plug', estimatedTokens: 9999 }], + skills: [ + { name: 'sk1', source: 'user', pluginName: null, estimatedTokens: 40 }, + { name: 'sk2', source: 'plugin', pluginName: 'plug', estimatedTokens: 30 }, + ], + rules: [ + { name: 'unscoped.md', source: 'project', pluginName: null, scoped: false, + estimatedTokens: 10, ...deriveLoadPattern('rule', { scoped: false }) }, + { name: 'scoped.md', source: 'project', pluginName: null, scoped: true, + estimatedTokens: 8, ...deriveLoadPattern('rule', { scoped: true }) }, + ], + agents: [ + { name: 'a1', source: 'project', pluginName: null, estimatedTokens: 5, + ...deriveLoadPattern('agent') }, + ], + outputStyles: [ + { name: 's1', source: 'project', pluginName: null, estimatedTokens: 7, + ...deriveLoadPattern('output-style') }, + ], + mcpServers: [ + { name: 'm1', source: 'project', estimatedTokens: 500, enabled: true }, + { name: 'm2', source: 'project', estimatedTokens: 999, enabled: false }, + ], + hooks: [ + { event: 'PreToolUse', matcher: 'Edit', source: 'project', estimatedTokens: 0 }, + ], + }; + + const built = buildManifest(activeConfig); + const byName = (kind, name) => built.sources.find(s => s.kind === kind && s.name === name); + + it('emits no kind:plugin source (roll-up dropped, component-level only)', () => { + assert.ok(built.sources.every(s => s.kind !== 'plugin')); + }); + + it('excludes the plugin roll-up and disabled MCP from the total', () => { + // 25+25 (claude-md) + 40+30 (skills) + 10+8 (rules) + 5 (agent) + 7 (style) + 500 (mcp m1) + assert.equal(built.total, 650); + }); + + it('tags skills as on-demand (body paid on invoke, not every turn)', () => { + assert.equal(byName('skill', 'sk1').loadPattern, 'on-demand'); + assert.equal(byName('skill', 'sk1').survivesCompaction, 'n/a'); + }); + + it('propagates rule load-pattern (unscoped=always, scoped=on-demand)', () => { + assert.equal(byName('rule', 'unscoped.md').loadPattern, 'always'); + assert.equal(byName('rule', 'scoped.md').loadPattern, 'on-demand'); + }); + + it('tags agents and output styles as always-loaded', () => { + assert.equal(byName('agent', 'a1').loadPattern, 'always'); + assert.equal(byName('output-style', 's1').loadPattern, 'always'); + }); + + it('tags MCP servers always and hooks external', () => { + assert.equal(byName('mcp-server', 'm1').loadPattern, 'always'); + assert.equal(byName('hook', 'PreToolUse:Edit').loadPattern, 'external'); + }); + + it('excludes the disabled MCP server from sources', () => { + assert.equal(byName('mcp-server', 'm2'), undefined); + }); + + it('maps CLAUDE.md scope to the correct load pattern', () => { + assert.equal(byName('claude-md', '/root/CLAUDE.md').loadPattern, 'always'); + assert.equal(byName('claude-md', '/root/CLAUDE.md').survivesCompaction, 'yes'); + assert.equal(byName('claude-md', '/home/.claude/CLAUDE.md').loadPattern, 'always'); + }); + + it('summary buckets reconcile with per-source tags', () => { + // always: 25+25+10+5+7+500 = 572 over 6 sources + assert.equal(built.summary.always.tokens, 572); + assert.equal(built.summary.always.count, 6); + // on-demand: 40+30+8 = 78 over 3 sources + assert.equal(built.summary.onDemand.tokens, 78); + assert.equal(built.summary.onDemand.count, 3); + // external: hook (0 tokens) over 1 source + assert.equal(built.summary.external.tokens, 0); + assert.equal(built.summary.external.count, 1); + }); }); describe('manifest CLI — fixture path (rich-repo with patched HOME)', () => { @@ -85,14 +212,20 @@ describe('manifest CLI — fixture path (rich-repo with patched HOME)', () => { `expected sources.length >= 5, got ${out.sources.length}: ${out.sources.map(s => `${s.kind}:${s.name}`).join(', ')}`); }); - it('includes both plugins (manifest-plugin-a + manifest-plugin-b)', () => { + it('drops the coarse plugin roll-up (component-level only, no kind:plugin)', () => { const proc = runCli([fixture.root, '--json'], { HOME: fixture.fakeHome }); const out = JSON.parse(proc.stdout); - const pluginNames = out.sources.filter(s => s.kind === 'plugin').map(s => s.name); - assert.ok(pluginNames.includes('manifest-plugin-a'), - `expected manifest-plugin-a in plugins; got: ${pluginNames.join(', ')}`); - assert.ok(pluginNames.includes('manifest-plugin-b'), - `expected manifest-plugin-b in plugins; got: ${pluginNames.join(', ')}`); + assert.ok(out.sources.every(s => s.kind !== 'plugin'), + `expected no kind:plugin source (roll-up dropped); got: ${out.sources.map(s => s.kind).join(', ')}`); + }); + + it('surfaces plugin-provided skills as component-level sources', () => { + const proc = runCli([fixture.root, '--json'], { HOME: fixture.fakeHome }); + const out = JSON.parse(proc.stdout); + const fromA = out.sources.find(s => s.kind === 'skill' && s.name === 'alpha-skill'); + assert.ok(fromA, 'expected alpha-skill as a component-level skill source'); + assert.ok(/^plugin:/.test(fromA.source), + `expected plugin-provided skill source to be namespaced plugin:*; got: ${fromA.source}`); }); it('includes 3 fixture skills (alpha-skill, beta-skill, gamma-skill)', () => {