From dcf8087972c4880f9ff7c22db5a50d2563971668 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Fri, 1 May 2026 09:39:08 +0200 Subject: [PATCH] chore(config-audit): bump version to 5.0.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - .claude-plugin/plugin.json: 4.0.0 → 5.0.0 - README.md: version badge bump + v5.0.0 row in Version History - CHANGELOG.md: consolidated `## [5.0.0]` entry covering alpha.1, alpha.2, beta.1, rc.1 — Summary, Added, Changed, Removed, Breaking changes, Migration notes, Tests, Notes (incl. SC-6b deferred-to-implementation-log) - root README.md: Config-Audit row v4.0.0 → v5.0.0; counts updated (8→12 scanners, 17→18 commands, 543→635 tests, 4→6 patterns, +manifest command, +--accurate-tokens, +CPS/DIS/COL coverage) No code changes in this commit. Co-Authored-By: Claude Opus 4.7 --- README.md | 13 ++-- .../config-audit/.claude-plugin/plugin.json | 2 +- plugins/config-audit/CHANGELOG.md | 75 +++++++++++++++++++ plugins/config-audit/README.md | 3 +- 4 files changed, 85 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 6c9a48e..0305aee 100644 --- a/README.md +++ b/README.md @@ -51,21 +51,22 @@ Key commands: `/security posture`, `/security audit`, `/security scan`, `/securi --- -### [Config-Audit](plugins/config-audit/) `v4.0.0` +### [Config-Audit](plugins/config-audit/) `v5.0.0` -Configuration intelligence for Claude Code — health checks, feature discovery, auto-fix, active-config inventory, and Opus-4.7-aware token-cost analysis. +Configuration intelligence for Claude Code — health checks, feature discovery, auto-fix, active-config inventory, and reality-based Opus-4.7 token analysis. Claude Code reads instructions from 7+ file types across multiple scopes. This plugin tells you what's wrong, what's missing, what's silently conflicting, what's actually loaded, and where you're burning tokens unnecessarily: -- **Health** — 8 deterministic scanners verify correctness across every configuration file (broken imports, deprecated settings, conflicting rules, permission contradictions, and Opus-4.7-era token waste) +- **Health** — 12 deterministic scanners verify correctness across every configuration file (broken imports, deprecated settings, conflicting rules, permission contradictions, Opus-4.7-era token waste, cache-prefix instability, dead tool grants, cross-plugin skill collisions) - **Opportunities** — context-aware recommendations for Claude Code features you're not using - **Action** — auto-fix with mandatory backups, syntax validation, rollback support, and human-in-the-loop workflow - **What's active** — read-only inventory of plugins, skills, MCP servers, hooks, and CLAUDE.md cascade for a repo, with token estimates -- **Token hotspots** — `/config-audit tokens` ranks files by estimated waste against 4 Opus-4.7 patterns (cache-breaking volatile content, redundant tool permissions, deep import chains, sonnet-era setups) +- **Token hotspots** — `/config-audit tokens` ranks files by estimated waste across 6 Opus-4.7 patterns (cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascades, bloated SKILL.md descriptions, MCP tool-schema budget). Optional `--accurate-tokens` calibrates against Anthropic's `count_tokens` API. +- **System-prompt manifest** — `/config-audit manifest` ranks every token source (CLAUDE.md cascade, plugins, skills, MCP servers, hooks) by estimated tokens -Key commands: `/config-audit posture`, `/config-audit feature-gap`, `/config-audit fix`, `/config-audit whats-active`, `/config-audit tokens` +Key commands: `/config-audit posture`, `/config-audit feature-gap`, `/config-audit fix`, `/config-audit whats-active`, `/config-audit tokens`, `/config-audit manifest` -6 agents · 9 scanners · 17 commands · 543+ tests +6 agents · 12 scanners · 18 commands · 635+ tests → [Full documentation](plugins/config-audit/README.md) diff --git a/plugins/config-audit/.claude-plugin/plugin.json b/plugins/config-audit/.claude-plugin/plugin.json index ae6564c..f7e8a9a 100644 --- a/plugins/config-audit/.claude-plugin/plugin.json +++ b/plugins/config-audit/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "config-audit", "description": "Multi-agent workflow for analyzing, reporting, and optimizing Claude Code configuration across your entire machine", - "version": "4.0.0", + "version": "5.0.0", "author": { "name": "Kjell Tore Guttormsen" }, diff --git a/plugins/config-audit/CHANGELOG.md b/plugins/config-audit/CHANGELOG.md index c6b60df..8973fd0 100644 --- a/plugins/config-audit/CHANGELOG.md +++ b/plugins/config-audit/CHANGELOG.md @@ -5,6 +5,81 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [5.0.0] - 2026-05-01 + +### Summary +Reality-based token-optimization release. v4.0.0 shipped Opus-4.7 token surfaces aligned to a Sonnet-era cost model; v5.0.0 rebuilds the foundations against verified Opus-4.7 cost dynamics. Three pillars: honest token estimation (severity-weighted scoring, MCP estimates 15 → 500+, optional `--accurate-tokens` API calibration), new structural scanners (cache-prefix stability, dead tool grants, plugin collisions), and new diagnostic surfaces (`/config-audit manifest`, `/config-audit tokens` extended, knowledge-base rensing aligned to Opus 4.7 cache dynamics). + +Consolidated from `5.0.0-alpha.1` (F1-F5 token-economy round), `5.0.0-alpha.2` (M1, M2, M4-M6, F6, F7 structural gaps + README self-audit), `5.0.0-beta.1` (N1-N4, N6 new scanners + manifest CLI), and `5.0.0-rc.1` (M7, M8 knowledge rensing + N5 tokenizer calibration). + +### Added +- **3 new scanners (9 → 12 deterministic):** + - **CPS — Cache-Prefix Stability** (`CA-CPS-NNN`): volatile content in lines 31–150 of CLAUDE.md cascade, beyond TOK Pattern A's top-30 window. Volatile-pattern set extends Pattern A with shell-exec lines (`!` prefix) and `${VAR}` substitutions. + - **DIS — Disabled-In-Schema** (`CA-DIS-NNN`): tools listed in BOTH `permissions.deny` AND `permissions.allow`. Tool identity uses bare name (`Bash(npm:*)` and `Bash` are the same tool). Severity low. + - **COL — Cross-Plugin Skill Collision** (`CA-COL-001`): plugin-vs-plugin same skill name → low; user-vs-plugin → medium. `details.namespaces` payload identifies conflicting sources. +- **TOK extensions:** + - **CA-TOK-005 MCP tool-schema budget:** per-server tiered finding (< 20 none, 20–49 low, 50–99 medium, 100+ high; null low + "tool count unknown"). Scoped to project-local `.mcp.json`. + - **Pattern E — Oversized cascade:** medium when `activeConfig.claudeMd.estimatedTokens > 10_000`. + - **Pattern F — Bloated SKILL.md description:** low when frontmatter `description > 500 chars` (loads every turn). Scoped to `discovery.files`. +- **`/config-audit manifest`** + `scanners/manifest.mjs` CLI — single ranked table of every system-prompt token source (CLAUDE.md cascade, plugins, skills, MCP servers, hooks) sorted DESC by `estimated_tokens`. CLAUDE.md per-file tokens distributed proportional to bytes. +- **`--accurate-tokens` flag** on `token-hotspots-cli.mjs` (N5): when `ANTHROPIC_API_KEY` is set, calls Anthropic's `count_tokens` for the top 3 hotspots and populates `output.calibration = { actual_tokens, source: 'count_tokens_api', sampled_hotspots: 3 }`. When absent: `calibration = { skipped: 'no-api-key' }` plus stderr warning. +- **`scanners/lib/tokenizer-api.mjs`** — `count_tokens` wrapper. 5s AbortController timeout. Exponential backoff on 429 (3 retries: 1s/2s/4s). API key masked to `${key.slice(0,8)}...` in every error; HTTP body never included in errors (it may echo the key on auth failures). `maskKey()` exported. +- **`--with-telemetry-recipe` flag** on the same CLI (M7): emits `telemetry_recipe_path` field pointing to `knowledge/cache-telemetry-recipe.md`. +- **`knowledge/cache-telemetry-recipe.md`** (M7): manual `jq` recipe summing `cache_read_input_tokens` + `cache_creation_input_tokens` per turn from session transcripts. Hit-rate interpretation table. +- **`'mcp'` kind on `estimateTokens`** (F2): active MCP servers estimate ≥ 500 tokens (base + schema overhead) instead of v4's flat 15. Optional `{toolCount}` raises to `500 + toolCount × 200`. +- **MCP tool-count detection** (M1): `readActiveMcpServers` resolves count via cache → `node_modules//package.json` → `{toolCount: null, toolCountUnknown: true}` fallback. +- **`additionalDirectories` settings key** (M6): added to `KNOWN_KEYS`; new low-severity finding when length > 2. +- **HKV verbose hook output** (M5): low-severity finding when referenced hook script contains > 50 `console.log`/`process.stdout.write` lines (static, no execution). +- **`self-audit --check-readme` flag** (F6): filesystem counts compared against README badges. Helper `checkReadmeBadges(pluginDir)`. Step 28 of v5 plan reconciled all badges. +- **`scoringVersion: 'v5'`** field on `scoreByArea` output for cross-version drift detection. +- **`WEIGHTS`** named export from `scanners/lib/severity.mjs` (frozen). +- **`details` field on findings** (`output.mjs:finding()`): optional structured payload for scanner-specific data (used by COL). +- **Plugin Hygiene** as 10th quality area (from COL). Posture JSON now reports 10 areas. +- **TOK-readActiveConfig integration** (F1): one hotspot per active MCP server; `result.activeConfig` summary (claudeMd cascade tokens, mcpServerCount, pluginCount, skillCount); try/catch fallback when scope-limited. + +### Changed +- **F3 — `scoreByArea` is severity-weighted.** Penalty = `Σ count[s] × WEIGHTS[s]`; `passRate = max(0, 100 − penalty / max(10, findingCount × 4) × 100)`. Lows no longer crater an area's grade; criticals/highs do. `baseline-all-a` fixture remains all-A (no critical/high present). +- **F7 — TOK pattern severities recalibrated** for tokens-per-turn impact: Pattern A `medium → high`, Pattern B `low → medium`, Pattern C `medium → low`. Each finding carries a `calibration_note` evidence field documenting the heuristic basis. +- **`scoreByArea` deduplicates by area name** (N3 prep): TOK + CPS share "Token Efficiency"; SET + DIS share "Settings". Combined row with merged finding counts. +- **M8 — knowledge rensing:** replaced "Keep CLAUDE.md under 200 lines" in `knowledge/configuration-best-practices.md` with cache-stability guidance (first 30 lines stable, volatile content below the cache threshold). Footnote explains the 200-line rule was a Sonnet-era adherence heuristic; Opus 4.7 uses prompt-cache structure as the dominant cost lever. Cross-references `knowledge/opus-4.7-patterns.md`. +- **`commands/tokens.md` next-steps:** documents `--with-telemetry-recipe` as the cache-verification path. +- **Scanner count: 9 → 12.** Command count: 17 → 18. Knowledge: 7 → 8. Quality areas: 8 → 10. +- **`.gitignore`** — unignore rules for `tests/fixtures/**/node_modules/` so the `mcp-tool-heavy` fixture stays under version control. + +### Removed +- **F4 — TOK hotspot padding loop and `take` dead-code.** Hotspots may now contain fewer than 3 entries for tiny projects (the honest answer); contract still bounds at ≤ 10. +- **F5 — Pattern D / `CA-TOK-004` (sonnet-era signature).** Catalogue entry removed from `knowledge/opus-4.7-patterns.md` and `commands/tokens.md`. Suppression entries for `CA-TOK-004` are now no-ops. + +### Breaking changes +- **F2 — MCP token estimates jump from flat 15 to ≥ 500.** Token Efficiency grades for projects with MCP servers may shift. `whats-active` totals report higher numbers. Documented in `commands/posture.md` next-steps. +- **F3 — `scoreByArea` is severity-weighted.** Posture JSON consumers reading `areas[*].score` will see different values for non-clean configs. Use `result.scoringVersion === 'v5'` to detect the change. Drift comparisons across v4↔v5 baselines may show artificial deltas — re-baseline after upgrade. +- **F5 — Pattern D / `CA-TOK-004` no longer emitted.** Existing exact `CA-TOK-004` suppression entries are harmless but obsolete. +- **N1 suppression backward-compat — `CA-TOK-*` glob now also matches `CA-TOK-005`.** To preserve prior behavior of suppressing only patterns A/B/C, replace the glob with explicit IDs: + ``` + CA-TOK-001 + CA-TOK-002 + CA-TOK-003 + ``` + A one-time runtime warning for this case is a v5.0.1 candidate. +- **Posture areas count: 9 → 10** (Plugin Hygiene from COL). Consumers hard-coding 9 must update. + +### Migration notes +- `CA-TOK-*` glob suppressions: explicit-ID list recommended if CA-TOK-005 should not be suppressed. +- `CA-TOK-004` exact-ID suppression entries: safe to remove. +- Drift baselines created against v4 should be re-saved post-upgrade to avoid artificial F3 weighting deltas. +- Posture JSON consumers must update any hardcoded `areas.length === 8` or `=== 9` assertions to `>= 10`. + +### Tests +- 543 → 635 (+92): F1-F7 (alpha rounds = +43), N1-N4 + N6 (beta = +39), M7 + M8 + N5 (rc = +10). 36 test files (12 lib + 23 scanner + 1 hook). +- New fixtures: `tok-active-config/`, `additional-dirs-many/`, `additional-dirs-ok/`, `large-cascade/`, `small-cascade/`, `skill-bloated/`, `skill-tight/`, `mcp-tool-heavy/` (with mocked `node_modules/`), `hooks-verbose/`, `hooks-quiet/`, `readme-desynced/`, `mcp-budget/{14,25,60,120,unknown}-tools/`, `volatile-mid-section/{volatile-line-60,volatile-line-200}/`, `denied-tools-in-schema/`, `collision-plugins/fake-home/` (plugin-a + plugin-b + plugin-c + user-level review skill). +- New test files: `tests/scanners/manifest.test.mjs`, `tests/scanners/cache-prefix.test.mjs`, `tests/scanners/disabled-in-schema.test.mjs`, `tests/scanners/collision.test.mjs`, `tests/scanners/accurate-tokens.test.mjs`. + +### Notes +- **`mock.method` against ESM module exports does not work** (Node 18+ ESM read-only export bindings). v5 tests use `globalThis.fetch` mocking for `--accurate-tokens` instead — equivalent coverage at the actual external-dependency boundary. +- **Plugin-vs-built-in collision detection is intentionally not implemented.** Step 22a research spike (`docs/v5-namespace-research.md`, gitignored) could not verify Claude Code's resolution behavior when a plugin command shares a name with a built-in. Treated as info-only; v5.0.1 candidate. +- **README/CLAUDE.md badge reconciliation** done in Step 28 (this release). `self-audit --check-readme` PASSES against the filesystem. Test count counter switched from file-count to test-case count via subprocess `node --test` parse. +- **SC-6b release-gate** (±5% tokenizer accuracy against real `count_tokens` API): documented separately in `docs/v5-implementation-log.md` Session 5. Either verified at release time with a live API key, or recorded as deferred — see implementation log for the actual outcome. + ## [5.0.0-rc.1] - 2026-05-01 ### Summary diff --git a/plugins/config-audit/README.md b/plugins/config-audit/README.md index 0011fec..dac0e47 100644 --- a/plugins/config-audit/README.md +++ b/plugins/config-audit/README.md @@ -6,7 +6,7 @@ *AI-generated: all code produced by Claude Code through dialog-driven development. [Full disclosure →](../../README.md#ai-generated-code-disclosure)* -![Version](https://img.shields.io/badge/version-4.0.0-blue) +![Version](https://img.shields.io/badge/version-5.0.0-blue) ![Platform](https://img.shields.io/badge/platform-Claude_Code_Plugin-purple) ![Scanners](https://img.shields.io/badge/scanners-12-cyan) ![Commands](https://img.shields.io/badge/commands-18-green) @@ -544,6 +544,7 @@ This plugin is cautious by design — configuration files are important, and a b | Version | Date | Highlights | |---------|------|-----------| +| **5.0.0** | 2026-05-01 | Reality-based token-optimization. 3 new scanners (CPS cache-prefix, DIS dead tools, COL plugin collisions) → 12 deterministic scanners. New `/config-audit manifest` and `--accurate-tokens` API calibration. Severity-weighted scoring (`scoringVersion: 'v5'`). MCP token estimates 15 → 500+. Plugin Hygiene as 10th quality area. Knowledge: cache-stability replaces 200-line rule, cache-telemetry recipe. **Breaking:** F2 token magnitude jump, F3 severity weighting, F5 Pattern D removed, N1 `CA-TOK-*` glob now matches CA-TOK-005. 635 tests | | **4.0.0** | 2026-04-19 | Opus 4.7 era: new TOK scanner (cache-breaking volatile content, redundant tool permissions, deep import chains, sonnet-era setups), `/config-audit tokens` command, Token Efficiency 8th quality area, scanner-agent + verifier-agent migrated haiku → sonnet. 543 tests | | **3.1.0** | 2026-04-14 | New `/config-audit whats-active` — read-only inventory of active plugins, skills, MCP, hooks, CLAUDE.md for a repo, with token estimates. 522 tests | | **3.0.1** | 2026-04-04 | Cross-platform fix: Windows path separators. 486 tests |