Commit graph

9 commits

Author SHA1 Message Date
Kjell Tore Guttormsen
6cfca82885 fix(config-audit): expose hotspot.path for --accurate-tokens calibration + SC-6b PASS
The v5.0.0-rc.1 N5 implementation looked up hotspot.path in
calibrateAgainstApi() but token-hotspots.mjs only emitted hotspot.source —
calibration silently produced 0 actual_tokens because every iteration hit
the `if (!hotspot?.path) continue` guard.

Fix: file-backed hotspots now expose `path: h.absPath` in the JSON output.
MCP-server hotspots intentionally leave path unset — their tokens are
runtime tool-schema (formula-based: 500 + toolCount × 200), not file
content readable by count_tokens.

SC-6b release-gate verified against tests/fixtures/marketplace-large:
- Actual (count_tokens, claude-opus-4-7): 589 tokens for CLAUDE.md
- Estimated (4-bytes/token byte heuristic): 594 tokens
- Delta: -5 tokens / -0.85% — well within ±5% gate. PASS.

CHANGELOG: documented the fix + SC-6b result inline under [5.0.0].

All 635 tests still green. No estimateTokens tuning required for v5.0.0.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 09:45:56 +02:00
Kjell Tore Guttormsen
b2407a09b3 feat(config-audit): CA-TOK-005 MCP tool-schema budget (v5 N1) [skip-docs]
Adds detectMcpToolBudget detection block in TOK scanner. Tiered severity
per project-local .mcp.json server based on toolCount:
- < 20: no finding
- 20-49: low
- 50-99: medium
- 100+: high
- null (manifest unparseable): low + "tool count unknown" message

Scoped to source==='.mcp.json' to keep findings actionable for the
audited path; plugin/user-level MCP servers are surfaced by the
manifest scanner (Step 19 / N2).

5 fixtures (mcp-budget/{14,25,60,120,unknown}-tools) use inline `tools`
arrays in .mcp.json — no node_modules needed for these tests.

Tests assert title+severity (not exact ID) since TOK IDs are sequential
per scan, not semantic per pattern.

[skip-docs] reason: v5 plan fences off README/CLAUDE.md badge updates
to Session 5; Forgejo pre-commit-docs-gate hook requires this tag on
feat commits without doc changes.

Tests: 586 → 593 (+7).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 07:29:57 +02:00
Kjell Tore Guttormsen
9a44df22ac feat(config-audit): TOK flags skill description > 500 chars (v5 M2) [skip-docs]
- New Pattern F in TOK: low-severity finding when SKILL.md description > 500 chars
- Scoped to discovery.files (project-local) — activeConfig.skills walk would
  pull in user/plugin skills out of project scope
- New fixtures: skill-bloated (594-char desc) + skill-tight (46-char baseline)

574 → 576 tests, all green.
2026-05-01 06:58:42 +02:00
Kjell Tore Guttormsen
25ca6139b4 feat(config-audit): TOK flags CLAUDE.md cascade > 10k tokens (v5 M4) [skip-docs]
- New Pattern E in TOK: emits medium finding when activeConfig.claudeMd.estimatedTokens > 10_000
- Uses cascade tokens, file count, and calibration note as evidence
- New fixtures: large-cascade (37k bytes / 14475 cascade tokens) + small-cascade (5k baseline)

572 → 574 tests, all green.
2026-05-01 06:53:12 +02:00
Kjell Tore Guttormsen
58d6b5b9ea feat(config-audit): recalibrate TOK severities for tokens/turn (v5 F7) [skip-docs]
- Pattern A (cache-breaking volatile top): medium → high
- Pattern B (redundant permissions): low → medium
- Pattern C (deep @import chain): medium → low
- Add calibration_note evidence on every TOK finding
- Table-driven severity tests (identify by title, IDs are sequential)

563 → 569 tests, all green. Doc sweep deferred to Session 5 (Step 28).
2026-05-01 06:47:32 +02:00
Kjell Tore Guttormsen
2810ee6f62 feat(config-audit): remove TOK Pattern D detectSonnetEra (v5 F5)
Pattern D was the v4 sonnet-era signature: 'config is structurally
clean but uses no Opus-4.7-specific features'. Two problems:
- It triggered on any minimal config that happened to lack skills/MCP
- The advice was generic and not actionable

The hotspots ranking and per-pattern findings (A/B/C) cover the same
ground with concrete, file-anchored signal. Dropping the noise.

BREAKING (intentional): scanners no longer emit the sonnet-era info
finding. Suppression entries and downstream tooling that reference
the v4 finding ID should be updated. Doc sweep follows in Step 8b.

Tests: sonnet-era fixture now asserts zero findings.
2026-05-01 06:31:43 +02:00
Kjell Tore Guttormsen
0d8a9af3d6 fix(config-audit): remove TOK dead take + hotspot padding (v5 F4)
The buildHotspots padding loop and unused 'take' variable were dead
code from the v3 hotspots-min contract. Replaced with a clean
ranked.slice(0, HOTSPOTS_MAX). Tiny fixtures may now return fewer
than 3 hotspots, which is the honest answer; the contract now only
asserts <= 10.

Tests: +2 cases — every hotspot.source is unique (no padding); length
never exceeds HOTSPOTS_MAX.
2026-05-01 06:29:33 +02:00
Kjell Tore Guttormsen
34669d596c feat(config-audit): TOK consumes readActiveConfig (v5 F1)
Removes the v4 'void readActiveConfig' placeholder and wires the
active-config snapshot into the TOK scanner.

Per-turn behavior changes:
- Each enabled MCP server becomes its own hotspot entry (richer than
  the parent .mcp.json file alone)
- total_estimated_tokens now includes MCP server cost
- result.activeConfig exposes a small summary
  (claudeMdEstimatedTokens, mcpServerCount, pluginCount, skillCount)

Failures of readActiveConfig are non-fatal — the scanner falls back
to the discovery-only path used in v4.

Tests: +3 cases on the new tok-active-config fixture
(.mcp.json with 2 servers, CLAUDE.md, plugin skeleton).
2026-05-01 06:27:34 +02:00
Kjell Tore Guttormsen
a090ed3a9f feat(config-audit): add token-hotspots (TOK) scanner — Opus 4.7 pattern catalogue + ranked hotspots 2026-04-19 22:43:17 +02:00