open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

feat(skill-listing): add SKL scanner for the skill-listing token budget

Browse source 
Fase 4 Items 2+3 (CC 2.1.114→181 gap-review). New orchestrated scanner
`skill-listing-scanner.mjs` (prefix SKL) flags every active skill whose
description exceeds the verified 1,536-char listing cap (CC 2.1.105, changelog
L1502). Past the cap, Claude Code silently truncates the description the model
reads to route skill invocation — dropping the trigger phrases at the tail.
HOME-scoped over all user + plugin skills via enumerateSkills (COL is the model).

- CA-SKL-001 (medium): description > 1,536 chars. Remediation folds in Item
  2(b) — recommends disableBundledSkills + skillOverrides + trimming
  (designvalg A: no standalone GAP-check, which would fire for nearly everyone).
- Designvalg B: v1 ships the verified cap ONLY. The aggregate 2%-of-context
  listing budget is deferred — it needs a context-window assumption that would
  turn a verified fact into a guess (would carry a CALIBRATION_NOTE if added).
- Choice C: recognize the skillOverrides settings key (CC 2.1.129) in
  KNOWN_KEYS. Left OUT of TYPE_CHECKS — the value is a per-skill object
  (off/user-invocable-only/name-only), not a string; a 'string' check (as the
  plan sketched) would create a NEW false positive. Verify-first deviation.

Registration: scan-orchestrator (13th scanner), humanizer (SKL → 'Wasted
tokens' + static/_default translations), scoring SCANNER_AREA_MAP (→ Token
Efficiency; no 11th area), README badge 12→13, CLAUDE.md (finding-id +
test-count), docs/scanner-internals.md, gap-matrix + plan status notes.

Snapshots reseeded hermetically (SEED_SNAPSHOT/UPDATE_SNAPSHOT): SKL entry with
0 findings in empty HOME, scanners_ok 11→12, claudeMdEstimatedTokens bump from
the CLAUDE.md edits flowing through the cascade. Contamination grep clean.

Suite 868/868 (856 baseline + 11 SKL + 1 skillOverrides). RED→GREEN logged
per cycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
This commit is contained in:
Kjell Tore Guttormsen 2026-06-18 17:36:28 +02:00
commit 7bb254780a
22 changed files with 390 additions and 37 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tests/snapshots/default-output/posture.json
View file

@ -1,4 +1,4 @@
{
"kind": "text",
"payload": "`[CML] CLAUDE.md Linter`: 1 finding(s) (0ms)\n `[SET] Settings Validator`: 0 finding(s) (0ms)\n `[HKV] Hook Validator`: 0 finding(s) (0ms)\n `[RUL] Rules Validator`: 0 finding(s) (0ms)\n `[MCP] MCP Config Validator`: 0 finding(s) (0ms)\n `[IMP] Import Resolver`: 0 finding(s) (0ms)\n `[CNF] Conflict Detector`: 0 finding(s) (0ms)\n `[GAP] Feature Gap Scanner`: 17 finding(s) (0ms)\n `[TOK] Token Hotspots`: 1 finding(s) (0ms)\n `[CPS] Cache-Prefix Stability`: 0 finding(s) (0ms)\n `[DIS] Disabled-In-Schema`: 0 finding(s) (0ms)\n `[COL] Plugin Skill Collision`: 0 finding(s) (0ms)\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Configuration health\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n Health: A (98/100) — Healthy setup, only minor polish needed\n 9 areas reviewed\n\n Area scores\n ───────────\n `CLAUDE.md` ........... A (90) `Settings` ............ A (100)\n `Hooks` ............... A (100) `Rules` ............... A (100)\n `MCP` ................. A (100) `Imports` ............. A (100)\n `Conflicts` ........... A (100) `Token Efficiency` .... A (90)\n `Plugin Hygiene` ...... A (100)\n\n 17 ways you could get more out of Claude Code — see /config-audit feature-gap\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
"payload": "`[CML] CLAUDE.md Linter`: 1 finding(s) (0ms)\n `[SET] Settings Validator`: 0 finding(s) (0ms)\n `[HKV] Hook Validator`: 0 finding(s) (0ms)\n `[RUL] Rules Validator`: 0 finding(s) (0ms)\n `[MCP] MCP Config Validator`: 0 finding(s) (0ms)\n `[IMP] Import Resolver`: 0 finding(s) (0ms)\n `[CNF] Conflict Detector`: 0 finding(s) (0ms)\n `[GAP] Feature Gap Scanner`: 17 finding(s) (0ms)\n `[TOK] Token Hotspots`: 1 finding(s) (0ms)\n `[CPS] Cache-Prefix Stability`: 0 finding(s) (0ms)\n `[DIS] Disabled-In-Schema`: 0 finding(s) (0ms)\n `[COL] Plugin Skill Collision`: 0 finding(s) (0ms)\n `[SKL] Skill-Listing Budget`: 0 finding(s) (0ms)\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Configuration health\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n Health: A (98/100) — Healthy setup, only minor polish needed\n 9 areas reviewed\n\n Area scores\n ───────────\n `CLAUDE.md` ........... A (90) `Settings` ............ A (100)\n `Hooks` ............... A (100) `Rules` ............... A (100)\n `MCP` ................. A (100) `Imports` ............. A (100)\n `Conflicts` ........... A (100) `Token Efficiency` .... A (90)\n `Plugin Hygiene` ...... A (100)\n\n 17 ways you could get more out of Claude Code — see /config-audit feature-gap\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
}

16
tests/snapshots/default-output/scan-orchestrator.json
View file

@ -539,6 +539,20 @@
"low": 0,
"info": 0
}
},
{
"scanner": "SKL",
"status": "ok",
"files_scanned": 0,
"duration_ms": 0,
"findings": [],
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"info": 0
}
}
],
"aggregate": {
@ -553,7 +567,7 @@
"risk_score": 11,
"risk_band": "Medium",
"verdict": "PASS",
"scanners_ok": 11,
"scanners_ok": 12,
"scanners_error": 0,
"scanners_skipped": 1
}

13
tests/snapshots/v5.0.0-stderr/posture.txt
View file

@ -1,15 +1,16 @@
[CML] CLAUDE.md Linter: 1 finding(s) (14ms)
[SET] Settings Validator: 0 finding(s) (2ms)
[HKV] Hook Validator: 0 finding(s) (1ms)
[SET] Settings Validator: 0 finding(s) (1ms)
[HKV] Hook Validator: 0 finding(s) (0ms)
[RUL] Rules Validator: 0 finding(s) (0ms)
[MCP] MCP Config Validator: 0 finding(s) (0ms)
[IMP] Import Resolver: 0 finding(s) (2ms)
[IMP] Import Resolver: 0 finding(s) (1ms)
[CNF] Conflict Detector: 0 finding(s) (1ms)
[GAP] Feature Gap Scanner: 17 finding(s) (2ms)
[TOK] Token Hotspots: 1 finding(s) (8ms)
[CPS] Cache-Prefix Stability: 0 finding(s) (0ms)
[DIS] Disabled-In-Schema: 0 finding(s) (1ms)
[COL] Plugin Skill Collision: 0 finding(s) (0ms)
[CPS] Cache-Prefix Stability: 0 finding(s) (1ms)
[DIS] Disabled-In-Schema: 0 finding(s) (0ms)
[COL] Plugin Skill Collision: 0 finding(s) (1ms)
[SKL] Skill-Listing Budget: 0 finding(s) (0ms)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Config-Audit Health Score

30
tests/snapshots/v5.0.0/posture.json
View file

@ -94,7 +94,7 @@
"scannerEnvelope": {
"meta": {
"target": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium",
"timestamp": "2026-06-18T12:19:30.125Z",
"timestamp": "2026-06-18T15:31:48.663Z",
"version": "2.2.0",
"tool": "config-audit"
},
@ -103,7 +103,7 @@
"scanner": "CML",
"status": "ok",
"files_scanned": 1,
"duration_ms": 2,
"duration_ms": 3,
"findings": [
{
"id": "CA-CML-001",
@ -145,7 +145,7 @@
"scanner": "HKV",
"status": "ok",
"files_scanned": 1,
"duration_ms": 2,
"duration_ms": 1,
"findings": [],
"counts": {
"critical": 0,
@ -173,7 +173,7 @@
"scanner": "MCP",
"status": "ok",
"files_scanned": 1,
"duration_ms": 0,
"duration_ms": 1,
"findings": [],
"counts": {
"critical": 0,
@ -187,7 +187,7 @@
"scanner": "IMP",
"status": "ok",
"files_scanned": 1,
"duration_ms": 2,
"duration_ms": 1,
"findings": [],
"counts": {
"critical": 0,
@ -526,7 +526,7 @@
],
"total_estimated_tokens": 801,
"activeConfig": {
"claudeMdEstimatedTokens": 1639,
"claudeMdEstimatedTokens": 1647,
"mcpServerCount": 1,
"pluginCount": 0,
"skillCount": 0
@ -536,7 +536,7 @@
"scanner": "CPS",
"status": "ok",
"files_scanned": 1,
"duration_ms": 0,
"duration_ms": 1,
"findings": [],
"counts": {
"critical": 0,
@ -573,6 +573,20 @@
"low": 0,
"info": 0
}
},
{
"scanner": "SKL",
"status": "ok",
"files_scanned": 0,
"duration_ms": 0,
"findings": [],
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"info": 0
}
}
],
"aggregate": {
@ -587,7 +601,7 @@
"risk_score": 11,
"risk_band": "Medium",
"verdict": "PASS",
"scanners_ok": 11,
"scanners_ok": 12,
"scanners_error": 0,
"scanners_skipped": 1
}

26
tests/snapshots/v5.0.0/scan-orchestrator.json
View file

@ -1,7 +1,7 @@
{
"meta": {
"target": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium",
"timestamp": "2026-06-18T12:19:29.934Z",
"timestamp": "2026-06-18T15:31:48.479Z",
"version": "2.2.0",
"tool": "config-audit"
},
@ -80,7 +80,7 @@
"scanner": "MCP",
"status": "ok",
"files_scanned": 1,
"duration_ms": 0,
"duration_ms": 1,
"findings": [],
"counts": {
"critical": 0,
@ -94,7 +94,7 @@
"scanner": "IMP",
"status": "ok",
"files_scanned": 1,
"duration_ms": 2,
"duration_ms": 1,
"findings": [],
"counts": {
"critical": 0,
@ -108,7 +108,7 @@
"scanner": "CNF",
"status": "ok",
"files_scanned": 2,
"duration_ms": 1,
"duration_ms": 0,
"findings": [],
"counts": {
"critical": 0,
@ -433,7 +433,7 @@
],
"total_estimated_tokens": 801,
"activeConfig": {
"claudeMdEstimatedTokens": 1639,
"claudeMdEstimatedTokens": 1647,
"mcpServerCount": 1,
"pluginCount": 0,
"skillCount": 0
@ -480,6 +480,20 @@
"low": 0,
"info": 0
}
},
{
"scanner": "SKL",
"status": "ok",
"files_scanned": 0,
"duration_ms": 0,
"findings": [],
"counts": {
"critical": 0,
"high": 0,
"medium": 0,
"low": 0,
"info": 0
}
}
],
"aggregate": {
@ -494,7 +508,7 @@
"risk_score": 11,
"risk_band": "Medium",
"verdict": "PASS",
"scanners_ok": 11,
"scanners_ok": 12,
"scanners_error": 0,
"scanners_skipped": 1
}