open/config-audit
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions

feat(skill-listing): add CA-SKL-002 aggregate listing-budget check

Browse source 
Syklus 2 of Fase 4 Items 2+3. Flags when the sum of active skill
descriptions exceeds the listing budget (~2% of context, CC 2.1.32).

Design (operator-confirmed "fact-first, 200k anchor"):
- low severity (estimate) vs medium for the verified 1,536-char cap
- each description counted up to the 1,536 cap (what actually loads in
  the listing) — avoids double-counting the tail CA-SKL-001 flags
- fires when sum > 2% x 200k = 4000 tok; evidence leads with the measured
  sum + a calibration note that the budget scales 5x on 1M-context models
- aggregate emitted after the per-skill loop so the common case reads
  001=cap, 002=aggregate (finding IDs are a sequential counter, not stable
  semantic IDs — tests match on title, never NNN)

Also:
- tailored humanizer static entry for the aggregate title
- fix latent HOME leak in posture-grade-stability.test.mjs: it spawned
  posture.mjs without hermeticEnv(), so a real ~/.claude leaked HOME-scoped
  SKL/COL findings into the baseline grade (Token Efficiency A->B). Now
  isolated like the 8 other CLI-spawning tests.
- docs sync: test count 868->875, scanner-internals, gap-matrix, plan status

Suite 875/875, no snapshot drift, self-audit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
This commit is contained in:
Kjell Tore Guttormsen 2026-06-18 18:06:17 +02:00
commit 66433fee48
8 changed files with 215 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

4
docs/cc-2.1.x-fase4-items-2-3-plan.md
View file

@ -1,6 +1,8 @@
# Plan — Fase 4 Items 2 + 3 (skill-listing token-styring)
**Skrevet:** 2026-06-18 · **Scope:** kun `config-audit/` · **Status:** ✅ **LEVERT 2026-06-18.** Designvalg bekreftet: A=fold inn i SKL, B=v1 KUN 1 536-cap (aggregat utsatt), C=`skillOverrides` i KNOWN_KEYS (IKKE TYPE_CHECKS — verdien er objekt; avvik fra §C-skissen, verify-først). Syklus 1+3+4 ferdig; Syklus 2 (aggregat CA-SKL-002) gjenstår som valgfri stretch. Suite 868/868. Detaljer i `STATE.md`.
**Skrevet:** 2026-06-18 · **Scope:** kun `config-audit/` · **Status:** ✅ **LEVERT 2026-06-18 (alle 4 sykluser).** Designvalg bekreftet: A=fold inn i SKL, B=**Syklus 2 NÅ LEVERT** — `CA-SKL-002` aggregat-budsjett implementert med «faktum-først, 200k-anker» (operatør-bekreftet 2026-06-18): low severity, sum av beskrivelser (hver tellet opp til 1 536-cap) > 2 %×200k = 4000 tok → funn m/ CALIBRATION_NOTE om 5×-skalering på 1M-kontekst. C=`skillOverrides` i KNOWN_KEYS (IKKE TYPE_CHECKS — verdien er objekt; avvik fra §C-skissen, verify-først). Suite **875/875**. Detaljer i `STATE.md`.
**Syklus 2-notat (avvik fra §B-skissen, verify-først):** (1) Aggregatet teller hver beskrivelse **capped på 1 536** (= det som faktisk lastes i listingen; unngår dobbelttelling av halen `CA-SKL-001` allerede flagger). (2) Finding-IDs er **sekvensiell teller** (`output.mjs:33`), ikke stabil semantisk ID — tester matcher på **tittel**, aldri NNN; aggregatet emitteres ETTER cap-loopen så vanlig-tilfellet leser 001=cap, 002=aggregat. (3) Eksponerte en latent **HOME-lekkasje** i `posture-grade-stability.test.mjs` (kjørte CLI uten hermetisk HOME → ekte ~/.claude lekket inn → Token Efficiency falt A→B); fikset med `hermeticEnv()` (samme mønster som 8 andre CLI-tester). (4) Egen tilpasset humanizer-`static`-entry for aggregat-tittelen (ikke generisk `_default`).
**Forrige:** Fase 4 Item 1 (4.7→modell-nøytral) levert som `8376dab`. Baseline **856/856 grønn**.
## Mål

2
docs/cc-2.1.x-gap-matrix.md
View file

@ -97,7 +97,7 @@ Netto-nytt scanner-arbeid (DIS param-bevissthet, PLH shadow-folder-check, permis
| settings.json | env MAX_THINKING_TOKENS=0 disables thinking | 2.1.166 | missing | update-knowledge | knowledge/claude-code-capabilities.md | L | S |
| settings.json | parentSettingsBehavior / policyHelper (UNVERIFIED) | 2.1.133 | missing | new-feature-gap-check | scanners/settings-validator.mjs | L | S |
| settings.json | skillOverrides | 2.1.129 | DONE | added to KNOWN_KEYS (NOT TYPE_CHECKS — value is per-skill object off/user-invocable-only/name-only) | scanners/settings-validator.mjs | L | S |
| settings.json | ~~skillListingBudgetFraction / maxSkillDescriptionChars~~ (keys DO NOT EXIST — verified) | 2.1.105 | DONE | budget=2% ctx + cap=1536 are internal CC mechanisms, not settings keys; the verified 1,536-char cap is enforced by the new SKL scanner (CA-SKL-001) | scanners/skill-listing-scanner.mjs | L | M |
| settings.json | ~~skillListingBudgetFraction / maxSkillDescriptionChars~~ (keys DO NOT EXIST — verified) | 2.1.105 + 2.1.32 | DONE | budget=2% ctx + cap=1536 are internal CC mechanisms, not settings keys; both enforced by the SKL scanner — `CA-SKL-001` (1,536-char cap) + `CA-SKL-002` (aggregate 2%-of-context budget, 200k-anchored estimate) | scanners/skill-listing-scanner.mjs | L | M |
| CLAUDE.md / memory | 'too long' threshold now scales with context window | 2.1.169 | now-wrong | fix-false-positive | scanners/claude-md-linter.mjs | H | M |
| CLAUDE.md / memory | --safe-mode / CLAUDE_CODE_SAFE_MODE disables customizations | 2.1.169 | missing | update-knowledge | knowledge/configuration-best-practices.md | L | S |
| CLAUDE.md / memory | plugins in .claude/skills auto-load (cascade/token accounting) | 2.1.157 | partial | new-feature-gap-check | knowledge/configuration-best-practices.md | M | M |

2
docs/scanner-internals.md
View file

@ -22,7 +22,7 @@ Scanner CLI: `node scanners/scan-orchestrator.mjs <path> [--global] [--full-mach
| `cache-prefix-scanner.mjs` | CPS | Volatile content in lines 31–150 of CLAUDE.md cascade (beyond Pattern A's top-30 window) |
| `disabled-in-schema-scanner.mjs` | DIS | Tools listed in BOTH `permissions.deny` AND `permissions.allow` — deny wins, allow entries are dead config |
| `collision-scanner.mjs` | COL | Cross-plugin skill name collisions (low); user-vs-plugin overlaps (medium); `details.namespaces` payload |
| `skill-listing-scanner.mjs` | SKL | Active skill descriptions over the verified 1,536-char listing cap (CC 2.1.105) → silently truncated in the model's skill listing (medium). HOME-scoped (all user + plugin skills). Remediation surfaces `disableBundledSkills` / `skillOverrides`. Distinct lens from TOK pattern F (project-local 500-char bloat heuristic) |
| `skill-listing-scanner.mjs` | SKL | (1) `CA-SKL-001` (medium): active skill descriptions over the verified 1,536-char listing cap (CC 2.1.105) → silently truncated in the model's skill listing. (2) `CA-SKL-002` (low): sum of active descriptions (each counted up to the cap) over the listing budget (~2% of context, CC 2.1.32), anchored on a conservative 200k window with a calibration note that the budget scales 5× on 1M-context models — leads with the measured sum, an estimate not telemetry. HOME-scoped (all user + plugin skills). Remediation surfaces `disableBundledSkills` / `skillOverrides` / trim. Distinct lens from TOK pattern F (project-local 500-char bloat heuristic) |
## Scanner Lib (`scanners/lib/`)