open/config-audit
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions
config-audit/docs/scanner-internals.md
Kjell Tore Guttormsen 66433fee48 feat(skill-listing): add CA-SKL-002 aggregate listing-budget check 
Syklus 2 of Fase 4 Items 2+3. Flags when the sum of active skill
descriptions exceeds the listing budget (~2% of context, CC 2.1.32).

Design (operator-confirmed "fact-first, 200k anchor"):
- low severity (estimate) vs medium for the verified 1,536-char cap
- each description counted up to the 1,536 cap (what actually loads in
  the listing) — avoids double-counting the tail CA-SKL-001 flags
- fires when sum > 2% x 200k = 4000 tok; evidence leads with the measured
  sum + a calibration note that the budget scales 5x on 1M-context models
- aggregate emitted after the per-skill loop so the common case reads
  001=cap, 002=aggregate (finding IDs are a sequential counter, not stable
  semantic IDs — tests match on title, never NNN)

Also:
- tailored humanizer static entry for the aggregate title
- fix latent HOME leak in posture-grade-stability.test.mjs: it spawned
  posture.mjs without hermeticEnv(), so a real ~/.claude leaked HOME-scoped
  SKL/COL findings into the baseline grade (Token Efficiency A->B). Now
  isolated like the 8 other CLI-spawning tests.
- docs sync: test count 868->875, scanner-internals, gap-matrix, plan status

Suite 875/875, no snapshot drift, self-audit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
2026-06-18 18:06:17 +02:00

77 lines
6.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Config-Audit — Scanner internals
Detailed scanner inventory, lib modules, action engines, knowledge base. Imported from `CLAUDE.md` via pointer.
## Deterministic Scanners
Node.js scanners (zero external dependencies), run via `node scanners/scan-orchestrator.mjs <path>`.
Posture CLI: `node scanners/posture.mjs <path> [--json] [--global] [--full-machine] [--output-file path]`.
Scanner CLI: `node scanners/scan-orchestrator.mjs <path> [--global] [--full-machine] [--no-suppress]`.
| Scanner | Prefix | Detects |
|---------|--------|---------|
| `claude-md-linter.mjs` | CML | Structure, length, sections, @imports, duplicates, TODOs |
| `settings-validator.mjs` | SET | Schema, unknown/deprecated keys, type mismatches, permissions |
| `hook-validator.mjs` | HKV | Format, script existence, event validity, timeouts |
| `rules-validator.mjs` | RUL | Glob matching, orphan rules, deprecated fields, unscoped rules |
| `mcp-config-validator.mjs` | MCP | Server types, env vars, unknown fields |
| `import-resolver.mjs` | IMP | Broken @imports, circular refs, deep chains, tilde paths |
| `conflict-detector.mjs` | CNF | Settings conflicts, permission contradictions, hook duplicates |
| `feature-gap-scanner.mjs` | GAP | 25 feature checks across 4 tiers — shown as opportunities, not grades |
| `token-hotspots.mjs` | TOK | Cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascade, bloated SKILL.md descriptions, MCP tool-schema budget (prompt-cache patterns) |
| `cache-prefix-scanner.mjs` | CPS | Volatile content in lines 31–150 of CLAUDE.md cascade (beyond Pattern A's top-30 window) |
| `disabled-in-schema-scanner.mjs` | DIS | Tools listed in BOTH `permissions.deny` AND `permissions.allow` — deny wins, allow entries are dead config |
| `collision-scanner.mjs` | COL | Cross-plugin skill name collisions (low); user-vs-plugin overlaps (medium); `details.namespaces` payload |
| `skill-listing-scanner.mjs` | SKL | (1) `CA-SKL-001` (medium): active skill descriptions over the verified 1,536-char listing cap (CC 2.1.105) → silently truncated in the model's skill listing. (2) `CA-SKL-002` (low): sum of active descriptions (each counted up to the cap) over the listing budget (~2% of context, CC 2.1.32), anchored on a conservative 200k window with a calibration note that the budget scales 5× on 1M-context models — leads with the measured sum, an estimate not telemetry. HOME-scoped (all user + plugin skills). Remediation surfaces `disableBundledSkills` / `skillOverrides` / trim. Distinct lens from TOK pattern F (project-local 500-char bloat heuristic) |
## Scanner Lib (`scanners/lib/`)
| Module | Purpose |
|--------|---------|
| `severity.mjs` | Severity constants, risk scoring, verdict logic, `WEIGHTS` named export (v5 F3) |
| `output.mjs` | Finding objects (CA-XXX-NNN format), scanner results, envelope, optional `details` payload (v5 N6) |
| `file-discovery.mjs` | Config file discovery: single-path, multi-path (`discoverConfigFilesMulti`), full-machine (`discoverFullMachinePaths`) |
| `yaml-parser.mjs` | Frontmatter parsing, JSON parsing, @import/section extraction |
| `string-utils.mjs` | Line counting, truncation, similarity, key extraction |
| `scoring.mjs` | Severity-weighted `scoreByArea` (v5 F3), health scorecard, dedup-by-area (v5 N3), `scoringVersion: 'v5'` |
| `backup.mjs` | Backup creation, manifest parsing, checksum verification |
| `diff-engine.mjs` | Drift diffing: diffEnvelopes(), formatDiffReport() |
| `baseline.mjs` | Baseline save/load/list/delete for drift detection |
| `report-generator.mjs` | Unified markdown reports: posture, drift, plugin health |
| `suppression.mjs` | .config-audit-ignore parsing, finding suppression, audit trail |
| `active-config-reader.mjs` | Read-only inventory: readActiveConfig(), detectGitRoot(), walkClaudeMdCascade(), readClaudeJsonProjectSlice() (longest-prefix match), enumeratePlugins(), enumerateSkills(), readActiveHooks(), readActiveMcpServers() (with cache → package.json tool-count fallback), estimateTokens() (v5: `'mcp'` kind = 500 + toolCount × 200) |
| `tokenizer-api.mjs` | Anthropic `count_tokens` wrapper for `--accurate-tokens` (v5 N5); 5s AbortController timeout, exponential 429 backoff, key masking |
| `humanizer.mjs` | Plain-language output translator (v5.1.0): `humanizeFinding`, `humanizeFindings`, `humanizeEnvelope`, `computeRelevanceContext`. Pure functions; never mutate inputs. Adds `userImpactCategory`, `userActionLanguage`, `relevanceContext` fields and replaces title/description/recommendation when a translation exists. Bypassed by `--raw` and `--json` paths. |
| `humanizer-data.mjs` | TRANSLATIONS table for 14 scanner prefixes (CML/SET/HKV/RUL/MCP/IMP/CNF/COL/TOK/CPS/DIS/GAP/PLH/SKL). Three-step lookup: exact title → regex pattern → `_default` → fall through to original |
## Action Engines (`scanners/`)
| Module | Purpose |
|--------|---------|
| `fix-engine.mjs` | planFixes(), applyFixes(), verifyFixes() — 9 fix types |
| `rollback-engine.mjs` | listBackups(), restoreBackup(), deleteBackup() |
| `fix-cli.mjs` | CLI: `node fix-cli.mjs <path> [--apply] [--json] [--global]` |
| `drift-cli.mjs` | CLI: `node drift-cli.mjs <path> [--save] [--baseline name] [--json]` |
| `whats-active.mjs` | CLI: `node whats-active.mjs <path> [--json] [--verbose] [--suggest-disables]` — read-only active-config inventory |
| `token-hotspots-cli.mjs` | CLI: `node token-hotspots-cli.mjs <path> [--json] [--global] [--output-file path] [--accurate-tokens] [--with-telemetry-recipe]` — prompt-cache token hotspots ranking with optional API calibration |
| `manifest.mjs` | CLI: `node manifest.mjs <path> [--json]` — ranked system-prompt token-source table (v5 N2) |
## Standalone Scanner
| Module | Prefix | Purpose |
|--------|--------|---------|
| `plugin-health-scanner.mjs` | PLH | Plugin structure, frontmatter, cross-plugin conflicts (runs independently) |
| `self-audit.mjs` | — | Runs all scanners + plugin health on this plugin itself |
## Knowledge Base (`knowledge/`)
| File | Content |
|------|---------|
| `claude-code-capabilities.md` | Feature register: 18 config surfaces, Anthropic guidance, relevance table |
| `configuration-best-practices.md` | Per-layer best practices (v5: cache-stability guidance replaces Sonnet-era 200-line rule) |
| `anti-patterns.md` | Common mistakes mapped to scanner IDs |
| `hook-events-reference.md` | All 26 hook events with details |
| `feature-evolution.md` | Feature timeline for staleness detection |
| `gap-closure-templates.md` | Config-specific templates for closing gaps |
| `prompt-cache-patterns.md` | Token-cost dynamics (prompt-cache patterns) — patterns powering the TOK scanner |
| `cache-telemetry-recipe.md` | Manual `jq` recipe for verifying prompt-cache hit rate from session transcripts (v5 M7) |
Reference in a new issue View git blame Copy permalink