Commit graph

7 commits

Author SHA1 Message Date
7bb254780a feat(skill-listing): add SKL scanner for the skill-listing token budget
Fase 4 Items 2+3 (CC 2.1.114→181 gap-review). New orchestrated scanner
`skill-listing-scanner.mjs` (prefix SKL) flags every active skill whose
description exceeds the verified 1,536-char listing cap (CC 2.1.105, changelog
L1502). Past the cap, Claude Code silently truncates the description the model
reads to route skill invocation — dropping the trigger phrases at the tail.
HOME-scoped over all user + plugin skills via enumerateSkills (COL is the model).

- CA-SKL-001 (medium): description > 1,536 chars. Remediation folds in Item
  2(b) — recommends disableBundledSkills + skillOverrides + trimming
  (designvalg A: no standalone GAP-check, which would fire for nearly everyone).
- Designvalg B: v1 ships the verified cap ONLY. The aggregate 2%-of-context
  listing budget is deferred — it needs a context-window assumption that would
  turn a verified fact into a guess (would carry a CALIBRATION_NOTE if added).
- Choice C: recognize the skillOverrides settings key (CC 2.1.129) in
  KNOWN_KEYS. Left OUT of TYPE_CHECKS — the value is a per-skill object
  (off/user-invocable-only/name-only), not a string; a 'string' check (as the
  plan sketched) would create a NEW false positive. Verify-first deviation.

Registration: scan-orchestrator (13th scanner), humanizer (SKL → 'Wasted
tokens' + static/_default translations), scoring SCANNER_AREA_MAP (→ Token
Efficiency; no 11th area), README badge 12→13, CLAUDE.md (finding-id +
test-count), docs/scanner-internals.md, gap-matrix + plan status notes.

Snapshots reseeded hermetically (SEED_SNAPSHOT/UPDATE_SNAPSHOT): SKL entry with
0 findings in empty HOME, scanners_ok 11→12, claudeMdEstimatedTokens bump from
the CLAUDE.md edits flowing through the cascade. Contamination grep clean.

Suite 868/868 (856 baseline + 11 SKL + 1 skillOverrides). RED→GREEN logged
per cycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
2026-06-18 17:36:28 +02:00
b3c572ad46 fix(mcp-config-validator): remove invented trust field (verify-first)
`.mcp.json` has no per-server `trust` key — verified 2026-06-18 against
code.claude.com/docs/en/mcp + /settings. MCP server approval is
dialog/settings-based (enableAllProjectMcpServers / enabledMcpjsonServers /
disabledMcpjsonServers), never a JSON field. The scanner's "Missing trust
level" (CA-MCP-001, medium) and "Invalid trust level" (high) were false
positives flagging a field that does not exist.

- scanner: delete both trust checks + VALID_TRUST_LEVELS; drop `trust` from
  VALID_SERVER_FIELDS so a stray `trust` is now flagged as an unknown field
- humanizer: remove the two trust-level entries
- knowledge (5 files): point to the real approval mechanism, not a trust field
- fixtures: scrub `trust` (incl. the invalid "local" in optimal-setup)
- tests: flip assertions (no trust-level finding; stray trust -> unknown
  field) + add knowledge-staleness re-freeze guards
- snapshots: reseed (marketplace-medium .mcp.json -8 tokens, hermetic)
- gap-matrix: mark the trust verify-first item DONE

Suite: 853/853 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
2026-06-18 14:22:56 +02:00
bec3f45329 fix(permissions): param-aware DIS dead-allow + CNF conflict matching
The DIS scanner collapsed Tool(param) rules to the bare tool name, so
Agent(model:opus) deny + Agent(model:sonnet) allow (and the same for
WebFetch(domain:...)) were flagged as dead config — a false positive now
that CC 2.1.178 matches Tool(param:value) and 2.1.172 adds domain rules.
The conflict-detector shared the blind spot from the other side: a
wildcard deny like WebFetch(domain:*) did not cover a
WebFetch(domain:good.com) allow, so a genuine cross-scope conflict was
missed (false negative).

New shared scanners/lib/permission-rules.mjs:
- parseRule / paramMatches (glob)
- dominates(deny, allow) -> DIS dead-allow (deny fully covers allow)
- rulesIntersect(a, b)   -> CNF cross-scope conflict (match sets intersect)

DIS now delegates to dominates; conflict-detector :156 delegates to
rulesIntersect. A bare deny still covers all params, so true positives
are preserved (Bash deny + Bash(npm:*) allow still flagged).

Re-seeded the marketplace-medium snapshots: the false-positive CA-DIS
finding (Read(src/**) allow + Read(./.env) deny) is correctly gone. This
changes snapshot CONTENT only — envelope schema is unchanged, so --json
and --raw stay byte-stable.

Full suite: 837/837 green (+25). self-audit PASS, A(100)/A(97).
2026-06-18 13:06:20 +02:00
8216fb4175 test(snapshots): make byte/snapshot tests hermetic + re-seed baseline
The COL collision-scanner and the CLAUDE.md cascade resolve ~/.claude from
process.env.HOME (active-config-reader). Snapshot/byte CLIs were spawned with
the developer's real HOME, so they picked up installed plugins/skills and the
user CLAUDE.md — making the v5.0.0 + default-output snapshots machine- and
time-dependent. They were seeded 2026-05-01 with COL=1 (a real ~/.claude skill
collision) and drifted to COL=0 after the polyrepo split: 26 pre-existing
failures unrelated to Batch 1.

Fix (test-only, no production change):
- tests/helpers/hermetic-home.mjs — empty temp HOME, mirroring the pattern
  collision.test.mjs already uses for the COL unit test.
- 7 harnesses spawn CLIs (or call lint()) under the hermetic HOME, so output
  depends only on committed fixtures. Determinism verified across runs.
- Re-seeded all snapshots under hermetic HOME via SEED_SNAPSHOT/UPDATE_SNAPSHOT
  (added a SEED guard to the frozen v5.0.0 byte tests). Snapshots now reflect
  the fixture alone (COL=0, fixture-only activeConfig counts).
- Also re-seeded the unused env-aware snapshots (manifest/whats-active/
  plugin-health), which had baked dozens of real ~/.claude skill/plugin names
  into the committed repo — privacy cleanup.

Full suite: 812/812 green, stable across 3 runs.
2026-06-18 12:26:00 +02:00
Kjell Tore Guttormsen
4df7f2480f docs(humanizer): v5.1.0 release notes across plugin + marketplace docs
- Plugin README: add "What's New in v5.1.0" section with humanizer overview,
  before/after example, plain-language vocabulary table, --raw flag docs.
  Bump version badge 5.0.0 → 5.1.0. Add Version History row.
- Plugin CLAUDE.md: add humanizer.mjs + humanizer-data.mjs to Scanner Lib
  table. Add "Plain-Language Output (v5.1.0)" section documenting output
  modes, vocabularies, and Wave 5 lessons. Bump test count 635 → 792 across
  52 test files.
- Marketplace root README: bump config-audit entry 5.0.0 → 5.1.0, update
  one-line description to mention plain-language UX, add bullet for the
  v5.1.0 humanizer, bump test count 635+ → 792+.

Test-normalizer hardening (consequence of growing CLAUDE.md):
walkClaudeMdCascade walks upward from the marketplace-medium fixture into
this plugin's own CLAUDE.md, so any docs edit ripples into
`scanners[*].activeConfig.claudeMdEstimatedTokens`. The v5.0.0 byte-stability
contract is about scanner internals being unchanged, not ancestor input
content being frozen. Normalizers in json-backcompat, raw-backcompat,
posture-humanizer, scan-orchestrator-humanizer, and snapshot-default-output
now strip claudeMdEstimatedTokens to <ANCESTOR_DERIVED>. The
default-output snapshot for scan-orchestrator was re-seeded via
UPDATE_SNAPSHOT=1 (intent: Wave 6 docs additions; humanizer prose
unchanged).

Verify:
- grep -E "5\.1\.0|v5\.1\.0" README.md CLAUDE.md ../../README.md | wc -l = 12
- node --test 'tests/**/*.test.mjs' = 792/792 pass
- self-audit configGrade A (97), pluginGrade A (100), readmeCheck.passed true
2026-05-01 20:35:24 +02:00
Kjell Tore Guttormsen
3e7d35fb3e test(humanizer): default-output snapshot test (SC-5) [skip-docs]
Step 12 of v5.1.0 humanizer Wave 4. Adds tests/snapshot-default-output
.test.mjs and seeds three snapshots in tests/snapshots/default-output/
that capture humanized default-mode output for representative CLIs.

Coverage:

- scan-orchestrator: stdout JSON envelope (humanized findings); time
  fields normalized.
- token-hotspots-cli: stdout JSON envelope (humanized payload.findings);
  duration_ms normalized.
- posture: stderr humanized scorecard; (Xms) durations normalized.

Snapshot envelope is uniform on disk: { kind: 'json', payload: ... }
for JSON streams and { kind: 'text', payload: '...' } for stderr text.
This keeps the snapshot files self-describing and easy to read.

Re-seeding requires UPDATE_SNAPSHOT=1 — drift fails the test by design,
so any humanizer prose change is intentional and re-approved.

Tests: 764 to 767 (+3 SC-5 cases). Full suite passes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 18:21:31 +02:00
Kjell Tore Guttormsen
5f0f32d7fc chore(humanizer): pre-flight snapshots + test audit for v5.1.0
Wave 0 / Step 0 of the v5.1.0 plain-language UX humanizer plan.

Captures v5.0.0 baseline output for all 8 CLIs at
tests/snapshots/v5.0.0/ — these snapshots are immutable references
for SC-6 (--json byte-equal) and SC-7 (--raw byte-equal) tests in
later waves.

- 5 CLIs captured via --output-file: scan-orchestrator, posture,
  token-hotspots-cli, manifest, whats-active
- 3 CLIs captured via stdout redirect (no --output-file support):
  drift-cli (after baseline seed), fix-cli, plugin-health-scanner
- Posture stderr scorecard captured separately for SC-7 stderr-mode
  comparison

docs/v5.1.0-test-audit.md classifies all 42 .title references in
7 known test files: 34 will break under humanization (literal
string equality / substring), 8 are safe (test fixtures or error
formatting). This document is the change list for Step 4.

Project: .claude/projects/2026-05-01-config-audit-ux-redesign/
2026-05-01 16:47:13 +02:00