Commit graph

16 commits

Author SHA1 Message Date
Kjell Tore Guttormsen
c5c937e94e feat(humanizer): forbidden-words lint runner + test wrapper (SC-3) [skip-docs]
Step 8 of v5.1.0 humanizer Wave 4. Adds tests/lint-default-output.mjs
runner and tests/scanners/lint-default-output.test.mjs wrapper that
exercise SC-3 against the 6 prose CLIs (scan-orchestrator, posture,
token-hotspots-cli, plugin-health-scanner, drift-cli, fix-cli) running
in default (humanized) mode against tests/fixtures/marketplace-medium.

Lint scope is stderr only — JSON envelope keys ("scanner", "severity")
are structural, not prose. Humanized prose fields embedded inside JSON
are already covered by tests/lib/humanizer-data.test.mjs tier1/tier3
checks. Code references inside backticks pass the lint
(stripBacktickSpans) so technical identifiers can appear when wrapped.

Default-mode prose fixes to land lint at zero violations:

- scan-orchestrator: top banner switches to "Config-Audit v2.2.0" and
  per-scanner progress wraps "[XXX] Label" in backticks. --raw and
  --json paths preserve the v5.0.0 verbatim banner via new
  opts.humanizedProgress flag on runAllScanners.
- plugin-health-scanner: top banner switches to "Plugin Health v2.1.0"
  in default mode; --raw/--json keep "Plugin Health Scanner v2.1.0".
- scoring.mjs generateHealthScorecard humanized branch: area names
  (CLAUDE.md, Hooks, MCP, Settings, Rules, Imports, Conflicts, Token
  Efficiency, Plugin Hygiene) are wrapped in backticks; dot-padding
  compensates so column alignment matches v5.0.0 layout.
- posture / drift-cli / fix-cli: thread humanizedProgress flag through
  their runAllScanners calls so default mode emits humanized progress
  and --raw/--json preserve the v5.0.0 stderr snapshot.

Test infrastructure only — user-facing docs land in Wave 5/6 once
commands and agents consume the humanized payload.

Tests: 735 to 736 (+1 SC-3 wrapper). Full suite passes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 18:11:15 +02:00
Kjell Tore Guttormsen
70ff900578 feat(humanizer): wire humanizer into posture and scoring scorecard
generateHealthScorecard signature: 2-arg → 3-arg (areaScores, opportunityCount,
options = {}). options.humanized=true renders friendlier title, grade-context
line per overall grade, and rephrased opportunity line. options.humanized=false
(or 2-arg call) preserves v5.0.0 verbatim output for backwards-compat.

topActions also gets an optional options.humanized that swaps recommendations
through humanizeFinding lookup.

posture.mjs main():
  --json → write JSON to stdout, suppress stderr scorecard
  --raw  → write JSON to stdout (byte-identical to --json), write v5.0.0
           verbatim scorecard to stderr
  default → humanized scorecard to stderr, no stdout

posture.test.mjs scorecard-prose assertions re-anchored to --raw mode (the
explicit v5.0.0 path) — Wave 0 audit only covered finding-title strings;
scorecard prose surfaces here for the first time.

Wave 3 / Step 6 of v5.1.0 humanizer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 17:38:03 +02:00
Kjell Tore Guttormsen
1a45caf18b feat(humanizer): translation module with category, action, relevance
Wave 1 / Step 3 of v5.1.0 plain-language UX humanizer.

scanners/lib/humanizer.mjs exports three pure functions:

- humanizeFinding(f) -> new finding object with translated
  title/description/recommendation + three new fields
  (userImpactCategory, userActionLanguage, relevanceContext).
- humanizeFindings(findings) -> mapped array.
- humanizeEnvelope(env) -> walks env.scanners[].findings.

Plus computeRelevanceContext(filePath) as a named export for
unit testing.

Field semantics:
- userImpactCategory: from scanner prefix per research/02 line 124
  (Configuration mistake / Conflict / Wasted tokens / Dead config /
  Missed opportunity / Other).
- userActionLanguage: from severity per research/02 line 134
  (Fix this now / Fix soon / Fix when convenient / Optional cleanup
  / FYI).
- relevanceContext: deterministic file-path heuristic — looks for
  /tests/fixtures/ or /test/fixtures/ substring (test-fixture-no-impact),
  *.local.* basename (affects-this-machine-only), defaults to
  affects-everyone. No subprocess, no network.

Lookup order per scanner: static[title] -> patterns regex match ->
_default -> fall through to original strings (when scanner prefix
absent).

Original id, scanner, severity, file, line, evidence, category,
autoFixable, and optional details are preserved exactly. Pure —
verified by deepEqual of input before/after.

Test (32 cases): purity, field preservation across all paths,
known/unknown scanner handling, all 5 severities, all 6 categories,
relevance heuristic for 4 path types, envelope walking, ANSI-free
guarantee. All pass.
Regression: 689/689 tests (657 + 32 new = 54 new across Wave 1).

Project: .claude/projects/2026-05-01-config-audit-ux-redesign/
2026-05-01 17:03:49 +02:00
Kjell Tore Guttormsen
02ee2a8b83 feat(humanizer): translation table for 12 scanners + plugin-health
Wave 1 / Step 2 of v5.1.0 plain-language UX humanizer.

scanners/lib/humanizer-data.mjs exports TRANSLATIONS keyed by
scanner prefix (CML, SET, HKV, RUL, MCP, IMP, CNF, GAP, TOK, CPS,
DIS, COL, PLH). Each scanner has:

- static: exact-title -> {title, description, recommendation}
- patterns: array of {regex, translation} for template-literal titles
- _default: graceful fallback for unknown findings

Architectural change vs. plan: keys translations by exact scanner
title (not finding ID). Reason: finding IDs are sequence-based
(global counter in lib/output.mjs:34), not stable per finding-type
— two runs can produce different IDs for the same logical issue.
Title strings ARE stable (defined as string literals or template
patterns in the scanner source).

Translations follow research/03 SR-1..SR-17:
- active voice, second person, present tense
- sentences <= 25 words
- tier1 absolute prohibitions and tier3 domain jargon are kept out
  of prose
- tier1/tier3 terms are permitted inside `backtick spans` (code
  references like filenames and field names) — established
  technical-doc convention

Test (12 cases): all 13 scanners covered; every static and pattern
entry has the 3 required fields; tier1 and tier3 forbidden-word
checks pass (with backtick-span exclusion); reference-stable
imports. All pass.
Regression: 657/657 tests (645 + 12 new).

Project: .claude/projects/2026-05-01-config-audit-ux-redesign/
2026-05-01 17:00:59 +02:00
Kjell Tore Guttormsen
b7414303de feat(config-audit): --accurate-tokens API calibration (v5 N5) [skip-docs] 2026-05-01 09:15:02 +02:00
Kjell Tore Guttormsen
cd25c1e934 feat(config-audit): cross-plugin collision scanner COL (v5 N6) [skip-docs]
New COL scanner detects skill-name collisions across plugins and
between user-level skills (~/.claude/skills/) and plugin-bundled
skills. Skill identity is the directory basename — matches how
enumerateSkills resolves names.

Detection rules (per docs/v5-namespace-research.md, confidence: medium):
- Plugin-vs-plugin same skill name → severity low (CA-COL-001)
- User-vs-plugin same skill name → severity medium (CA-COL-001)
- Plugin-vs-built-in collisions: out of scope for v5.0.0 (insufficient
  verification — recorded for v5.0.1 follow-up).

Findings carry details.namespaces array with {source, name, path} for
every conflicting source — supports per-collision reporting downstream.

output.mjs: finding() helper now passes through optional `details`
field (scanner-specific structured payload).

scoring.mjs: COL → "Plugin Hygiene" (new area, 10 total). Posture test
updated from 9 → 10 area scores.

.gitignore: docs/v5-namespace-research.md is local-only (Step 22a
research output, gitignored per plan).

Fixture collision-plugins/fake-home/ has user skill `review` colliding
with plugin-a + plugin-b's `review` (medium severity), plus plugin-c's
unique `summarize` (no collision).

[skip-docs] reason: v5 plan fences off README/CLAUDE.md badge updates
to Session 5; Forgejo pre-commit-docs-gate hook requires this tag.

Tests: 617 → 625 (+8).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 07:46:15 +02:00
Kjell Tore Guttormsen
cc349d6fe1 feat(config-audit): disabled-in-schema scanner DIS (v5 N4) [skip-docs]
New DIS scanner detects tools that appear in BOTH permissions.deny
and permissions.allow within the same settings.json file. The deny
list wins, so allow entries are dead config but still load on every
turn and confuse intent.

Tool identity = bare name (everything before "("). `Bash(npm:*)` and
`Bash` are treated as the same tool, so a deny on `Bash` flags any
`Bash(...)` allow entry.

Severity: low. Wired into scan-orchestrator + scoring (area: Settings).
Fixture denied-tools-in-schema has Bash in both arrays; healthy-project
serves as the negative case.

[skip-docs] reason: v5 plan fences off README/CLAUDE.md badge updates
to Session 5; Forgejo pre-commit-docs-gate hook requires this tag.

Tests: 611 → 617 (+6).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 07:39:58 +02:00
Kjell Tore Guttormsen
65087e624f feat(config-audit): cache-prefix stability scanner CPS (v5 N3) [skip-docs]
New CPS scanner walks CLAUDE.md cascade and flags volatile content
between lines 31 and 150 — the cache-prefix window beyond TOK Pattern
A's top-30 territory. Volatile content anywhere in the cached prefix
forces a fresh cache write from that line down on every turn.

Volatile-pattern set extends TOK Pattern A with:
- shell-exec lines (! prefix) — common in CLAUDE.md to inject git/date
- ${VAR} substitutions — vary per-shell, defeat cache reuse

Severity: medium per finding. Skips lines 1-30 to avoid duplicating
Pattern A's range; CPS' value is in the 31-150 zone.

Wired into scan-orchestrator + scoring SCANNER_AREA_MAP. CPS shares
the "Token Efficiency" area with TOK; scoreByArea now deduplicates by
area name and combines counts across scanners contributing to the
same area, so the 9-area scorecard contract holds.

Fixtures volatile-mid-section/{volatile-line-60, volatile-line-200}
verify both positive (line 60) and out-of-window (line 200) cases.

[skip-docs] reason: v5 plan fences off README/CLAUDE.md badge updates
to Session 5; Forgejo pre-commit-docs-gate hook requires this tag.

Tests: 604 → 611 (+7).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 07:37:54 +02:00
Kjell Tore Guttormsen
1422daf895 feat(config-audit): MCP tool-count detection with manifest fallback (v5 M1) [skip-docs]
readActiveMcpServers now resolves tool count via:
  1. In-config tools array
  2. Cached tools/list at \$HOME/.claude/config-audit/mcp-cache/<name>.json
  3. node_modules/<pkg>/package.json (resolved from npx <pkg>)
  4. Fallback: { toolCount: null, toolCountUnknown: true }

estimateTokens uses detected toolCount (heavy server > light server).

New fixture: mcp-tool-heavy/ with mocked node_modules/mcp-heavy/package.json (20 tools).

576 → 580 tests, all green.
2026-05-01 07:02:08 +02:00
Kjell Tore Guttormsen
ce7c42f517 fix(config-audit): MCP token callers use 'mcp' kind (v5 F2)
Two MCP enumeration paths in readActiveMcpServers now pass kind='mcp'
to estimateTokens with optional toolCount derived from def.tools array
(populated when callers cache MCP discovery — Step 14 wires that up).

Hook callers keep kind='item' (no schema overhead).

Visible effect: every active MCP server jumps from estimatedTokens=15
to >= 500 (or higher when toolCount is known). The whats-active output
and TOK hotspots now reflect actual MCP cost.

Tests: assert mcpServers[].estimatedTokens >= 500 in fixture.
2026-05-01 06:22:54 +02:00
Kjell Tore Guttormsen
48d560a209 feat(config-audit): add 'mcp' kind to estimateTokens (v5 F2)
Differentiate MCP servers from generic 'item' (flat 15) — they actually
cost 500+ tokens per turn for protocol metadata and tool schemas.

estimateTokens(bytes, 'mcp', {toolCount}) returns max of:
- 500 token floor (base overhead)
- ceil(bytes / 3.5) (json-rate when bytes known)
- 500 + toolCount * 200 (when tool count is detected; Step 14 wires this)

Caller-side migration in next commit (Step 5).

Tests: +4 cases for mcp kind.
2026-05-01 06:21:30 +02:00
Kjell Tore Guttormsen
a65c7f4080 feat(config-audit): severity-weighted scoreByArea (v5 F3)
Replace count-based pass-rate with severity-weighted penalty:
- penalty = sum(count[s] * WEIGHTS[s])
- maxBudget = max(10, findingCount * 4)
- passRate = max(0, 100 - penalty / maxBudget * 100)

A few lows no longer crater an area's grade; a single high or critical
consumes a large fraction of budget. Mirrors the operator intuition that
severity, not count, is the signal.

BREAKING (intentional): scoring semantics differ from v4 for non-clean
configs. Add scoringVersion: 'v5' to the returned struct so consumers
can detect the version. baseline-all-a remains all-A (no critical/high
on that fixture).

Tests: +6 cases for severity weighting; existing "many findings" test
updated to use highs (where v5 still drops the grade as expected).
2026-05-01 06:20:08 +02:00
Kjell Tore Guttormsen
e5efc2ff64 feat(config-audit): export WEIGHTS from severity.mjs (v5 F3 prep)
Promote WEIGHTS const to named export with Object.freeze for downstream
use in scoring.mjs (severity-weighted scoreByArea, F3).

Tests: +2 cases asserting WEIGHTS shape.
2026-05-01 06:16:28 +02:00
Kjell Tore Guttormsen
4b385bf456 feat(config-audit): wire TOK into posture scorecard as 8th quality area (Token Efficiency) 2026-04-19 22:45:12 +02:00
Kjell Tore Guttormsen
4f1cc7e0b7 feat(config-audit): v3.1.0 — /config-audit whats-active inventory command
New read-only command that shows everything Claude Code actually loads for a
given repo — plugins, skills, MCP servers, hooks, CLAUDE.md cascade — with
source attribution (user/project/plugin) and rough token estimates. Helps
identify candidates for disabling without guessing.

Added:
- scanners/lib/active-config-reader.mjs — pure async helper: readActiveConfig,
  detectGitRoot, walkClaudeMdCascade, readClaudeJsonProjectSlice (longest-prefix
  matching for .claude.json projects), enumeratePlugins, enumerateSkills,
  readActiveHooks, readActiveMcpServers, estimateTokens (markdown 4 c/tok,
  json 3.5 c/tok, frontmatter cap 150 tokens, item flat 15)
- scanners/whats-active.mjs — thin CLI shim: --json, --output-file, --verbose,
  --suggest-disables
- commands/whats-active.md — renders tables via Read tool; honors UX rules
- tests/lib/active-config-reader.test.mjs — 36 tests, all green (integration
  fixture built in tmpdir with fake HOME, .claude.json prefix matching,
  plugin discovery, hook/MCP merge from all scopes)

Verified:
- Performance budget: <2s wall-clock (smoke test: 102ms on real repo)
- Token estimates within ±20% of hand-computed values
- Read-only: no writeFile/mkdir/unlink in production code
- Self-audit: Plugin Health scanner reports 0 findings (Grade A)
- Full test suite: 522 tests, 512 pass (10 pre-existing conflict-detector
  failures on main — unrelated to this change, reproducible on clean HEAD)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 21:50:20 +02:00
Kjell Tore Guttormsen
f93d6abdae feat: initial open marketplace with llm-security, config-audit, ultraplan-local 2026-04-06 18:47:49 +02:00