Kjell Tore Guttormsen
7bb254780a
Fase 4 Items 2+3 (CC 2.1.114→181 gap-review). New orchestrated scanner `skill-listing-scanner.mjs` (prefix SKL) flags every active skill whose description exceeds the verified 1,536-char listing cap (CC 2.1.105, changelog L1502). Past the cap, Claude Code silently truncates the description the model reads to route skill invocation — dropping the trigger phrases at the tail. HOME-scoped over all user + plugin skills via enumerateSkills (COL is the model). - CA-SKL-001 (medium): description > 1,536 chars. Remediation folds in Item 2(b) — recommends disableBundledSkills + skillOverrides + trimming (designvalg A: no standalone GAP-check, which would fire for nearly everyone). - Designvalg B: v1 ships the verified cap ONLY. The aggregate 2%-of-context listing budget is deferred — it needs a context-window assumption that would turn a verified fact into a guess (would carry a CALIBRATION_NOTE if added). - Choice C: recognize the skillOverrides settings key (CC 2.1.129) in KNOWN_KEYS. Left OUT of TYPE_CHECKS — the value is a per-skill object (off/user-invocable-only/name-only), not a string; a 'string' check (as the plan sketched) would create a NEW false positive. Verify-first deviation. Registration: scan-orchestrator (13th scanner), humanizer (SKL → 'Wasted tokens' + static/_default translations), scoring SCANNER_AREA_MAP (→ Token Efficiency; no 11th area), README badge 12→13, CLAUDE.md (finding-id + test-count), docs/scanner-internals.md, gap-matrix + plan status notes. Snapshots reseeded hermetically (SEED_SNAPSHOT/UPDATE_SNAPSHOT): SKL entry with 0 findings in empty HOME, scanners_ok 11→12, claudeMdEstimatedTokens bump from the CLAUDE.md edits flowing through the cascade. Contamination grep clean. Suite 868/868 (856 baseline + 11 SKL + 1 skillOverrides). RED→GREEN logged per cycle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
575 lines
20 KiB
JSON
575 lines
20 KiB
JSON
{
|
|
"kind": "json",
|
|
"payload": {
|
|
"meta": {
|
|
"target": "<TARGET>",
|
|
"timestamp": "<TIMESTAMP>",
|
|
"version": "2.2.0",
|
|
"tool": "config-audit"
|
|
},
|
|
"scanners": [
|
|
{
|
|
"scanner": "CML",
|
|
"status": "ok",
|
|
"files_scanned": 1,
|
|
"duration_ms": 0,
|
|
"findings": [
|
|
{
|
|
"id": "CA-CML-001",
|
|
"scanner": "CML",
|
|
"severity": "low",
|
|
"title": "Your instructions file is missing common sections",
|
|
"description": "Sections like Project Overview, Commands, and Conventions help Claude apply your guidance consistently across tasks.",
|
|
"file": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/CLAUDE.md",
|
|
"line": null,
|
|
"evidence": "Present sections: Marketplace Medium, Plugins, Commands, Conventions",
|
|
"category": null,
|
|
"recommendation": "Add the missing sections noted in the details.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Configuration mistake",
|
|
"userActionLanguage": "Optional cleanup",
|
|
"relevanceContext": "test-fixture-no-impact"
|
|
}
|
|
],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 1,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "SET",
|
|
"status": "ok",
|
|
"files_scanned": 1,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "HKV",
|
|
"status": "ok",
|
|
"files_scanned": 1,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "RUL",
|
|
"status": "skipped",
|
|
"files_scanned": 0,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "MCP",
|
|
"status": "ok",
|
|
"files_scanned": 1,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "IMP",
|
|
"status": "ok",
|
|
"files_scanned": 1,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "CNF",
|
|
"status": "ok",
|
|
"files_scanned": 2,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "GAP",
|
|
"status": "ok",
|
|
"files_scanned": 4,
|
|
"duration_ms": 0,
|
|
"findings": [
|
|
{
|
|
"id": "CA-GAP-001",
|
|
"scanner": "GAP",
|
|
"severity": "medium",
|
|
"title": "You haven't added any custom shortcuts yet",
|
|
"description": "Custom skills give you `/your-shortcut` invocations for tasks you do often.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t1",
|
|
"recommendation": "Create a skill in `.claude/skills/` for a workflow you find yourself repeating.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "Fix when convenient",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-002",
|
|
"scanner": "GAP",
|
|
"severity": "low",
|
|
"title": "You only have settings at one level",
|
|
"description": "Settings can live at user, project, or local-only scope. Using more than one lets you keep personal preferences separate from team-shared ones.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t2",
|
|
"recommendation": "Consider moving team-wide settings to project scope and keeping personal ones at user or local scope.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "Optional cleanup",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-003",
|
|
"scanner": "GAP",
|
|
"severity": "low",
|
|
"title": "Your rules all load on every conversation",
|
|
"description": "Path-scoped rules only load when you're working with files that match — keeps each conversation focused.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t2",
|
|
"recommendation": "Add scoping to your rules so they only load for the files they apply to.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "Optional cleanup",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-004",
|
|
"scanner": "GAP",
|
|
"severity": "low",
|
|
"title": "Your automations all listen to similar events",
|
|
"description": "Listening to a wider range of events (before-tool, after-tool, session-start, etc.) lets you catch more workflow opportunities.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t2",
|
|
"recommendation": "Look at the events your current automations skip and consider adding one or two.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "Optional cleanup",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-005",
|
|
"scanner": "GAP",
|
|
"severity": "low",
|
|
"title": "You haven't set up any specialized helper agents yet",
|
|
"description": "Subagents handle parallel work in separate contexts (research, code review, testing) without crowding your main conversation.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t2",
|
|
"recommendation": "Create a subagent in `.claude/agents/` for a task you delegate often.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "Optional cleanup",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-006",
|
|
"scanner": "GAP",
|
|
"severity": "low",
|
|
"title": "You haven't pinned a model preference",
|
|
"description": "Setting a default model lets you choose between speed and depth of reasoning for your work.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t2",
|
|
"recommendation": "Add a `model` setting in your settings file.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "Optional cleanup",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-007",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "You haven't set up a status line yet",
|
|
"description": "A status line shows live context (token usage, current branch, time) at the bottom of your terminal.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t3",
|
|
"recommendation": "Add a `statusLine` setting if you want this information at a glance.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-008",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "You haven't set up any custom keybindings",
|
|
"description": "Custom keybindings let you trigger your most-used skills with a keystroke.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t3",
|
|
"recommendation": "Add bindings in your settings for skills you run often.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-009",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "You're using the default output style",
|
|
"description": "Output styles let you change how Claude formats responses (concise, verbose, bullet-heavy, etc.).",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t3",
|
|
"recommendation": "Try a different `outputStyle` setting if you have a strong preference.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-010",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "You haven't set up parallel worktree support",
|
|
"description": "Worktrees let Claude work on a branch in an isolated copy of the repo without disturbing your main checkout.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t3",
|
|
"recommendation": "Enable worktrees if you regularly work on multiple branches at once.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-011",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "Your skills don't use the richer settings block",
|
|
"description": "Adding richer settings at the top of a skill lets you control when it loads, what tools it uses, and more.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t3",
|
|
"recommendation": "Add fields like `model`, `tools`, or `description` to your skill files where useful.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-012",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "Your subagents share Claude's main work folder",
|
|
"description": "Isolated subagents run in their own copy of the repo so they can't accidentally disturb your main work.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t3",
|
|
"recommendation": "Add `isolation: worktree` to subagents that do destructive or experimental work.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-013",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "Your skills don't include live context",
|
|
"description": "Dynamic context lets a skill see fresh information (file contents, command output) at the moment it runs, not at the time it was written.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t3",
|
|
"recommendation": "Use the dynamic-context block in skills that need up-to-date information.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-014",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "You haven't set up auto-mode classification",
|
|
"description": "Auto-mode classification helps Claude decide when to act on its own vs. ask you, based on the kind of task.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t3",
|
|
"recommendation": "Add an auto-mode classifier in your settings if you want this nuance.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-015",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "You haven't built a custom plugin yet",
|
|
"description": "Plugins let you bundle skills, automations, and connected services that you want available across many projects.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t4",
|
|
"recommendation": "If you have workflows you repeat across projects, consider packaging them as a plugin.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-016",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "Your project has no settings managed by your organization",
|
|
"description": "Managed settings let your organization apply rules everyone has to follow.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t4",
|
|
"recommendation": "If you work in a team setting, consider whether managed settings would help.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
},
|
|
{
|
|
"id": "CA-GAP-017",
|
|
"scanner": "GAP",
|
|
"severity": "info",
|
|
"title": "You haven't connected Claude to your editor's language servers",
|
|
"description": "Language-server connections let Claude see types, error messages, and definitions the same way your editor does.",
|
|
"file": null,
|
|
"line": null,
|
|
"evidence": null,
|
|
"category": "t4",
|
|
"recommendation": "Set up LSP integration if you work in a typed language.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Missed opportunity",
|
|
"userActionLanguage": "FYI",
|
|
"relevanceContext": "affects-everyone"
|
|
}
|
|
],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 1,
|
|
"low": 5,
|
|
"info": 11
|
|
}
|
|
},
|
|
{
|
|
"scanner": "TOK",
|
|
"status": "ok",
|
|
"files_scanned": 2,
|
|
"duration_ms": 0,
|
|
"findings": [
|
|
{
|
|
"id": "CA-TOK-001",
|
|
"scanner": "TOK",
|
|
"severity": "low",
|
|
"title": "A connected service exposes many tools, all loading on every turn",
|
|
"description": "Each tool a connected service exposes adds its description to every turn. Services with many tools eat space fast.",
|
|
"file": ".mcp.json",
|
|
"line": null,
|
|
"evidence": "tool_count=unknown; server=\"memory\"; source=\".mcp.json\" — severity reflects estimated tokens/turn based on structural heuristic; not measured against runtime telemetry",
|
|
"category": "token-efficiency",
|
|
"recommendation": "Limit which tools the service exposes (often via a `tools` allow-list), or disconnect services you rarely use.",
|
|
"autoFixable": false,
|
|
"userImpactCategory": "Wasted tokens",
|
|
"userActionLanguage": "Optional cleanup",
|
|
"relevanceContext": "affects-everyone"
|
|
}
|
|
],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 1,
|
|
"info": 0
|
|
},
|
|
"hotspots": [
|
|
{
|
|
"source": "mcp:memory (.mcp.json)",
|
|
"estimated_tokens": 500,
|
|
"rank": 1,
|
|
"recommendations": [
|
|
"Review whether this source needs to load on every turn."
|
|
]
|
|
},
|
|
{
|
|
"source": "CLAUDE.md",
|
|
"estimated_tokens": 116,
|
|
"rank": 2,
|
|
"recommendations": [
|
|
"Move volatile top-of-file content to the bottom or extract to an @import-ed file.",
|
|
"Split overlong CLAUDE.md into focused @imports (≤200 lines each)."
|
|
],
|
|
"path": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/CLAUDE.md"
|
|
},
|
|
{
|
|
"source": "hooks/hooks.json",
|
|
"estimated_tokens": 81,
|
|
"rank": 3,
|
|
"recommendations": [
|
|
"Deduplicate overlapping entries — each duplicate inflates the per-turn schema payload.",
|
|
"Move rarely-used permissions to a project-local override."
|
|
],
|
|
"path": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/hooks/hooks.json"
|
|
},
|
|
{
|
|
"source": ".claude/settings.json",
|
|
"estimated_tokens": 59,
|
|
"rank": 4,
|
|
"recommendations": [
|
|
"Deduplicate overlapping entries — each duplicate inflates the per-turn schema payload.",
|
|
"Move rarely-used permissions to a project-local override."
|
|
],
|
|
"path": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/.claude/settings.json"
|
|
},
|
|
{
|
|
"source": ".mcp.json",
|
|
"estimated_tokens": 45,
|
|
"rank": 5,
|
|
"recommendations": [
|
|
"Deduplicate overlapping entries — each duplicate inflates the per-turn schema payload.",
|
|
"Move rarely-used permissions to a project-local override."
|
|
],
|
|
"path": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/.mcp.json"
|
|
}
|
|
],
|
|
"total_estimated_tokens": 801,
|
|
"activeConfig": {
|
|
"claudeMdEstimatedTokens": "<ANCESTOR_DERIVED>",
|
|
"mcpServerCount": 1,
|
|
"pluginCount": 0,
|
|
"skillCount": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "CPS",
|
|
"status": "ok",
|
|
"files_scanned": 1,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "DIS",
|
|
"status": "ok",
|
|
"files_scanned": 1,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "COL",
|
|
"status": "ok",
|
|
"files_scanned": 0,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
},
|
|
{
|
|
"scanner": "SKL",
|
|
"status": "ok",
|
|
"files_scanned": 0,
|
|
"duration_ms": 0,
|
|
"findings": [],
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 0,
|
|
"low": 0,
|
|
"info": 0
|
|
}
|
|
}
|
|
],
|
|
"aggregate": {
|
|
"total_findings": 19,
|
|
"counts": {
|
|
"critical": 0,
|
|
"high": 0,
|
|
"medium": 1,
|
|
"low": 7,
|
|
"info": 11
|
|
},
|
|
"risk_score": 11,
|
|
"risk_band": "Medium",
|
|
"verdict": "PASS",
|
|
"scanners_ok": 12,
|
|
"scanners_error": 0,
|
|
"scanners_skipped": 1
|
|
}
|
|
}
|
|
}
|