fix(scanners)!: a finding ID names the check, not the emission (M-BUG-28)
BREAKING CHANGE: the {NNN} in CA-{SCANNER}-{NNN} identifies the check that
produced the finding. It used to be the finding's position in that scanner's
output for that run, which made it unstable across CONFIGURATIONS, not just
across releases as STATE framed it. Measured on two fixtures: "No custom
subagents" was CA-GAP-007 on minimal-project and CA-GAP-004 on healthy-project.
A user who fixed an unrelated earlier gap silently renumbered every later one,
so a .config-audit-ignore pin retargeted to a neighbouring finding with no
version change at all.
Second measured arm: README already documented the opposite scheme. It and the
scanner headers describe ~20 numbers as check codes (CA-SKL-003 = oversized
body, CA-PLH-015 = folder shadowing, CA-TOK-006 = schema deferral), and the
counter could only produce those in the all-fire case -- source-order positions
are 4, 3 and 8. The documentation described the scheme; the implementation was
what was wrong. Every published number is preserved by construction and pinned
exhaustively in tests/lib/finding-codes.test.mjs.
scanners/lib/finding-codes.mjs is the single authority. Every finding() call
passes a `code`; an undeclared or missing one THROWS. No counter fallback --
that would reproduce D1's findGapId -> 'unknown' silent degradation and let a
half-converted scanner ship IDs that look valid. findingCounter/resetCounter
are deleted outright, not left as no-ops. Retirement is now a mechanism:
RETIRED_CODES tombstones a withdrawn key so its number is never reissued,
seeded with GAP t3_8 -- the D1 removal that opened this chunk.
IDs are consequently NOT unique per finding: one check failing in three files
emits three findings sharing an ID. That inverts which consumer is correct, so
every f.id/findingId site was classified before the change. diff-engine and
most of fix-engine already keyed on scanner+title+file (drift was never lying);
fix-engine's verification did not, and keyed on the ID alone -- fixing one of
two sibling instances marked both fixed, and the untouched one, still present
in the re-scan, was reported as a REGRESSION. Red test first, then keyed on
(findingId, file), which both planFixes and applyFixes already carry.
plugin-health's crossIds Set was measured and is a clean negative: cross
findings are allFindings.slice(crossPluginStart) and codes 18/19 are emitted
only in that tail, so the partition holds by construction.
unknownSuppressions() reports a pin that names no declared check, in the
--output-file payload (ux-rules rule 2 -- a stderr-only warning is invisible to
the commands) and only when one exists, so a clean config is byte-identical.
That is what makes the break safe: a stale pin goes loud instead of dying quiet.
Frozen tests/snapshots/v5.0.0/ untouched on disk. IDs are masked out of that
comparison (mask-finding-ids.mjs) rather than re-derived -- re-deriving
positional IDs would assert the retired scheme against itself, and #58's
isGapEntry off-by-one is the measured example of that misfiring. The dead
re-derivation is removed from strip-retired-gap.mjs. default-output snapshots
re-approved after confirming the diff is IDs and nothing else.
Guards, each seen red against its own defect: a missing code (scanner errors
out mid-sweep), an orphan declaration, a resurrected retired key, and a
documented ID naming no check. The sweep asserts the union across all 16
scanners, never per scanner -- a per-scanner assertion goes green on a partial
conversion.
Fasit written before implementation: docs/mbug28-id-semantics-fasit.local.md,
including one correction made before running (CML has 12 checks over 13 call
sites -- the anchored and calibrated char-budget arms are one check, which a
repeated-title sweep found and my call-site count had missed).
Suite 1535 -> 1573, 0 failing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MyqCQKK2ornJ1jFWwqx17E
This commit is contained in:
parent
4027cdcf54
commit
7a794b47eb
62 changed files with 1092 additions and 279 deletions
|
|
@ -79,6 +79,8 @@ Workflow: `/config-audit → discover + analyze (auto) → plan → implement
|
|||
|
||||
Finding ID format: `CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-SKL-001`, `CA-OST-001`, `CA-OPT-001`, `CA-AGT-001`.
|
||||
|
||||
**`{NNN}` names the CHECK, never the emission position (invariant).** `scanners/lib/finding-codes.mjs` is the single authority: every `finding()` call passes a `code`, and an undeclared or missing one **throws** — there is no counter fallback, because a fallback lets a half-converted scanner ship IDs that look valid. Adding a check takes the next free number for that scanner, never the next source-order position; removing one moves its key to `RETIRED_CODES` and its number is never reissued. IDs are therefore **not unique per finding** — one check failing in three files emits three findings sharing an ID, and `(id, file, line)` is the instance key that `fix-engine` verification uses. Frozen `v5.0.0` baselines mask IDs (`tests/helpers/mask-finding-ids.mjs`) instead of re-deriving them; the check→number pairs are pinned exhaustively in `tests/lib/finding-codes.test.mjs`.
|
||||
|
||||
## Conventions
|
||||
|
||||
Enforced conventions live in `.claude/rules/` (auto-loaded as project instructions):
|
||||
|
|
|
|||
|
|
@ -465,7 +465,12 @@ Skills activate automatically when your question matches their trigger patterns.
|
|||
|
||||
### Finding ID Format
|
||||
|
||||
Every finding has a unique ID: `CA-{SCANNER}-{NNN}` — where `{SCANNER}` is the scanner prefix (see table above) and `{NNN}` is a sequential number. Examples: `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`.
|
||||
Every finding carries an ID of the form `CA-{SCANNER}-{NNN}` — where `{SCANNER}` is the scanner prefix (see table above) and `{NNN}` **names the check**, not the finding's position in a run. Examples: `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`.
|
||||
|
||||
Two consequences worth knowing before you pin one:
|
||||
|
||||
- **The same check always has the same ID.** It does not move when you fix an unrelated finding, when a check stops firing, or when a later release adds or retires one. That is what makes an ID safe to write into `.config-audit-ignore`. A number withdrawn from service is never reissued — a suppression naming a retired check goes dead rather than quietly matching a different one.
|
||||
- **An ID is not unique per finding.** One check failing in three files produces three findings that share an ID; `file` and `line` tell them apart. Suppressing by ID suppresses the check everywhere in scope.
|
||||
|
||||
### Suppression
|
||||
|
||||
|
|
@ -484,6 +489,8 @@ CA-PLH-*
|
|||
|
||||
Suppressed findings are tracked in the scan envelope's `suppressed_findings` array for audit trail — nothing is silently hidden. Use `--no-suppress` to see everything.
|
||||
|
||||
A pattern that names no known check is reported back in the envelope's `unknown_suppressions` array, so a pin that has gone stale (a typo, or a check retired in a later release) is visible instead of silently protecting nothing. Scanner-wide globs like `CA-GAP-*` are validated only down to the prefix, which is why a glob is the durable way to pin a whole scanner.
|
||||
|
||||
---
|
||||
|
||||
## Examples & Self-Audit
|
||||
|
|
|
|||
|
|
@ -65,6 +65,7 @@ export async function scan(_targetPath, _discovery) {
|
|||
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'description-bloat',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Agent description is long (re-sent every turn in the always-loaded listing)',
|
||||
description:
|
||||
|
|
@ -91,6 +92,7 @@ export async function scan(_targetPath, _discovery) {
|
|||
if (aggregate.overBudget) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'aggregate-listing-budget',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Aggregate agent listing may exceed the always-loaded budget',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -157,6 +157,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'volatile-in-prefix',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Volatile content inside cached prefix breaks reuse',
|
||||
description:
|
||||
|
|
@ -199,6 +200,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'volatile-in-import',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Volatile content in @imported file breaks cached prefix',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
*/
|
||||
|
||||
import { readTextFile } from './lib/file-discovery.mjs';
|
||||
import { finding, scannerResult, resetCounter } from './lib/output.mjs';
|
||||
import { finding, scannerResult } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
import { parseFrontmatter, extractSections, findImports } from './lib/yaml-parser.mjs';
|
||||
import { lineCount, truncate } from './lib/string-utils.mjs';
|
||||
|
|
@ -62,6 +62,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
return scannerResult(SCANNER, 'ok', [
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-claude-md',
|
||||
severity: SEVERITY.high,
|
||||
title: 'No CLAUDE.md found',
|
||||
description: 'No CLAUDE.md files were discovered. This is the primary configuration surface for Claude Code.',
|
||||
|
|
@ -91,6 +92,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (file.scope === 'project' && relDir !== '.' && relDir !== '.claude' && lines > 5) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'nested-not-reinjected',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Nested CLAUDE.md is not re-injected after compaction',
|
||||
description: `${file.relPath} is a nested (subdirectory) CLAUDE.md. It loads when Claude reads a file in that directory, but after a context compaction it is not re-injected (only the project-root CLAUDE.md is) — its instructions silently drop until a file in that directory is read again.`,
|
||||
|
|
@ -109,6 +111,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (lines > MAX_ABSOLUTE_LINES) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'over-500-lines',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md exceeds 500 lines',
|
||||
description: `${file.relPath} has ${lines} lines. A file this size loads in full on every turn (token cost) and, on smaller-context models, can crowd out instructions. Large-context models tolerate longer files when the cache prefix stays stable — raw line count is no longer an absolute adherence threshold (CC 2.1.169 scales it by context window).`,
|
||||
|
|
@ -120,6 +123,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
} else if (lines > MAX_RECOMMENDED_LINES) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'over-200-lines',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md exceeds recommended 200 lines',
|
||||
description: `${file.relPath} has ${lines} lines. Under ~200 lines is the safe default across models; larger is fine on large-context models when the cache prefix stays stable. A long file still costs tokens every turn.`,
|
||||
|
|
@ -142,6 +146,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
// Conservative 200k anchor — byte-identical to the pre-B8 finding.
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'over-char-budget',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md exceeds Claude Code\'s performance-warning threshold',
|
||||
description: `${file.relPath} is ${withCommas(chars)} chars. Claude Code shows a startup warning ("Large CLAUDE.md will impact performance ... chars > 40.0k") once a CLAUDE.md passes ~40.0k chars on a 200k-context model — it loads in full on every turn. CC 2.1.169 scales that threshold with the context window, so on a ${withCommas(LARGE_CONTEXT_WINDOW)}-token model it relaxes to ~${withCommas(CLAUDE_MD_CHAR_WARN_LARGE)} chars and you are likely within it.`,
|
||||
|
|
@ -156,6 +161,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
const threshLabel = withCommas(charThreshold);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'over-char-budget',
|
||||
severity: advisory ? SEVERITY.info : SEVERITY.medium,
|
||||
title: 'CLAUDE.md exceeds Claude Code\'s performance-warning threshold',
|
||||
description: `${file.relPath} is ${withCommas(chars)} chars, over the ~${threshLabel}-char performance-warning threshold Claude Code applies at a ${winLabel}-token context window (it scales the ~40.0k-char @ 200k warning by the context window, CC 2.1.169) — it loads in full on every turn.` +
|
||||
|
|
@ -172,6 +178,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (lines < 3) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'nearly-empty',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md is nearly empty',
|
||||
description: `${file.relPath} has only ${lines} lines.`,
|
||||
|
|
@ -197,6 +204,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (missingSections.length > 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-sections',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Missing recommended sections',
|
||||
description: `${file.relPath} is missing: ${missingSections.join(', ')}`,
|
||||
|
|
@ -212,6 +220,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (sections.length === 0 && lines > 10) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-headings',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md has no markdown headings',
|
||||
description: `${file.relPath} has ${lines} lines but no ## headings. Structured content with headers improves Claude's ability to find and follow instructions.`,
|
||||
|
|
@ -228,6 +237,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (imp.path.includes('..') && imp.path.split('..').length > 3) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deep-relative-import',
|
||||
severity: SEVERITY.low,
|
||||
title: '@import with deep relative path',
|
||||
description: `${file.relPath}:${imp.line} imports "${truncate(imp.path, 60)}" with multiple parent traversals.`,
|
||||
|
|
@ -245,6 +255,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (htmlComments > 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'html-comments',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Uses HTML comments',
|
||||
description: `${file.relPath} uses ${htmlComments} HTML comment(s). These are stripped before injection, saving tokens.`,
|
||||
|
|
@ -266,6 +277,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (duplicates.length > 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'repeated-content',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Repeated content detected',
|
||||
description: `${file.relPath} has ${duplicates.length} line(s) repeated 3+ times.`,
|
||||
|
|
@ -281,6 +293,7 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
if (todos.length > 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'todo-markers',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Contains TODO/FIXME markers',
|
||||
description: `${file.relPath} has ${todos.length} TODO/FIXME/HACK marker(s).`,
|
||||
|
|
|
|||
|
|
@ -74,6 +74,7 @@ export async function scan(_targetPath, _discovery) {
|
|||
];
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'skill-user-vs-plugin',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Skill name "${name}" collides between user-level and plugin sources`,
|
||||
description:
|
||||
|
|
@ -97,6 +98,7 @@ export async function scan(_targetPath, _discovery) {
|
|||
const pluginNames = pluginSkills.map(s => s.pluginName);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'skill-multi-plugin',
|
||||
severity: SEVERITY.low,
|
||||
title: `Skill name "${name}" used by multiple plugins`,
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -129,6 +129,7 @@ export async function scan(targetPath, discovery) {
|
|||
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'settings-key-conflict',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Settings key conflict: "${key}"`,
|
||||
description: `Key "${key}" has different values across scopes. ${details}`,
|
||||
|
|
@ -160,6 +161,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (rulesIntersect(allowRule, denyRule)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'permission-allow-deny',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Permission allow/deny conflict',
|
||||
description: `"${allowRule}" is allowed in ${a.scope} (${a.file}) but denied in ${b.scope} (${b.file}).`,
|
||||
|
|
@ -177,6 +179,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (rulesIntersect(allowRule, denyRule)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'permission-allow-deny',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Permission allow/deny conflict',
|
||||
description: `"${allowRule}" is allowed in ${b.scope} (${b.file}) but denied in ${a.scope} (${a.file}).`,
|
||||
|
|
@ -227,6 +230,7 @@ export async function scan(targetPath, discovery) {
|
|||
const [event, matcher] = key.split(':');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'duplicate-hook',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Duplicate hook definition',
|
||||
description: `Hook "${event}" with matcher "${matcher}" is defined in ${uniqueSources.length} sources.`,
|
||||
|
|
|
|||
|
|
@ -113,6 +113,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deny-and-allow',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Tool listed in both permissions.deny and permissions.allow',
|
||||
description:
|
||||
|
|
@ -134,6 +135,7 @@ export async function scan(targetPath, discovery) {
|
|||
const evidence = `allow: ${ineffective.slice(0, 5).map(e => `"${e}"`).join(', ')}`;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'ineffective-allow-wildcard',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Ineffective allow wildcard — Claude Code ignores this rule',
|
||||
description:
|
||||
|
|
@ -160,6 +162,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'forbidden-param-deny',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Permission rule silently ignored — deny/ask uses a forbidden param key',
|
||||
description:
|
||||
|
|
@ -184,6 +187,7 @@ export async function scan(targetPath, discovery) {
|
|||
.join('; ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'forbidden-param-allow',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Permission rule silently ignored — allow uses a forbidden param key (dead config)',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -177,6 +177,7 @@ export function bundledSkillsLeverFinding({ leverPulled, aggregate }) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.low,
|
||||
code: 'bundled-skills-lever',
|
||||
title: 'Bundled skills add to an over-budget skill listing',
|
||||
description:
|
||||
`Your ${aggregate.scanned} active skills already carry ~${aggregate.aggregateTokens} tokens of ` +
|
||||
|
|
@ -221,6 +222,7 @@ export function cliOverMcpLeverFinding({ assessment } = {}) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.low,
|
||||
code: 'cli-over-mcp-lever',
|
||||
title: 'Prefer CLI over MCP for common operations',
|
||||
description:
|
||||
`Your active project MCP tool schemas (~${assessment.aggregateTokens} tokens) are forced into the ` +
|
||||
|
|
@ -258,6 +260,7 @@ export function filterHookLeverFinding({ flaggedHooks } = {}) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
code: 'filter-hook-output-lever',
|
||||
title: 'Filter hook output before it enters context',
|
||||
description:
|
||||
`${hooks.length} active hook${hooks.length === 1 ? '' : 's'} build hookSpecificOutput.additionalContext ` +
|
||||
|
|
@ -574,6 +577,7 @@ export async function scan(targetPath, sharedDiscovery) {
|
|||
if (!present) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: gap.id,
|
||||
severity: TIER_SEVERITY[gap.tier],
|
||||
title: gap.title,
|
||||
description: `Feature gap: ${gap.title}. ${gap.recommendation}`,
|
||||
|
|
|
|||
|
|
@ -629,9 +629,13 @@ export async function verifyFixes(originalEnvelope, appliedResults, opts = {}) {
|
|||
// "fixed" report for files nothing had touched.
|
||||
const newEnvelope = await runAllScanners(targetPath, { includeGlobal: opts.includeGlobal === true });
|
||||
|
||||
// Build set of original finding IDs that were fixed
|
||||
const fixedIds = new Set(
|
||||
appliedResults.filter(r => r.status === 'applied').map(r => r.findingId),
|
||||
// Build the set of fixed finding INSTANCES. A finding ID names the check, so
|
||||
// one check failing in two files yields two findings sharing an ID; keying on
|
||||
// the ID alone marks both fixed when one was, and the untouched sibling — still
|
||||
// present in the re-scan — is then reported as a regression (M-BUG-28).
|
||||
const instanceKey = (findingId, file) => `${findingId}::${file || ''}`;
|
||||
const fixedInstances = new Set(
|
||||
appliedResults.filter(r => r.status === 'applied').map(r => instanceKey(r.findingId, r.file)),
|
||||
);
|
||||
|
||||
// Build set of new finding titles for comparison
|
||||
|
|
@ -645,11 +649,13 @@ export async function verifyFixes(originalEnvelope, appliedResults, opts = {}) {
|
|||
// Check that fixed findings are gone
|
||||
for (const scanner of originalEnvelope.scanners) {
|
||||
for (const f of scanner.findings) {
|
||||
if (!fixedIds.has(f.id)) continue;
|
||||
if (!fixedInstances.has(instanceKey(f.id, f.file))) continue;
|
||||
|
||||
const key = `${f.scanner}:${f.title}:${f.file}`;
|
||||
// For file-rename fixes, the original file path won't exist anymore
|
||||
const fixResult = appliedResults.find(r => r.findingId === f.id);
|
||||
const fixResult = appliedResults.find(
|
||||
r => instanceKey(r.findingId, r.file) === instanceKey(f.id, f.file),
|
||||
);
|
||||
if (fixResult && fixResult.type === 'file-rename') {
|
||||
// Check that the finding doesn't reappear at the new path
|
||||
verified.push(f.id);
|
||||
|
|
|
|||
|
|
@ -70,6 +70,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed === null) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Invalid JSON in hooks.json',
|
||||
description: `${file.relPath} contains invalid JSON. All hooks in this file will be ignored.`,
|
||||
|
|
@ -120,6 +121,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (typeof hooks !== 'object' || Array.isArray(hooks)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-not-object',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Hooks must be an object with event keys',
|
||||
description: `${file.relPath}: hooks is ${Array.isArray(hooks) ? 'an array' : typeof hooks}. Expected object with event names as keys.`,
|
||||
|
|
@ -135,6 +137,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (!VALID_EVENTS.has(event)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unknown-event',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Unknown hook event',
|
||||
description: `${file.relPath}: "${event}" is not a valid hook event. This hook will never fire.`,
|
||||
|
|
@ -149,6 +152,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (!Array.isArray(handlers)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'handlers-not-array',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Hook handlers must be an array',
|
||||
description: `${file.relPath}: handlers for "${event}" is not an array.`,
|
||||
|
|
@ -166,6 +170,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (typeof handlerGroup.matcher === 'object') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'matcher-not-string',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Matcher must be a string, not an object',
|
||||
description: `${file.relPath}: "${event}" has a matcher that is an object. Matcher should be a simple string like "Bash" or "Edit|Write".`,
|
||||
|
|
@ -180,6 +185,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (!handlerGroup.hooks || !Array.isArray(handlerGroup.hooks)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-hooks-array',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Missing hooks array in handler group',
|
||||
description: `${file.relPath}: "${event}" handler group is missing the "hooks" array.`,
|
||||
|
|
@ -195,6 +201,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (!hook.type || !VALID_TYPES.has(hook.type)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-handler-type',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Invalid hook handler type',
|
||||
description: `${file.relPath}: "${event}" has handler with type "${hook.type || '(missing)'}".`,
|
||||
|
|
@ -216,6 +223,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'script-not-found',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Hook script not found',
|
||||
description: `${file.relPath}: "${event}" references script that does not exist.`,
|
||||
|
|
@ -232,6 +240,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (verboseCount > VERBOSE_HOOK_LINE_THRESHOLD) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'verbose-output',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Verbose hook output (loud script)',
|
||||
description:
|
||||
|
|
@ -259,6 +268,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (ac.flagged) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unfiltered-additional-context',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Hook injects unfiltered output into context',
|
||||
description:
|
||||
|
|
@ -287,6 +297,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
if (typeof hook.timeout !== 'number') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'timeout-not-number',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Hook timeout must be a number',
|
||||
description: `${file.relPath}: "${event}" has non-numeric timeout.`,
|
||||
|
|
@ -298,6 +309,7 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
} else if (hook.timeout < MIN_TIMEOUT || hook.timeout > MAX_TIMEOUT) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'timeout-out-of-range',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Hook timeout outside recommended range',
|
||||
description: `${file.relPath}: "${event}" timeout is ${hook.timeout}ms. Recommended range: ${MIN_TIMEOUT}-${MAX_TIMEOUT}ms.`,
|
||||
|
|
|
|||
|
|
@ -74,6 +74,7 @@ async function walkImports(file, chain, reported, findings) {
|
|||
reported.add(`tilde::${resolved}`);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'tilde-path',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Tilde path in @import',
|
||||
description: `@${imp.path} uses ~ which may not expand correctly in all contexts.`,
|
||||
|
|
@ -91,6 +92,7 @@ async function walkImports(file, chain, reported, findings) {
|
|||
reported.add(reportKey);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'broken-link',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Broken @import link',
|
||||
description: `@${imp.path} references a file that does not exist.`,
|
||||
|
|
@ -111,6 +113,7 @@ async function walkImports(file, chain, reported, findings) {
|
|||
const cycle = chain.slice(cycleStart).map(f => basename(f)).join(' → ');
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'circular-reference',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Circular @import reference',
|
||||
description: `@${imp.path} creates a circular import chain.`,
|
||||
|
|
@ -129,6 +132,7 @@ async function walkImports(file, chain, reported, findings) {
|
|||
reported.add(`deep::${resolved}`);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deep-chain',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Deep @import chain',
|
||||
description: `@${imp.path} is at depth ${chain.length} (>${MAX_CHAIN_DEPTH} hops).`,
|
||||
|
|
|
|||
316
scanners/lib/finding-codes.mjs
Normal file
316
scanners/lib/finding-codes.mjs
Normal file
|
|
@ -0,0 +1,316 @@
|
|||
/**
|
||||
* Finding-code registry — the authority for the {NNN} in `CA-{SCANNER}-{NNN}`.
|
||||
*
|
||||
* A finding ID names the CHECK, not the finding's position in a run (M-BUG-28).
|
||||
* Before this registry, `{NNN}` came from an emission counter, so the same check
|
||||
* carried different IDs on different configurations: fixing an unrelated earlier
|
||||
* gap silently renumbered every later one, and a `.config-audit-ignore` entry
|
||||
* retargeted to a neighbouring finding without the user changing anything.
|
||||
*
|
||||
* Rules for editing this file:
|
||||
*
|
||||
* 1. A number, once published, belongs to its check forever. Adding a check
|
||||
* takes the next free number for that scanner — never the next source-order
|
||||
* position, and never a number listed in RETIRED.
|
||||
* 2. Removing a check moves its key to RETIRED. The number is never reissued;
|
||||
* a user's suppression must go dead rather than quietly point at a
|
||||
* different finding. (D1 retired GAP `t3_8` under the old scheme, which is
|
||||
* the incident that motivated the registry.)
|
||||
* 3. Several call sites may share one code when they are arms of one check —
|
||||
* e.g. the forward/reverse arms of a permission conflict. Duplicate
|
||||
* emission is legal; a finding is identified by (id, file, line).
|
||||
* 4. Numbers below are NOT all source order: the ones marked "documented"
|
||||
* are pinned by README / command copy that shipped before the registry.
|
||||
*
|
||||
* GAP keys are the `GAP_CHECKS[].id` values from `feature-gap-scanner.mjs`,
|
||||
* which were already stable. They are declared here rather than derived, so
|
||||
* numbers live in exactly one place; `tests/lib/finding-codes.test.mjs` binds
|
||||
* the two together instead of a second copy of the table drifting.
|
||||
*/
|
||||
|
||||
/**
|
||||
* @type {Record<string, Record<string, number>>}
|
||||
* scanner prefix → check key → number
|
||||
*/
|
||||
export const FINDING_CODES = {
|
||||
// ── CML: claude-md-linter (source order) ────────────────────────────────
|
||||
// `over-char-budget` has two call sites: the conservative 200k anchor and the
|
||||
// `--context-window` calibrated variant. One check, one code.
|
||||
CML: {
|
||||
'no-claude-md': 1,
|
||||
'nested-not-reinjected': 2,
|
||||
'over-500-lines': 3,
|
||||
'over-200-lines': 4,
|
||||
'over-char-budget': 5,
|
||||
'nearly-empty': 6,
|
||||
'missing-sections': 7,
|
||||
'no-headings': 8,
|
||||
'deep-relative-import': 9,
|
||||
'html-comments': 10,
|
||||
'repeated-content': 11,
|
||||
'todo-markers': 12,
|
||||
},
|
||||
|
||||
// ── SET: settings-validator (source order) ──────────────────────────────
|
||||
SET: {
|
||||
'invalid-json': 1,
|
||||
'key-typo': 2,
|
||||
'deprecated-key': 3,
|
||||
'type-mismatch': 4,
|
||||
'invalid-effort-level': 5,
|
||||
'missing-schema': 6,
|
||||
'no-deny-rules': 7,
|
||||
'no-allow-rules': 8,
|
||||
'many-additional-dirs': 9,
|
||||
'automode-not-object': 10,
|
||||
'automode-unknown-subkey': 11,
|
||||
'automode-subkey-not-string-array': 12,
|
||||
'automode-in-shared-settings': 13,
|
||||
'hooks-as-array': 14,
|
||||
},
|
||||
|
||||
// ── HKV: hook-validator (source order) ──────────────────────────────────
|
||||
HKV: {
|
||||
'invalid-json': 1,
|
||||
'hooks-not-object': 2,
|
||||
'unknown-event': 3,
|
||||
'handlers-not-array': 4,
|
||||
'matcher-not-string': 5,
|
||||
'missing-hooks-array': 6,
|
||||
'invalid-handler-type': 7,
|
||||
'script-not-found': 8,
|
||||
'verbose-output': 9,
|
||||
'unfiltered-additional-context': 10,
|
||||
'timeout-not-number': 11,
|
||||
'timeout-out-of-range': 12,
|
||||
},
|
||||
|
||||
// ── RUL: rules-validator (source order) ─────────────────────────────────
|
||||
RUL: {
|
||||
'no-frontmatter': 1,
|
||||
'globs-instead-of-paths': 2,
|
||||
'pattern-matches-nothing': 3,
|
||||
'nearly-empty': 4,
|
||||
'large-unscoped': 5,
|
||||
'large-scoped-lost-after-compaction': 6,
|
||||
'not-markdown': 7,
|
||||
},
|
||||
|
||||
// ── MCP: mcp-config-validator (source order) ────────────────────────────
|
||||
MCP: {
|
||||
'invalid-json': 1,
|
||||
'unknown-server-type': 2,
|
||||
'sse-transport': 3,
|
||||
'unreferenced-env-var': 4,
|
||||
'unknown-server-field': 5,
|
||||
},
|
||||
|
||||
// ── IMP: import-resolver (source order) ─────────────────────────────────
|
||||
IMP: {
|
||||
'tilde-path': 1,
|
||||
'broken-link': 2,
|
||||
'circular-reference': 3,
|
||||
'deep-chain': 4,
|
||||
},
|
||||
|
||||
// ── CNF: conflict-detector ──────────────────────────────────────────────
|
||||
// `permission-allow-deny` covers both arms (allow-in-A/deny-in-B and reverse).
|
||||
CNF: {
|
||||
'settings-key-conflict': 1,
|
||||
'permission-allow-deny': 2,
|
||||
'duplicate-hook': 3,
|
||||
},
|
||||
|
||||
// ── DIS: disabled-in-schema-scanner (source order) ──────────────────────
|
||||
DIS: {
|
||||
'deny-and-allow': 1,
|
||||
'ineffective-allow-wildcard': 2,
|
||||
'forbidden-param-deny': 3,
|
||||
'forbidden-param-allow': 4,
|
||||
},
|
||||
|
||||
// ── CPS: cache-prefix-scanner (CPS-001 documented) ──────────────────────
|
||||
CPS: {
|
||||
'volatile-in-prefix': 1,
|
||||
'volatile-in-import': 2,
|
||||
},
|
||||
|
||||
// ── COL: collision-scanner (source order) ───────────────────────────────
|
||||
COL: {
|
||||
'skill-user-vs-plugin': 1,
|
||||
'skill-multi-plugin': 2,
|
||||
},
|
||||
|
||||
// ── AGT: agent-listing-scanner (both documented; source order matches) ──
|
||||
AGT: {
|
||||
'description-bloat': 1,
|
||||
'aggregate-listing-budget': 2,
|
||||
},
|
||||
|
||||
// ── OST: output-style-scanner (all three documented) ────────────────────
|
||||
OST: {
|
||||
'strips-coding-instructions': 1,
|
||||
'plugin-forces-style': 2,
|
||||
'style-not-found': 3,
|
||||
},
|
||||
|
||||
// ── OPT: optimization-lens-scanner (documented) ─────────────────────────
|
||||
OPT: {
|
||||
'procedure-should-be-skill': 1,
|
||||
},
|
||||
|
||||
// ── SKL: skill-listing-scanner (all three documented) ───────────────────
|
||||
// `aggregate-listing-budget` has two call sites: the conservative 200k anchor
|
||||
// and the calibrated `--context-window` variant. One check, one code.
|
||||
SKL: {
|
||||
'description-over-cap': 1,
|
||||
'aggregate-listing-budget': 2,
|
||||
'oversized-body': 3,
|
||||
},
|
||||
|
||||
// ── TOK: token-hotspots ─────────────────────────────────────────────────
|
||||
// 1/2/3/5/6 are documented (README + commands/tokens.md). `mcp-schema-deferral`
|
||||
// is documented as 006 although it is the 8th call site in source order, so
|
||||
// `cascade-over-budget` and `stale-plugin-cache` take the free 7 and 8.
|
||||
TOK: {
|
||||
'volatile-top': 1,
|
||||
'redundant-permissions': 2,
|
||||
'deep-import-chain': 3,
|
||||
'bloated-skill-description': 4,
|
||||
'mcp-schema-budget': 5,
|
||||
'mcp-schema-deferral': 6,
|
||||
'cascade-over-budget': 7,
|
||||
'stale-plugin-cache': 8,
|
||||
},
|
||||
|
||||
// ── PLH: plugin-health-scanner ──────────────────────────────────────────
|
||||
// 15 and 16 are documented (README v5.4.0 entry) but sit at source positions
|
||||
// 3 and 4; the remaining checks take {1…14, 17, 18, 19} in source order.
|
||||
PLH: {
|
||||
'invalid-plugin-json': 1,
|
||||
'missing-required-field': 2,
|
||||
'missing-plugin-json': 3,
|
||||
'claude-md-missing-section': 4,
|
||||
'missing-claude-md': 5,
|
||||
'command-missing-frontmatter': 6,
|
||||
'command-missing-field': 7,
|
||||
'agent-missing-frontmatter': 8,
|
||||
'agent-missing-field': 9,
|
||||
'agent-ignored-key': 10,
|
||||
'hooks-json-invalid-structure': 11,
|
||||
'hooks-json-array': 12,
|
||||
'hooks-json-invalid': 13,
|
||||
'unknown-plugin-file': 14,
|
||||
'plugin-json-shadows-default': 15,
|
||||
'skills-array-entry': 16,
|
||||
'no-plugins-found': 17,
|
||||
'command-name-collision': 18,
|
||||
'namespace-collision': 19,
|
||||
},
|
||||
|
||||
// ── GAP: feature-gap-scanner ────────────────────────────────────────────
|
||||
// Keys are GAP_CHECKS[].id (already stable). Dimensions 1–24 in table order,
|
||||
// then the three conditional levers, which the scanner emits after the loop.
|
||||
GAP: {
|
||||
t1_1: 1,
|
||||
t1_2: 2,
|
||||
t1_3: 3,
|
||||
t1_4: 4,
|
||||
t1_5: 5,
|
||||
t2_1: 6,
|
||||
t2_2: 7,
|
||||
t2_3: 8,
|
||||
t2_4: 9,
|
||||
t2_5: 10,
|
||||
t2_6: 11,
|
||||
t2_7: 12,
|
||||
t3_1: 13,
|
||||
t3_2: 14,
|
||||
t3_3: 15,
|
||||
t3_4: 16,
|
||||
t3_5: 17,
|
||||
t3_6: 18,
|
||||
t3_7: 19,
|
||||
t4_1: 20,
|
||||
t4_2: 21,
|
||||
t4_3: 22,
|
||||
t4_4: 23,
|
||||
t4_5: 24,
|
||||
'bundled-skills-lever': 25,
|
||||
'cli-over-mcp-lever': 26,
|
||||
'filter-hook-output-lever': 27,
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Keys withdrawn from a scanner. Their numbers are never reissued, so a stale
|
||||
* suppression goes dead instead of silently naming a different check.
|
||||
* @type {Record<string, string[]>}
|
||||
*/
|
||||
export const RETIRED_CODES = {
|
||||
// D1 (4027cdc, 2026-08-09): "No autoMode classifier" — /doctor Check 8 covers
|
||||
// auto mode with usage-weighted judgement, so the nudge went. The number it
|
||||
// occupied under the old counter scheme is not reused.
|
||||
GAP: ['t3_8'],
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve a check key to its published number.
|
||||
* Throws rather than falling back: a fallback would let a half-converted scanner
|
||||
* ship IDs that look valid, which is the silent-degradation class this registry
|
||||
* exists to remove.
|
||||
* @param {string} scanner - scanner prefix, e.g. 'GAP'
|
||||
* @param {string} code - check key, e.g. 't3_7'
|
||||
* @returns {number}
|
||||
*/
|
||||
export function codeNumber(scanner, code) {
|
||||
const table = FINDING_CODES[scanner];
|
||||
if (!table) {
|
||||
throw new Error(`finding(): unknown scanner "${scanner}" — add it to FINDING_CODES`);
|
||||
}
|
||||
if (code === undefined || code === null || code === '') {
|
||||
throw new Error(`finding(): missing "code" for scanner ${scanner} — every finding must name its check`);
|
||||
}
|
||||
if (!Object.prototype.hasOwnProperty.call(table, code)) {
|
||||
const retired = (RETIRED_CODES[scanner] || []).includes(code);
|
||||
throw new Error(
|
||||
retired
|
||||
? `finding(): check "${code}" is RETIRED for ${scanner} — retired numbers are never reissued`
|
||||
: `finding(): undeclared check "${code}" for ${scanner} — add it to FINDING_CODES with the next free number`
|
||||
);
|
||||
}
|
||||
return table[code];
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a finding ID from a check key.
|
||||
* @param {string} scanner
|
||||
* @param {string} code
|
||||
* @returns {string} e.g. 'CA-GAP-019'
|
||||
*/
|
||||
export function findingId(scanner, code) {
|
||||
return `CA-${scanner}-${String(codeNumber(scanner, code)).padStart(3, '0')}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every declared ID, as a flat set — used to validate suppression patterns so a
|
||||
* stale pin is reported instead of silently matching nothing.
|
||||
* @returns {Set<string>}
|
||||
*/
|
||||
export function allFindingIds() {
|
||||
const ids = new Set();
|
||||
for (const [scanner, table] of Object.entries(FINDING_CODES)) {
|
||||
for (const n of Object.values(table)) {
|
||||
ids.add(`CA-${scanner}-${String(n).padStart(3, '0')}`);
|
||||
}
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scanner prefixes the registry knows about.
|
||||
* @returns {string[]}
|
||||
*/
|
||||
export function knownScanners() {
|
||||
return Object.keys(FINDING_CODES);
|
||||
}
|
||||
|
|
@ -5,18 +5,13 @@
|
|||
*/
|
||||
|
||||
import { riskScore, riskBand, verdict } from './severity.mjs';
|
||||
|
||||
let findingCounter = 0;
|
||||
|
||||
/** Reset the finding counter. Call in beforeEach of tests and before each scanner run. */
|
||||
export function resetCounter() {
|
||||
findingCounter = 0;
|
||||
}
|
||||
import { findingId } from './finding-codes.mjs';
|
||||
|
||||
/**
|
||||
* Create a finding object with auto-incremented ID.
|
||||
* Create a finding object. The ID names the CHECK — see `finding-codes.mjs`.
|
||||
* @param {object} opts
|
||||
* @param {string} opts.scanner - 3-letter scanner prefix (CML, SET, HKV, RUL, etc.)
|
||||
* @param {string} opts.code - check key declared in FINDING_CODES for this scanner
|
||||
* @param {string} opts.severity - critical | high | medium | low | info
|
||||
* @param {string} opts.title
|
||||
* @param {string} opts.description
|
||||
|
|
@ -30,10 +25,8 @@ export function resetCounter() {
|
|||
* @returns {object}
|
||||
*/
|
||||
export function finding(opts) {
|
||||
findingCounter++;
|
||||
const id = `CA-${opts.scanner}-${String(findingCounter).padStart(3, '0')}`;
|
||||
const result = {
|
||||
id,
|
||||
id: findingId(opts.scanner, opts.code),
|
||||
scanner: opts.scanner,
|
||||
severity: opts.severity,
|
||||
title: opts.title,
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@
|
|||
import { readFile } from 'node:fs/promises';
|
||||
import { join } from 'node:path';
|
||||
import { homedir } from 'node:os';
|
||||
import { allFindingIds, knownScanners } from './finding-codes.mjs';
|
||||
|
||||
/**
|
||||
* Load suppressions from .config-audit-ignore files.
|
||||
|
|
@ -69,6 +70,39 @@ export function parseIgnoreFile(content) {
|
|||
return suppressions;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find suppression patterns that can never match anything.
|
||||
*
|
||||
* A finding ID names a check (see `finding-codes.mjs`), so an exact pin either
|
||||
* names a declared check or names nothing at all. Silently keeping a dead pin
|
||||
* would reproduce, in the other direction, the very failure the check-code
|
||||
* scheme removed: the user believes a finding is suppressed when it is not.
|
||||
*
|
||||
* Globs are validated only down to the scanner prefix — `CA-GAP-*` stays valid
|
||||
* however GAP's checks change, which is why a glob is the safe way to pin.
|
||||
*
|
||||
* @param {Array<{ pattern: string }>} suppressions
|
||||
* @returns {string[]} patterns that match no declared check
|
||||
*/
|
||||
export function unknownSuppressions(suppressions) {
|
||||
if (!suppressions || suppressions.length === 0) return [];
|
||||
|
||||
const ids = allFindingIds();
|
||||
const scanners = new Set(knownScanners());
|
||||
const unknown = [];
|
||||
|
||||
for (const { pattern } of suppressions) {
|
||||
if (pattern.endsWith('-*')) {
|
||||
const scanner = pattern.slice(3, -2); // "CA-GAP-*" → "GAP"
|
||||
if (!scanners.has(scanner)) unknown.push(pattern);
|
||||
continue;
|
||||
}
|
||||
if (!ids.has(pattern)) unknown.push(pattern);
|
||||
}
|
||||
|
||||
return unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply suppressions to a findings array.
|
||||
* @param {object[]} findings - Array of finding objects with .id
|
||||
|
|
|
|||
|
|
@ -56,6 +56,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!parsed) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Invalid JSON in MCP config',
|
||||
description: `${file.relPath}: Failed to parse as JSON.`,
|
||||
|
|
@ -75,6 +76,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (config.type && !VALID_SERVER_TYPES.has(config.type)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unknown-server-type',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Unknown MCP server type',
|
||||
description: `${file.relPath}: Server "${name}" has unknown type "${config.type}".`,
|
||||
|
|
@ -88,6 +90,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (config.type === 'sse') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'sse-transport',
|
||||
severity: SEVERITY.info,
|
||||
title: 'SSE server type — consider HTTP',
|
||||
description: `${file.relPath}: Server "${name}" uses "sse" type. The "http" type is the current standard.`,
|
||||
|
|
@ -110,6 +113,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!hasEnvBlock) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unreferenced-env-var',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Unreferenced env var in args',
|
||||
description: `${file.relPath}: Server "${name}" references \${${varName}} in args but has no env block defining it.`,
|
||||
|
|
@ -127,6 +131,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!VALID_SERVER_FIELDS.has(key)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unknown-server-field',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Unknown MCP server field',
|
||||
description: `${file.relPath}: Server "${name}" has unknown field "${key}".`,
|
||||
|
|
|
|||
|
|
@ -122,6 +122,7 @@ export async function scan(targetPath, discovery) {
|
|||
findings.push(
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
code: 'procedure-should-be-skill',
|
||||
severity: SEVERITY.low,
|
||||
title: PROCEDURE_TITLE,
|
||||
description: claim,
|
||||
|
|
|
|||
|
|
@ -28,7 +28,6 @@ import { resolve, sep } from 'node:path';
|
|||
import { readFile, stat } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { discoverConfigFiles } from './lib/file-discovery.mjs';
|
||||
import { resetCounter } from './lib/output.mjs';
|
||||
import { parseFrontmatter } from './lib/yaml-parser.mjs';
|
||||
import { loadRegister, getEntry } from './lib/best-practices-register.mjs';
|
||||
import { prefilterClaudeMd, LENS_DETECTORS } from './lib/lens-prefilter.mjs';
|
||||
|
|
@ -91,7 +90,6 @@ async function main() {
|
|||
register = null;
|
||||
}
|
||||
|
||||
resetCounter();
|
||||
const rawDiscovery = await discoverConfigFiles(absPath, { includeGlobal });
|
||||
// Scope the lens to the user's authored config: drop plugin-bundled files for
|
||||
// BOTH halves of the motor (the OPT scanner reads discovery.files directly).
|
||||
|
|
|
|||
|
|
@ -100,6 +100,7 @@ export async function scan(targetPath, _discovery) {
|
|||
if (kci === true) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'strips-coding-instructions',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Custom output style removes built-in coding instructions',
|
||||
description:
|
||||
|
|
@ -127,6 +128,7 @@ export async function scan(targetPath, _discovery) {
|
|||
if (ffp !== true) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'plugin-forces-style',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Plugin output style overrides your selected output style',
|
||||
description:
|
||||
|
|
@ -155,6 +157,7 @@ export async function scan(targetPath, _discovery) {
|
|||
const customNames = styles.map(s => s.name);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'style-not-found',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Configured output style does not exist',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ import { readdir, stat, readFile } from 'node:fs/promises';
|
|||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireTargetDir } from './lib/require-target-dir.mjs';
|
||||
import { join, basename, resolve, sep } from 'node:path';
|
||||
import { finding, scannerResult, resetCounter } from './lib/output.mjs';
|
||||
import { finding, scannerResult } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
import { parseFrontmatter } from './lib/yaml-parser.mjs';
|
||||
import { humanizeFindings } from './lib/humanizer.mjs';
|
||||
|
|
@ -222,6 +222,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-plugin-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Invalid plugin.json',
|
||||
description: `plugin.json is not valid JSON in ${pluginName}`,
|
||||
|
|
@ -238,6 +239,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!parsed[field]) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-required-field',
|
||||
severity: SEVERITY.high,
|
||||
title: `Missing required field in plugin.json: ${field}`,
|
||||
description: `Plugin "${pluginName}" plugin.json is missing required field "${field}"`,
|
||||
|
|
@ -259,6 +261,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!(await dirExists(join(pluginDir, defaultDir)))) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'plugin-json-shadows-default',
|
||||
severity: SEVERITY.medium,
|
||||
title: `plugin.json "${key}" path shadows the default ${defaultDir}/ folder`,
|
||||
description:
|
||||
|
|
@ -298,6 +301,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
const m = SKILLS_ENTRY_MESSAGES[problem];
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'skills-array-entry',
|
||||
severity: SEVERITY.medium,
|
||||
title: m.title(entry),
|
||||
description: `Plugin "${pluginName}": ${m.description}`,
|
||||
|
|
@ -313,6 +317,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-plugin-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Missing plugin.json',
|
||||
description: `No .claude-plugin/plugin.json found in ${pluginName}`,
|
||||
|
|
@ -338,6 +343,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!hasSection) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'claude-md-missing-section',
|
||||
severity: SEVERITY.medium,
|
||||
title: `CLAUDE.md missing ${section} section`,
|
||||
description: `Plugin "${pluginName}" CLAUDE.md should have a ${section} table or section`,
|
||||
|
|
@ -349,6 +355,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-claude-md',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Missing CLAUDE.md',
|
||||
description: `Plugin "${pluginName}" has no CLAUDE.md`,
|
||||
|
|
@ -372,6 +379,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!frontmatter) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'command-missing-frontmatter',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Command missing frontmatter',
|
||||
description: `Command "${file}" in plugin "${pluginName}" has no frontmatter`,
|
||||
|
|
@ -385,6 +393,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!frontmatter[key]) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'command-missing-field',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Command missing frontmatter field: ${display}`,
|
||||
description: `Command "${file}" in plugin "${pluginName}" is missing "${display}" in frontmatter`,
|
||||
|
|
@ -411,6 +420,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!frontmatter) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'agent-missing-frontmatter',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Agent missing frontmatter',
|
||||
description: `Agent "${file}" in plugin "${pluginName}" has no frontmatter`,
|
||||
|
|
@ -424,6 +434,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!frontmatter[key]) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'agent-missing-field',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Agent missing frontmatter field: ${display}`,
|
||||
description: `Agent "${file}" in plugin "${pluginName}" is missing "${display}" in frontmatter`,
|
||||
|
|
@ -439,6 +450,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (frontmatter[key] !== undefined) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'agent-ignored-key',
|
||||
severity,
|
||||
title: `Plugin agent sets "${key}", which Claude Code ignores`,
|
||||
description: `Agent "${file}" in plugin "${pluginName}" sets "${key}" in frontmatter, but Claude Code ignores ${key} for plugin subagents — ${key === 'permissionMode' ? 'the agent runs with default permissions, not the restricted mode this implies' : 'this configuration has no effect'}.`,
|
||||
|
|
@ -461,6 +473,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!parsed.hooks || typeof parsed.hooks !== 'object') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-json-invalid-structure',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Invalid hooks.json structure',
|
||||
description: `hooks.json in "${pluginName}" missing "hooks" object`,
|
||||
|
|
@ -470,6 +483,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} else if (Array.isArray(parsed.hooks)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-json-array',
|
||||
severity: SEVERITY.high,
|
||||
title: 'hooks.json uses array instead of object',
|
||||
description: `hooks.json "hooks" in "${pluginName}" is an array — must be object with event keys`,
|
||||
|
|
@ -480,6 +494,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
} catch {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-json-invalid',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Invalid hooks.json',
|
||||
description: `hooks.json is not valid JSON in "${pluginName}"`,
|
||||
|
|
@ -503,6 +518,7 @@ async function scanSinglePlugin(pluginDir) {
|
|||
if (!known.has(entry)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'unknown-plugin-file',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Unknown file in .claude-plugin/',
|
||||
description: `Unexpected file "${entry}" in .claude-plugin/ of "${pluginName}"`,
|
||||
|
|
@ -554,7 +570,6 @@ export async function scan(targetPath) {
|
|||
*/
|
||||
export async function scanDetailed(targetPath) {
|
||||
const start = Date.now();
|
||||
resetCounter();
|
||||
|
||||
const pluginDirs = await discoverPlugins(resolve(targetPath));
|
||||
|
||||
|
|
@ -563,6 +578,7 @@ export async function scanDetailed(targetPath) {
|
|||
result: scannerResult(SCANNER, 'ok', [
|
||||
finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-plugins-found',
|
||||
severity: SEVERITY.info,
|
||||
title: 'No plugins found',
|
||||
description: `No Claude Code plugins found under ${targetPath}`,
|
||||
|
|
@ -621,6 +637,7 @@ export async function scanDetailed(targetPath) {
|
|||
const namespaceList = entries.map(e => e.namespace).join(', ');
|
||||
allFindings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'command-name-collision',
|
||||
severity: SEVERITY.low,
|
||||
title: `Command name "${cmdName}" used by multiple plugins`,
|
||||
description:
|
||||
|
|
@ -656,6 +673,7 @@ export async function scanDetailed(targetPath) {
|
|||
if (dirs.length < 2) continue;
|
||||
allFindings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'namespace-collision',
|
||||
severity: SEVERITY.medium,
|
||||
title: `Plugin namespace collision: "${declaredName}"`,
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -55,6 +55,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (lines > 5) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-frontmatter',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Rule has no frontmatter (always active)',
|
||||
description: `${file.relPath} has no YAML frontmatter. It will be loaded for ALL files. Add paths: frontmatter to scope it.`,
|
||||
|
|
@ -69,6 +70,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (frontmatter.globs && !frontmatter.paths) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'globs-instead-of-paths',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Rule uses "globs" instead of documented "paths"',
|
||||
description: `${file.relPath} uses "globs:" for scoping. Claude Code's documentation specifies "paths:" as the rule-scoping field; "globs:" is not documented. Rename to "paths:" so the rule scopes as intended.`,
|
||||
|
|
@ -99,6 +101,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (matchCount === 0) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'pattern-matches-nothing',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Rule path pattern matches no files',
|
||||
description: `${file.relPath}: pattern "${pattern}" matches 0 files. This rule will never activate.`,
|
||||
|
|
@ -117,6 +120,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (lines < 2) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'nearly-empty',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Rule file is nearly empty',
|
||||
description: `${file.relPath} has only ${lines} line(s).`,
|
||||
|
|
@ -130,6 +134,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!frontmatter?.paths && !frontmatter?.globs && lines > 50) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'large-unscoped',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Large unscoped rule file',
|
||||
description: `${file.relPath} has ${lines} lines and no path scoping. It loads into context for every file interaction.`,
|
||||
|
|
@ -147,6 +152,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (frontmatter?.paths && lines > 50) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'large-scoped-lost-after-compaction',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Large path-scoped rule is lost after compaction',
|
||||
description: `${file.relPath} is path-scoped (${lines} lines). Path-scoped rules load only when a matching file is read, and after a context compaction they are not re-injected until a matching file is read again — so a large scoped rule carrying must-always-hold instructions can silently drop out mid-session.`,
|
||||
|
|
@ -161,6 +167,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!file.absPath.endsWith('.md')) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'not-markdown',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Rule file is not .md',
|
||||
description: `${file.relPath} is not a .md file. Only .md files are loaded from rules/.`,
|
||||
|
|
|
|||
|
|
@ -11,10 +11,9 @@ import { resolve, sep } from 'node:path';
|
|||
import { readFile, writeFile } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { requireTargetDir } from './lib/require-target-dir.mjs';
|
||||
import { resetCounter } from './lib/output.mjs';
|
||||
import { envelope } from './lib/output.mjs';
|
||||
import { discoverConfigFiles, discoverConfigFilesMulti, discoverFullMachinePaths } from './lib/file-discovery.mjs';
|
||||
import { loadSuppressions, applySuppressions, formatSuppressionSummary } from './lib/suppression.mjs';
|
||||
import { loadSuppressions, applySuppressions, formatSuppressionSummary, unknownSuppressions } from './lib/suppression.mjs';
|
||||
import { humanizeEnvelope } from './lib/humanizer.mjs';
|
||||
import { resolveContextWindow } from './lib/context-window.mjs';
|
||||
import { resolveActiveModel } from './lib/active-model.mjs';
|
||||
|
|
@ -134,7 +133,6 @@ export async function runAllScanners(targetPath, opts = {}) {
|
|||
const results = [];
|
||||
|
||||
for (const scanner of SCANNERS) {
|
||||
resetCounter();
|
||||
const scanStart = Date.now();
|
||||
try {
|
||||
const result = await scanner.fn(resolvedPath, discovery, { contextWindow });
|
||||
|
|
@ -194,8 +192,14 @@ export async function runAllScanners(targetPath, opts = {}) {
|
|||
const shouldSuppress = opts.suppress !== false;
|
||||
let suppressedFindings = [];
|
||||
|
||||
let deadSuppressions = [];
|
||||
|
||||
if (shouldSuppress) {
|
||||
const { suppressions } = await loadSuppressions(resolvedPath);
|
||||
// A pin that names no declared check can never match. Report it: a silently
|
||||
// dead suppression leaves the user believing a finding is hidden when it is
|
||||
// not (M-BUG-28).
|
||||
deadSuppressions = unknownSuppressions(suppressions);
|
||||
if (suppressions.length > 0) {
|
||||
for (const result of results) {
|
||||
const { active, suppressed } = applySuppressions(result.findings, suppressions);
|
||||
|
|
@ -221,6 +225,12 @@ export async function runAllScanners(targetPath, opts = {}) {
|
|||
if (suppressedFindings.length > 0) {
|
||||
env.suppressed_findings = suppressedFindings;
|
||||
}
|
||||
// ux-rules rule 2: commands run scanners with `2>/dev/null`, so anything they
|
||||
// must ACT on rides in the payload, never in a stderr-only warning. Added only
|
||||
// when a dead pin exists, so a config without one is byte-identical.
|
||||
if (deadSuppressions.length > 0) {
|
||||
env.unknown_suppressions = deadSuppressions;
|
||||
}
|
||||
return env;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -121,6 +121,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed === null) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-json',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Invalid JSON in settings file',
|
||||
description: `${file.relPath} contains invalid JSON and will be ignored by Claude Code.`,
|
||||
|
|
@ -151,6 +152,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (nearest) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'key-typo',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Possible typo in settings key',
|
||||
description: `${file.relPath}: "${key}" is not a recognized settings.json key, but it closely matches "${nearest}". Claude Code forwards unrecognized keys unchanged (it does not reject them), so if "${key}" is a typo of "${nearest}" the intended setting silently has no effect.`,
|
||||
|
|
@ -167,6 +169,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed[key] !== undefined) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deprecated-key',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Deprecated settings key',
|
||||
description: `${file.relPath}: "${key}" is deprecated. ${migration}`,
|
||||
|
|
@ -183,6 +186,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed[key] !== undefined && typeof parsed[key] !== expectedType) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'type-mismatch',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Type mismatch in settings',
|
||||
description: `${file.relPath}: "${key}" should be ${expectedType}, got ${typeof parsed[key]}.`,
|
||||
|
|
@ -198,6 +202,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (parsed.effortLevel && !VALID_EFFORT_LEVELS.has(parsed.effortLevel)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'invalid-effort-level',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Invalid effortLevel value',
|
||||
description: `${file.relPath}: effortLevel "${parsed.effortLevel}" is not valid.`,
|
||||
|
|
@ -212,6 +217,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!parsed.$schema) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'missing-schema',
|
||||
severity: SEVERITY.info,
|
||||
title: 'Missing $schema reference',
|
||||
description: `${file.relPath} lacks a $schema reference. Adding one enables autocomplete in VS Code/Cursor.`,
|
||||
|
|
@ -228,6 +234,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!perms.deny || (Array.isArray(perms.deny) && perms.deny.length === 0)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-deny-rules',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'No deny rules configured',
|
||||
description: `${file.relPath}: No permission deny rules. Claude can access all files including .env and secrets.`,
|
||||
|
|
@ -240,6 +247,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!perms.allow || (Array.isArray(perms.allow) && perms.allow.length === 0)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'no-allow-rules',
|
||||
severity: SEVERITY.low,
|
||||
title: 'No allow rules configured',
|
||||
description: `${file.relPath}: No permission allow rules. This means frequent permission prompts for common operations.`,
|
||||
|
|
@ -255,6 +263,7 @@ export async function scan(targetPath, discovery) {
|
|||
parsed.additionalDirectories.length > ADDITIONAL_DIRS_THRESHOLD) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'many-additional-dirs',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Many additionalDirectories entries',
|
||||
description:
|
||||
|
|
@ -281,6 +290,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (typeof am !== 'object' || am === null || Array.isArray(am)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'automode-not-object',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'autoMode must be an object',
|
||||
description: `${file.relPath}: "autoMode" must be an object with environment/allow/soft_deny/hard_deny arrays, got ${Array.isArray(am) ? 'array' : typeof am}.`,
|
||||
|
|
@ -295,6 +305,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!AUTO_MODE_SUBKEYS.has(subKey)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'automode-unknown-subkey',
|
||||
severity: SEVERITY.medium,
|
||||
title: `autoMode has an unknown sub-key: ${subKey}`,
|
||||
description: `${file.relPath}: "autoMode.${subKey}" is not a recognized sub-key. Valid keys are environment, allow, soft_deny, hard_deny. It is silently ignored — a typo of a real key (e.g. "hard_denies") means those rules never apply.`,
|
||||
|
|
@ -311,6 +322,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (!isStringArray) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'automode-subkey-not-string-array',
|
||||
severity: SEVERITY.medium,
|
||||
title: `autoMode.${subKey} must be an array of strings`,
|
||||
description: `${file.relPath}: "autoMode.${subKey}" must be an array of prose-rule strings (the literal "$defaults" is allowed), got ${Array.isArray(val) ? 'an array with a non-string entry' : typeof val}.`,
|
||||
|
|
@ -327,6 +339,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (file.scope === 'project') {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'automode-in-shared-settings',
|
||||
severity: SEVERITY.low,
|
||||
title: 'autoMode in shared project settings is ignored by Claude Code',
|
||||
description: `${file.relPath}: Claude Code does not read "autoMode" from shared project settings (.claude/settings.json), so a checked-in repo cannot inject its own rules. This autoMode block has no effect where it is.`,
|
||||
|
|
@ -344,6 +357,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (Array.isArray(parsed.hooks)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'hooks-as-array',
|
||||
severity: SEVERITY.critical,
|
||||
title: 'Hooks configured as array instead of object',
|
||||
description: `${file.relPath}: "hooks" must be an object with event keys, not an array. All hooks will be ignored.`,
|
||||
|
|
|
|||
|
|
@ -87,6 +87,7 @@ export async function scan(_targetPath, _discovery, opts = {}) {
|
|||
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'description-over-cap',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Skill description exceeds the listing cap (Claude Code truncates it)',
|
||||
description:
|
||||
|
|
@ -116,6 +117,7 @@ export async function scan(_targetPath, _discovery, opts = {}) {
|
|||
// Conservative 200k anchor — byte-identical to the pre-B8 finding.
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'aggregate-listing-budget',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Aggregate skill descriptions may exceed the listing budget',
|
||||
description:
|
||||
|
|
@ -138,6 +140,7 @@ export async function scan(_targetPath, _discovery, opts = {}) {
|
|||
const winLabel = withCommas(window);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'aggregate-listing-budget',
|
||||
severity: advisory ? SEVERITY.info : SEVERITY.low,
|
||||
title: 'Aggregate skill descriptions may exceed the listing budget',
|
||||
description:
|
||||
|
|
@ -173,6 +176,7 @@ export async function scan(_targetPath, _discovery, opts = {}) {
|
|||
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'oversized-body',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Skill body is large (loads on demand when the skill runs)',
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -17,7 +17,6 @@ import { fileURLToPath } from 'node:url';
|
|||
import { readFile, stat } from 'node:fs/promises';
|
||||
import { writeOutputFile } from './lib/write-output.mjs';
|
||||
import { discoverConfigFiles } from './lib/file-discovery.mjs';
|
||||
import { resetCounter } from './lib/output.mjs';
|
||||
import { scan } from './token-hotspots.mjs';
|
||||
import * as tokenizerApi from './lib/tokenizer-api.mjs';
|
||||
import { humanizeFindings } from './lib/humanizer.mjs';
|
||||
|
|
@ -99,7 +98,6 @@ async function main() {
|
|||
return;
|
||||
}
|
||||
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(absPath, { includeGlobal, excludeCache });
|
||||
const result = await scan(absPath, discovery);
|
||||
|
||||
|
|
|
|||
|
|
@ -400,6 +400,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (detectVolatileTop(content)) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'volatile-top',
|
||||
severity: SEVERITY.high,
|
||||
title: 'Cache-breaking volatile content at top of CLAUDE.md',
|
||||
description:
|
||||
|
|
@ -428,6 +429,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (issues.length === 0) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'redundant-permissions',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Redundant permission declarations',
|
||||
description:
|
||||
|
|
@ -452,6 +454,7 @@ export async function scan(targetPath, discovery) {
|
|||
if (depth > MAX_IMPORT_DEPTH) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'deep-import-chain',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Deep @import chain defeats prompt-cache reuse',
|
||||
description:
|
||||
|
|
@ -484,6 +487,7 @@ export async function scan(targetPath, discovery) {
|
|||
const skillName = (fm && fm.name) || f.absPath.split('/').slice(-2, -1)[0] || f.absPath;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'bloated-skill-description',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Bloated skill description (loads on every turn)',
|
||||
description:
|
||||
|
|
@ -535,6 +539,7 @@ export async function scan(targetPath, discovery) {
|
|||
'and user-scopes so per-project budget stays tight.';
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'mcp-schema-budget',
|
||||
severity,
|
||||
title: `High MCP tool-schema budget on server "${m.name}"`,
|
||||
description,
|
||||
|
|
@ -552,6 +557,7 @@ export async function scan(targetPath, discovery) {
|
|||
const fileCount = activeConfig.claudeMd.files?.length ?? 0;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'cascade-over-budget',
|
||||
severity: SEVERITY.medium,
|
||||
title: 'CLAUDE.md cascade exceeds 10k tokens per turn',
|
||||
description:
|
||||
|
|
@ -584,6 +590,7 @@ export async function scan(targetPath, discovery) {
|
|||
const keys = stale.map(v => v.key);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'stale-plugin-cache',
|
||||
severity: SEVERITY.low,
|
||||
title: 'Stale plugin-cache versions (disk cleanup, zero live-context impact)',
|
||||
description:
|
||||
|
|
@ -654,6 +661,7 @@ export async function scan(targetPath, discovery) {
|
|||
'(gh / aws / gcloud) over MCP for common operations.';
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'mcp-schema-deferral',
|
||||
severity,
|
||||
title: 'MCP tool schemas forced into the always-loaded prefix',
|
||||
file: null,
|
||||
|
|
|
|||
47
tests/helpers/mask-finding-ids.mjs
Normal file
47
tests/helpers/mask-finding-ids.mjs
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
/**
|
||||
* Drop finding IDs from the frozen-baseline comparison (M-BUG-28).
|
||||
*
|
||||
* v5.0.0's baselines were captured when `{NNN}` came from an emission counter,
|
||||
* so a frozen ID records where a finding happened to land in that run — not
|
||||
* which check produced it. The ID now names the check, so those recorded
|
||||
* numbers are not a contract the live output can or should reproduce.
|
||||
*
|
||||
* The alternative — re-deriving positional IDs on the frozen side, the way
|
||||
* `strip-retired-gap.mjs` originally did — would assert the retired scheme
|
||||
* against itself, and #58's `isGapEntry` off-by-one is the measured example of
|
||||
* that misfiring. Both sides run through this normalizer, so masking (not
|
||||
* deleting) keeps the key present: a finding that loses its `id` entirely still
|
||||
* fails the comparison.
|
||||
*
|
||||
* What replaces the coverage: `tests/lib/finding-codes.test.mjs` pins every
|
||||
* published check→number pair exhaustively, and
|
||||
* `tests/scanners/finding-code-coverage.test.mjs` asserts no finding escapes the
|
||||
* registry. Arithmetic and identity are each asserted where they belong.
|
||||
*/
|
||||
|
||||
const CA_ID = /^CA-[A-Z]{2,4}-\d{3}$/;
|
||||
const MASK = '<CA-ID>';
|
||||
|
||||
/**
|
||||
* Replace every finding ID with a constant, in place.
|
||||
* @template T
|
||||
* @param {T} payload
|
||||
* @returns {T}
|
||||
*/
|
||||
export function maskFindingIds(payload) {
|
||||
walk(payload);
|
||||
return payload;
|
||||
}
|
||||
|
||||
function walk(node) {
|
||||
if (Array.isArray(node)) {
|
||||
for (const item of node) walk(item);
|
||||
return;
|
||||
}
|
||||
if (!node || typeof node !== 'object') return;
|
||||
|
||||
for (const key of ['id', 'findingId']) {
|
||||
if (typeof node[key] === 'string' && CA_ID.test(node[key])) node[key] = MASK;
|
||||
}
|
||||
for (const v of Object.values(node)) walk(v);
|
||||
}
|
||||
16
tests/helpers/scanner-run.mjs
Normal file
16
tests/helpers/scanner-run.mjs
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
/**
|
||||
* Per-scanner run boundary for tests that call a scanner's `scan()` directly.
|
||||
*
|
||||
* `scan-orchestrator` establishes this boundary before every scanner; a test that
|
||||
* calls `scan()` twice in one process otherwise carries state across the two runs
|
||||
* and confounds what it measures (M-BUG-28, session #59).
|
||||
*
|
||||
* Finding IDs are now derived from the check registry rather than from emission
|
||||
* order, so there is no per-run state left and this IS a deliberate no-op. It is
|
||||
* kept as the seam: the tests that call it assert the same thing either way, and
|
||||
* that is the point — the assertion never depended on the counter.
|
||||
*/
|
||||
|
||||
export function startScannerRun() {
|
||||
// Intentionally empty: `finding()` holds no per-run state.
|
||||
}
|
||||
|
|
@ -24,11 +24,12 @@
|
|||
* A single traversal handles all three rather than three special cases.
|
||||
*
|
||||
* What is adjusted, and what is deliberately NOT:
|
||||
* - the retired finding is dropped, and GAP IDs are re-derived densely in
|
||||
* traversal order. GAP IDs are already position-derived (`output.mjs`
|
||||
* assigns them from an emission counter), so this re-derivation applies the
|
||||
* scanner's own rule, not a new one. Retiring a dimension from the middle
|
||||
* of the list shifts every later ID by one.
|
||||
* - the retired finding is dropped. IDs are NOT re-derived: this helper used
|
||||
* to renumber the survivors positionally, which was the scanner's own rule
|
||||
* at the time. Since M-BUG-28 an ID names the check, so renumbering here
|
||||
* would assert a retired scheme against itself — the frozen comparison masks
|
||||
* IDs instead (`mask-finding-ids.mjs`), and the check→number pairs are
|
||||
* pinned exhaustively in tests/lib/finding-codes.test.mjs.
|
||||
* - counter bookkeeping is decremented mechanically, exactly as
|
||||
* strip-added-scanner.mjs decrements `scanners_ok`.
|
||||
* - the DERIVED utilization figures (`utilization.score`/`overhang` and the
|
||||
|
|
@ -145,12 +146,6 @@ export function stripRetiredGap(payload) {
|
|||
walk(payload, state);
|
||||
|
||||
if (state.removed > 0) {
|
||||
// Re-derive GAP IDs densely — the same position rule output.mjs applies.
|
||||
state.survivors.forEach((f, i) => {
|
||||
const id = `CA-GAP-${String(i + 1).padStart(3, '0')}`;
|
||||
if ('id' in f) f.id = id;
|
||||
if ('findingId' in f) f.findingId = id;
|
||||
});
|
||||
adjustCounters(payload, state);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@ import { hermeticEnv, HERMETIC_HOME } from './helpers/hermetic-home.mjs';
|
|||
import { stripHotspotLoadPattern } from './helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from './helpers/strip-added-scanner.mjs';
|
||||
import { stripRetiredGap } from './helpers/strip-retired-gap.mjs';
|
||||
import { maskFindingIds } from './helpers/mask-finding-ids.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -105,7 +106,7 @@ function normalizeScanOrchestrator(env) {
|
|||
}
|
||||
}
|
||||
stripAncestorDerived(out);
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
return maskFindingIds(stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out))));
|
||||
}
|
||||
|
||||
function normalizePosture(p) {
|
||||
|
|
@ -122,29 +123,29 @@ function normalizePosture(p) {
|
|||
}
|
||||
stripAncestorDerived(out.scannerEnvelope);
|
||||
}
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
return maskFindingIds(stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out))));
|
||||
}
|
||||
|
||||
function normalizeTokenHotspots(p) {
|
||||
const out = JSON.parse(JSON.stringify(p));
|
||||
out.duration_ms = 0;
|
||||
return stripHotspotLoadPattern(out);
|
||||
return maskFindingIds(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
function normalizeDrift(p) {
|
||||
// Drift result has no time fields — round-trip through JSON for safety.
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(p)));
|
||||
return maskFindingIds(stripRetiredGap(JSON.parse(JSON.stringify(p))));
|
||||
}
|
||||
|
||||
function normalizeFix(p) {
|
||||
// Fix-cli stdout is the planFixes result with no time fields.
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(p)));
|
||||
return maskFindingIds(stripRetiredGap(JSON.parse(JSON.stringify(p))));
|
||||
}
|
||||
|
||||
function normalizePluginHealth(p) {
|
||||
const out = JSON.parse(JSON.stringify(p));
|
||||
out.duration_ms = 0;
|
||||
return out;
|
||||
return maskFindingIds(out);
|
||||
}
|
||||
|
||||
function normalizeManifest(o) {
|
||||
|
|
|
|||
50
tests/lib/documented-ids-resolve.test.mjs
Normal file
50
tests/lib/documented-ids-resolve.test.mjs
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
/**
|
||||
* Every finding ID printed in shipped prose must name a declared check.
|
||||
*
|
||||
* README, CLAUDE.md, the command files and the scanner header comments are a
|
||||
* second copy of the registry, and two copies of one table drift (#57/C2). This
|
||||
* is the direction that matters to a user: a documented `CA-PLH-016` that no
|
||||
* check emits sends them to write a suppression that can never match.
|
||||
*/
|
||||
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFile, readdir } from 'node:fs/promises';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { allFindingIds } from '../../scanners/lib/finding-codes.mjs';
|
||||
|
||||
const ROOT = resolve(fileURLToPath(new URL('.', import.meta.url)), '../..');
|
||||
const ID_RE = /CA-[A-Z]{2,4}-\d{3}/g;
|
||||
|
||||
async function documentedFiles() {
|
||||
const files = ['README.md', 'CLAUDE.md'];
|
||||
for (const dir of ['commands', 'scanners', 'scanners/lib']) {
|
||||
const entries = await readdir(resolve(ROOT, dir));
|
||||
for (const e of entries) {
|
||||
if (e.endsWith('.md') || e.endsWith('.mjs')) files.push(`${dir}/${e}`);
|
||||
}
|
||||
}
|
||||
return files;
|
||||
}
|
||||
|
||||
describe('documented finding IDs', () => {
|
||||
it('all resolve to a declared check', async () => {
|
||||
const ids = allFindingIds();
|
||||
const dangling = [];
|
||||
|
||||
for (const rel of await documentedFiles()) {
|
||||
let src;
|
||||
try {
|
||||
src = await readFile(resolve(ROOT, rel), 'utf-8');
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
for (const [id] of src.matchAll(ID_RE)) {
|
||||
if (!ids.has(id)) dangling.push(`${rel}: ${id}`);
|
||||
}
|
||||
}
|
||||
|
||||
assert.deepEqual([...new Set(dangling)], [], 'documented ID names no declared check');
|
||||
});
|
||||
});
|
||||
113
tests/lib/finding-codes.test.mjs
Normal file
113
tests/lib/finding-codes.test.mjs
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
/**
|
||||
* Registry invariants for the finding-code scheme (M-BUG-28).
|
||||
*
|
||||
* These are blanket assertions over the whole registry, never a relation between
|
||||
* two chosen entries: a per-entry check goes green on a partial conversion, which
|
||||
* is the failure mode #51/#57/#58 kept reproducing.
|
||||
*/
|
||||
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { FINDING_CODES, RETIRED_CODES, codeNumber, findingId, allFindingIds } from '../../scanners/lib/finding-codes.mjs';
|
||||
import { GAP_CHECKS } from '../../scanners/feature-gap-scanner.mjs';
|
||||
|
||||
describe('finding-code registry', () => {
|
||||
it('gives every check a distinct number within its scanner', () => {
|
||||
const collisions = [];
|
||||
for (const [scanner, table] of Object.entries(FINDING_CODES)) {
|
||||
const seen = new Map();
|
||||
for (const [key, n] of Object.entries(table)) {
|
||||
if (seen.has(n)) collisions.push(`${scanner}: ${seen.get(n)} and ${key} both claim ${n}`);
|
||||
seen.set(n, key);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(collisions, []);
|
||||
});
|
||||
|
||||
it('uses positive integers only', () => {
|
||||
const bad = [];
|
||||
for (const [scanner, table] of Object.entries(FINDING_CODES)) {
|
||||
for (const [key, n] of Object.entries(table)) {
|
||||
if (!Number.isInteger(n) || n < 1) bad.push(`${scanner}.${key} = ${n}`);
|
||||
}
|
||||
}
|
||||
assert.deepEqual(bad, []);
|
||||
});
|
||||
|
||||
it('never lets a retired key stay active', () => {
|
||||
const resurrected = [];
|
||||
for (const [scanner, keys] of Object.entries(RETIRED_CODES)) {
|
||||
for (const key of keys) {
|
||||
if (FINDING_CODES[scanner] && key in FINDING_CODES[scanner]) {
|
||||
resurrected.push(`${scanner}.${key}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
assert.deepEqual(resurrected, []);
|
||||
});
|
||||
|
||||
it('never reissues a retired number', () => {
|
||||
// The whole point of the tombstone: D1 retired GAP t3_8 and shifted three
|
||||
// live IDs down by one. Reusing a retired number would repeat that silently.
|
||||
assert.ok(RETIRED_CODES.GAP.includes('t3_8'), 'D1 tombstone missing');
|
||||
});
|
||||
|
||||
it('declares exactly the GAP dimensions the scanner ships, plus its levers', () => {
|
||||
const declared = new Set(Object.keys(FINDING_CODES.GAP));
|
||||
const shipped = GAP_CHECKS.map((g) => g.id);
|
||||
|
||||
const missing = shipped.filter((id) => !declared.has(id));
|
||||
assert.deepEqual(missing, [], 'a GAP dimension has no declared code');
|
||||
|
||||
const levers = ['bundled-skills-lever', 'cli-over-mcp-lever', 'filter-hook-output-lever'];
|
||||
const orphans = [...declared].filter((k) => !shipped.includes(k) && !levers.includes(k));
|
||||
assert.deepEqual(orphans, [], 'a declared GAP code matches no shipped dimension');
|
||||
});
|
||||
|
||||
it('throws on an undeclared code instead of inventing an ID', () => {
|
||||
assert.throws(() => codeNumber('GAP', 'nope'), /undeclared check/);
|
||||
assert.throws(() => codeNumber('NOSUCH', 't1_1'), /unknown scanner/);
|
||||
assert.throws(() => codeNumber('GAP', undefined), /missing "code"/);
|
||||
});
|
||||
|
||||
it('names retirement explicitly when a retired key is used', () => {
|
||||
assert.throws(() => codeNumber('GAP', 't3_8'), /RETIRED/);
|
||||
});
|
||||
|
||||
it('renders the published ID format', () => {
|
||||
assert.equal(findingId('GAP', 't1_1'), 'CA-GAP-001');
|
||||
assert.equal(findingId('PLH', 'skills-array-entry'), 'CA-PLH-016');
|
||||
assert.ok(allFindingIds().has('CA-SKL-003'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('published finding IDs (pinned exhaustively — README is a contract)', () => {
|
||||
// Every number that shipped in README, CLAUDE.md or command copy before the
|
||||
// registry existed. Spot-checking these would let a renumber through.
|
||||
const PUBLISHED = [
|
||||
['SKL', 'description-over-cap', 'CA-SKL-001'],
|
||||
['SKL', 'aggregate-listing-budget', 'CA-SKL-002'],
|
||||
['SKL', 'oversized-body', 'CA-SKL-003'],
|
||||
['OST', 'strips-coding-instructions', 'CA-OST-001'],
|
||||
['OST', 'plugin-forces-style', 'CA-OST-002'],
|
||||
['OST', 'style-not-found', 'CA-OST-003'],
|
||||
['TOK', 'volatile-top', 'CA-TOK-001'],
|
||||
['TOK', 'redundant-permissions', 'CA-TOK-002'],
|
||||
['TOK', 'deep-import-chain', 'CA-TOK-003'],
|
||||
['TOK', 'mcp-schema-budget', 'CA-TOK-005'],
|
||||
['TOK', 'mcp-schema-deferral', 'CA-TOK-006'],
|
||||
['PLH', 'plugin-json-shadows-default', 'CA-PLH-015'],
|
||||
['PLH', 'skills-array-entry', 'CA-PLH-016'],
|
||||
['OPT', 'procedure-should-be-skill', 'CA-OPT-001'],
|
||||
['AGT', 'description-bloat', 'CA-AGT-001'],
|
||||
['AGT', 'aggregate-listing-budget', 'CA-AGT-002'],
|
||||
['CPS', 'volatile-in-prefix', 'CA-CPS-001'],
|
||||
['COL', 'skill-user-vs-plugin', 'CA-COL-001'],
|
||||
];
|
||||
|
||||
for (const [scanner, key, expected] of PUBLISHED) {
|
||||
it(`${expected} still names ${key}`, () => {
|
||||
assert.equal(findingId(scanner, key), expected);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
|
@ -1,36 +1,34 @@
|
|||
import { describe, it, beforeEach } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { finding, scannerResult, envelope, resetCounter } from '../../scanners/lib/output.mjs';
|
||||
|
||||
describe('resetCounter', () => {
|
||||
it('resets finding ID counter', () => {
|
||||
resetCounter();
|
||||
const f1 = finding({ scanner: 'TST', severity: 'info', title: 'a', description: 'b' });
|
||||
assert.strictEqual(f1.id, 'CA-TST-001');
|
||||
resetCounter();
|
||||
const f2 = finding({ scanner: 'TST', severity: 'info', title: 'c', description: 'd' });
|
||||
assert.strictEqual(f2.id, 'CA-TST-001');
|
||||
});
|
||||
});
|
||||
import { finding, scannerResult, envelope } from '../../scanners/lib/output.mjs';
|
||||
|
||||
describe('finding', () => {
|
||||
beforeEach(() => { resetCounter(); });
|
||||
|
||||
it('generates correct ID format', () => {
|
||||
const f = finding({ scanner: 'CML', severity: 'high', title: 't', description: 'd' });
|
||||
it('generates correct ID format', () => {
|
||||
const f = finding({ scanner: 'CML', code: 'no-claude-md', severity: 'high', title: 't', description: 'd' });
|
||||
assert.match(f.id, /^CA-CML-\d{3}$/);
|
||||
});
|
||||
|
||||
it('auto-increments IDs', () => {
|
||||
const f1 = finding({ scanner: 'CML', severity: 'info', title: 'a', description: 'b' });
|
||||
const f2 = finding({ scanner: 'CML', severity: 'info', title: 'c', description: 'd' });
|
||||
assert.strictEqual(f1.id, 'CA-CML-001');
|
||||
assert.strictEqual(f2.id, 'CA-CML-002');
|
||||
it('gives a check the same ID however many findings precede it', () => {
|
||||
// The ID names the check, not the emission (M-BUG-28). Emitting other
|
||||
// findings first must not move it.
|
||||
const first = finding({ scanner: 'CML', code: 'todo-markers', severity: 'info', title: 'a', description: 'b' });
|
||||
finding({ scanner: 'CML', code: 'no-headings', severity: 'info', title: 'c', description: 'd' });
|
||||
finding({ scanner: 'CML', code: 'html-comments', severity: 'info', title: 'e', description: 'f' });
|
||||
const again = finding({ scanner: 'CML', code: 'todo-markers', severity: 'info', title: 'a', description: 'b' });
|
||||
assert.strictEqual(first.id, again.id);
|
||||
});
|
||||
|
||||
it('refuses to invent an ID when the check is not named', () => {
|
||||
assert.throws(
|
||||
() => finding({ scanner: 'CML', severity: 'info', title: 'a', description: 'b' }),
|
||||
/missing "code"/,
|
||||
);
|
||||
});
|
||||
|
||||
it('includes all required fields', () => {
|
||||
const f = finding({
|
||||
scanner: 'SET',
|
||||
code: 'invalid-json',
|
||||
severity: 'critical',
|
||||
title: 'Test',
|
||||
description: 'Desc',
|
||||
|
|
@ -54,7 +52,7 @@ describe('finding', () => {
|
|||
});
|
||||
|
||||
it('defaults nullable fields to null', () => {
|
||||
const f = finding({ scanner: 'TST', severity: 'info', title: 't', description: 'd' });
|
||||
const f = finding({ scanner: 'SET', code: 'missing-schema', severity: 'info', title: 't', description: 'd' });
|
||||
assert.strictEqual(f.file, null);
|
||||
assert.strictEqual(f.line, null);
|
||||
assert.strictEqual(f.evidence, null);
|
||||
|
|
@ -65,16 +63,14 @@ describe('finding', () => {
|
|||
});
|
||||
|
||||
describe('scannerResult', () => {
|
||||
beforeEach(() => { resetCounter(); });
|
||||
|
||||
it('counts severity correctly', () => {
|
||||
it('counts severity correctly', () => {
|
||||
const findings = [
|
||||
finding({ scanner: 'TST', severity: 'critical', title: 'a', description: 'b' }),
|
||||
finding({ scanner: 'TST', severity: 'high', title: 'c', description: 'd' }),
|
||||
finding({ scanner: 'TST', severity: 'high', title: 'e', description: 'f' }),
|
||||
finding({ scanner: 'TST', severity: 'info', title: 'g', description: 'h' }),
|
||||
finding({ scanner: 'SET', code: 'invalid-json', severity: 'critical', title: 'a', description: 'b' }),
|
||||
finding({ scanner: 'SET', code: 'key-typo', severity: 'high', title: 'c', description: 'd' }),
|
||||
finding({ scanner: 'SET', code: 'deprecated-key', severity: 'high', title: 'e', description: 'f' }),
|
||||
finding({ scanner: 'SET', code: 'missing-schema', severity: 'info', title: 'g', description: 'h' }),
|
||||
];
|
||||
const r = scannerResult('TST', 'ok', findings, 5, 100);
|
||||
const r = scannerResult('SET', 'ok', findings, 5, 100);
|
||||
assert.strictEqual(r.counts.critical, 1);
|
||||
assert.strictEqual(r.counts.high, 2);
|
||||
assert.strictEqual(r.counts.medium, 0);
|
||||
|
|
@ -83,12 +79,12 @@ describe('scannerResult', () => {
|
|||
});
|
||||
|
||||
it('includes error message when provided', () => {
|
||||
const r = scannerResult('TST', 'error', [], 0, 50, 'boom');
|
||||
const r = scannerResult('SET', 'error', [], 0, 50, 'boom');
|
||||
assert.strictEqual(r.error, 'boom');
|
||||
});
|
||||
|
||||
it('omits error when not provided', () => {
|
||||
const r = scannerResult('TST', 'ok', [], 3, 100);
|
||||
const r = scannerResult('SET', 'ok', [], 3, 100);
|
||||
assert.strictEqual(r.error, undefined);
|
||||
});
|
||||
|
||||
|
|
@ -103,16 +99,13 @@ describe('scannerResult', () => {
|
|||
});
|
||||
|
||||
describe('envelope', () => {
|
||||
beforeEach(() => { resetCounter(); });
|
||||
|
||||
it('aggregates across scanners', () => {
|
||||
const r1 = scannerResult('A', 'ok', [
|
||||
finding({ scanner: 'A', severity: 'high', title: 'x', description: 'y' }),
|
||||
it('aggregates across scanners', () => {
|
||||
const r1 = scannerResult('CML', 'ok', [
|
||||
finding({ scanner: 'CML', code: 'no-claude-md', severity: 'high', title: 'x', description: 'y' }),
|
||||
], 1, 10);
|
||||
resetCounter();
|
||||
const r2 = scannerResult('B', 'ok', [
|
||||
finding({ scanner: 'B', severity: 'critical', title: 'a', description: 'b' }),
|
||||
finding({ scanner: 'B', severity: 'low', title: 'c', description: 'd' }),
|
||||
const r2 = scannerResult('SET', 'ok', [
|
||||
finding({ scanner: 'SET', code: 'invalid-json', severity: 'critical', title: 'a', description: 'b' }),
|
||||
finding({ scanner: 'SET', code: 'no-deny-rules', severity: 'low', title: 'c', description: 'd' }),
|
||||
], 2, 20);
|
||||
|
||||
const env = envelope('/target', [r1, r2], 50);
|
||||
|
|
|
|||
48
tests/lib/suppression-validation.test.mjs
Normal file
48
tests/lib/suppression-validation.test.mjs
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
/**
|
||||
* A suppression that names no known check must be reported, not silently
|
||||
* ignored (M-BUG-28, prediction 8).
|
||||
*
|
||||
* This is what makes the ID-semantics change safe to ship: pins written against
|
||||
* the old positional numbering either still name a real check, or they now name
|
||||
* nothing — and "nothing" has to be visible. A silently-dead suppression is the
|
||||
* same failure the old scheme had, just in the other direction.
|
||||
*/
|
||||
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { parseIgnoreFile, unknownSuppressions } from '../../scanners/lib/suppression.mjs';
|
||||
|
||||
describe('unknownSuppressions', () => {
|
||||
it('accepts an exact ID that names a declared check', () => {
|
||||
const s = parseIgnoreFile('CA-SKL-003\n');
|
||||
assert.deepEqual(unknownSuppressions(s), []);
|
||||
});
|
||||
|
||||
it('reports an exact ID that names no declared check', () => {
|
||||
// CA-GAP-099 has never existed; CA-PLH-021 is past the end of PLH's range.
|
||||
const s = parseIgnoreFile('CA-GAP-099\nCA-PLH-021\n');
|
||||
assert.deepEqual(unknownSuppressions(s), ['CA-GAP-099', 'CA-PLH-021']);
|
||||
});
|
||||
|
||||
it('reports an ID whose number was retired rather than pretending it matches', () => {
|
||||
// GAP's retired autoMode dimension sat at 25 under the registry's numbering
|
||||
// had it survived; nothing occupies it now.
|
||||
const s = parseIgnoreFile('CA-GAP-028\n');
|
||||
assert.deepEqual(unknownSuppressions(s), ['CA-GAP-028']);
|
||||
});
|
||||
|
||||
it('accepts a scanner-wide glob for a real scanner', () => {
|
||||
const s = parseIgnoreFile('CA-GAP-*\nCA-PLH-*\n');
|
||||
assert.deepEqual(unknownSuppressions(s), []);
|
||||
});
|
||||
|
||||
it('reports a glob for a scanner that does not exist', () => {
|
||||
const s = parseIgnoreFile('CA-XYZ-*\n');
|
||||
assert.deepEqual(unknownSuppressions(s), ['CA-XYZ-*']);
|
||||
});
|
||||
|
||||
it('stays quiet on an empty ignore file', () => {
|
||||
assert.deepEqual(unknownSuppressions(parseIgnoreFile('# just a comment\n')), []);
|
||||
assert.deepEqual(unknownSuppressions([]), []);
|
||||
});
|
||||
});
|
||||
|
|
@ -29,6 +29,7 @@ import { hermeticEnv, HERMETIC_HOME } from './helpers/hermetic-home.mjs';
|
|||
import { stripHotspotLoadPattern } from './helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from './helpers/strip-added-scanner.mjs';
|
||||
import { stripRetiredGap, maskGapTallyStderr } from './helpers/strip-retired-gap.mjs';
|
||||
import { maskFindingIds } from './helpers/mask-finding-ids.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -100,7 +101,7 @@ function normalizeScanOrchestrator(env) {
|
|||
}
|
||||
}
|
||||
stripAncestorDerived(out);
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
return maskFindingIds(stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out))));
|
||||
}
|
||||
|
||||
function normalizePosture(p) {
|
||||
|
|
@ -117,27 +118,27 @@ function normalizePosture(p) {
|
|||
}
|
||||
stripAncestorDerived(out.scannerEnvelope);
|
||||
}
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
return maskFindingIds(stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out))));
|
||||
}
|
||||
|
||||
function normalizeTokenHotspots(p) {
|
||||
const out = JSON.parse(JSON.stringify(p));
|
||||
out.duration_ms = 0;
|
||||
return stripHotspotLoadPattern(out);
|
||||
return maskFindingIds(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
function normalizeDrift(p) {
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(p)));
|
||||
return maskFindingIds(stripRetiredGap(JSON.parse(JSON.stringify(p))));
|
||||
}
|
||||
|
||||
function normalizeFix(p) {
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(p)));
|
||||
return maskFindingIds(stripRetiredGap(JSON.parse(JSON.stringify(p))));
|
||||
}
|
||||
|
||||
function normalizePluginHealth(p) {
|
||||
const out = JSON.parse(JSON.stringify(p));
|
||||
out.duration_ms = 0;
|
||||
return out;
|
||||
return maskFindingIds(out);
|
||||
}
|
||||
|
||||
function normalizeManifest(o) {
|
||||
|
|
|
|||
|
|
@ -3,7 +3,6 @@ import assert from 'node:assert/strict';
|
|||
import { join } from 'node:path';
|
||||
import { mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/agent-listing-scanner.mjs';
|
||||
import { PER_AGENT_DESC_SOFT_CAP } from '../../scanners/lib/agent-listing-budget.mjs';
|
||||
|
||||
|
|
@ -33,7 +32,6 @@ function uniqueDir(suffix) {
|
|||
* override HOME, run, restore — never rely on the developer's real ~/.claude.
|
||||
*/
|
||||
async function runScannerWithHome(home) {
|
||||
resetCounter();
|
||||
const original = process.env.HOME;
|
||||
process.env.HOME = home;
|
||||
try {
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@ import { describe, it } from 'node:test';
|
|||
import assert from 'node:assert/strict';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/cache-prefix-scanner.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
|
||||
|
|
@ -10,7 +9,6 @@ const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
|||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
|
||||
async function runScanner(fixtureName) {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, fixtureName);
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
return scan(path, discovery);
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ import { resolve, join } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { scan } from '../../scanners/claude-md-linter.mjs';
|
||||
|
||||
|
|
@ -14,7 +13,6 @@ const FIXTURES = resolve(__dirname, '../fixtures');
|
|||
describe('CML scanner — healthy project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'healthy-project'));
|
||||
result = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
});
|
||||
|
|
@ -52,7 +50,6 @@ describe('CML scanner — healthy project', () => {
|
|||
describe('CML scanner — broken project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'broken-project'));
|
||||
result = await scan(resolve(FIXTURES, 'broken-project'), discovery);
|
||||
});
|
||||
|
|
@ -81,7 +78,6 @@ describe('CML scanner — broken project', () => {
|
|||
describe('CML scanner — broken project: 200-tier stays MEDIUM (regression lock)', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'broken-project'));
|
||||
result = await scan(resolve(FIXTURES, 'broken-project'), discovery);
|
||||
});
|
||||
|
|
@ -101,7 +97,6 @@ describe('CML scanner — large cascade (>500 lines): reframed, not absolute-adh
|
|||
// HIGH@500 + "significantly reduce adherence" claim is now-wrong.
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'large-cascade'));
|
||||
result = await scan(resolve(FIXTURES, 'large-cascade'), discovery);
|
||||
});
|
||||
|
|
@ -137,7 +132,6 @@ describe('CML scanner — char budget mirrors CC startup warning (CC 2.1.169)',
|
|||
// 200-line count, so it isolates the char-budget check from the line check.
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'large-claude-chars'));
|
||||
result = await scan(resolve(FIXTURES, 'large-claude-chars'), discovery);
|
||||
});
|
||||
|
|
@ -178,7 +172,6 @@ describe('CML scanner — context-window calibration (B8)', () => {
|
|||
r.findings.find((f) => /performance-warning threshold/i.test(f.title || ''));
|
||||
|
||||
async function scanWithCtx(contextWindow) {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(FIXTURE);
|
||||
return scan(FIXTURE, discovery, { contextWindow });
|
||||
}
|
||||
|
|
@ -197,7 +190,6 @@ describe('CML scanner — context-window calibration (B8)', () => {
|
|||
});
|
||||
|
||||
it('no opts (default) is unchanged: fires medium at the 40k anchor', async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(FIXTURE);
|
||||
const result = await scan(FIXTURE, discovery);
|
||||
assert.equal(charFinding(result)?.severity, 'medium', 'default must stay byte-stable: medium');
|
||||
|
|
@ -210,7 +202,6 @@ describe('CML scanner — large-by-lines but under the char budget (no false cha
|
|||
// proving the check keys on chars, not raw size or line count.
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'large-cascade'));
|
||||
result = await scan(resolve(FIXTURES, 'large-cascade'), discovery);
|
||||
});
|
||||
|
|
@ -224,7 +215,6 @@ describe('CML scanner — large-by-lines but under the char budget (no false cha
|
|||
describe('CML scanner — empty project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'empty-project'));
|
||||
result = await scan(resolve(FIXTURES, 'empty-project'), discovery);
|
||||
});
|
||||
|
|
@ -243,7 +233,6 @@ describe('CML scanner — empty project', () => {
|
|||
describe('CML scanner — minimal project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'minimal-project'));
|
||||
result = await scan(resolve(FIXTURES, 'minimal-project'), discovery);
|
||||
});
|
||||
|
|
@ -262,7 +251,6 @@ describe('CML — nested CLAUDE.md not re-injected after compaction (A)', () =>
|
|||
let result;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-cml-nested-'));
|
||||
await mkdir(join(tmpRoot, 'src'), { recursive: true });
|
||||
await writeFile(join(tmpRoot, 'CLAUDE.md'), '# Root\n\nProject overview goes here.\n', 'utf8');
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import { readFile, writeFile, unlink, mkdir, access } from 'node:fs/promises';
|
|||
import { hermeticEnv, HERMETIC_HOME } from '../helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from '../helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripRetiredGap } from '../helpers/strip-retired-gap.mjs';
|
||||
import { maskFindingIds } from '../helpers/mask-finding-ids.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -44,7 +45,7 @@ async function runCli(cliPath, args, env = {}) {
|
|||
function normalizeTokenHotspotsPayload(p) {
|
||||
const out = JSON.parse(JSON.stringify(p));
|
||||
out.duration_ms = 0;
|
||||
return stripRetiredGap(stripHotspotLoadPattern(out));
|
||||
return maskFindingIds(stripRetiredGap(stripHotspotLoadPattern(out)));
|
||||
}
|
||||
|
||||
function normalizeManifestOutput(o) {
|
||||
|
|
@ -77,7 +78,7 @@ function normalizePluginHealthOutput(o) {
|
|||
|
||||
function normalizeDriftOutput(o) {
|
||||
// Drift result has no time fields; just round-trip through JSON.
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(o)));
|
||||
return maskFindingIds(stripRetiredGap(JSON.parse(JSON.stringify(o))));
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
|
|
@ -319,14 +320,14 @@ describe('fix-cli humanizer (Step 7)', () => {
|
|||
const { stdout } = await runCli(CLI, [FIXTURE, '--json']);
|
||||
const actual = JSON.parse(stdout);
|
||||
const expected = JSON.parse(await readFile(SNAPSHOT, 'utf-8'));
|
||||
assert.deepStrictEqual(stripRetiredGap(actual), stripRetiredGap(expected));
|
||||
assert.deepStrictEqual(maskFindingIds(stripRetiredGap(actual)), maskFindingIds(stripRetiredGap(expected)));
|
||||
});
|
||||
|
||||
it('--raw: stdout JSON byte-equal v5.0.0 snapshot', async () => {
|
||||
const { stdout } = await runCli(CLI, [FIXTURE, '--raw']);
|
||||
const actual = JSON.parse(stdout);
|
||||
const expected = JSON.parse(await readFile(SNAPSHOT, 'utf-8'));
|
||||
assert.deepStrictEqual(stripRetiredGap(actual), stripRetiredGap(expected));
|
||||
assert.deepStrictEqual(maskFindingIds(stripRetiredGap(actual)), maskFindingIds(stripRetiredGap(expected)));
|
||||
});
|
||||
|
||||
it('default mode stderr differs from --raw stderr when findings have humanizer translations', async () => {
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ import { resolve, join } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/collision-scanner.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
|
|
@ -20,7 +19,6 @@ function uniqueDir(suffix) {
|
|||
* Tests must override HOME, run, and restore — never rely on user-state.
|
||||
*/
|
||||
async function runScannerWithHome(home) {
|
||||
resetCounter();
|
||||
const original = process.env.HOME;
|
||||
process.env.HOME = home;
|
||||
try {
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ import { resolve, join, sep } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { scan } from '../../scanners/conflict-detector.mjs';
|
||||
|
||||
|
|
@ -14,7 +13,6 @@ const FIXTURES = resolve(__dirname, '../fixtures');
|
|||
describe('CNF scanner — conflict project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'conflict-project'));
|
||||
result = await scan(resolve(FIXTURES, 'conflict-project'), discovery);
|
||||
});
|
||||
|
|
@ -77,7 +75,6 @@ describe('CNF scanner — conflict project', () => {
|
|||
describe('CNF scanner — healthy project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'healthy-project'));
|
||||
result = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
});
|
||||
|
|
@ -94,7 +91,6 @@ describe('CNF scanner — healthy project', () => {
|
|||
describe('CNF scanner — empty project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'empty-project'));
|
||||
result = await scan(resolve(FIXTURES, 'empty-project'), discovery);
|
||||
});
|
||||
|
|
@ -111,7 +107,6 @@ describe('CNF scanner — empty project', () => {
|
|||
describe('CNF scanner — minimal project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'minimal-project'));
|
||||
result = await scan(resolve(FIXTURES, 'minimal-project'), discovery);
|
||||
});
|
||||
|
|
@ -132,7 +127,6 @@ describe('CNF scanner — param-qualified cross-scope conflicts', () => {
|
|||
// Agent: deny model:opus vs allow model:sonnet → disjoint → NO conflict.
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'param-conflict-project'));
|
||||
result = await scan(resolve(FIXTURES, 'param-conflict-project'), discovery);
|
||||
});
|
||||
|
|
@ -186,11 +180,9 @@ describe('CNF scanner — cache exclusion drops duplicate-hook count (B3)', () =
|
|||
},
|
||||
}));
|
||||
|
||||
resetCounter();
|
||||
const dIncl = await discoverConfigFiles(dir);
|
||||
includeCount = (await scan(dir, dIncl)).findings.filter(f => f.title.includes('Duplicate hook')).length;
|
||||
|
||||
resetCounter();
|
||||
const dExcl = await discoverConfigFiles(dir, { excludeCache: true });
|
||||
excludeCount = (await scan(dir, dExcl)).findings.filter(f => f.title.includes('Duplicate hook')).length;
|
||||
});
|
||||
|
|
@ -255,7 +247,6 @@ describe('CNF scanner — excludes plugin-bundled configs (M-BUG-2)', () => {
|
|||
permissions: { allow: ['Bash(rm:*)'] },
|
||||
}));
|
||||
|
||||
resetCounter();
|
||||
discovery = await discoverConfigFiles(dir);
|
||||
result = await scan(dir, discovery);
|
||||
});
|
||||
|
|
@ -289,7 +280,6 @@ describe('CNF scanner — genuine live conflict still flagged (M-BUG-2 guard)',
|
|||
await mkdir(claude, { recursive: true });
|
||||
await writeFile(join(claude, 'settings.json'), JSON.stringify({ model: 'opus' }));
|
||||
await writeFile(join(claude, 'settings.local.json'), JSON.stringify({ model: 'haiku' }));
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(dir);
|
||||
result = await scan(dir, discovery);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@ import { describe, it } from 'node:test';
|
|||
import assert from 'node:assert/strict';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/disabled-in-schema-scanner.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
|
||||
|
|
@ -10,7 +9,6 @@ const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
|||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
|
||||
async function runScanner(fixtureName) {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, fixtureName);
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
return scan(path, discovery);
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ import { resolve, join, dirname } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan, opportunitySummary, bundledSkillsLeverFinding, cliOverMcpLeverFinding, filterHookLeverFinding, GAP_CHECKS } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { TITLE_TO_ID as GAP_TITLE_TO_ID, TIER_COUNTS, TOTAL_DIMENSIONS } from '../../scanners/lib/scoring.mjs';
|
||||
import { TRANSLATIONS } from '../../scanners/lib/humanizer-data.mjs';
|
||||
|
|
@ -23,7 +22,6 @@ async function fixtureDiscovery(name) {
|
|||
describe('GAP scanner — healthy project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('healthy-project');
|
||||
// Hermetic HOME: scan() now enumerates active skills via process.env.HOME
|
||||
// (the disableBundledSkills lever check). An empty HOME keeps these fixture
|
||||
|
|
@ -97,7 +95,6 @@ describe('GAP scanner — healthy project', () => {
|
|||
describe('GAP scanner — minimal project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('minimal-project');
|
||||
result = await withHermeticHome(() => scan(resolve(FIXTURES, 'minimal-project'), discovery));
|
||||
});
|
||||
|
|
@ -131,7 +128,6 @@ describe('GAP scanner — minimal project', () => {
|
|||
});
|
||||
|
||||
it('has more findings than healthy project', async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('healthy-project');
|
||||
const healthyResult = await withHermeticHome(() => scan(resolve(FIXTURES, 'healthy-project'), discovery));
|
||||
assert.ok(result.findings.length > healthyResult.findings.length);
|
||||
|
|
@ -141,7 +137,6 @@ describe('GAP scanner — minimal project', () => {
|
|||
describe('GAP scanner — empty project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('empty-project');
|
||||
result = await withHermeticHome(() => scan(resolve(FIXTURES, 'empty-project'), discovery));
|
||||
});
|
||||
|
|
@ -230,7 +225,6 @@ describe('bundledSkillsLeverFinding — pure decision', () => {
|
|||
});
|
||||
|
||||
it('returns a finding when the lever is un-pulled AND the listing is over budget', () => {
|
||||
resetCounter();
|
||||
const f = bundledSkillsLeverFinding({ leverPulled: false, aggregate: overBudget });
|
||||
assert.ok(f, 'expected a finding');
|
||||
assert.match(f.id, /^CA-GAP-\d{3}$/);
|
||||
|
|
@ -267,7 +261,6 @@ describe('GAP scanner — disableBundledSkills lever wiring (HOME-scoped)', () =
|
|||
const original = process.env.HOME;
|
||||
process.env.HOME = home;
|
||||
try {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(project, { includeGlobal: true });
|
||||
const result = await scan(project, discovery);
|
||||
return result;
|
||||
|
|
@ -351,7 +344,6 @@ describe('cliOverMcpLeverFinding (CLI-over-MCP lever, v5.10 B4)', () => {
|
|||
});
|
||||
|
||||
it('fires a low-severity opportunity when MCP schemas are forced upfront', () => {
|
||||
resetCounter();
|
||||
const f = cliOverMcpLeverFinding({
|
||||
assessment: {
|
||||
forcedUpfront: true,
|
||||
|
|
@ -375,7 +367,6 @@ describe('filterHookLeverFinding (filter-before-Claude-reads lever, v5.10 B5)',
|
|||
});
|
||||
|
||||
it('fires an info opportunity when ≥1 chatty hook is detected', () => {
|
||||
resetCounter();
|
||||
const f = filterHookLeverFinding({
|
||||
flaggedHooks: [{ event: 'SessionStart', scriptPath: '/x/hooks/scripts/chatty.sh' }],
|
||||
});
|
||||
|
|
@ -389,7 +380,6 @@ describe('filterHookLeverFinding (filter-before-Claude-reads lever, v5.10 B5)',
|
|||
|
||||
describe('GAP scanner — filter-before lever wiring (chatty hook fixture)', () => {
|
||||
it('emits the filter-before lever when scanning a repo with a chatty hook', async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('hooks-additional-context');
|
||||
const result = await withHermeticHome(() =>
|
||||
scan(resolve(FIXTURES, 'hooks-additional-context'), discovery),
|
||||
|
|
@ -402,7 +392,6 @@ describe('GAP scanner — filter-before lever wiring (chatty hook fixture)', ()
|
|||
});
|
||||
|
||||
it('does NOT emit the lever for a repo with only quiet hooks', async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('hooks-quiet');
|
||||
const result = await withHermeticHome(() =>
|
||||
scan(resolve(FIXTURES, 'hooks-quiet'), discovery),
|
||||
|
|
@ -432,7 +421,6 @@ describe('GAP scanner — test/demo data must not mask real gaps (M-BUG-13)', ()
|
|||
const original = process.env.HOME;
|
||||
process.env.HOME = home;
|
||||
try {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(project, { includeGlobal: true });
|
||||
const result = await scan(project, discovery);
|
||||
return result;
|
||||
|
|
@ -529,7 +517,6 @@ describe('GAP scanner — retired dimensions (D1)', () => {
|
|||
const RETIRED_TITLES = ['No autoMode classifier'];
|
||||
|
||||
it('emits no retired dimension on a fixture that lacks the feature', async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('healthy-project');
|
||||
const result = await withHermeticHome(
|
||||
() => scan(resolve(FIXTURES, 'healthy-project'), discovery),
|
||||
|
|
|
|||
82
tests/scanners/finding-code-coverage.test.mjs
Normal file
82
tests/scanners/finding-code-coverage.test.mjs
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
/**
|
||||
* Blanket invariants over the whole scanner set (M-BUG-28).
|
||||
*
|
||||
* Deliberately NOT per-scanner: a per-scanner assertion goes green on a partial
|
||||
* conversion, which is the class #51/#57/#58 kept reproducing. Both directions
|
||||
* are asserted — no finding without a declared code, and no declared code
|
||||
* without a call site (an orphan declaration is as wrong as a missing one).
|
||||
*/
|
||||
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFile, readdir } from 'node:fs/promises';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { runAllScanners } from '../../scanners/scan-orchestrator.mjs';
|
||||
import { FINDING_CODES, allFindingIds } from '../../scanners/lib/finding-codes.mjs';
|
||||
import { GAP_CHECKS } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
const SCANNERS_DIR = resolve(__dirname, '../../scanners');
|
||||
|
||||
// Fixtures chosen to spread across scanners: broken/conflicting config fires the
|
||||
// validators, healthy config fires the gap dimensions.
|
||||
const SWEEP = [
|
||||
'broken-project',
|
||||
'conflict-project',
|
||||
'healthy-project',
|
||||
'minimal-project',
|
||||
'fixable-project',
|
||||
'large-cascade',
|
||||
];
|
||||
|
||||
describe('every emitted finding carries a declared check code', () => {
|
||||
it('holds across all orchestrated scanners and a spread of fixtures', async () => {
|
||||
const declaredIds = allFindingIds();
|
||||
const offenders = [];
|
||||
const scannerErrors = [];
|
||||
|
||||
for (const fixture of SWEEP) {
|
||||
const env = await withHermeticHome(async () =>
|
||||
runAllScanners(resolve(FIXTURES, fixture), { filterFixtures: false, suppress: false })
|
||||
);
|
||||
|
||||
for (const r of env.scanners || []) {
|
||||
if (r.status === 'error') scannerErrors.push(`${fixture}/${r.scanner}: ${r.error}`);
|
||||
for (const f of r.findings || []) {
|
||||
if (!declaredIds.has(f.id)) offenders.push(`${fixture}/${r.scanner}: ${f.id} — ${f.title}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A scanner that throws returns status:'error' with zero findings, so an
|
||||
// unconverted call site would otherwise hide as "no findings".
|
||||
assert.deepEqual(scannerErrors, [], 'a scanner failed during the sweep');
|
||||
assert.deepEqual(offenders, [], 'a finding carries an ID outside the registry');
|
||||
});
|
||||
});
|
||||
|
||||
describe('every declared code is claimed by a call site', () => {
|
||||
it('finds no orphan declarations', async () => {
|
||||
const files = (await readdir(SCANNERS_DIR)).filter((f) => f.endsWith('.mjs'));
|
||||
const sources = await Promise.all(
|
||||
files.map((f) => readFile(resolve(SCANNERS_DIR, f), 'utf-8'))
|
||||
);
|
||||
const blob = sources.join('\n');
|
||||
|
||||
const gapIds = new Set(GAP_CHECKS.map((g) => g.id));
|
||||
const orphans = [];
|
||||
|
||||
for (const [scanner, table] of Object.entries(FINDING_CODES)) {
|
||||
for (const key of Object.keys(table)) {
|
||||
// GAP dimension codes are claimed via `code: gap.id` over GAP_CHECKS.
|
||||
if (scanner === 'GAP' && gapIds.has(key)) continue;
|
||||
if (!blob.includes(`code: '${key}'`)) orphans.push(`${scanner}.${key}`);
|
||||
}
|
||||
}
|
||||
|
||||
assert.deepEqual(orphans, [], 'a declared code is emitted by nothing');
|
||||
});
|
||||
});
|
||||
69
tests/scanners/finding-id-stability.test.mjs
Normal file
69
tests/scanners/finding-id-stability.test.mjs
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
/**
|
||||
* M-BUG-28 — finding IDs identify the CHECK, not the emission position.
|
||||
*
|
||||
* The defect these tests pin (measured 2026-08-09, session #59): `CA-{SCANNER}-{NNN}`
|
||||
* was rendered from a module-global emission counter, so NNN was the finding's
|
||||
* position in that scanner's output for THAT run. Two configurations that both
|
||||
* fail the same check got different IDs for it, and a `.config-audit-ignore`
|
||||
* entry silently retargeted to a neighbouring finding whenever an earlier check
|
||||
* started or stopped firing.
|
||||
*
|
||||
* These assert BEHAVIOUR (run the scanner, read the id), never the mechanism.
|
||||
*/
|
||||
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { scan as gapScan } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
||||
import { startScannerRun } from '../helpers/scanner-run.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
|
||||
async function gapIdsByTitle(fixture) {
|
||||
return withHermeticHome(async () => {
|
||||
// Mirror what scan-orchestrator does before each scanner, so this measures
|
||||
// ONLY config-driven ID drift and not counter carry-over between two scans
|
||||
// in one test process. Becomes a no-op once IDs are check-derived.
|
||||
startScannerRun();
|
||||
const target = resolve(FIXTURES, fixture);
|
||||
const discovery = await discoverConfigFiles(target);
|
||||
const result = await gapScan(target, discovery);
|
||||
const map = new Map();
|
||||
for (const f of result.findings) map.set(f.title, f.id);
|
||||
return map;
|
||||
});
|
||||
}
|
||||
|
||||
describe('finding IDs are stable across configurations (M-BUG-28)', () => {
|
||||
it('gives a check the same ID no matter which other checks fired', async () => {
|
||||
// minimal-project fails more early checks than healthy-project, so under the
|
||||
// emission-counter scheme every shared check downstream of the first
|
||||
// divergence was renumbered.
|
||||
const minimal = await gapIdsByTitle('minimal-project');
|
||||
const healthy = await gapIdsByTitle('healthy-project');
|
||||
|
||||
const shared = [...minimal.keys()].filter((t) => healthy.has(t));
|
||||
assert.ok(shared.length >= 10, `expected overlapping checks, got ${shared.length}`);
|
||||
|
||||
const drifted = shared
|
||||
.filter((t) => minimal.get(t) !== healthy.get(t))
|
||||
.map((t) => `${t}: minimal=${minimal.get(t)} healthy=${healthy.get(t)}`);
|
||||
|
||||
assert.deepEqual(drifted, [], 'a check must carry one ID regardless of config');
|
||||
});
|
||||
|
||||
it('keeps a suppression pinned to the check the user pinned', async () => {
|
||||
// The user-visible consequence: pin the ID you see, fix an unrelated earlier
|
||||
// gap, and the pin must still name the same check.
|
||||
const minimal = await gapIdsByTitle('minimal-project');
|
||||
const healthy = await gapIdsByTitle('healthy-project');
|
||||
|
||||
const pinned = 'No custom subagents';
|
||||
assert.ok(minimal.has(pinned) && healthy.has(pinned), 'fixture drifted — check still fires?');
|
||||
assert.equal(minimal.get(pinned), healthy.get(pinned));
|
||||
});
|
||||
});
|
||||
|
|
@ -5,7 +5,6 @@ import { fileURLToPath } from 'node:url';
|
|||
import { cp, rm, readFile, writeFile, stat } from 'node:fs/promises';
|
||||
import { mkdirSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { runAllScanners } from '../../scanners/scan-orchestrator.mjs';
|
||||
import { planFixes, applyFixes, verifyFixes, FIX_TYPES } from '../../scanners/fix-engine.mjs';
|
||||
|
|
@ -30,7 +29,6 @@ describe('planFixes', () => {
|
|||
let envelope;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
envelope = await runAllScanners(FIXABLE);
|
||||
});
|
||||
|
||||
|
|
@ -159,7 +157,6 @@ describe('applyFixes dry-run', () => {
|
|||
let envelope;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
envelope = await runAllScanners(FIXABLE);
|
||||
});
|
||||
|
||||
|
|
@ -190,7 +187,6 @@ describe('applyFixes on tmp copy', () => {
|
|||
|
||||
beforeEach(async () => {
|
||||
tmpDir = await createTmpCopy();
|
||||
resetCounter();
|
||||
envelope = await runAllScanners(tmpDir);
|
||||
});
|
||||
|
||||
|
|
@ -282,7 +278,6 @@ describe('applyFixes on tmp copy', () => {
|
|||
before.effortLevel = raw;
|
||||
await writeFile(settingsPath, `${JSON.stringify(before, null, 2)}\n`, 'utf-8');
|
||||
|
||||
resetCounter();
|
||||
const env = await runAllScanners(dir);
|
||||
const { fixes } = planFixes(env);
|
||||
const effortFix = fixes.filter(f => f.key === 'effortLevel');
|
||||
|
|
@ -360,7 +355,6 @@ describe('verifyFixes', () => {
|
|||
});
|
||||
|
||||
it('confirms fixed findings are gone', async () => {
|
||||
resetCounter();
|
||||
const envelope = await runAllScanners(tmpDir);
|
||||
const { fixes } = planFixes(envelope);
|
||||
|
||||
|
|
@ -383,7 +377,6 @@ describe('planFixes ordering (M-BUG-29)', () => {
|
|||
// Both defects at once: undocumented `globs:` AND a non-.md extension.
|
||||
await writeFile(join(rulesDir, 'both.txt'), '---\nglobs: "**/*.ts"\n---\n\nBody.\n');
|
||||
|
||||
resetCounter();
|
||||
const envelope = await runAllScanners(dir, { includeGlobal: false });
|
||||
const { fixes } = planFixes(envelope);
|
||||
|
||||
|
|
@ -419,7 +412,6 @@ describe('verifyFixes scope (F5)', () => {
|
|||
const originalHome = process.env.HOME;
|
||||
process.env.HOME = fakeHome;
|
||||
try {
|
||||
resetCounter();
|
||||
const envelope = await runAllScanners(dir, { includeGlobal: true });
|
||||
const globalOnly = envelope.scanners
|
||||
.flatMap((s) => s.findings)
|
||||
|
|
|
|||
77
tests/scanners/fix-verify-instance-identity.test.mjs
Normal file
77
tests/scanners/fix-verify-instance-identity.test.mjs
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
/**
|
||||
* Fix verification must identify a finding INSTANCE, not just its check
|
||||
* (M-BUG-28, prediction 7).
|
||||
*
|
||||
* A finding ID names the check, so one check failing in two files yields two
|
||||
* findings sharing an ID. If verification keys on the ID alone, fixing one of
|
||||
* them marks both as fixed — and the untouched one, still present in the
|
||||
* re-scan, is then reported as a REGRESSION. The user is told a fix came back
|
||||
* undone when it was never applied.
|
||||
*
|
||||
* `(findingId, file)` is the instance key, and both `planFixes` and `applyFixes`
|
||||
* already carry `file` next to `findingId`.
|
||||
*/
|
||||
|
||||
import { describe, it, before, after } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { runAllScanners } from '../../scanners/scan-orchestrator.mjs';
|
||||
import { planFixes, applyFixes, verifyFixes } from '../../scanners/fix-engine.mjs';
|
||||
|
||||
const RULE = (name) => `---
|
||||
globs:
|
||||
- "src/**/*.ts"
|
||||
---
|
||||
|
||||
# ${name}
|
||||
|
||||
A path-scoped rule that uses the undocumented \`globs:\` key.
|
||||
`;
|
||||
|
||||
describe('fix verification distinguishes two instances of one check', () => {
|
||||
let dir;
|
||||
|
||||
before(async () => {
|
||||
dir = await mkdtemp(join(tmpdir(), 'ca-mbug28-'));
|
||||
await mkdir(join(dir, '.claude', 'rules'), { recursive: true });
|
||||
await writeFile(join(dir, 'CLAUDE.md'), '# Project\n\nInstructions.\n');
|
||||
await writeFile(join(dir, '.claude', 'rules', 'alpha.md'), RULE('Alpha'));
|
||||
await writeFile(join(dir, '.claude', 'rules', 'beta.md'), RULE('Beta'));
|
||||
});
|
||||
|
||||
after(async () => {
|
||||
if (dir) await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('does not report the untouched instance as a regression', async () => {
|
||||
const envelope = await runAllScanners(dir, { filterFixtures: false, suppress: false });
|
||||
|
||||
const globsFindings = envelope.scanners
|
||||
.flatMap((s) => s.findings)
|
||||
.filter((f) => f.title === 'Rule uses "globs" instead of documented "paths"');
|
||||
|
||||
assert.equal(globsFindings.length, 2, 'fixture must produce one finding per rule file');
|
||||
assert.equal(
|
||||
globsFindings[0].id,
|
||||
globsFindings[1].id,
|
||||
'both instances name the same check — that is the scheme, not a defect'
|
||||
);
|
||||
|
||||
const { fixes } = planFixes(envelope);
|
||||
const alphaFix = fixes.filter((p) => p.file && p.file.endsWith('alpha.md'));
|
||||
assert.equal(alphaFix.length, 1, 'expected exactly one fix plan for alpha.md');
|
||||
|
||||
const backupDir = join(dir, '.backups');
|
||||
await mkdir(backupDir, { recursive: true });
|
||||
const { applied } = await applyFixes(alphaFix, { backupDir });
|
||||
assert.equal(applied.length, 1);
|
||||
|
||||
const { verified, regressions } = await verifyFixes(envelope, applied);
|
||||
|
||||
// beta.md was never fixed. It is still open — which is not a regression.
|
||||
assert.deepEqual(regressions, [], 'an unfixed sibling instance must not count as a regression');
|
||||
assert.equal(verified.length, 1, 'exactly the fixed instance is verified');
|
||||
});
|
||||
});
|
||||
|
|
@ -4,9 +4,9 @@ import { resolve, join } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { scan } from '../../scanners/hook-validator.mjs';
|
||||
import { findingId } from '../../scanners/lib/finding-codes.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
|
|
@ -14,7 +14,6 @@ const FIXTURES = resolve(__dirname, '../fixtures');
|
|||
describe('HKV scanner — healthy project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'healthy-project'));
|
||||
result = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
});
|
||||
|
|
@ -42,7 +41,6 @@ describe('HKV scanner — healthy project', () => {
|
|||
describe('HKV scanner — broken project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'broken-project'));
|
||||
result = await scan(resolve(FIXTURES, 'broken-project'), discovery);
|
||||
});
|
||||
|
|
@ -54,8 +52,7 @@ describe('HKV scanner — broken project', () => {
|
|||
});
|
||||
|
||||
it('detects object matcher (should be string)', () => {
|
||||
// CA-HKV-002 in broken-project, evidence contains the object matcher snippet.
|
||||
const found = result.findings.some(f => f.scanner === 'HKV' && f.id === 'CA-HKV-002');
|
||||
const found = result.findings.some(f => f.scanner === 'HKV' && f.id === findingId('HKV', 'matcher-not-string'));
|
||||
assert.ok(found, 'Should detect nested object matcher');
|
||||
});
|
||||
|
||||
|
|
@ -80,7 +77,6 @@ describe('HKV scanner — broken project', () => {
|
|||
|
||||
describe('HKV scanner — verbose hook output (v5 M5)', () => {
|
||||
it('flags hook script with > 50 console.log/stdout.write lines (low)', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'hooks-verbose');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -92,7 +88,6 @@ describe('HKV scanner — verbose hook output (v5 M5)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag a quiet hook script', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'hooks-quiet');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -103,7 +98,6 @@ describe('HKV scanner — verbose hook output (v5 M5)', () => {
|
|||
|
||||
describe('HKV scanner — additionalContext injection advisory (v5.10 B5)', () => {
|
||||
it('flags a hook that injects unfiltered command output into additionalContext (info advisory)', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'hooks-additional-context');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -119,7 +113,6 @@ describe('HKV scanner — additionalContext injection advisory (v5.10 B5)', () =
|
|||
});
|
||||
|
||||
it('does NOT flag the filtered (grep|head) sibling hook', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'hooks-additional-context');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -130,7 +123,6 @@ describe('HKV scanner — additionalContext injection advisory (v5.10 B5)', () =
|
|||
});
|
||||
|
||||
it('does NOT flag a quiet hook with no additionalContext', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'hooks-quiet');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -150,7 +142,6 @@ describe('HKV scanner — CC 2.1.152 MessageDisplay event (Batch 1 false-positiv
|
|||
const VALID_NEW = ['MessageDisplay'];
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-hkv-events-'));
|
||||
await mkdir(join(tmpRoot, '.claude'), { recursive: true });
|
||||
const settings = {
|
||||
|
|
@ -197,7 +188,6 @@ describe('HKV scanner — post-session is NOT a settings.json hook event (Verifi
|
|||
let result;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-hkv-postsession-'));
|
||||
await mkdir(join(tmpRoot, '.claude'), { recursive: true });
|
||||
const settings = {
|
||||
|
|
@ -236,7 +226,6 @@ describe('HKV scanner — Setup/UserPromptExpansion/PostToolBatch events (Batch
|
|||
const NEW_EVENTS = ['Setup', 'UserPromptExpansion', 'PostToolBatch'];
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-hkv-events2-'));
|
||||
await mkdir(join(tmpRoot, '.claude'), { recursive: true });
|
||||
const settings = {
|
||||
|
|
@ -278,7 +267,6 @@ describe('HKV scanner — Setup/UserPromptExpansion/PostToolBatch events (Batch
|
|||
describe('HKV scanner — empty project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'empty-project'));
|
||||
result = await scan(resolve(FIXTURES, 'empty-project'), discovery);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@ import { describe, it, beforeEach } from 'node:test';
|
|||
import assert from 'node:assert/strict';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { scan } from '../../scanners/import-resolver.mjs';
|
||||
|
||||
|
|
@ -12,7 +11,6 @@ const FIXTURES = resolve(__dirname, '../fixtures');
|
|||
describe('IMP scanner — healthy project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'healthy-project'));
|
||||
result = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
});
|
||||
|
|
@ -44,7 +42,6 @@ describe('IMP scanner — healthy project', () => {
|
|||
describe('IMP scanner — broken project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'broken-project'));
|
||||
result = await scan(resolve(FIXTURES, 'broken-project'), discovery);
|
||||
});
|
||||
|
|
@ -85,7 +82,6 @@ describe('IMP scanner — broken project', () => {
|
|||
describe('IMP scanner — minimal project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'minimal-project'));
|
||||
result = await scan(resolve(FIXTURES, 'minimal-project'), discovery);
|
||||
});
|
||||
|
|
@ -102,7 +98,6 @@ describe('IMP scanner — minimal project', () => {
|
|||
describe('IMP scanner — empty project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'empty-project'));
|
||||
result = await scan(resolve(FIXTURES, 'empty-project'), discovery);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ import { resolve, join } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { scan } from '../../scanners/mcp-config-validator.mjs';
|
||||
|
||||
|
|
@ -14,7 +13,6 @@ const FIXTURES = resolve(__dirname, '../fixtures');
|
|||
describe('MCP scanner — healthy project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'healthy-project'));
|
||||
result = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
});
|
||||
|
|
@ -54,7 +52,6 @@ describe('MCP scanner — healthy project', () => {
|
|||
describe('MCP scanner — broken project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'broken-project'));
|
||||
result = await scan(resolve(FIXTURES, 'broken-project'), discovery);
|
||||
});
|
||||
|
|
@ -104,7 +101,6 @@ describe('MCP scanner — stray `trust` field is an unknown field (verify-first,
|
|||
let tmpRoot;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-mcp-trust-'));
|
||||
// `trust` is NOT a field in the official .mcp.json schema (verified against
|
||||
// code.claude.com/docs/en/mcp + /settings, 2026-06-18). Approval is
|
||||
|
|
@ -141,7 +137,6 @@ describe('MCP scanner — `alwaysLoad` is a valid field (v5.10 B4, verify-first
|
|||
let tmpRoot;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-mcp-alwaysload-'));
|
||||
// alwaysLoad exempts a server from MCP tool-schema deferral (CC v2.1.121+).
|
||||
// Verified against code.claude.com/docs/en/mcp.md#exempt-a-server-from-deferral.
|
||||
|
|
@ -171,7 +166,6 @@ describe('MCP scanner — env-var false positives (CC 2.1.139/2.1.142, Batch 1)'
|
|||
let envFindings;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-mcp-env-'));
|
||||
const mcp = {
|
||||
mcpServers: {
|
||||
|
|
@ -219,7 +213,6 @@ describe('MCP scanner — env-var false positives (CC 2.1.139/2.1.142, Batch 1)'
|
|||
describe('MCP scanner — empty project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'empty-project'));
|
||||
result = await scan(resolve(FIXTURES, 'empty-project'), discovery);
|
||||
});
|
||||
|
|
@ -236,7 +229,6 @@ describe('MCP scanner — empty project', () => {
|
|||
describe('MCP scanner — minimal project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'minimal-project'));
|
||||
result = await scan(resolve(FIXTURES, 'minimal-project'), discovery);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -15,7 +15,6 @@ import assert from 'node:assert/strict';
|
|||
import { join } from 'node:path';
|
||||
import { mkdtemp, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/optimization-lens-scanner.mjs';
|
||||
|
||||
const tmp = () => mkdtemp(join(tmpdir(), 'ca-opt-'));
|
||||
|
|
@ -29,7 +28,6 @@ async function runOpt(claudeMd, relPath = 'CLAUDE.md') {
|
|||
const repo = await tmp();
|
||||
const abs = join(repo, relPath.replace(/\//g, '-'));
|
||||
await writeFile(abs, claudeMd, 'utf-8');
|
||||
resetCounter();
|
||||
try {
|
||||
return await scan(repo, {
|
||||
files: [{ absPath: abs, relPath, type: 'claude-md', scope: 'project', size: claudeMd.length }],
|
||||
|
|
@ -50,7 +48,6 @@ describe('OPT scanner — result shape', () => {
|
|||
|
||||
it('emits nothing and stays ok when no claude-md files are discovered', async () => {
|
||||
const repo = await tmp();
|
||||
resetCounter();
|
||||
try {
|
||||
const result = await scan(repo, { files: [] });
|
||||
assert.equal(result.status, 'ok');
|
||||
|
|
|
|||
|
|
@ -16,7 +16,6 @@ import assert from 'node:assert/strict';
|
|||
import { join } from 'node:path';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/output-style-scanner.mjs';
|
||||
|
||||
function tmp(prefix) {
|
||||
|
|
@ -66,7 +65,6 @@ async function runOst(opts = {}) {
|
|||
const originalProfile = process.env.USERPROFILE;
|
||||
process.env.HOME = home;
|
||||
process.env.USERPROFILE = home;
|
||||
resetCounter();
|
||||
try {
|
||||
return await scan(repo, { files: [] });
|
||||
} finally {
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@ import { fileURLToPath } from 'node:url';
|
|||
import { mkdtemp, mkdir, writeFile, rm, readFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan, discoverPlugins } from '../../scanners/plugin-health-scanner.mjs';
|
||||
import { findingId } from '../../scanners/lib/finding-codes.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
|
|
@ -39,14 +39,12 @@ describe('discoverPlugins', () => {
|
|||
|
||||
describe('scan on valid test-plugin', () => {
|
||||
it('returns ok status', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(TEST_PLUGIN);
|
||||
assert.equal(result.scanner, 'PLH');
|
||||
assert.equal(result.status, 'ok');
|
||||
});
|
||||
|
||||
it('finds commands and agents', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(TEST_PLUGIN);
|
||||
assert.ok(result.files_scanned >= 1, 'Should scan at least 1 plugin');
|
||||
// Valid plugin should have few or no findings
|
||||
|
|
@ -55,7 +53,6 @@ describe('scan on valid test-plugin', () => {
|
|||
});
|
||||
|
||||
it('no findings for missing plugin.json fields', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(TEST_PLUGIN);
|
||||
// Anchor on PLH + a title-substring stable across humanizer rewrites.
|
||||
// Raw: "Missing required field in plugin.json: <field>". Humanized: "A plugin's manifest is missing a required field".
|
||||
|
|
@ -66,7 +63,6 @@ describe('scan on valid test-plugin', () => {
|
|||
});
|
||||
|
||||
it('no findings for missing CLAUDE.md sections', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(TEST_PLUGIN);
|
||||
// Raw: "CLAUDE.md missing '<name>' section". Humanized: "A plugin's instructions file is missing a recommended section".
|
||||
const missingSections = result.findings.filter(f =>
|
||||
|
|
@ -78,7 +74,6 @@ describe('scan on valid test-plugin', () => {
|
|||
|
||||
describe('CLAUDE.md section findings track present components', () => {
|
||||
it('flags a missing section only for components the plugin actually ships', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(SECTION_COV);
|
||||
// section-coverage ships commands/ but no agents/ or hooks, and its CLAUDE.md omits the
|
||||
// Commands section. Per the component-aware rule: flag the present component's missing
|
||||
|
|
@ -97,33 +92,27 @@ describe('CLAUDE.md section findings track present components', () => {
|
|||
|
||||
describe('scan on broken-plugin', () => {
|
||||
it('detects missing plugin.json fields', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(BROKEN_PLUGIN);
|
||||
// CA-PLH-001 (description) and CA-PLH-002 (version) in broken-plugin.
|
||||
// One check ("missing required field"), two instances (description, version).
|
||||
const missingFields = result.findings.filter(f =>
|
||||
f.scanner === 'PLH' && (f.id === 'CA-PLH-001' || f.id === 'CA-PLH-002')
|
||||
f.scanner === 'PLH' && f.id === findingId('PLH', 'missing-required-field')
|
||||
);
|
||||
assert.ok(missingFields.length >= 2, 'Should detect missing description and version');
|
||||
});
|
||||
|
||||
it('detects missing CLAUDE.md', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(BROKEN_PLUGIN);
|
||||
// CA-PLH-003 in broken-plugin = Missing CLAUDE.md.
|
||||
const missingMd = result.findings.filter(f => f.scanner === 'PLH' && f.id === 'CA-PLH-003');
|
||||
const missingMd = result.findings.filter(f => f.scanner === 'PLH' && f.id === findingId('PLH', 'missing-claude-md'));
|
||||
assert.equal(missingMd.length, 1, 'Should detect missing CLAUDE.md');
|
||||
});
|
||||
|
||||
it('detects command without frontmatter', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(BROKEN_PLUGIN);
|
||||
// CA-PLH-004 in broken-plugin = Command missing frontmatter.
|
||||
const noFrontmatter = result.findings.filter(f => f.scanner === 'PLH' && f.id === 'CA-PLH-004');
|
||||
const noFrontmatter = result.findings.filter(f => f.scanner === 'PLH' && f.id === findingId('PLH', 'command-missing-frontmatter'));
|
||||
assert.equal(noFrontmatter.length, 1, 'Should detect command without frontmatter');
|
||||
});
|
||||
|
||||
it('flags missing required agent field (description) but not optional model/tools', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(BROKEN_PLUGIN);
|
||||
// Per CC sub-agents docs: only `name` and `description` are required; `model` and `tools`
|
||||
// are optional (inherit / all-tools by default). bad-agent.md has `name` only.
|
||||
|
|
@ -138,11 +127,9 @@ describe('scan on broken-plugin', () => {
|
|||
|
||||
describe('scan with no plugins', () => {
|
||||
it('returns info finding for empty directory', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(resolve(FIXTURES, 'empty-project'));
|
||||
assert.equal(result.findings.length, 1);
|
||||
// CA-PLH-001 in empty-project = No plugins found.
|
||||
assert.equal(result.findings[0].id, 'CA-PLH-001');
|
||||
assert.equal(result.findings[0].id, findingId('PLH', 'no-plugins-found'));
|
||||
assert.equal(result.findings[0].scanner, 'PLH');
|
||||
assert.equal(result.findings[0].severity, 'info');
|
||||
});
|
||||
|
|
@ -150,7 +137,6 @@ describe('scan with no plugins', () => {
|
|||
|
||||
describe('cross-plugin command conflict detection', () => {
|
||||
it('scans fixtures dir and reports findings for all plugins', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(FIXTURES);
|
||||
assert.equal(result.scanner, 'PLH');
|
||||
assert.ok(result.files_scanned >= 2, 'Should scan multiple plugins');
|
||||
|
|
@ -161,7 +147,6 @@ describe('plugin namespace collision detection', () => {
|
|||
const COLLISION_RE = /namespace collision/i;
|
||||
|
||||
it('flags two plugins that declare the same name', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(DUP_NAME);
|
||||
const collisions = result.findings.filter(f =>
|
||||
f.scanner === 'PLH' && COLLISION_RE.test(f.title || '')
|
||||
|
|
@ -181,7 +166,6 @@ describe('plugin namespace collision detection', () => {
|
|||
});
|
||||
|
||||
it('excludes name-less plugins from the collision map', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(DUP_NAME);
|
||||
// gamma + delta declare no name; they must NOT form an undefined/empty collision.
|
||||
const collisions = result.findings.filter(f =>
|
||||
|
|
@ -195,7 +179,6 @@ describe('plugin namespace collision detection', () => {
|
|||
});
|
||||
|
||||
it('does not flag a single plugin as a collision', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(TEST_PLUGIN);
|
||||
const collisions = result.findings.filter(f =>
|
||||
f.scanner === 'PLH' && COLLISION_RE.test(f.title || '')
|
||||
|
|
@ -208,7 +191,6 @@ describe('cross-plugin command name ambiguity (COL-level)', () => {
|
|||
const CMD_RE = /used by multiple plugins/i;
|
||||
|
||||
it('flags a command name shared across different plugin namespaces as low', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(DUP_CMD);
|
||||
const amb = result.findings.filter(f => f.scanner === 'PLH' && CMD_RE.test(f.title || ''));
|
||||
assert.equal(amb.length, 1, `Expected one command-ambiguity finding, got ${amb.length}`);
|
||||
|
|
@ -221,7 +203,6 @@ describe('cross-plugin command name ambiguity (COL-level)', () => {
|
|||
});
|
||||
|
||||
it('labels plugins by declared name, not folder basename', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(DUP_CMD);
|
||||
const f = result.findings.find(x => x.scanner === 'PLH' && CMD_RE.test(x.title || ''));
|
||||
const sources = f.details.namespaces.map(n => n.source).sort();
|
||||
|
|
@ -230,7 +211,6 @@ describe('cross-plugin command name ambiguity (COL-level)', () => {
|
|||
});
|
||||
|
||||
it('does not emit a high-severity command conflict (legacy behavior removed)', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(DUP_CMD);
|
||||
const high = result.findings.filter(f =>
|
||||
f.scanner === 'PLH' && /command name conflict/i.test(f.title || '') && f.severity === 'high'
|
||||
|
|
@ -239,7 +219,6 @@ describe('cross-plugin command name ambiguity (COL-level)', () => {
|
|||
});
|
||||
|
||||
it('does not flag a shared command WITHIN a colliding namespace (namespace-collision covers it)', async () => {
|
||||
resetCounter();
|
||||
// alpha + beta both declare name "dup" and both ship a "hello" command.
|
||||
const result = await scan(DUP_NAME);
|
||||
const amb = result.findings.filter(f => f.scanner === 'PLH' && CMD_RE.test(f.title || ''));
|
||||
|
|
@ -252,7 +231,6 @@ describe('plugin-folder shadowing (CA-PLH-015)', () => {
|
|||
const SHADOW_RE = /shadows the default/i;
|
||||
|
||||
it('flags a manifest path that shadows the default commands/ folder', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(SHADOW);
|
||||
const shadows = result.findings.filter(f => f.scanner === 'PLH' && SHADOW_RE.test(f.title || ''));
|
||||
assert.equal(shadows.length, 1, `Expected exactly one shadow finding, got ${shadows.length}: ${shadows.map(f => f.title).join(' | ')}`);
|
||||
|
|
@ -266,7 +244,6 @@ describe('plugin-folder shadowing (CA-PLH-015)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag a default folder addressed explicitly in the manifest array', async () => {
|
||||
resetCounter();
|
||||
// agents: ["./agents/", "./more-agents/"] addresses the default agents/ folder → no warning.
|
||||
const result = await scan(SHADOW);
|
||||
const agentShadows = result.findings.filter(f =>
|
||||
|
|
@ -276,7 +253,6 @@ describe('plugin-folder shadowing (CA-PLH-015)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag skills (adds to default, not replace)', async () => {
|
||||
resetCounter();
|
||||
// skills: "./custom-skills/" ADDS to the default skills/ scan; both load → no shadow.
|
||||
const result = await scan(SHADOW);
|
||||
const skillShadows = result.findings.filter(f =>
|
||||
|
|
@ -286,7 +262,6 @@ describe('plugin-folder shadowing (CA-PLH-015)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag when the default folder is absent', async () => {
|
||||
resetCounter();
|
||||
// outputStyles: "./styles/" is declared, but there is no output-styles/ folder → nothing ignored.
|
||||
const result = await scan(SHADOW);
|
||||
const osShadows = result.findings.filter(f =>
|
||||
|
|
@ -296,7 +271,6 @@ describe('plugin-folder shadowing (CA-PLH-015)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag a plugin with no component-path keys', async () => {
|
||||
resetCounter();
|
||||
// test-plugin has commands/ and agents/ folders but declares no custom paths → nothing shadowed.
|
||||
const result = await scan(TEST_PLUGIN);
|
||||
const shadows = result.findings.filter(f => f.scanner === 'PLH' && SHADOW_RE.test(f.title || ''));
|
||||
|
|
@ -309,7 +283,6 @@ describe('skills:-array entry validation (CA-PLH-016)', () => {
|
|||
const SKILLS_RE = /^plugin\.json "skills" entry/;
|
||||
|
||||
it('flags one finding per bad entry (file, missing, escape, non-string) and none for a valid dir', async () => {
|
||||
resetCounter();
|
||||
// skills: ["./valid-skill/", "./a-file.md", "./missing-dir/", "../escape", 42]
|
||||
const result = await scan(SKILLS_ARR);
|
||||
const bad = result.findings.filter(f => f.scanner === 'PLH' && SKILLS_RE.test(f.title || ''));
|
||||
|
|
@ -329,7 +302,6 @@ describe('skills:-array entry validation (CA-PLH-016)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag the valid skill directory', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(SKILLS_ARR);
|
||||
const validFlagged = result.findings.some(f =>
|
||||
f.scanner === 'PLH' && SKILLS_RE.test(f.title || '') && /valid-skill/.test(f.title || '')
|
||||
|
|
@ -338,7 +310,6 @@ describe('skills:-array entry validation (CA-PLH-016)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag a plugin with no skills: key', async () => {
|
||||
resetCounter();
|
||||
// test-plugin declares no skills: field.
|
||||
const result = await scan(TEST_PLUGIN);
|
||||
const skillsFindings = result.findings.filter(f => f.scanner === 'PLH' && SKILLS_RE.test(f.title || ''));
|
||||
|
|
@ -348,7 +319,6 @@ describe('skills:-array entry validation (CA-PLH-016)', () => {
|
|||
|
||||
describe('finding format', () => {
|
||||
it('findings have standard fields', async () => {
|
||||
resetCounter();
|
||||
const result = await scan(BROKEN_PLUGIN);
|
||||
assert.ok(result.findings.length > 0);
|
||||
const f = result.findings[0];
|
||||
|
|
@ -383,7 +353,6 @@ describe('PLH — plugin agent declares fields Claude Code ignores (E)', () => {
|
|||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-plh-agentdead-'));
|
||||
await writePlugin(tmpRoot, 'permissionMode: plan\nhooks: present\nmcpServers: present\n');
|
||||
result = await scan(tmpRoot);
|
||||
|
|
@ -410,7 +379,6 @@ describe('PLH — plugin agent declares fields Claude Code ignores (E)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag a clean agent (name + description only)', async () => {
|
||||
resetCounter();
|
||||
const clean = await mkdtemp(join(tmpdir(), 'ca-plh-agentclean-'));
|
||||
try {
|
||||
await writePlugin(clean, '');
|
||||
|
|
@ -578,7 +546,6 @@ describe('.claude-plugin/marketplace.json is not an unknown file (F9)', () => {
|
|||
}
|
||||
|
||||
it('does not flag marketplace.json (documented catalog location)', async () => {
|
||||
resetCounter();
|
||||
await writeMarketplacePlugin(dir);
|
||||
const result = await scan(dir);
|
||||
const unknown = result.findings.filter(f => /Unknown file/i.test(f.title || ''));
|
||||
|
|
@ -587,7 +554,6 @@ describe('.claude-plugin/marketplace.json is not an unknown file (F9)', () => {
|
|||
});
|
||||
|
||||
it('still flags a genuinely unexpected file', async () => {
|
||||
resetCounter();
|
||||
await writeMarketplacePlugin(dir, { 'notes.txt': 'scratch' });
|
||||
const result = await scan(dir);
|
||||
const unknown = result.findings.filter(f => /Unknown file/i.test(f.title || ''));
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ import { hermeticEnv } from '../helpers/hermetic-home.mjs';
|
|||
import { stripHotspotLoadPattern } from '../helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from '../helpers/strip-added-scanner.mjs';
|
||||
import { stripRetiredGap, maskGapTallyStderr } from '../helpers/strip-retired-gap.mjs';
|
||||
import { maskFindingIds } from '../helpers/mask-finding-ids.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -42,7 +43,7 @@ function normalizePosture(p) {
|
|||
}
|
||||
}
|
||||
}
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
return maskFindingIds(stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out))));
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ import { resolve, join } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { scan } from '../../scanners/rules-validator.mjs';
|
||||
|
||||
|
|
@ -14,7 +13,6 @@ const FIXTURES = resolve(__dirname, '../fixtures');
|
|||
describe('RUL scanner — healthy project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'healthy-project'));
|
||||
result = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
});
|
||||
|
|
@ -42,7 +40,6 @@ describe('RUL scanner — healthy project', () => {
|
|||
describe('RUL scanner — broken project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'broken-project'));
|
||||
result = await scan(resolve(FIXTURES, 'broken-project'), discovery);
|
||||
});
|
||||
|
|
@ -83,7 +80,6 @@ describe('RUL scanner — broken project', () => {
|
|||
describe('RUL scanner — empty project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'empty-project'));
|
||||
result = await scan(resolve(FIXTURES, 'empty-project'), discovery);
|
||||
});
|
||||
|
|
@ -120,7 +116,6 @@ describe('RUL — large path-scoped rule lost after compaction (A)', () => {
|
|||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-rul-durability-'));
|
||||
await writeProject(tmpRoot, 60);
|
||||
const discovery = await discoverConfigFiles(tmpRoot);
|
||||
|
|
@ -138,7 +133,6 @@ describe('RUL — large path-scoped rule lost after compaction (A)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag a small path-scoped rule', async () => {
|
||||
resetCounter();
|
||||
const small = await mkdtemp(join(tmpdir(), 'ca-rul-small-'));
|
||||
try {
|
||||
await writeProject(small, 3);
|
||||
|
|
@ -177,7 +171,6 @@ describe('RUL — block-sequence-scoped rule is correctly scoped (parser regress
|
|||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-rul-blockseq-'));
|
||||
await writeBlockSeqProject(tmpRoot, 60);
|
||||
const discovery = await discoverConfigFiles(tmpRoot);
|
||||
|
|
@ -235,7 +228,6 @@ describe('RUL — nested-repo rule glob resolves to its own project root (M-BUG-
|
|||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-rul-nested-'));
|
||||
await writeNestedRepo(tmpRoot);
|
||||
const discovery = await discoverConfigFiles(tmpRoot);
|
||||
|
|
@ -266,7 +258,6 @@ describe('RUL — user-global rule is not flagged "matches no files" (M-BUG-9 gu
|
|||
let result;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpHome = await mkdtemp(join(tmpdir(), 'ca-rul-home-'));
|
||||
savedHome = process.env.HOME;
|
||||
process.env.HOME = tmpHome;
|
||||
|
|
@ -302,7 +293,6 @@ describe('RUL — mid-pattern /**/ glob matches intermediate dirs (M-BUG-19)', (
|
|||
let result;
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-rul-globstar-'));
|
||||
await mkdir(join(tmpRoot, '.claude', 'rules'), { recursive: true });
|
||||
await mkdir(join(tmpRoot, 'posts', '2026-01-23-slug'), { recursive: true });
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import { hermeticEnv } from '../helpers/hermetic-home.mjs';
|
|||
import { stripHotspotLoadPattern } from '../helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners } from '../helpers/strip-added-scanner.mjs';
|
||||
import { stripRetiredGap } from '../helpers/strip-retired-gap.mjs';
|
||||
import { maskFindingIds } from '../helpers/mask-finding-ids.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -39,7 +40,7 @@ function normalizeEnvelope(env) {
|
|||
}
|
||||
}
|
||||
}
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
return maskFindingIds(stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out))));
|
||||
}
|
||||
|
||||
async function runOrchestrator(flags) {
|
||||
|
|
|
|||
|
|
@ -4,9 +4,9 @@ import { resolve, join } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { scan } from '../../scanners/settings-validator.mjs';
|
||||
import { findingId } from '../../scanners/lib/finding-codes.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
|
|
@ -14,7 +14,6 @@ const FIXTURES = resolve(__dirname, '../fixtures');
|
|||
describe('SET scanner — healthy project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'healthy-project'));
|
||||
result = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
});
|
||||
|
|
@ -42,7 +41,6 @@ describe('SET scanner — healthy project', () => {
|
|||
describe('SET scanner — broken project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'broken-project'));
|
||||
result = await scan(resolve(FIXTURES, 'broken-project'), discovery);
|
||||
});
|
||||
|
|
@ -87,7 +85,6 @@ describe('SET scanner — broken project', () => {
|
|||
|
||||
describe('SET scanner — additionalDirectories (v5 M6)', () => {
|
||||
it('does NOT flag additionalDirectories as unknown key', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'additional-dirs-ok');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -100,7 +97,6 @@ describe('SET scanner — additionalDirectories (v5 M6)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag 2 entries as too many', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'additional-dirs-ok');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -112,12 +108,10 @@ describe('SET scanner — additionalDirectories (v5 M6)', () => {
|
|||
});
|
||||
|
||||
it('flags > 2 entries as low finding', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'additional-dirs-many');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
// CA-SET-001 in additional-dirs-many = the additionalDirectories threshold finding.
|
||||
const f = result.findings.find(x => x.scanner === 'SET' && x.id === 'CA-SET-001');
|
||||
const f = result.findings.find(x => x.scanner === 'SET' && x.id === findingId('SET', 'many-additional-dirs'));
|
||||
assert.ok(f, `expected additionalDirectories threshold finding; got: ${result.findings.map(x => x.id).join(' | ')}`);
|
||||
assert.equal(f.severity, 'low', `expected low severity, got ${f.severity}`);
|
||||
});
|
||||
|
|
@ -137,7 +131,6 @@ describe('SET scanner — CC 2.1.114→181 valid keys (Batch 1 false-positive fi
|
|||
];
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-set-2181-'));
|
||||
await mkdir(join(tmpRoot, '.claude'), { recursive: true });
|
||||
const settings = {
|
||||
|
|
@ -206,7 +199,6 @@ describe('SET scanner — unknown-key typo gate (M-BUG-10)', () => {
|
|||
const TYPO_TITLE = 'Possible typo in settings key';
|
||||
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-set-typo-'));
|
||||
await mkdir(join(tmpRoot, '.claude'), { recursive: true });
|
||||
const settings = {
|
||||
|
|
@ -254,7 +246,6 @@ describe('SET scanner — unknown-key typo gate (M-BUG-10)', () => {
|
|||
describe('SET scanner — empty project', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(resolve(FIXTURES, 'empty-project'));
|
||||
result = await scan(resolve(FIXTURES, 'empty-project'), discovery);
|
||||
});
|
||||
|
|
@ -274,7 +265,6 @@ describe('SET scanner — autoMode validation (CA-SET-NNN)', () => {
|
|||
const amFindings = (result) => result.findings.filter(f => f.scanner === 'SET' && AM_RE.test(f.title || ''));
|
||||
|
||||
it('flags structural problems: wrong sub-key type, non-string array entry, unknown sub-key', async () => {
|
||||
resetCounter();
|
||||
// automode-structure/.claude/settings.local.json (scope=local → no dead-config):
|
||||
// environment valid; allow not-array; soft_deny ["ok",7]; bogus unknown sub-key.
|
||||
const path = resolve(FIXTURES, 'automode-structure');
|
||||
|
|
@ -295,7 +285,6 @@ describe('SET scanner — autoMode validation (CA-SET-NNN)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag a valid string-array sub-key (incl. "$defaults")', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'automode-structure');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -305,7 +294,6 @@ describe('SET scanner — autoMode validation (CA-SET-NNN)', () => {
|
|||
});
|
||||
|
||||
it('flags autoMode that is not an object', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'automode-nonobject');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
|
|
@ -316,7 +304,6 @@ describe('SET scanner — autoMode validation (CA-SET-NNN)', () => {
|
|||
});
|
||||
|
||||
it('flags autoMode in shared project settings as dead config (and not in local scope)', async () => {
|
||||
resetCounter();
|
||||
// automode-shared/.claude/settings.json → scope=project → dead-config.
|
||||
const sharedPath = resolve(FIXTURES, 'automode-shared');
|
||||
const sharedResult = await scan(sharedPath, await discoverConfigFiles(sharedPath));
|
||||
|
|
@ -327,7 +314,6 @@ describe('SET scanner — autoMode validation (CA-SET-NNN)', () => {
|
|||
assert.equal(amFindings(sharedResult).length, 1, 'valid structure in shared scope yields only the dead-config finding');
|
||||
|
||||
// The same-shaped autoMode in LOCAL scope (automode-structure) must NOT be dead config.
|
||||
resetCounter();
|
||||
const localPath = resolve(FIXTURES, 'automode-structure');
|
||||
const localResult = await scan(localPath, await discoverConfigFiles(localPath));
|
||||
const localDead = amFindings(localResult).filter(f => f.details && f.details.problem === 'shared-project-scope');
|
||||
|
|
@ -335,7 +321,6 @@ describe('SET scanner — autoMode validation (CA-SET-NNN)', () => {
|
|||
});
|
||||
|
||||
it('does NOT flag a settings file without autoMode', async () => {
|
||||
resetCounter();
|
||||
// healthy-project has .claude/settings.json but no autoMode.
|
||||
const path = resolve(FIXTURES, 'healthy-project');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
|
|
|
|||
|
|
@ -3,7 +3,6 @@ import assert from 'node:assert/strict';
|
|||
import { join } from 'node:path';
|
||||
import { mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/skill-listing-scanner.mjs';
|
||||
|
||||
const CAP = 1536; // verified per-description listing cap (CC 2.1.105, changelog L1502)
|
||||
|
|
@ -18,7 +17,6 @@ function uniqueDir(suffix) {
|
|||
* never rely on the developer's real ~/.claude.
|
||||
*/
|
||||
async function runScannerWithHome(home) {
|
||||
resetCounter();
|
||||
const original = process.env.HOME;
|
||||
process.env.HOME = home;
|
||||
try {
|
||||
|
|
@ -30,7 +28,6 @@ async function runScannerWithHome(home) {
|
|||
|
||||
/** Like runScannerWithHome but threads a resolved { window, advisory } (B8 calibration). */
|
||||
async function runScannerWithCtx(home, contextWindow) {
|
||||
resetCounter();
|
||||
const original = process.env.HOME;
|
||||
process.env.HOME = home;
|
||||
try {
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@ import { describe, it } from 'node:test';
|
|||
import assert from 'node:assert/strict';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/token-hotspots.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
||||
|
|
@ -14,7 +13,6 @@ const FIXTURES = resolve(__dirname, '../fixtures');
|
|||
// ~/.claude.json plugin MCP servers into the fixture result (mirrors the other
|
||||
// TOK tests). The deferral check reads project+local settings + project .mcp.json.
|
||||
async function runScanner(fixtureName) {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, fixtureName);
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
return withHermeticHome(() => scan(path, discovery));
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ import { resolve, join } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/token-hotspots.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
||||
|
|
@ -17,7 +16,6 @@ async function fixtureDiscovery(name) {
|
|||
}
|
||||
|
||||
async function runScanner(fixtureName) {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, fixtureName);
|
||||
const discovery = await fixtureDiscovery(fixtureName);
|
||||
// Hermetic HOME: scan() calls readActiveConfig, which resolves the user
|
||||
|
|
@ -368,7 +366,6 @@ describe('TOK scanner — H stale plugin-cache versions (v5.9 B3)', () => {
|
|||
};
|
||||
|
||||
it('emits a low-severity stale-cache finding when stale versions exist', async () => {
|
||||
resetCounter();
|
||||
const result = await withHermeticHome(() => scan('/tmp/x', staleDiscovery));
|
||||
const f = result.findings.find(x => /stale plugin-cache/i.test(x.title || ''));
|
||||
assert.ok(f, 'expected a stale plugin-cache finding');
|
||||
|
|
@ -377,7 +374,6 @@ describe('TOK scanner — H stale plugin-cache versions (v5.9 B3)', () => {
|
|||
});
|
||||
|
||||
it('states zero live-context impact and lists the stale keys', async () => {
|
||||
resetCounter();
|
||||
const result = await withHermeticHome(() => scan('/tmp/x', staleDiscovery));
|
||||
const f = result.findings.find(x => /stale plugin-cache/i.test(x.title || ''));
|
||||
assert.match(f.evidence, /zero live-context impact/i);
|
||||
|
|
@ -386,7 +382,6 @@ describe('TOK scanner — H stale plugin-cache versions (v5.9 B3)', () => {
|
|||
});
|
||||
|
||||
it('warns against deleting a version a running session still uses (/exit caveat)', async () => {
|
||||
resetCounter();
|
||||
const result = await withHermeticHome(() => scan('/tmp/x', staleDiscovery));
|
||||
const f = result.findings.find(x => /stale plugin-cache/i.test(x.title || ''));
|
||||
assert.match(f.recommendation, /running session|active session/i,
|
||||
|
|
@ -396,13 +391,11 @@ describe('TOK scanner — H stale plugin-cache versions (v5.9 B3)', () => {
|
|||
});
|
||||
|
||||
it('does NOT fire when there are no stale versions', async () => {
|
||||
resetCounter();
|
||||
const result = await withHermeticHome(() => scan('/tmp/x', { files: [], staleCacheVersions: [] }));
|
||||
assert.ok(!result.findings.some(x => /stale plugin-cache/i.test(x.title || '')));
|
||||
});
|
||||
|
||||
it('does NOT fire when discovery omits staleCacheVersions entirely', async () => {
|
||||
resetCounter();
|
||||
const result = await withHermeticHome(() => scan('/tmp/x', { files: [] }));
|
||||
assert.ok(!result.findings.some(x => /stale plugin-cache/i.test(x.title || '')));
|
||||
});
|
||||
|
|
@ -423,7 +416,6 @@ describe('TOK scanner — CLAUDE.md HTML-comment token discount (M-BUG-6)', () =
|
|||
const content = `# Root\n\n${comment}\n\nReal instruction body.\n`;
|
||||
const rawBytes = Buffer.byteLength(content, 'utf8');
|
||||
await writeFile(join(dir, 'CLAUDE.md'), content);
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(dir);
|
||||
const result = await withHermeticHome(() => scan(dir, discovery));
|
||||
const hs = result.hotspots.find(
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
"duration_ms": 0,
|
||||
"findings": [
|
||||
{
|
||||
"id": "CA-CML-001",
|
||||
"id": "CA-CML-007",
|
||||
"scanner": "CML",
|
||||
"severity": "low",
|
||||
"title": "Your instructions file is missing common sections",
|
||||
|
|
@ -130,7 +130,7 @@
|
|||
"duration_ms": 0,
|
||||
"findings": [
|
||||
{
|
||||
"id": "CA-GAP-001",
|
||||
"id": "CA-GAP-004",
|
||||
"scanner": "GAP",
|
||||
"severity": "medium",
|
||||
"title": "You haven't added any custom shortcuts yet",
|
||||
|
|
@ -146,7 +146,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-002",
|
||||
"id": "CA-GAP-006",
|
||||
"scanner": "GAP",
|
||||
"severity": "low",
|
||||
"title": "You only have settings at one level",
|
||||
|
|
@ -162,7 +162,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-003",
|
||||
"id": "CA-GAP-008",
|
||||
"scanner": "GAP",
|
||||
"severity": "low",
|
||||
"title": "You haven't set up path-scoped rules yet",
|
||||
|
|
@ -178,7 +178,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-004",
|
||||
"id": "CA-GAP-010",
|
||||
"scanner": "GAP",
|
||||
"severity": "low",
|
||||
"title": "Your automations all listen to similar events",
|
||||
|
|
@ -194,7 +194,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-005",
|
||||
"id": "CA-GAP-011",
|
||||
"scanner": "GAP",
|
||||
"severity": "low",
|
||||
"title": "You haven't set up any specialized helper agents yet",
|
||||
|
|
@ -210,7 +210,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-006",
|
||||
"id": "CA-GAP-012",
|
||||
"scanner": "GAP",
|
||||
"severity": "low",
|
||||
"title": "You haven't pinned a model preference",
|
||||
|
|
@ -226,7 +226,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-007",
|
||||
"id": "CA-GAP-013",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't set up a status line yet",
|
||||
|
|
@ -242,7 +242,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-008",
|
||||
"id": "CA-GAP-014",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't set up any custom keybindings",
|
||||
|
|
@ -258,7 +258,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-009",
|
||||
"id": "CA-GAP-015",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You're using the default output style",
|
||||
|
|
@ -274,7 +274,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-010",
|
||||
"id": "CA-GAP-016",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't set up parallel worktree support",
|
||||
|
|
@ -290,7 +290,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-011",
|
||||
"id": "CA-GAP-017",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "Your skills don't use the richer settings block",
|
||||
|
|
@ -306,7 +306,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-012",
|
||||
"id": "CA-GAP-018",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't set up subagent isolation yet",
|
||||
|
|
@ -322,7 +322,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-013",
|
||||
"id": "CA-GAP-019",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "Your skills don't include live context",
|
||||
|
|
@ -338,7 +338,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-014",
|
||||
"id": "CA-GAP-021",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't built a custom plugin yet",
|
||||
|
|
@ -354,7 +354,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-015",
|
||||
"id": "CA-GAP-023",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "Your project has no settings managed by your organization",
|
||||
|
|
@ -370,7 +370,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-016",
|
||||
"id": "CA-GAP-024",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't connected Claude to your editor's language servers",
|
||||
|
|
@ -401,7 +401,7 @@
|
|||
"duration_ms": 0,
|
||||
"findings": [
|
||||
{
|
||||
"id": "CA-TOK-001",
|
||||
"id": "CA-TOK-005",
|
||||
"scanner": "TOK",
|
||||
"severity": "low",
|
||||
"title": "A connected service exposes many tools, all loading on every turn",
|
||||
|
|
|
|||
|
|
@ -73,7 +73,7 @@
|
|||
],
|
||||
"findings": [
|
||||
{
|
||||
"id": "CA-TOK-001",
|
||||
"id": "CA-TOK-005",
|
||||
"scanner": "TOK",
|
||||
"severity": "low",
|
||||
"title": "A connected service exposes many tools, all loading on every turn",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue