BREAKING CHANGE: the {NNN} in CA-{SCANNER}-{NNN} identifies the check that
produced the finding. It used to be the finding's position in that scanner's
output for that run, which made it unstable across CONFIGURATIONS, not just
across releases as STATE framed it. Measured on two fixtures: "No custom
subagents" was CA-GAP-007 on minimal-project and CA-GAP-004 on healthy-project.
A user who fixed an unrelated earlier gap silently renumbered every later one,
so a .config-audit-ignore pin retargeted to a neighbouring finding with no
version change at all.
Second measured arm: README already documented the opposite scheme. It and the
scanner headers describe ~20 numbers as check codes (CA-SKL-003 = oversized
body, CA-PLH-015 = folder shadowing, CA-TOK-006 = schema deferral), and the
counter could only produce those in the all-fire case -- source-order positions
are 4, 3 and 8. The documentation described the scheme; the implementation was
what was wrong. Every published number is preserved by construction and pinned
exhaustively in tests/lib/finding-codes.test.mjs.
scanners/lib/finding-codes.mjs is the single authority. Every finding() call
passes a `code`; an undeclared or missing one THROWS. No counter fallback --
that would reproduce D1's findGapId -> 'unknown' silent degradation and let a
half-converted scanner ship IDs that look valid. findingCounter/resetCounter
are deleted outright, not left as no-ops. Retirement is now a mechanism:
RETIRED_CODES tombstones a withdrawn key so its number is never reissued,
seeded with GAP t3_8 -- the D1 removal that opened this chunk.
IDs are consequently NOT unique per finding: one check failing in three files
emits three findings sharing an ID. That inverts which consumer is correct, so
every f.id/findingId site was classified before the change. diff-engine and
most of fix-engine already keyed on scanner+title+file (drift was never lying);
fix-engine's verification did not, and keyed on the ID alone -- fixing one of
two sibling instances marked both fixed, and the untouched one, still present
in the re-scan, was reported as a REGRESSION. Red test first, then keyed on
(findingId, file), which both planFixes and applyFixes already carry.
plugin-health's crossIds Set was measured and is a clean negative: cross
findings are allFindings.slice(crossPluginStart) and codes 18/19 are emitted
only in that tail, so the partition holds by construction.
unknownSuppressions() reports a pin that names no declared check, in the
--output-file payload (ux-rules rule 2 -- a stderr-only warning is invisible to
the commands) and only when one exists, so a clean config is byte-identical.
That is what makes the break safe: a stale pin goes loud instead of dying quiet.
Frozen tests/snapshots/v5.0.0/ untouched on disk. IDs are masked out of that
comparison (mask-finding-ids.mjs) rather than re-derived -- re-deriving
positional IDs would assert the retired scheme against itself, and #58's
isGapEntry off-by-one is the measured example of that misfiring. The dead
re-derivation is removed from strip-retired-gap.mjs. default-output snapshots
re-approved after confirming the diff is IDs and nothing else.
Guards, each seen red against its own defect: a missing code (scanner errors
out mid-sweep), an orphan declaration, a resurrected retired key, and a
documented ID naming no check. The sweep asserts the union across all 16
scanners, never per scanner -- a per-scanner assertion goes green on a partial
conversion.
Fasit written before implementation: docs/mbug28-id-semantics-fasit.local.md,
including one correction made before running (CML has 12 checks over 13 call
sites -- the anchored and calibrated char-budget arms are one check, which a
repeated-title sweep found and my call-site count had missed).
Suite 1535 -> 1573, 0 failing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MyqCQKK2ornJ1jFWwqx17E
224 lines
9.7 KiB
JavaScript
224 lines
9.7 KiB
JavaScript
/**
|
||
* CPS Scanner — Cache-Prefix Stability Analyzer (v5 N3)
|
||
*
|
||
* Walks the CLAUDE.md cascade and flags volatile content anywhere in the
|
||
* cached prefix (≤ CACHED_PREFIX_LINES). Distinguishes from TOK Pattern A,
|
||
* which only inspects the top 30 lines: CPS catches a `!git log` at line 60
|
||
* or a `${TIMESTAMP}` at line 100. Volatile content anywhere in the cached
|
||
* prefix breaks prompt-cache reuse from that line forward.
|
||
*
|
||
* Volatile patterns extend the TOK set with shell-exec `!` prefix and
|
||
* `${VAR}` substitutions — both common cache-busters in real CLAUDE.md files.
|
||
*
|
||
* Finding ID: CA-CPS-NNN. Severity: medium.
|
||
*
|
||
* Zero external dependencies.
|
||
*/
|
||
|
||
import { resolve, dirname } from 'node:path';
|
||
import { tmpdir } from 'node:os';
|
||
import { readTextFile } from './lib/file-discovery.mjs';
|
||
import { finding, scannerResult } from './lib/output.mjs';
|
||
import { SEVERITY } from './lib/severity.mjs';
|
||
import { findImports } from './lib/yaml-parser.mjs';
|
||
|
||
const SCANNER = 'CPS';
|
||
|
||
// Cache-prefix line threshold: content below this line is unlikely to be
|
||
// part of a stable cached prefix in typical sessions. The number is
|
||
// heuristic — the goal is to flag volatility that genuinely costs cache
|
||
// hits per turn, not to chase every inline date in a long backlog file.
|
||
const CACHED_PREFIX_LINES = 150;
|
||
|
||
// CC-provided substitution variables that resolve to a stable per-install or
|
||
// per-project path (e.g. "${CLAUDE_PLUGIN_ROOT}/hooks/x.mjs"). CC expands them
|
||
// to the same value every turn, so they never break the prompt cache — unlike a
|
||
// runtime ${TIMESTAMP}. Excluded from the ${VAR} volatile flag (M-BUG-7).
|
||
const STABLE_CC_VARS = new Set(['CLAUDE_PLUGIN_ROOT', 'CLAUDE_PROJECT_DIR']);
|
||
|
||
// Matches every ${VAR} occurrence on a line so a line carrying only stable CC
|
||
// vars is not mistaken for a runtime cache-buster.
|
||
const VAR_RX = /\$\{([A-Z_][A-Z0-9_]*)\}/g;
|
||
|
||
/** True when a line contains at least one non-CC-stable ${VAR} substitution. */
|
||
function hasVolatileVar(line) {
|
||
VAR_RX.lastIndex = 0;
|
||
let m;
|
||
while ((m = VAR_RX.exec(line)) !== null) {
|
||
if (!STABLE_CC_VARS.has(m[1])) return true;
|
||
}
|
||
return false;
|
||
}
|
||
|
||
// Volatile-pattern set (extends token-hotspots.mjs Pattern A). The ${VAR} entry
|
||
// is `varAware` — flagged via hasVolatileVar() so CC-stable vars are excluded.
|
||
const VOLATILE_PATTERNS = [
|
||
{ rx: /\{timestamp\}/i, label: '{timestamp} placeholder' },
|
||
{ rx: /\{uuid\}/i, label: '{uuid} placeholder' },
|
||
{ rx: /\{date\}/i, label: '{date} placeholder' },
|
||
{ rx: /\{session(?:_id)?\}/i, label: '{session_id} placeholder' },
|
||
{ rx: /\bactivity log\b/i, label: 'activity-log section' },
|
||
{ rx: /^\s*\d{4}-\d{2}-\d{2}T\d{2}:\d{2}/, label: 'ISO timestamp at line start' },
|
||
{ rx: /^\s*\[\d{4}-\d{2}-\d{2}/, label: 'dated log line [YYYY-MM-DD ...]' },
|
||
// v5 N3 extensions:
|
||
{ rx: /^\s*!/, label: 'shell-exec line (! prefix)' },
|
||
{ rx: /\$\{[A-Z_][A-Z0-9_]*\}/, label: '${VAR} substitution', varAware: true },
|
||
];
|
||
|
||
/**
|
||
* Resolve an @import path relative to the file that declares it.
|
||
* Mirrors import-resolver.mjs / token-hotspots.mjs path semantics.
|
||
* @param {string} importPath
|
||
* @param {string} containingFile
|
||
* @returns {string} absolute resolved path
|
||
*/
|
||
function resolveImportPath(importPath, containingFile) {
|
||
if (importPath.startsWith('~')) {
|
||
const home = process.env.HOME || process.env.USERPROFILE || tmpdir();
|
||
return resolve(importPath.replace(/^~/, home));
|
||
}
|
||
if (importPath.startsWith('/')) return importPath;
|
||
return resolve(dirname(containingFile), importPath);
|
||
}
|
||
|
||
/**
|
||
* Scan content for volatile lines within the cached prefix window.
|
||
* Returns array of {line, label, snippet}.
|
||
*/
|
||
function findVolatileLines(content) {
|
||
const out = [];
|
||
if (!content) return out;
|
||
const lines = content.split('\n').slice(0, CACHED_PREFIX_LINES);
|
||
let inFence = false;
|
||
for (let i = 0; i < lines.length; i++) {
|
||
const line = lines[i];
|
||
// Fenced code blocks (``` or ~~~) hold illustrative, byte-stable literal
|
||
// text — a ${VAR} or timestamp shown inside one is documentation, not a
|
||
// runtime cache-buster — so the fence delimiters and their content are
|
||
// skipped (M-BUG-7).
|
||
if (/^\s*(```|~~~)/.test(line)) {
|
||
inFence = !inFence;
|
||
continue;
|
||
}
|
||
if (inFence) continue;
|
||
// Strip `inline code` spans before pattern-testing: a {date} or ${VAR}
|
||
// shown inside backticks is literal documentation text, byte-stable, not a
|
||
// runtime cache-buster (M-BUG-7). The original line is still reported as the
|
||
// snippet so context is preserved.
|
||
const probe = line.replace(/`[^`]*`/g, '');
|
||
for (const { rx, label, varAware } of VOLATILE_PATTERNS) {
|
||
// The ${VAR} pattern flags only non-CC-stable substitutions; every other
|
||
// pattern keeps its plain line test.
|
||
if (varAware ? !hasVolatileVar(probe) : !rx.test(probe)) continue;
|
||
out.push({
|
||
line: i + 1,
|
||
label,
|
||
snippet: line.length > 120 ? line.slice(0, 117) + '...' : line,
|
||
});
|
||
break;
|
||
}
|
||
}
|
||
return out;
|
||
}
|
||
|
||
/**
|
||
* Main scanner entry point.
|
||
*
|
||
* @param {string} targetPath
|
||
* @param {{files: Array<{absPath:string, relPath:string, type:string, scope:string, size:number}>}} discovery
|
||
*/
|
||
export async function scan(targetPath, discovery) {
|
||
const start = Date.now();
|
||
const findings = [];
|
||
let filesScanned = 0;
|
||
|
||
// Files already scanned in-file below — an @import resolving to one of these
|
||
// is reported by its own iteration, not duplicated as an import finding.
|
||
const discoveredClaudeMd = new Set(
|
||
discovery.files.filter(f => f.type === 'claude-md').map(f => f.absPath));
|
||
// @imported files reported once, even when several CLAUDE.md files import them.
|
||
const reportedImports = new Set();
|
||
|
||
for (const f of discovery.files) {
|
||
if (f.type !== 'claude-md') continue;
|
||
filesScanned++;
|
||
const content = await readTextFile(f.absPath);
|
||
if (!content) continue;
|
||
|
||
// --- In-file volatility (unchanged behavior) ---
|
||
const volatile = findVolatileLines(content);
|
||
// Skip volatility that's already covered by TOK Pattern A (lines 1–30) —
|
||
// CPS' value is in the 31–150 range. Pattern A handles 1–30.
|
||
const beyondTopThirty = volatile.filter(v => v.line > 30);
|
||
if (beyondTopThirty.length > 0) {
|
||
const evidence =
|
||
beyondTopThirty.slice(0, 5)
|
||
.map(v => `line ${v.line} (${v.label}): ${v.snippet}`)
|
||
.join('; ');
|
||
findings.push(finding({
|
||
scanner: SCANNER,
|
||
code: 'volatile-in-prefix',
|
||
severity: SEVERITY.medium,
|
||
title: 'Volatile content inside cached prefix breaks reuse',
|
||
description:
|
||
`${f.relPath || f.absPath} contains ${beyondTopThirty.length} volatile ` +
|
||
`entr${beyondTopThirty.length === 1 ? 'y' : 'ies'} between lines 31 and ` +
|
||
`${CACHED_PREFIX_LINES}. The prompt cache covers the file's prefix; ` +
|
||
'any volatility forces a fresh cache write from that line down on every turn.',
|
||
file: f.absPath,
|
||
evidence,
|
||
recommendation:
|
||
'Move volatile sections (timestamps, !shell-exec, ${VAR} substitutions, dated logs) ' +
|
||
`below line ${CACHED_PREFIX_LINES} or extract them to an @import-ed file outside the ` +
|
||
'cached prefix. Stable content above, volatile content below.',
|
||
category: 'token-efficiency',
|
||
}));
|
||
}
|
||
|
||
// --- v5.10 B6: volatility inside @imported files ---
|
||
// @import-ed content is inlined into the cached prefix at the import site.
|
||
// TOK Pattern A and the in-file scan above never look past the importing
|
||
// file, so volatility in an imported file is otherwise invisible. We scan
|
||
// direct imports only (one hop); IMP owns deep-chain analysis. The whole
|
||
// imported-file prefix counts (no lines-1–30 skip — that exclusion is
|
||
// root-file-specific to avoid Pattern A overlap, which does not reach here).
|
||
for (const imp of findImports(content)) {
|
||
if (imp.line > CACHED_PREFIX_LINES) continue; // import site outside prefix
|
||
const resolved = resolveImportPath(imp.path, f.absPath);
|
||
if (discoveredClaudeMd.has(resolved)) continue; // scanned in its own iteration
|
||
if (reportedImports.has(resolved)) continue;
|
||
reportedImports.add(resolved);
|
||
const importedContent = await readTextFile(resolved);
|
||
if (!importedContent) continue;
|
||
const importedVolatile = findVolatileLines(importedContent);
|
||
if (importedVolatile.length === 0) continue;
|
||
|
||
const importEvidence =
|
||
`imported by ${f.relPath || f.absPath} (@${imp.path} at line ${imp.line}); ` +
|
||
importedVolatile.slice(0, 5)
|
||
.map(v => `line ${v.line} (${v.label}): ${v.snippet}`)
|
||
.join('; ');
|
||
findings.push(finding({
|
||
scanner: SCANNER,
|
||
code: 'volatile-in-import',
|
||
severity: SEVERITY.medium,
|
||
title: 'Volatile content in @imported file breaks cached prefix',
|
||
description:
|
||
`@${imp.path} (imported by ${f.relPath || f.absPath} at line ${imp.line}) contains ` +
|
||
`${importedVolatile.length} volatile entr${importedVolatile.length === 1 ? 'y' : 'ies'} ` +
|
||
`within its first ${CACHED_PREFIX_LINES} lines. @import-ed content is inlined into the ` +
|
||
'prompt-cache prefix, so volatility there forces a fresh cache write every turn — even ' +
|
||
'when the importing CLAUDE.md is itself byte-stable.',
|
||
file: resolved,
|
||
evidence: importEvidence,
|
||
recommendation:
|
||
'Move volatile content (timestamps, !shell-exec, ${VAR} substitutions, dated logs) out ' +
|
||
'of the @imported file, or import it below the cached-prefix window. Keep imported config ' +
|
||
'byte-stable so the importing file\'s cache survives.',
|
||
category: 'token-efficiency',
|
||
}));
|
||
}
|
||
}
|
||
|
||
return scannerResult(SCANNER, 'ok', findings, filesScanned, Date.now() - start);
|
||
}
|