feat(scanners): a path written in prose is now resolved, not assumed (C3)
`import-resolver` follows @import targets; a path written in ordinary prose was checked by nothing. CA-CML-013 resolves those too — one finding per file, severity low, against both the CLAUDE.md's own directory and the scan root, because a nested file may legitimately write repo-root-relative paths. The design work here is the SILENCE list, and every entry on it was measured against 407 real CLAUDE.md files rather than argued for: - Bare filenames excluded: admitting them tripled the output (2350 vs 810), led by name-drops of tools that exist elsewhere on the machine. - Org/repo slugs, npm packages, pytest node ids and prose enumerations excluded: 111 fires, inspected, all false positives. - Bare folder names excluded on the same reasoning one level up: 183 of the remaining 699 fires (26%), led by `open/` — a Forgejo remote namespace prefix, not a directory. This one overturned a premise the fasit had asserted without measuring; the deviation is recorded rather than the prediction quietly edited. - Containment is checked against the scan root, not the file's own dir: a base a `..` chain can escape is not a base. Measured — without it, `../../../../etc/passwd` resolved to the real file and silenced its own finding, while a legitimate `../docs/x.md` still resolves. Rule ORDER is the reported reason (first match wins), so `npm test` is silenced as a command rather than as a bare token, and two silences with different causes keep their own fixtures. Twelve classes, pinned by name. Both load-bearing rules were seen RED against their own defect: deleting containment fails 1 test, deleting the slug rule fails 6. Dogfooded through the argv the command template itself constructs, which found a true positive in our own CLAUDE.md — `lib/humanizer.mjs` where the file is `scanners/lib/humanizer.mjs`. Fixed here. Suite 1662 -> 1701, 0 failing. Frozen v5.0.0 and default-output baselines: 0 changed files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HJbfM3N8zWQ1wA2voTrZxz
This commit is contained in:
parent
33bfd5ff5b
commit
dbb6a6a3cf
10 changed files with 478 additions and 3 deletions
19
CLAUDE.md
19
CLAUDE.md
|
|
@ -67,7 +67,7 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
|
|||
|
||||
## Plain-Language Output (v5.1.0)
|
||||
|
||||
Default output of all commands routes through `humanizeEnvelope` (`lib/humanizer.mjs`), decorating each finding with `userImpactCategory`, `userActionLanguage`, and `relevanceContext`. `--raw` and `--json` bypass the humanizer for byte-stable v5.0.0 output. Full detail: `docs/humanizer.md`.
|
||||
Default output of all commands routes through `humanizeEnvelope` (`scanners/lib/humanizer.mjs`), decorating each finding with `userImpactCategory`, `userActionLanguage`, and `relevanceContext`. `--raw` and `--json` bypass the humanizer for byte-stable v5.0.0 output. Full detail: `docs/humanizer.md`.
|
||||
|
||||
## Suppressions
|
||||
|
||||
|
|
@ -95,6 +95,23 @@ Coding style: scanners are zero-dependency Node ESM; new findings use the `CA-{S
|
|||
|
||||
**Write-scope gate (invariant).** Every write target is classified by `scanners/lib/write-scope.mjs` before it reaches an approval surface, and the **scope class decides the gate's strength — never the command asking**. Five command-owned policies would drift apart the way five copies of the lever table did. `SCOPE_CLASSES` is the single source for class, gate (`silent`/`disclose`/`require-ok`), wording and predicate; templates render `disclosures[]` from `write-scope-cli.mjs` rather than restating what a class means. Two orderings in that object are load-bearing and were measured, not reasoned about: `plugin-managed` before `user-scope` (both `~/.claude/config-audit/` and legacy `~/.config-audit/` are live, so the other order fires the gate on every session write and gets it switched off), and `user-scope` before `cross-repo` (`~/.claude/.git` exists, so a plain `.git`-upward walk calls `~/.claude/CLAUDE.md` merely "another repo" and silently downgrades the strongest gate). `disclose` ≠ `require-ok`: `campaign export` is cross-repo *by design*, so tightening it into a refusal breaks the feature. Distinct from the `require-target-dir.mjs` guard, which asks whether a scan **root** is readable (exit 3) — a different invariant, not to be merged.
|
||||
|
||||
**Dead-prose-reference silence list (invariant).** `CA-CML-013` is a precision-first check, so its
|
||||
design lives in what it *declines* to flag, and that list is measured (407 real CLAUDE.md files),
|
||||
never argued. Three rules are load-bearing and each has a guard seen red against its own defect.
|
||||
(1) **Containment is checked against the scan root, not the file's own directory** — a `../` chain
|
||||
that leaves the tree is silenced (`outside-scan-tree`) rather than resolved, because a base a `..`
|
||||
chain can escape is not a base: measured, `../../../../etc/passwd` resolved to the real file and
|
||||
silenced its own finding. A legitimate `../docs/x.md` inside the same repo still resolves.
|
||||
(2) **A bare token is a concept, not a reference** — `README.md` (no separator) and `docs/`
|
||||
(single segment) are excluded on the same reasoning one level apart; admitting bare filenames
|
||||
tripled the output with name-drops of tools living elsewhere, and single-segment folders are 26 %
|
||||
of the remainder, led by a remote namespace prefix. (3) **Rule ORDER is the reported reason** —
|
||||
first match wins, so `npm test` is silenced as `whitespace` (a command), not as `no-separator`,
|
||||
and two silences with different causes keep their own fixtures. The check emits **one finding per
|
||||
file** (the `todo-markers` / `repeated-content` idiom), because per-token emission measured 699
|
||||
findings where per-file measured 128. Silence is the safe failure direction here: a path carrying
|
||||
a trailing `:54-56` locator is a recorded v1 miss, not a bug to fix by loosening a rule.
|
||||
|
||||
**Subtraction floor (invariant).** `optimize --subtract` is the only lens that proposes removing config, so `scanners/lib/floor-exclusion.mjs` runs as a deterministic pre-step *before* the judge — a load-bearing block is never a candidate, and that guarantee must not be moved into the agent prompt. Two rules follow from it: (1) **staleness is not a deletion signal** — an outdated version pin inside a floor block is a `drift`/`CA-CML` dead-reference concern; (2) **tier 2 ≠ tier 3** — a compensatory block that keeps earning its place returns, and reporting it as dead weight is wrong even when the label matches. Norwegian keywords need the Unicode boundaries in `subtraction-prefilter.mjs`; JS `\b` is ASCII-only, so `/\bunngå\b/` silently never matches.
|
||||
|
||||
**Subtraction write path (invariant).** `--apply` routes through `scanners/lib/subtraction-write.mjs`, never through `fix-engine` or the `plan`/`implement` pipeline, and both exclusions are **measured**: the subtraction axis is absent from the orchestrated envelope, so `verifyFixes`' re-scan would mark every removal `verified` whether or not it happened (a success-shaped no-op), and the findings pipeline needs a finding code — which names a deterministic check, not a prose judgement. Three properties are load-bearing and each has a guard seen red against its own defect: removals are validated against the ORIGINAL content and applied in **descending** line order (an ascending pass shifts later spans out from under themselves); the **range** check is not redundant with the text check (`line: 0` makes `slice(-1, 0)` empty, so an empty `text` matches and `splice(-1, 1)` deletes the file's LAST line); and `createBackup` skips a nonexistent path while still returning an id, so coverage of every file about to be written is **asserted from the manifest** before a byte changes. The floor is *repeated* here, not moved: `floor-exclusion` still vetoes before anything is proposed, and the engine refuses a load-bearing block again so a hand-built approval cannot route around it. The archive rule (`mv` to `_archive/`) is file-level and does not apply to a block excision — the timestamped backup is the recovery artifact, and inventing a second copy with no restorer behind it would be worse than none.
|
||||
|
|
|
|||
23
README.md
23
README.md
|
|
@ -301,7 +301,7 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
|
||||
| Scanner | Prefix | What It Catches |
|
||||
|---------|--------|-----------------|
|
||||
| `claude-md-linter.mjs` | CML | Oversized files (line count **plus** a context-window-scaled char budget mirroring Claude Code's ~40.0k-char startup warning), missing sections, broken @imports, duplicates, stale TODOs |
|
||||
| `claude-md-linter.mjs` | CML | Oversized files (line count **plus** a context-window-scaled char budget mirroring Claude Code's ~40.0k-char startup warning), missing sections, broken @imports, duplicates, stale TODOs, and **dead prose references** (`CA-CML-013`) — backtick-quoted relative paths in prose that resolve to nothing, next to the file or from the scan root |
|
||||
| `settings-validator.mjs` | SET | Schema violations, unknown/deprecated keys, type mismatches, permission issues |
|
||||
| `hook-validator.mjs` | HKV | Invalid format, missing scripts, wrong event names, timeout risks, verbose-stdout scripts, and a low-precision **advisory** (info) when a hook injects un-grepped command output into `hookSpecificOutput.additionalContext` — that payload enters context on every fire (plain stdout does not) |
|
||||
| `rules-validator.mjs` | RUL | Bad glob patterns, orphaned rules, deprecated fields, unscoped rules |
|
||||
|
|
@ -318,6 +318,27 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
| `optimization-lens-scanner.mjs` | OPT | Optimization lens (mechanism-fit): a multi-step procedure in CLAUDE.md that would fit better as a skill (`CA-OPT-001`) — reads the machine-readable best-practices register, framed as an opportunity, not a failure. The deterministic half of the lens; prose-judgment cases (lifecycle→hook, unscoped path→rule, "never"→permission) are judged by the opus `optimization-lens-agent` via `/config-audit optimize` |
|
||||
| `agent-listing-scanner.mjs` | AGT | Always-loaded agent-listing budget: a per-agent description over the soft bloat cap (`CA-AGT-001`, advisory) and the summed active-agent name+description listing — re-sent every turn — exceeding the listing budget (`CA-AGT-002`). Both LOW and explicitly **inferred / upper-bound**: the agent-listing mechanism is undocumented, so the evidence discloses the estimate and heuristic budget rather than overstating certainty |
|
||||
|
||||
> **Dead prose references — a check designed around what it stays silent about.**
|
||||
> `import-resolver` follows `@import` targets; a path written in ordinary prose was
|
||||
> checked by nothing, so `CA-CML-013` (low) resolves those too. A reference has to be
|
||||
> unambiguously path-shaped to qualify: a separator, plus either a trailing `/` or a
|
||||
> known file extension, resolved both next to the CLAUDE.md and from the scan root —
|
||||
> a nested file may legitimately write repo-root-relative paths. One finding per file,
|
||||
> carrying the count and the first few paths.
|
||||
>
|
||||
> The design work is the silence list, and every entry on it was measured against 407
|
||||
> real CLAUDE.md files rather than argued for. Left alone: commands (`npm test`), URLs,
|
||||
> globs and placeholders (`CA-GAP-*`, `${CLAUDE_PLUGIN_ROOT}/…`), absolute and `~/`
|
||||
> paths, config keys and flags (`model:`, `--raw`), bare filenames (`README.md` — a
|
||||
> filename in prose is a concept, and admitting them tripled the output with name-drops
|
||||
> of tools that exist elsewhere), org/repo slugs and package names
|
||||
> (`ktg/some-repo`, `@anthropic-ai/claude-agent-sdk`), bare folder names (`docs/`,
|
||||
> `open/` — the same reasoning one level up), fenced code blocks, and anything resolving
|
||||
> outside the scanned tree. That last rule is not fussiness: without it a `../../../../etc/passwd`
|
||||
> resolved to the real file and silenced its own finding. A path carrying a trailing
|
||||
> `:54-56` locator is a known v1 miss — for a precision-first check, silence is the safe
|
||||
> failure direction.
|
||||
|
||||
> **Cross-scanner remediation — diagnosis meets the fix.** SKL diagnoses an over-budget
|
||||
> skill listing (`CA-SKL-002`); GAP prescribes the remedy. When the active skill listing
|
||||
> exceeds its ~2%-of-context budget and `disableBundledSkills` is not already set (in the
|
||||
|
|
|
|||
|
|
@ -10,7 +10,8 @@ import { SEVERITY } from './lib/severity.mjs';
|
|||
import { parseFrontmatter, extractSections, findImports } from './lib/yaml-parser.mjs';
|
||||
import { lineCount, truncate } from './lib/string-utils.mjs';
|
||||
import { CONTEXT_WINDOW_ANCHOR, LARGE_CONTEXT_WINDOW, LARGE_CONTEXT_SCALE, scaleForWindow, withCommas } from './lib/context-window.mjs';
|
||||
import { dirname } from 'node:path';
|
||||
import { dirname, resolve as resolvePath, sep } from 'node:path';
|
||||
import { stat } from 'node:fs/promises';
|
||||
|
||||
const SCANNER = 'CML';
|
||||
const MAX_RECOMMENDED_LINES = 200;
|
||||
|
|
@ -30,6 +31,134 @@ const CHAR_BUDGET_RECOMMENDATION =
|
|||
const CLAUDE_MD_CHAR_WARN_ANCHOR = 40_000; // chars @ 200k context (CC startup warning)
|
||||
const CLAUDE_MD_CHAR_WARN_LARGE = CLAUDE_MD_CHAR_WARN_ANCHOR * LARGE_CONTEXT_SCALE; // 200,000 @ 1M
|
||||
|
||||
// ── C3: dead prose references ───────────────────────────────────────────────
|
||||
// `import-resolver` resolves @import targets; a path written in prose is not
|
||||
// checked by anything. The whole design here is the SILENCE taxonomy — a
|
||||
// precision-first check whose failure mode must be a miss, never a false alarm.
|
||||
// Each rule below was measured against 407 real CLAUDE.md files, not reasoned
|
||||
// about; the numbers live in docs/c3-deadref-fasit.local.md §2.
|
||||
const KNOWN_EXTENSIONS = /\.(?:md|mjs|js|ts|tsx|jsx|json|ya?ml|sh|py|toml|txt|html|css)$/i;
|
||||
|
||||
// How many dead references the evidence names before it summarises the rest.
|
||||
const MAX_LISTED_DEAD_REFS = 5;
|
||||
|
||||
/**
|
||||
* Inline-code spans that sit in prose, i.e. outside fenced code blocks.
|
||||
* Fenced code is illustrative — a dead path in a `bash` sample is a sample,
|
||||
* not a reference (silence class S1).
|
||||
* @param {string} content
|
||||
* @returns {Array<{text: string, line: number}>}
|
||||
*/
|
||||
export function extractInlineSpans(content) {
|
||||
const spans = [];
|
||||
const lines = String(content == null ? '' : content).split('\n');
|
||||
let inFence = false;
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const raw = lines[i];
|
||||
if (/^\s*(?:```|~~~)/.test(raw)) {
|
||||
inFence = !inFence;
|
||||
continue;
|
||||
}
|
||||
if (inFence) continue;
|
||||
|
||||
const re = /`([^`\n]+)`/g;
|
||||
let m;
|
||||
while ((m = re.exec(raw)) !== null) {
|
||||
const text = m[1].trim();
|
||||
if (text) spans.push({ text, line: i + 1 });
|
||||
}
|
||||
}
|
||||
return spans;
|
||||
}
|
||||
|
||||
/**
|
||||
* Lexical half of the taxonomy: is this token even a path reference?
|
||||
* Order is load-bearing — the FIRST matching rule is the reported reason, so
|
||||
* `npm test` is silenced as `whitespace` (a command) rather than as
|
||||
* `no-separator`, and the taxonomy keeps describing what actually happened.
|
||||
*
|
||||
* @param {string} token - the text inside one backtick span
|
||||
* @returns {{rule: string|null}} rule name, or null when the token is a
|
||||
* candidate that still needs resolving against the filesystem
|
||||
*/
|
||||
export function classifyProseReference(token) {
|
||||
const t = String(token == null ? '' : token);
|
||||
|
||||
// S2 — a command invocation, not a path.
|
||||
if (/\s/.test(t)) return { rule: 'whitespace' };
|
||||
// S3 — an external resource; on-disk existence is meaningless.
|
||||
if (/^[a-z][a-z0-9+.-]*:\/\//i.test(t) || /^(?:www\.|mailto:)/i.test(t)) return { rule: 'url' };
|
||||
// S4 — a pattern or template: resolves to many, or to nothing until expanded.
|
||||
if (/[*?[\]{}<>$]/.test(t)) return { rule: 'glob-or-placeholder' };
|
||||
// S5 — outside project scope, and machine-dependent.
|
||||
if (t.startsWith('/') || t.startsWith('~')) return { rule: 'absolute-or-home' };
|
||||
// S6 — a config key or a CLI flag.
|
||||
if (t.endsWith(':') || t.startsWith('-')) return { rule: 'key-or-flag' };
|
||||
// S7 — a bare filename in prose is a concept or a tool name, not a reference.
|
||||
// Measured: admitting bare names triples the output, and its top entries are
|
||||
// name-drops of tools that exist elsewhere on the machine.
|
||||
if (!t.includes('/')) return { rule: 'no-separator' };
|
||||
// S8 — has a separator but no unambiguous path shape: org/repo slugs, npm
|
||||
// packages, pytest node ids, prose enumerations. Also swallows S10, a path
|
||||
// carrying a trailing `:54-56` locator — a known v1 gap, and a miss rather
|
||||
// than a false alarm.
|
||||
if (!t.endsWith('/') && !KNOWN_EXTENSIONS.test(t)) return { rule: 'ambiguous-slug' };
|
||||
// S8b — a BARE folder name is a concept one level up from a bare filename,
|
||||
// and the same D-A reasoning applies. Measured on the same corpus: 183 of 699
|
||||
// fires (26 %) are single-segment directory tokens, led by `open/` (39x, a
|
||||
// remote namespace prefix) and generic names — `tests/`, `src/`, `docs/`,
|
||||
// `scripts/` — that prose almost always MENTIONS rather than references. A
|
||||
// specific path like `tools/wiki_ingest/` still qualifies.
|
||||
if (t.endsWith('/') && t.replace(/^\.\//, '').split('/').filter(Boolean).length === 1) {
|
||||
return { rule: 'single-segment-directory' };
|
||||
}
|
||||
|
||||
return { rule: null };
|
||||
}
|
||||
|
||||
/** @returns {Promise<boolean>} */
|
||||
async function pathExists(p) {
|
||||
try {
|
||||
await stat(p);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/** Is `p` the root itself or below it? */
|
||||
function isInside(p, root) {
|
||||
return p === root || p.startsWith(root.endsWith(sep) ? root : root + sep);
|
||||
}
|
||||
|
||||
/**
|
||||
* Filesystem half of the taxonomy. Two bases, because a nested CLAUDE.md
|
||||
* routinely writes repo-root-relative paths.
|
||||
*
|
||||
* Containment is checked against the SCAN ROOT, not the file's own directory:
|
||||
* a legitimate `../docs/x.md` inside the same repo must still resolve, while a
|
||||
* `..` chain that leaves the tree must not. Measured: without this,
|
||||
* `../../../../etc/passwd` resolved to the real /etc/passwd and silenced the
|
||||
* finding by accident. A base a `..` chain can escape is not a base.
|
||||
*
|
||||
* @param {string} token
|
||||
* @param {{fileDir: string, scanRoot: string}} bases
|
||||
* @returns {Promise<{rule: string|null}>} null means the reference is dead
|
||||
*/
|
||||
export async function resolveProseReference(token, { fileDir, scanRoot }) {
|
||||
const root = resolvePath(scanRoot);
|
||||
const ownAbs = resolvePath(fileDir, token);
|
||||
|
||||
if (!isInside(ownAbs, root)) return { rule: 'outside-scan-tree' };
|
||||
if (await pathExists(ownAbs)) return { rule: 'resolves-own-dir' };
|
||||
|
||||
const rootAbs = resolvePath(root, token);
|
||||
if (isInside(rootAbs, root) && await pathExists(rootAbs)) return { rule: 'resolves-scan-root' };
|
||||
|
||||
return { rule: null };
|
||||
}
|
||||
|
||||
/** Recommended sections for a project CLAUDE.md */
|
||||
const RECOMMENDED_SECTIONS = [
|
||||
{ pattern: /project|overview|description|what/i, label: 'Project overview' },
|
||||
|
|
@ -301,6 +430,40 @@ export async function scan(targetPath, discovery, opts = {}) {
|
|||
evidence: truncate(todos[0].trim(), 80),
|
||||
}));
|
||||
}
|
||||
|
||||
// --- Dead prose references (C3) ---
|
||||
// One finding per FILE, matching the idiom of the two checks above: a
|
||||
// machine-wide scan measured 699 dead references across 128 files, and
|
||||
// per-token emission would bury the file that has ten of them.
|
||||
const deadRefs = [];
|
||||
for (const span of extractInlineSpans(content)) {
|
||||
if (classifyProseReference(span.text).rule !== null) continue;
|
||||
const { rule } = await resolveProseReference(span.text, {
|
||||
fileDir: dirname(file.absPath),
|
||||
scanRoot: targetPath,
|
||||
});
|
||||
if (rule === null) deadRefs.push(span);
|
||||
}
|
||||
|
||||
if (deadRefs.length > 0) {
|
||||
const listed = deadRefs
|
||||
.slice(0, MAX_LISTED_DEAD_REFS)
|
||||
.map(r => `${r.text} (line ${r.line})`)
|
||||
.join(', ');
|
||||
const rest = deadRefs.length - Math.min(deadRefs.length, MAX_LISTED_DEAD_REFS);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
code: 'dead-prose-reference',
|
||||
severity: SEVERITY.low,
|
||||
title: 'CLAUDE.md points at files that are not there',
|
||||
description: `${file.relPath} has ${deadRefs.length} backtick-quoted path reference(s) in prose that resolve to nothing — neither next to the file nor from the scan root. Anyone following them, human or Claude, finds nothing.`,
|
||||
file: file.absPath,
|
||||
line: deadRefs[0].line,
|
||||
evidence: `${listed}${rest > 0 ? `, +${rest} more` : ''}`,
|
||||
recommendation: 'Point each reference at where the file actually lives, or drop it. Only unambiguous relative paths are checked — URLs, globs, absolute paths and bare filenames are left alone.',
|
||||
autoFixable: false,
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
||||
return scannerResult(SCANNER, 'ok', findings, filesScanned, Date.now() - start);
|
||||
|
|
|
|||
|
|
@ -49,6 +49,7 @@ export const FINDING_CODES = {
|
|||
'html-comments': 10,
|
||||
'repeated-content': 11,
|
||||
'todo-markers': 12,
|
||||
'dead-prose-reference': 13,
|
||||
},
|
||||
|
||||
// ── SET: settings-validator (source order) ──────────────────────────────
|
||||
|
|
|
|||
|
|
@ -77,6 +77,11 @@ export const TRANSLATIONS = {
|
|||
description: 'HTML comments still count as text sent to Claude on every turn — they don\'t actually hide anything.',
|
||||
recommendation: 'Delete the comment text if you don\'t want it sent, or convert it to a regular note.',
|
||||
},
|
||||
'CLAUDE.md points at files that are not there': {
|
||||
title: 'Your instructions file links to files that are not there',
|
||||
description: 'Some file paths written in `CLAUDE.md` point at files that do not exist — not next to the file, and not from your project root. Anyone following them finds nothing.',
|
||||
recommendation: 'Point each path at where the file actually lives, or drop the reference. Only clear relative paths are checked; web links, wildcards and plain file names are left alone.',
|
||||
},
|
||||
'Contains TODO/FIXME markers': {
|
||||
title: 'Your file has TODO or FIXME notes',
|
||||
description: 'These notes are sent to Claude on every turn even when they\'re internal reminders.',
|
||||
|
|
|
|||
40
tests/fixtures/dead-prose-ref/CLAUDE.md
vendored
Normal file
40
tests/fixtures/dead-prose-ref/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
# Dead prose reference fixture
|
||||
|
||||
## Project overview
|
||||
|
||||
This fixture pins the C3 silence taxonomy. Every token below is here on purpose;
|
||||
each silence class carries a distinct cause and gets its own token.
|
||||
|
||||
## Commands and workflows
|
||||
|
||||
Three dead path-shaped references (these, and only these, must fire):
|
||||
|
||||
- The runbook lives in `docs/missing-runbook.md`.
|
||||
- Deploy with the script at `scripts/deploy.sh`.
|
||||
- Generated output lands in `build/artifacts/`.
|
||||
|
||||
A live reference that must stay silent: `docs/real.md`.
|
||||
|
||||
## Architecture
|
||||
|
||||
One token per silence class:
|
||||
|
||||
- whitespace, a command not a path: `node scripts/build.mjs`
|
||||
- url, existence on disk is meaningless: `https://example.com/a/b.md`
|
||||
- glob, a pattern not a path: `CA-GAP-*`
|
||||
- placeholder, unresolved until expanded: `${CLAUDE_PLUGIN_ROOT}/hooks/x.mjs`
|
||||
- absolute or home, outside project scope: `~/.claude/settings.json`
|
||||
- key or flag, not a path at all: `model:`
|
||||
- bare token, a concept not a reference: `README.md`
|
||||
- ambiguous slug, a Forgejo remote: `ktg/from-ai-to-chitta`
|
||||
- bare folder name, a concept not a reference: `vendor/`
|
||||
- escapes the scanned tree: `../../../../etc/passwd`
|
||||
- trailing locator, a known v1 gap: `docs/plan.md:54-56`
|
||||
|
||||
## Conventions and patterns
|
||||
|
||||
Fenced code is illustrative, never a reference. The dead path below must stay silent:
|
||||
|
||||
```bash
|
||||
cat docs/fenced-and-dead.md
|
||||
```
|
||||
3
tests/fixtures/dead-prose-ref/docs/real.md
vendored
Normal file
3
tests/fixtures/dead-prose-ref/docs/real.md
vendored
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# Real
|
||||
|
||||
This file exists so `docs/real.md` resolves and stays silent.
|
||||
4
tests/fixtures/dead-prose-ref/nested/CLAUDE.md
vendored
Normal file
4
tests/fixtures/dead-prose-ref/nested/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
# Nested
|
||||
|
||||
Repo-root-relative reference: `docs/real.md` resolves from the scan root,
|
||||
not from this directory. It must stay silent.
|
||||
|
|
@ -105,6 +105,7 @@ describe('published finding IDs (pinned exhaustively — README is a contract)',
|
|||
['AGT', 'aggregate-listing-budget', 'CA-AGT-002'],
|
||||
['CPS', 'volatile-in-prefix', 'CA-CPS-001'],
|
||||
['COL', 'skill-user-vs-plugin', 'CA-COL-001'],
|
||||
['CML', 'dead-prose-reference', 'CA-CML-013'],
|
||||
];
|
||||
|
||||
for (const [scanner, key, expected] of PUBLISHED) {
|
||||
|
|
|
|||
220
tests/scanners/claude-md-dead-prose-ref.test.mjs
Normal file
220
tests/scanners/claude-md-dead-prose-ref.test.mjs
Normal file
|
|
@ -0,0 +1,220 @@
|
|||
import { describe, it, beforeEach, afterEach } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { resolve, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import {
|
||||
scan,
|
||||
extractInlineSpans,
|
||||
classifyProseReference,
|
||||
resolveProseReference,
|
||||
} from '../../scanners/claude-md-linter.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
const DEAD_REF = resolve(FIXTURES, 'dead-prose-ref');
|
||||
|
||||
const deadRefFindings = (r) => r.findings.filter((f) => f.id === 'CA-CML-013');
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// The silence taxonomy. One assertion per class, each pinning the rule BY NAME.
|
||||
//
|
||||
// A single "produces no findings" assertion would go green for the whole table
|
||||
// while telling us nothing about WHICH class regressed — and two silences with
|
||||
// different causes must never share a fixture (C4 §P5, #61, #62 §5). The four
|
||||
// classes STATE named land on four different rules: `npm test` → whitespace,
|
||||
// `model:` → key-or-flag, `CA-GAP-*` → glob-or-placeholder, a URL → url.
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
describe('CML dead prose references — silence taxonomy (S1: fenced code)', () => {
|
||||
it('S1 — a dead path inside a fence is illustrative, not a reference', () => {
|
||||
const spans = extractInlineSpans([
|
||||
'Prose mentions `docs/live.md`.',
|
||||
'```bash',
|
||||
'echo `docs/fenced.md`',
|
||||
'```',
|
||||
'Prose again mentions `docs/after.md`.',
|
||||
].join('\n'));
|
||||
const texts = spans.map((s) => s.text);
|
||||
assert.deepEqual(texts, ['docs/live.md', 'docs/after.md']);
|
||||
});
|
||||
|
||||
it('S1 — spans carry 1-based line numbers for the finding evidence', () => {
|
||||
const spans = extractInlineSpans('intro\nsee `docs/x.md` here\n');
|
||||
assert.equal(spans.length, 1);
|
||||
assert.equal(spans[0].line, 2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CML dead prose references — silence taxonomy (lexical rules)', () => {
|
||||
const cases = [
|
||||
['S2 whitespace — a command invocation, not a path', 'node scripts/build.mjs', 'whitespace'],
|
||||
['S2 whitespace — the generic-command case STATE named', 'npm test', 'whitespace'],
|
||||
['S3 url — an external resource, existence on disk is meaningless', 'https://example.com/a/b.md', 'url'],
|
||||
['S4 glob — a pattern, resolving to many or to none', 'CA-GAP-*', 'glob-or-placeholder'],
|
||||
['S4 placeholder — unresolved until expanded', '${CLAUDE_PLUGIN_ROOT}/hooks/x.mjs', 'glob-or-placeholder'],
|
||||
['S5 absolute-or-home — outside project scope, machine-dependent', '~/.claude/settings.json', 'absolute-or-home'],
|
||||
['S5 absolute-or-home — a rooted path', '/usr/local/bin/node', 'absolute-or-home'],
|
||||
['S6 key-or-flag — a frontmatter key, not a path', 'model:', 'key-or-flag'],
|
||||
['S6 key-or-flag — a CLI flag', '--output-file', 'key-or-flag'],
|
||||
['S7 no-separator — a bare filename in prose is a concept', 'README.md', 'no-separator'],
|
||||
['S8 ambiguous-slug — a Forgejo org/repo remote', 'ktg/from-ai-to-chitta', 'ambiguous-slug'],
|
||||
['S8 ambiguous-slug — an npm scoped package', '@anthropic-ai/claude-agent-sdk', 'ambiguous-slug'],
|
||||
['S8 ambiguous-slug — a prose enumeration that happens to use slashes', 'known/none/cheap/local', 'ambiguous-slug'],
|
||||
['S10 trailing-locator — a known v1 gap, recorded not hidden', 'docs/plan.md:54-56', 'ambiguous-slug'],
|
||||
['S8b single-segment-directory — a remote namespace prefix', 'open/', 'single-segment-directory'],
|
||||
['S8b single-segment-directory — a bare folder name used as a concept', 'docs/', 'single-segment-directory'],
|
||||
['S8b single-segment-directory — a convention name, not a path here', '_archive/', 'single-segment-directory'],
|
||||
];
|
||||
|
||||
for (const [name, token, rule] of cases) {
|
||||
it(name, () => {
|
||||
assert.equal(classifyProseReference(token).rule, rule, `token: ${token}`);
|
||||
});
|
||||
}
|
||||
|
||||
it('ordering is load-bearing: whitespace wins over no-separator', () => {
|
||||
// `npm test` is silenced by BOTH rules. The reported reason must be the
|
||||
// first match, or the taxonomy stops describing what actually happened.
|
||||
assert.equal(classifyProseReference('npm test').rule, 'whitespace');
|
||||
});
|
||||
|
||||
it('a path-shaped relative token is a CANDIDATE, not silenced', () => {
|
||||
for (const token of ['docs/missing.md', 'scripts/deploy.sh', 'build/artifacts/', './docs/x.md']) {
|
||||
assert.equal(classifyProseReference(token).rule, null, `token: ${token}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('S8b does not swallow a multi-segment directory reference', () => {
|
||||
// The exclusion is about a BARE folder name being a concept, one level up
|
||||
// from the bare-filename rule. A specific path stays a candidate.
|
||||
assert.equal(classifyProseReference('tools/wiki_ingest/').rule, null);
|
||||
assert.equal(classifyProseReference('build/artifacts/').rule, null);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CML dead prose references — silence taxonomy (resolution rules)', () => {
|
||||
let root;
|
||||
beforeEach(async () => {
|
||||
root = await mkdtemp(join(tmpdir(), 'ca-cml-deadref-'));
|
||||
await mkdir(join(root, 'docs'), { recursive: true });
|
||||
await mkdir(join(root, 'nested'), { recursive: true });
|
||||
await writeFile(join(root, 'docs', 'real.md'), '# real\n', 'utf8');
|
||||
});
|
||||
afterEach(async () => {
|
||||
if (root) await rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('S9 outside-scan-tree — a `..` chain that escapes the root is not ours to judge', async () => {
|
||||
// Measured in the C3 corpus sweep: `../../../../etc/passwd` resolved to the
|
||||
// real /etc/passwd and SILENCED the finding by accident. A base a `..` chain
|
||||
// can escape is not a base.
|
||||
const r = await resolveProseReference('../../../../etc/passwd', {
|
||||
fileDir: join(root, 'nested'),
|
||||
scanRoot: root,
|
||||
});
|
||||
assert.equal(r.rule, 'outside-scan-tree');
|
||||
});
|
||||
|
||||
it('S9 does NOT swallow a `..` that stays inside the scan root', async () => {
|
||||
const r = await resolveProseReference('../docs/real.md', {
|
||||
fileDir: join(root, 'nested'),
|
||||
scanRoot: root,
|
||||
});
|
||||
assert.equal(r.rule, 'resolves-own-dir');
|
||||
});
|
||||
|
||||
it('S11 resolves-own-dir — it exists next to the CLAUDE.md', async () => {
|
||||
const r = await resolveProseReference('docs/real.md', { fileDir: root, scanRoot: root });
|
||||
assert.equal(r.rule, 'resolves-own-dir');
|
||||
});
|
||||
|
||||
it('S12 resolves-scan-root — a nested file may write repo-root-relative paths', async () => {
|
||||
const r = await resolveProseReference('docs/real.md', {
|
||||
fileDir: join(root, 'nested'),
|
||||
scanRoot: root,
|
||||
});
|
||||
assert.equal(r.rule, 'resolves-scan-root');
|
||||
});
|
||||
|
||||
it('a reference absent from BOTH bases is dead (rule null)', async () => {
|
||||
const r = await resolveProseReference('docs/missing.md', {
|
||||
fileDir: join(root, 'nested'),
|
||||
scanRoot: root,
|
||||
});
|
||||
assert.equal(r.rule, null);
|
||||
});
|
||||
});
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// Integration — the fixture is the fasit's prediction 1.
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
describe('CML dead prose references — fixture integration', () => {
|
||||
let result;
|
||||
beforeEach(async () => {
|
||||
const discovery = await discoverConfigFiles(DEAD_REF);
|
||||
result = await scan(DEAD_REF, discovery);
|
||||
});
|
||||
|
||||
it('emits exactly ONE finding for the file that has dead references', () => {
|
||||
const found = deadRefFindings(result);
|
||||
assert.equal(found.length, 1,
|
||||
`expected one per-file finding, got: ${found.map((f) => f.evidence).join(' | ')}`);
|
||||
});
|
||||
|
||||
it('is severity low and not auto-fixable', () => {
|
||||
const f = deadRefFindings(result)[0];
|
||||
assert.equal(f.severity, 'low');
|
||||
assert.equal(f.autoFixable, false);
|
||||
});
|
||||
|
||||
it('counts all three dead references and names the first', () => {
|
||||
const f = deadRefFindings(result)[0];
|
||||
const text = `${f.description} ${f.evidence}`;
|
||||
assert.match(text, /\b3\b/, `should carry the count of 3: ${text}`);
|
||||
assert.match(text, /docs\/missing-runbook\.md/, `should name the first dead path: ${text}`);
|
||||
});
|
||||
|
||||
it('names every dead reference, and only those', () => {
|
||||
const f = deadRefFindings(result)[0];
|
||||
const text = `${f.description} ${f.evidence}`;
|
||||
for (const dead of ['docs/missing-runbook.md', 'scripts/deploy.sh', 'build/artifacts/']) {
|
||||
assert.ok(text.includes(dead), `missing dead ref "${dead}" in: ${text}`);
|
||||
}
|
||||
for (const silent of ['docs/real.md', 'ktg/from-ai-to-chitta', 'etc/passwd', 'fenced-and-dead']) {
|
||||
assert.ok(!text.includes(silent), `silent token "${silent}" leaked into: ${text}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('does not fire on the nested CLAUDE.md whose path resolves from the scan root', () => {
|
||||
const nested = deadRefFindings(result).filter((f) => /nested/.test(f.file || ''));
|
||||
assert.equal(nested.length, 0, 'a repo-root-relative path in a nested file must stay silent');
|
||||
});
|
||||
});
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// Non-regression — a new check must not disturb the existing fixtures.
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
describe('CML dead prose references — existing fixtures stay clean', () => {
|
||||
const untouched = [
|
||||
'healthy-project',
|
||||
'broken-project',
|
||||
'large-cascade',
|
||||
'minimal-project',
|
||||
'large-claude-chars',
|
||||
];
|
||||
|
||||
for (const name of untouched) {
|
||||
it(`${name} emits no CA-CML-013`, async () => {
|
||||
const dir = resolve(FIXTURES, name);
|
||||
const discovery = await discoverConfigFiles(dir);
|
||||
const result = await scan(dir, discovery);
|
||||
assert.equal(deadRefFindings(result).length, 0);
|
||||
});
|
||||
}
|
||||
|
||||
it('the untouched list is non-empty (a sweep over an empty list certifies nothing)', () => {
|
||||
assert.ok(untouched.length >= 5);
|
||||
});
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue