feat(scanners): a path written in prose is now resolved, not assumed (C3)

`import-resolver` follows @import targets; a path written in ordinary prose
was checked by nothing. CA-CML-013 resolves those too — one finding per file,
severity low, against both the CLAUDE.md's own directory and the scan root,
because a nested file may legitimately write repo-root-relative paths.

The design work here is the SILENCE list, and every entry on it was measured
against 407 real CLAUDE.md files rather than argued for:

- Bare filenames excluded: admitting them tripled the output (2350 vs 810),
  led by name-drops of tools that exist elsewhere on the machine.
- Org/repo slugs, npm packages, pytest node ids and prose enumerations
  excluded: 111 fires, inspected, all false positives.
- Bare folder names excluded on the same reasoning one level up: 183 of the
  remaining 699 fires (26%), led by `open/` — a Forgejo remote namespace
  prefix, not a directory. This one overturned a premise the fasit had
  asserted without measuring; the deviation is recorded rather than the
  prediction quietly edited.
- Containment is checked against the scan root, not the file's own dir: a
  base a `..` chain can escape is not a base. Measured — without it,
  `../../../../etc/passwd` resolved to the real file and silenced its own
  finding, while a legitimate `../docs/x.md` still resolves.

Rule ORDER is the reported reason (first match wins), so `npm test` is
silenced as a command rather than as a bare token, and two silences with
different causes keep their own fixtures. Twelve classes, pinned by name.

Both load-bearing rules were seen RED against their own defect: deleting
containment fails 1 test, deleting the slug rule fails 6.

Dogfooded through the argv the command template itself constructs, which
found a true positive in our own CLAUDE.md — `lib/humanizer.mjs` where the
file is `scanners/lib/humanizer.mjs`. Fixed here.

Suite 1662 -> 1701, 0 failing. Frozen v5.0.0 and default-output baselines:
0 changed files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HJbfM3N8zWQ1wA2voTrZxz
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 20:11:12 +02:00
commit dbb6a6a3cf
10 changed files with 478 additions and 3 deletions

40
tests/fixtures/dead-prose-ref/CLAUDE.md vendored Normal file
View file

@ -0,0 +1,40 @@
# Dead prose reference fixture
## Project overview
This fixture pins the C3 silence taxonomy. Every token below is here on purpose;
each silence class carries a distinct cause and gets its own token.
## Commands and workflows
Three dead path-shaped references (these, and only these, must fire):
- The runbook lives in `docs/missing-runbook.md`.
- Deploy with the script at `scripts/deploy.sh`.
- Generated output lands in `build/artifacts/`.
A live reference that must stay silent: `docs/real.md`.
## Architecture
One token per silence class:
- whitespace, a command not a path: `node scripts/build.mjs`
- url, existence on disk is meaningless: `https://example.com/a/b.md`
- glob, a pattern not a path: `CA-GAP-*`
- placeholder, unresolved until expanded: `${CLAUDE_PLUGIN_ROOT}/hooks/x.mjs`
- absolute or home, outside project scope: `~/.claude/settings.json`
- key or flag, not a path at all: `model:`
- bare token, a concept not a reference: `README.md`
- ambiguous slug, a Forgejo remote: `ktg/from-ai-to-chitta`
- bare folder name, a concept not a reference: `vendor/`
- escapes the scanned tree: `../../../../etc/passwd`
- trailing locator, a known v1 gap: `docs/plan.md:54-56`
## Conventions and patterns
Fenced code is illustrative, never a reference. The dead path below must stay silent:
```bash
cat docs/fenced-and-dead.md
```

View file

@ -0,0 +1,3 @@
# Real
This file exists so `docs/real.md` resolves and stays silent.

View file

@ -0,0 +1,4 @@
# Nested
Repo-root-relative reference: `docs/real.md` resolves from the scan root,
not from this directory. It must stay silent.

View file

@ -105,6 +105,7 @@ describe('published finding IDs (pinned exhaustively — README is a contract)',
['AGT', 'aggregate-listing-budget', 'CA-AGT-002'],
['CPS', 'volatile-in-prefix', 'CA-CPS-001'],
['COL', 'skill-user-vs-plugin', 'CA-COL-001'],
['CML', 'dead-prose-reference', 'CA-CML-013'],
];
for (const [scanner, key, expected] of PUBLISHED) {

View file

@ -0,0 +1,220 @@
import { describe, it, beforeEach, afterEach } from 'node:test';
import assert from 'node:assert/strict';
import { resolve, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
import {
scan,
extractInlineSpans,
classifyProseReference,
resolveProseReference,
} from '../../scanners/claude-md-linter.mjs';
const __dirname = fileURLToPath(new URL('.', import.meta.url));
const FIXTURES = resolve(__dirname, '../fixtures');
const DEAD_REF = resolve(FIXTURES, 'dead-prose-ref');
const deadRefFindings = (r) => r.findings.filter((f) => f.id === 'CA-CML-013');
// ─────────────────────────────────────────────────────────────────────────────
// The silence taxonomy. One assertion per class, each pinning the rule BY NAME.
//
// A single "produces no findings" assertion would go green for the whole table
// while telling us nothing about WHICH class regressed — and two silences with
// different causes must never share a fixture (C4 §P5, #61, #62 §5). The four
// classes STATE named land on four different rules: `npm test` → whitespace,
// `model:` → key-or-flag, `CA-GAP-*` → glob-or-placeholder, a URL → url.
// ─────────────────────────────────────────────────────────────────────────────
describe('CML dead prose references — silence taxonomy (S1: fenced code)', () => {
it('S1 — a dead path inside a fence is illustrative, not a reference', () => {
const spans = extractInlineSpans([
'Prose mentions `docs/live.md`.',
'```bash',
'echo `docs/fenced.md`',
'```',
'Prose again mentions `docs/after.md`.',
].join('\n'));
const texts = spans.map((s) => s.text);
assert.deepEqual(texts, ['docs/live.md', 'docs/after.md']);
});
it('S1 — spans carry 1-based line numbers for the finding evidence', () => {
const spans = extractInlineSpans('intro\nsee `docs/x.md` here\n');
assert.equal(spans.length, 1);
assert.equal(spans[0].line, 2);
});
});
describe('CML dead prose references — silence taxonomy (lexical rules)', () => {
const cases = [
['S2 whitespace — a command invocation, not a path', 'node scripts/build.mjs', 'whitespace'],
['S2 whitespace — the generic-command case STATE named', 'npm test', 'whitespace'],
['S3 url — an external resource, existence on disk is meaningless', 'https://example.com/a/b.md', 'url'],
['S4 glob — a pattern, resolving to many or to none', 'CA-GAP-*', 'glob-or-placeholder'],
['S4 placeholder — unresolved until expanded', '${CLAUDE_PLUGIN_ROOT}/hooks/x.mjs', 'glob-or-placeholder'],
['S5 absolute-or-home — outside project scope, machine-dependent', '~/.claude/settings.json', 'absolute-or-home'],
['S5 absolute-or-home — a rooted path', '/usr/local/bin/node', 'absolute-or-home'],
['S6 key-or-flag — a frontmatter key, not a path', 'model:', 'key-or-flag'],
['S6 key-or-flag — a CLI flag', '--output-file', 'key-or-flag'],
['S7 no-separator — a bare filename in prose is a concept', 'README.md', 'no-separator'],
['S8 ambiguous-slug — a Forgejo org/repo remote', 'ktg/from-ai-to-chitta', 'ambiguous-slug'],
['S8 ambiguous-slug — an npm scoped package', '@anthropic-ai/claude-agent-sdk', 'ambiguous-slug'],
['S8 ambiguous-slug — a prose enumeration that happens to use slashes', 'known/none/cheap/local', 'ambiguous-slug'],
['S10 trailing-locator — a known v1 gap, recorded not hidden', 'docs/plan.md:54-56', 'ambiguous-slug'],
['S8b single-segment-directory — a remote namespace prefix', 'open/', 'single-segment-directory'],
['S8b single-segment-directory — a bare folder name used as a concept', 'docs/', 'single-segment-directory'],
['S8b single-segment-directory — a convention name, not a path here', '_archive/', 'single-segment-directory'],
];
for (const [name, token, rule] of cases) {
it(name, () => {
assert.equal(classifyProseReference(token).rule, rule, `token: ${token}`);
});
}
it('ordering is load-bearing: whitespace wins over no-separator', () => {
// `npm test` is silenced by BOTH rules. The reported reason must be the
// first match, or the taxonomy stops describing what actually happened.
assert.equal(classifyProseReference('npm test').rule, 'whitespace');
});
it('a path-shaped relative token is a CANDIDATE, not silenced', () => {
for (const token of ['docs/missing.md', 'scripts/deploy.sh', 'build/artifacts/', './docs/x.md']) {
assert.equal(classifyProseReference(token).rule, null, `token: ${token}`);
}
});
it('S8b does not swallow a multi-segment directory reference', () => {
// The exclusion is about a BARE folder name being a concept, one level up
// from the bare-filename rule. A specific path stays a candidate.
assert.equal(classifyProseReference('tools/wiki_ingest/').rule, null);
assert.equal(classifyProseReference('build/artifacts/').rule, null);
});
});
describe('CML dead prose references — silence taxonomy (resolution rules)', () => {
let root;
beforeEach(async () => {
root = await mkdtemp(join(tmpdir(), 'ca-cml-deadref-'));
await mkdir(join(root, 'docs'), { recursive: true });
await mkdir(join(root, 'nested'), { recursive: true });
await writeFile(join(root, 'docs', 'real.md'), '# real\n', 'utf8');
});
afterEach(async () => {
if (root) await rm(root, { recursive: true, force: true });
});
it('S9 outside-scan-tree — a `..` chain that escapes the root is not ours to judge', async () => {
// Measured in the C3 corpus sweep: `../../../../etc/passwd` resolved to the
// real /etc/passwd and SILENCED the finding by accident. A base a `..` chain
// can escape is not a base.
const r = await resolveProseReference('../../../../etc/passwd', {
fileDir: join(root, 'nested'),
scanRoot: root,
});
assert.equal(r.rule, 'outside-scan-tree');
});
it('S9 does NOT swallow a `..` that stays inside the scan root', async () => {
const r = await resolveProseReference('../docs/real.md', {
fileDir: join(root, 'nested'),
scanRoot: root,
});
assert.equal(r.rule, 'resolves-own-dir');
});
it('S11 resolves-own-dir — it exists next to the CLAUDE.md', async () => {
const r = await resolveProseReference('docs/real.md', { fileDir: root, scanRoot: root });
assert.equal(r.rule, 'resolves-own-dir');
});
it('S12 resolves-scan-root — a nested file may write repo-root-relative paths', async () => {
const r = await resolveProseReference('docs/real.md', {
fileDir: join(root, 'nested'),
scanRoot: root,
});
assert.equal(r.rule, 'resolves-scan-root');
});
it('a reference absent from BOTH bases is dead (rule null)', async () => {
const r = await resolveProseReference('docs/missing.md', {
fileDir: join(root, 'nested'),
scanRoot: root,
});
assert.equal(r.rule, null);
});
});
// ─────────────────────────────────────────────────────────────────────────────
// Integration — the fixture is the fasit's prediction 1.
// ─────────────────────────────────────────────────────────────────────────────
describe('CML dead prose references — fixture integration', () => {
let result;
beforeEach(async () => {
const discovery = await discoverConfigFiles(DEAD_REF);
result = await scan(DEAD_REF, discovery);
});
it('emits exactly ONE finding for the file that has dead references', () => {
const found = deadRefFindings(result);
assert.equal(found.length, 1,
`expected one per-file finding, got: ${found.map((f) => f.evidence).join(' | ')}`);
});
it('is severity low and not auto-fixable', () => {
const f = deadRefFindings(result)[0];
assert.equal(f.severity, 'low');
assert.equal(f.autoFixable, false);
});
it('counts all three dead references and names the first', () => {
const f = deadRefFindings(result)[0];
const text = `${f.description} ${f.evidence}`;
assert.match(text, /\b3\b/, `should carry the count of 3: ${text}`);
assert.match(text, /docs\/missing-runbook\.md/, `should name the first dead path: ${text}`);
});
it('names every dead reference, and only those', () => {
const f = deadRefFindings(result)[0];
const text = `${f.description} ${f.evidence}`;
for (const dead of ['docs/missing-runbook.md', 'scripts/deploy.sh', 'build/artifacts/']) {
assert.ok(text.includes(dead), `missing dead ref "${dead}" in: ${text}`);
}
for (const silent of ['docs/real.md', 'ktg/from-ai-to-chitta', 'etc/passwd', 'fenced-and-dead']) {
assert.ok(!text.includes(silent), `silent token "${silent}" leaked into: ${text}`);
}
});
it('does not fire on the nested CLAUDE.md whose path resolves from the scan root', () => {
const nested = deadRefFindings(result).filter((f) => /nested/.test(f.file || ''));
assert.equal(nested.length, 0, 'a repo-root-relative path in a nested file must stay silent');
});
});
// ─────────────────────────────────────────────────────────────────────────────
// Non-regression — a new check must not disturb the existing fixtures.
// ─────────────────────────────────────────────────────────────────────────────
describe('CML dead prose references — existing fixtures stay clean', () => {
const untouched = [
'healthy-project',
'broken-project',
'large-cascade',
'minimal-project',
'large-claude-chars',
];
for (const name of untouched) {
it(`${name} emits no CA-CML-013`, async () => {
const dir = resolve(FIXTURES, name);
const discovery = await discoverConfigFiles(dir);
const result = await scan(dir, discovery);
assert.equal(deadRefFindings(result).length, 0);
});
}
it('the untouched list is non-empty (a sweep over an empty list certifies nothing)', () => {
assert.ok(untouched.length >= 5);
});
});