docs(scoring): unify scan/audit/mcp-scanner/posture-assessor to v2 formula

Closes the v7.1.1 out-of-scope item: commands/scan.md:113-114 retained
the v1 formula. Exploration found two more v1 surfaces that v7.1.1
missed: commands/audit.md:46 and agents/mcp-scanner-agent.md:419, plus
agents/posture-assessor-agent.md:376 (caught by the new doc-consistency
test). Four files unified to v2 in one atomic commit.

Three-way → four-way verdict-divergence is now closed:
- scanners/lib/severity.mjs (v2, BLOCK ≥65, WARNING ≥15) — authoritative
- agents/skill-scanner-agent.md (v2 since v7.1.1)
- templates/unified-report.md (v2 since v7.1.1)
- commands/scan.md (v2 — this commit)
- commands/audit.md (v2 — this commit)
- agents/mcp-scanner-agent.md (v2 — this commit)
- agents/posture-assessor-agent.md (v2 — this commit)

New: tests/lib/doc-consistency.test.mjs walks commands/ + agents/ and
asserts NO file contains v1 formula tokens. Pinned regex set:
  - score >= 61, score >= 21, score ≥ 61, score ≥ 21
  - critical * 25, Critical × 25
  - min(100, critical*25 ...)

Plus three v2-cutoff anchors asserting commands/scan.md, commands/audit.md,
and agents/mcp-scanner-agent.md document the v2 BLOCK ≥65 cutoff (or
reference riskScore() helper).

Tests: 1523 → 1551 (+28 from doc-consistency: 25 file walks + 3 anchors).
All green.
This commit is contained in:
Kjell Tore Guttormsen 2026-04-29 13:58:25 +02:00
commit d3b1157a08
5 changed files with 92 additions and 6 deletions

View file

@ -0,0 +1,86 @@
// doc-consistency.test.mjs — Static asserts that prose documentation
// stays aligned with the v2 risk-scoring model in scanners/lib/severity.mjs.
//
// Background: v7.0.0 introduced the severity-dominated v2 risk-score model
// (BLOCK ≥65, WARNING ≥15) but several prose surfaces (commands/, agents/)
// continued to emit the v1 formula (`critical*25 + ...`, BLOCK ≥61,
// WARNING ≥21). v7.1.1 fixed two of them (agents/skill-scanner-agent.md,
// templates/unified-report.md). Batch B → v7.2.0 closes the trifecta:
// commands/scan.md, commands/audit.md, agents/mcp-scanner-agent.md.
//
// This test pins the closure. If any future edit re-introduces v1 formula
// tokens in commands/ or agents/, this test fails fast.
import { describe, it } from 'node:test';
import assert from 'node:assert/strict';
import { readdirSync, readFileSync, statSync } from 'node:fs';
import { join, dirname, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
const __filename = fileURLToPath(import.meta.url);
const PLUGIN_ROOT = resolve(dirname(__filename), '..', '..');
// v1 formula tokens that must NOT appear in commands/ or agents/.
// These are the patterns the brief's verification step 4 grep checks.
const V1_TOKENS = [
/\bscore\s*[><]?=\s*61\b/, // verdict cutoff
/\bscore\s*[><]?=\s*21\b/, // verdict cutoff
/score\s*≥\s*61/, // unicode variant
/score\s*≥\s*21/, // unicode variant
/critical\s*\*\s*25/, // formula multiplier
/Critical\s*[×x]\s*25/, // formula multiplier (table form)
/min\(\s*100\s*,\s*critical\s*\*\s*25/i, // full v1 formula prefix
];
function* walkMarkdown(dir) {
for (const entry of readdirSync(dir)) {
const full = join(dir, entry);
const stat = statSync(full);
if (stat.isDirectory()) {
yield* walkMarkdown(full);
} else if (entry.endsWith('.md')) {
yield full;
}
}
}
describe('doc-consistency — v1 risk-formula tokens are absent from prose', () => {
const COMMANDS_DIR = join(PLUGIN_ROOT, 'commands');
const AGENTS_DIR = join(PLUGIN_ROOT, 'agents');
for (const dir of [COMMANDS_DIR, AGENTS_DIR]) {
for (const file of walkMarkdown(dir)) {
const rel = file.replace(PLUGIN_ROOT + '/', '');
it(`${rel} contains no v1 formula tokens`, () => {
const content = readFileSync(file, 'utf-8');
for (const token of V1_TOKENS) {
assert.equal(
token.test(content),
false,
`${rel} still contains v1 formula token matching ${token}. ` +
`v7.2.0 unified all command/agent prose to v2 (BLOCK ≥65, WARNING ≥15). ` +
`If a new file legitimately needs to reference v1 (e.g. CHANGELOG history), ` +
`move that file out of commands/ or agents/.`,
);
}
});
}
}
});
describe('doc-consistency — v2 cutoffs are documented in unified prose', () => {
it('commands/scan.md mentions the v2 BLOCK cutoff (≥ 65)', () => {
const content = readFileSync(join(PLUGIN_ROOT, 'commands', 'scan.md'), 'utf-8');
assert.match(content, /score\s*[≥>=]+\s*65/);
});
it('commands/audit.md references riskScore() (v2 helper)', () => {
const content = readFileSync(join(PLUGIN_ROOT, 'commands', 'audit.md'), 'utf-8');
assert.match(content, /riskScore/);
});
it('agents/mcp-scanner-agent.md mentions the v2 BLOCK cutoff (≥ 65)', () => {
const content = readFileSync(join(PLUGIN_ROOT, 'agents', 'mcp-scanner-agent.md'), 'utf-8');
assert.match(content, /score\s*[≥>=]+\s*65/);
});
});