Kjell Tore Guttormsen
494a344700
Closes devils-advocate audit §Top changes #8 (MINOR×MED). Reports the behavior-test count separately from the doc-consistency prose-pin count so the cited total no longer oversells behavior coverage, and removes the clearest reword-fragile redundancies. Conservative scope (operator-chosen): no genuine regression guard dropped. 1. Suite census (report separately). lib/util/test-census.mjs + tests/lib/test-census.test.mjs walk tests/**/*.test.mjs and bucket top-level test() declarations into behavior vs doc-consistency-pins (bucket per file, regex PIN_FILE_RE), asserting the two sum to the total so neither can drift silently. Split emitted as a t.diagnostic: behavior=601 doc-consistency-pins=69. Metric = top-level declarations; node:test's runtime total counts subtests too and is therefore >= it. 2. Conservative prune (3 tests, each provably subsumed): - "trekexecute.md still parses v1.7 plan schema" — tautological OR-chain; real coverage = the plan_version:1.7 template pin + plan-validator / plan-schema behavior tests. - "CLAUDE.md mentions all six pipeline commands" — hardcoded six-string list subsumed by the filesystem-driven "commands table mentions every commands/*.md file" structural pin. - "CLAUDE.md mentions /trekcontinue command" — same subsumption. Each removal leaves an in-place note (why + where coverage lives). 3. doc-consistency.test.mjs header now documents the structural-invariant vs prose/existence-pin distinction so new pins land in the right kind. TDD: census test written failing-first (ERR_MODULE_NOT_FOUND before lib/util/test-census.mjs existed). Suite 699->698 (696 pass / 2 skip / 0 fail): -3 prune +2 census. plugin validate passes (1 accepted warning). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
43 lines
2.1 KiB
JavaScript
43 lines
2.1 KiB
JavaScript
// tests/lib/test-census.test.mjs
|
|
// Behavior test for the suite census + the honest-count invariant from the
|
|
// devil's-advocate audit §Top changes #8 (S19): the behavior-test count must
|
|
// be reported SEPARATELY from the doc-consistency prose-pin count. A prose-pin
|
|
// (a string/existence assertion against documentation) is not the same coverage
|
|
// as a behavior test, so a single conflated total oversells the behavior
|
|
// coverage. This census makes the split explicit and drift-proof.
|
|
|
|
import { test } from 'node:test';
|
|
import { strict as assert } from 'node:assert';
|
|
import { dirname, join } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
import { censusTests } from '../../lib/util/test-census.mjs';
|
|
|
|
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
const TESTS_ROOT = join(HERE, '..');
|
|
|
|
test('suite census splits behavior tests from doc-consistency pins (S19)', (t) => {
|
|
const c = censusTests(TESTS_ROOT);
|
|
// Honest-count invariant: the two buckets must account for every top-level
|
|
// test() declaration — no silent drift between behavior and pin counts.
|
|
assert.equal(c.behavior + c.docPins, c.total,
|
|
'census buckets must sum to the total declaration count');
|
|
assert.ok(c.docPins > 0, 'doc-consistency pin bucket must be non-empty (regex/glob sanity)');
|
|
assert.ok(c.behavior > 0, 'behavior bucket must be non-empty (regex/glob sanity)');
|
|
// Report the split so the cited count is honest (audit §Top changes #8).
|
|
// Metric = top-level test() declarations; node:test's runtime total counts
|
|
// subtests too and is therefore ≥ this number.
|
|
t.diagnostic(
|
|
`behavior=${c.behavior} doc-consistency-pins=${c.docPins} ` +
|
|
`total=${c.total} (top-level test() declarations)`,
|
|
);
|
|
});
|
|
|
|
test("doc-consistency.test.mjs is the suite's prose-pin home (S19)", () => {
|
|
const c = censusTests(TESTS_ROOT);
|
|
const pinFiles = Object.keys(c.byFile)
|
|
.filter((f) => /(doc-consistency|prose-pins)\.test\.mjs$/.test(f));
|
|
assert.ok(
|
|
pinFiles.some((f) => f.endsWith('doc-consistency.test.mjs')),
|
|
'doc-consistency.test.mjs must exist as the canonical prose-pin bucket',
|
|
);
|
|
});
|