Kjell Tore Guttormsen
83d82f121d
The cosmetic close came back one level up. The probe reads the test file from the same tree it measures, so a four-line file holding two EMPTY tests with the two named names closed D-03 and D-04 on a tree where `lib/verification/` did not exist at all - "defects 0 of 7, registry intact" (measured 2026-09-18). First a two-line stub exporting the right symbols; then an empty test with the right name. A name is always forgeable. So the named test is now run twice. Once on the tree, as before - and once in a sandbox where the module the condition declares in `stubs` is replaced by a stub exporting the same names, all inert. If the test still passes there, it binds the name and not the behaviour, and the condition THROWS: NOT FELLABLE, counted open. A condition that declares no `stubs`, or names a module that is not there, cannot fire either. The sandbox is a symlink overlay: every entry of the tree is symlinked, and only the test file and the stubbed module are materialised for real - Node resolves an ESM import through the realpath, so a symlinked test file would import the original module and never see the mutant. Nothing is ever written inside the measured tree, and the only directory removed is the one this code made under the system temp dir (pinned by a test). M7 is now a permanent mutant beside M6, in two forms: the checkpoint's own reproduction (unfixed tree + empty named tests) and the harder one (the real module present, so the stub can be built and the empty test passes against it). Both report `defects 2 of 7`. A positive control pins that D-03/D-04 still CLOSE on the real tree, so "not closed" everywhere cannot read as a working probe. The frozen denominator moves a third time, deliberately, and its `why` no longer claims authority it does not have: the second and third amendments were maintenance decisions by the maintainer, not operator decisions, and the tracked file now says exactly that. Measured after: real tree node scripts/end-state-gate.mjs -> defects 0 of 7, intact, exit 1 M6 (stub)8d1669e+ current gate/registry/frozen + lib/cosmetic/stub.mjs -> 2 of 7 M7 (empty)8d1669e+ current gate/registry/frozen + 3-line tests/lib/criteria-runner.test.mjs with the two named tests -> 2 of 7 Red first: 4 of the new tests failed before the change. Suite 1154 (1152/0/2). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
950 lines
43 KiB
JavaScript
950 lines
43 KiB
JavaScript
// tests/scripts/end-state-gate.test.mjs
|
|
// The end-state gate counts the distance to "Voyage is finished" as five tallies:
|
|
// open pipeline defects, unresolved experiments/opt-ins, dormant agents, open operator
|
|
// decisions, and feat commits after the freeze tag. "Finished" = all five at 0 AND the
|
|
// registry the gate counts from is intact against its frozen denominator.
|
|
//
|
|
// These tests pin the gate's MECHANICS in both directions — every tally can be 0 and can
|
|
// be > 0 — against throwaway fixture trees. They deliberately do NOT assert that the real
|
|
// repo is red today: the gate is supposed to go green, and a test that pins the current
|
|
// distance would fail on the day the work is done. What they DO pin about the real repo
|
|
// is the denominator itself (see "the frozen denominator" below).
|
|
|
|
import { test } from 'node:test';
|
|
import { strict as assert } from 'node:assert';
|
|
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, existsSync, readFileSync } from 'node:fs';
|
|
import { tmpdir } from 'node:os';
|
|
import { join, dirname, resolve } from 'node:path';
|
|
import { spawnSync } from 'node:child_process';
|
|
import { fileURLToPath } from 'node:url';
|
|
import {
|
|
evaluateCondition,
|
|
evaluateCheck,
|
|
measure,
|
|
loadRegistry,
|
|
loadFrozen,
|
|
spawnedNames,
|
|
checkSignature,
|
|
verifyIntegrity,
|
|
render,
|
|
PROBE_NOTES,
|
|
} from '../../scripts/end-state-gate.mjs';
|
|
|
|
const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
|
const GATE = join(ROOT, 'scripts', 'end-state-gate.mjs');
|
|
|
|
// --- the frozen denominator ------------------------------------------------
|
|
// Denominator frozen 2026-09-17 by the end-state work order, the day the end-state
|
|
// direction was chosen. An entry may CLOSE; it may never be removed or have its check
|
|
// changed, and the gate's agent/decision/freeze configuration may not move. To change
|
|
// anything below you must change this test — say in the commit message who decided it.
|
|
const FROZEN = {
|
|
defects: {
|
|
'D-01': '48f7f5cca070091e8f9b9ca0c7cdb7a1880733691935c25d07ea1d3e8d508fc9',
|
|
'D-02': 'dfc94dba04a9116ae9be2097a0c5a9d8313b02c7de1ca6b49a472c1823a6d1e1',
|
|
// Amended 2026-09-18 (operator decision): D-03/D-04 became behaviour probes.
|
|
// Amended a second time the same day, as a maintenance decision: those
|
|
// behaviour probes were greps for an export name, and a two-line stub
|
|
// exporting both names closed them on an unfixed tree (mutant M6, pinned
|
|
// below). They now RUN a named test.
|
|
// Amended a third time the same day, same standing: a named test can be
|
|
// EMPTY, and two empty tests with the right names closed both defects on a
|
|
// tree without lib/verification/ (mutant M7, pinned below). Each condition
|
|
// now also names the module whose STUB its test must fell. Every amendment
|
|
// moved a signature on purpose and removed nothing.
|
|
'D-03': '98698ce74c243353713786b8604160f6672568b090884e9428d9e5e9fbcad89d',
|
|
'D-04': 'df11a2e95db7b014ce077db0473e5dde4195ace92d8077cacdc1cee273947b47',
|
|
'D-05': '8fe6df52b43496c3302400507cb98002dcf8d2a12d92085e14bda4c7819bc7ba',
|
|
'D-06': '72a787c8154d1c18849e9856dcf1a67703d6b5541f88db81d6bab13cab14b418',
|
|
'D-07': '1bf3e0cd36c621720697475b0a3fd7db78611d66b58a9ce035e7695e57c69f1b',
|
|
},
|
|
experiments: {
|
|
'E-01': 'f7718897cec8ace6879fe79f5f32760ec957c3642e7186f8aa1688ce77b3f4e8',
|
|
'E-02': 'c8ff12c504ba7a9ac36f11bb8f9b713263e9b4eb28c07d984d9e2115980e740d',
|
|
'E-03': '17153ad8808615a833a49ae0c3523264ce055aa690e5542a75bae1874495e7e8',
|
|
},
|
|
agents: {
|
|
dir: 'agents',
|
|
spawnSites: 'commands/*.md',
|
|
referenceMarker: 'Reference document, not a spawnable capability',
|
|
},
|
|
decisions: {
|
|
file: 'STATE.md',
|
|
section: '## Åpne operatørbeslutninger',
|
|
open: '^- \\[ \\] ',
|
|
closed: '^- \\[x\\] ',
|
|
},
|
|
freeze: { tag: 'end-state-freeze', featPattern: '^feat(\\(|!|:)' },
|
|
};
|
|
|
|
// --- helpers ---------------------------------------------------------------
|
|
|
|
function fixture(files) {
|
|
const dir = mkdtempSync(join(tmpdir(), 'end-state-gate-'));
|
|
for (const [rel, body] of Object.entries(files)) {
|
|
const p = join(dir, rel);
|
|
mkdirSync(dirname(p), { recursive: true });
|
|
writeFileSync(p, body);
|
|
}
|
|
return dir;
|
|
}
|
|
|
|
const cleanup = (...dirs) => { for (const d of dirs) rmSync(d, { recursive: true, force: true }); };
|
|
|
|
const DECISIONS = { file: 'STATE.md', section: '## Open decisions', open: '^- \\[ \\] ', closed: '^- \\[x\\] ' };
|
|
const AGENTS = { dir: 'agents', spawnSites: 'commands/*.md', referenceMarker: 'Reference document, not a spawnable capability' };
|
|
const FREEZE = { tag: 'end-state-freeze', featPattern: '^feat(\\(|!|:)' };
|
|
|
|
function registry(overrides = {}) {
|
|
return { defects: [], experiments: [], agents: AGENTS, decisions: DECISIONS, freeze: FREEZE, ...overrides };
|
|
}
|
|
|
|
// A frozen manifest that matches `reg` exactly.
|
|
function frozenFor(reg) {
|
|
const sigs = (list) => Object.fromEntries(list.map((e) => [e.id, checkSignature(e.check)]));
|
|
return { defects: sigs(reg.defects), experiments: sigs(reg.experiments), agents: reg.agents, decisions: reg.decisions, freeze: reg.freeze };
|
|
}
|
|
|
|
function git(dir, ...args) {
|
|
const r = spawnSync('git', [
|
|
'-c', 'user.name=fixture', '-c', 'user.email=fixture@example.invalid',
|
|
'-c', 'core.hooksPath=/dev/null', '-c', 'commit.gpgsign=false', '-c', 'tag.gpgsign=false',
|
|
...args,
|
|
], { cwd: dir, encoding: 'utf8' });
|
|
assert.equal(r.status, 0, `git ${args.join(' ')}: ${r.stderr}`);
|
|
return r.stdout;
|
|
}
|
|
|
|
function gitInit(dir) {
|
|
git(dir, 'init', '-q');
|
|
git(dir, 'add', '-A');
|
|
git(dir, 'commit', '-q', '--allow-empty', '-m', 'chore: fixture baseline');
|
|
}
|
|
|
|
function gitCommit(dir, subject) {
|
|
git(dir, 'commit', '-q', '--allow-empty', '-m', subject);
|
|
}
|
|
|
|
const rowById = (result, id) => result.rows.find((r) => r.id === id);
|
|
|
|
// --- conditions -----------------------------------------------------------
|
|
|
|
test('evaluateCondition: match / no-match on a single file, multiline regex', () => {
|
|
const dir = fixture({ 'a.md': 'intro\n### Heading\nbody\n' });
|
|
try {
|
|
assert.equal(evaluateCondition(dir, { path: 'a.md', pattern: '^### Heading', flags: 'm', expect: 'match' }), true);
|
|
assert.equal(evaluateCondition(dir, { path: 'a.md', pattern: '^### Other', flags: 'm', expect: 'match' }), false);
|
|
assert.equal(evaluateCondition(dir, { path: 'a.md', pattern: '^### Other', flags: 'm', expect: 'no-match' }), true);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('evaluateCondition: globs (dir/*.ext, dir/**/*.ext) and path arrays match ANY file', () => {
|
|
const dir = fixture({
|
|
'commands/x.md': 'nothing here\n',
|
|
'lib/deep/y.mjs': 'const FLAG = process.env.SOME_FLAG;\n',
|
|
});
|
|
try {
|
|
assert.equal(evaluateCondition(dir, { path: 'commands/*.md', pattern: 'SOME_FLAG', expect: 'match' }), false);
|
|
assert.equal(evaluateCondition(dir, { path: 'lib/**/*.mjs', pattern: 'SOME_FLAG', expect: 'match' }), true);
|
|
assert.equal(evaluateCondition(dir, { path: ['commands/*.md', 'lib/**/*.mjs'], pattern: 'SOME_FLAG', expect: 'match' }), true);
|
|
assert.equal(evaluateCondition(dir, { path: ['commands/*.md', 'lib/**/*.mjs'], pattern: 'SOME_FLAG', expect: 'no-match' }), false);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('evaluateCondition: section scoping reads only from the heading to the next "## "', () => {
|
|
const dir = fixture({
|
|
'doc.md': '## 5. Design\nno status here\n## 6. PoC\n> **STATUS: RUN AND DECLINED**\n',
|
|
});
|
|
try {
|
|
const inFive = { path: 'doc.md', section: '## 5.', pattern: '^> \\*\\*STATUS:', flags: 'm', expect: 'no-match' };
|
|
const inSix = { ...inFive, section: '## 6.' };
|
|
assert.equal(evaluateCondition(dir, inFive), true, 'section 5 has no status block');
|
|
assert.equal(evaluateCondition(dir, inSix), false, 'section 6 has one');
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('evaluateCondition: a literal NUL byte is detectable', () => {
|
|
const dir = fixture({ 'nul.mjs': "const SEP = '\u0000';\n", 'clean.mjs': "const SEP = '\\x00';\n" });
|
|
try {
|
|
assert.equal(evaluateCondition(dir, { path: 'nul.mjs', pattern: '\\u0000', expect: 'match' }), true);
|
|
assert.equal(evaluateCondition(dir, { path: 'clean.mjs', pattern: '\\u0000', expect: 'match' }), false);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('evaluateCondition: a missing file, a missing glob base or a missing section throws', () => {
|
|
const dir = fixture({ 'a.md': 'x\n' });
|
|
try {
|
|
assert.throws(() => evaluateCondition(dir, { path: 'missing.md', pattern: 'x', expect: 'no-match' }), /missing\.md/);
|
|
assert.throws(() => evaluateCondition(dir, { path: 'nodir/*.md', pattern: 'x', expect: 'no-match' }), /nodir/);
|
|
assert.throws(() => evaluateCondition(dir, { path: 'a.md', section: '## 9.', pattern: 'x', expect: 'no-match' }), /## 9\./);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('evaluateCondition: a glob over an EXISTING dir with no matching files throws (no silent no-match)', () => {
|
|
// In a no-match condition, "silently nothing matched" would read as the condition holding.
|
|
const dir = fixture({ 'docs/readme.txt': 'x\n' });
|
|
try {
|
|
assert.throws(() => evaluateCondition(dir, { path: 'docs/*.md', pattern: 'x', expect: 'no-match' }), /docs\/\*\.md/);
|
|
assert.throws(() => evaluateCondition(dir, { path: 'docs/**/*.md', pattern: 'x', expect: 'no-match' }), /docs\/\*\*\/\*\.md/);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('evaluateCheck: open only when ALL conditions hold; null check is not fellable', () => {
|
|
const dir = fixture({
|
|
'r.md': 'run the `### Bridge agent` block\n',
|
|
'fixed.md': 'run the `### Bridge agent` block\n### Bridge agent\n',
|
|
});
|
|
const dangling = (file) => [
|
|
{ path: file, pattern: '`### Bridge agent`', expect: 'match' },
|
|
{ path: file, pattern: '^### Bridge agent', flags: 'm', expect: 'no-match' },
|
|
];
|
|
try {
|
|
assert.equal(evaluateCheck(dir, dangling('r.md')).status, 'open');
|
|
assert.equal(evaluateCheck(dir, dangling('fixed.md')).status, 'closed');
|
|
assert.equal(evaluateCheck(dir, null).status, 'not-fellable');
|
|
const broken = evaluateCheck(dir, dangling('gone.md'));
|
|
assert.equal(broken.status, 'not-fellable');
|
|
assert.match(broken.detail, /gone\.md/);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
// --- spawn instructions (what "referenced by a command" means) -------------
|
|
|
|
test('spawnedNames: a row of a table headed `Agent` is a spawn instruction; other tables are not', () => {
|
|
const md = [
|
|
'Launch the new agents:', '',
|
|
'| Agent | Purpose |', '|-------|---------|', '| `alpha-agent` | does a |', '| `beta-agent` | does b |', '',
|
|
'| Name | Purpose |', '|------|---------|', '| `gamma-agent` | just a list |', '',
|
|
].join('\n');
|
|
const names = spawnedNames(md);
|
|
assert.ok(names.has('alpha-agent') && names.has('beta-agent'));
|
|
assert.ok(!names.has('gamma-agent'), 'a table not headed "Agent" is not a roster');
|
|
});
|
|
|
|
test('spawnedNames: an imperative Launch/Spawn line names the agents it launches', () => {
|
|
const md = [
|
|
'Launch the **alpha-agent** agent:',
|
|
'- Launch `beta-agent` (Agent tool) with the findings.',
|
|
'3. Spawn `voyage:gamma-agent` in the background.',
|
|
'**Launch** the `delta-agent` agent now.',
|
|
'launch `epsilon-agent` too',
|
|
].join('\n');
|
|
const names = spawnedNames(md);
|
|
for (const n of ['alpha-agent', 'beta-agent', 'gamma-agent', 'delta-agent', 'epsilon-agent']) {
|
|
assert.ok(names.has(n), `${n} should count as spawned`);
|
|
}
|
|
});
|
|
|
|
test('spawnedNames: an agent block with a Prompt: line is a spawn instruction; a bare bold name is not', () => {
|
|
const md = [
|
|
'**alpha-agent** — adversarial review of the plan.',
|
|
'Prompt: "Review this plan."',
|
|
'',
|
|
'**beta-agent** — mentioned for context only.',
|
|
'It is not launched here.',
|
|
].join('\n');
|
|
const names = spawnedNames(md);
|
|
assert.ok(names.has('alpha-agent'));
|
|
assert.ok(!names.has('beta-agent'));
|
|
});
|
|
|
|
test('spawnedNames: prose, negation, comments, fences and quotes are never spawn instructions', () => {
|
|
const md = [
|
|
'The alpha-agent is dormant and is never spawned.',
|
|
'Never launch `beta-agent`.',
|
|
'In quick mode we used to launch `gamma-agent`.',
|
|
'<!-- Launch the `delta-agent` agent -->',
|
|
'<!--',
|
|
'Launch the `epsilon-agent` agent',
|
|
'-->',
|
|
'```',
|
|
'Launch the `zeta-agent` agent',
|
|
'| Agent |',
|
|
'|---|',
|
|
'| `eta-agent` |',
|
|
'```',
|
|
'> Launch the `theta-agent` agent',
|
|
].join('\n');
|
|
const names = spawnedNames(md);
|
|
for (const n of ['alpha-agent', 'beta-agent', 'gamma-agent', 'delta-agent', 'epsilon-agent', 'zeta-agent', 'eta-agent', 'theta-agent']) {
|
|
assert.ok(!names.has(n), `${n} must not count as spawned`);
|
|
}
|
|
});
|
|
|
|
// --- the tallies, both directions ------------------------------------------
|
|
|
|
test('defects tally: > 0 while a check holds, 0 once fixed; not-fellable counts as open', () => {
|
|
const defect = {
|
|
id: 'D-X',
|
|
summary: 'removed tool still referenced',
|
|
probe: 'phrase',
|
|
check: [{ path: 'commands/plan.md', pattern: '\\bTeamCreate\\b', expect: 'match' }],
|
|
};
|
|
const unfellable = { id: 'D-Y', summary: 'no runnable check yet', check: null };
|
|
const base = { 'STATE.md': '## Open decisions\n', 'agents/a.md': 'x\n', 'commands/plan.md': 'Launch `a`\n' };
|
|
|
|
const red = fixture({ ...base, 'commands/plan.md': 'Launch `a` and TeamCreate\n' });
|
|
const green = fixture(base);
|
|
try {
|
|
let row = rowById(measure(red, registry({ defects: [defect] })), 'defects');
|
|
assert.equal(row.open, 1);
|
|
assert.equal(row.total, 1);
|
|
row = rowById(measure(green, registry({ defects: [defect] })), 'defects');
|
|
assert.equal(row.open, 0);
|
|
row = rowById(measure(green, registry({ defects: [defect, unfellable] })), 'defects');
|
|
assert.equal(row.open, 1, 'a defect without a runnable check stays open');
|
|
assert.equal(row.items.find((i) => i.id === 'D-Y').status, 'not-fellable');
|
|
} finally { cleanup(red, green); }
|
|
});
|
|
|
|
test('experiments tally: an env-gated opt-in is open until its gate is gone', () => {
|
|
const exp = {
|
|
id: 'E-X',
|
|
summary: 'default-off loop',
|
|
probe: 'phrase',
|
|
check: [{ path: ['commands/*.md', 'lib/**/*.mjs'], pattern: 'LOOP_ENABLED', expect: 'match' }],
|
|
};
|
|
const base = { 'STATE.md': '## Open decisions\n', 'agents/a.md': 'x\n', 'commands/r.md': 'Launch `a`\n' };
|
|
const red = fixture({ ...base, 'lib/util/cap.mjs': 'if (process.env.LOOP_ENABLED) {}\n' });
|
|
const green = fixture({ ...base, 'lib/util/cap.mjs': 'export const x = 1;\n' });
|
|
try {
|
|
assert.equal(rowById(measure(red, registry({ experiments: [exp] })), 'experiments').open, 1);
|
|
assert.equal(rowById(measure(green, registry({ experiments: [exp] })), 'experiments').open, 0);
|
|
} finally { cleanup(red, green); }
|
|
});
|
|
|
|
test('dormant-agents tally: an agent no command spawns counts; reference docs never do', () => {
|
|
const files = {
|
|
'STATE.md': '## Open decisions\n',
|
|
'agents/used-agent.md': '---\nname: used-agent\ndescription: |\n does work\n---\n',
|
|
'agents/idle-agent.md': '---\nname: idle-agent\ndescription: |\n never wired\n---\n',
|
|
'agents/some-orchestrator.md': '---\nname: some-orchestrator\ndescription: Reference document, not a spawnable capability — docs\n---\n',
|
|
'commands/run.md': 'Launch the **used-agent** agent.\n',
|
|
};
|
|
const red = fixture(files);
|
|
const green = fixture({ ...files, 'commands/other.md': 'Spawn `idle-agent` in parallel.\n' });
|
|
try {
|
|
let row = rowById(measure(red, registry()), 'dormant-agents');
|
|
assert.equal(row.open, 1);
|
|
assert.equal(row.total, 2, 'denominator = spawnable agents (reference docs excluded)');
|
|
assert.deepEqual(row.items.filter((i) => i.status === 'open').map((i) => i.id), ['idle-agent']);
|
|
row = rowById(measure(green, registry()), 'dormant-agents');
|
|
assert.equal(row.open, 0);
|
|
} finally { cleanup(red, green); }
|
|
});
|
|
|
|
test('dormant-agents tally: a prose sentence or an HTML comment does not wake a dormant agent', () => {
|
|
const files = {
|
|
'STATE.md': '## Open decisions\n',
|
|
'agents/idle-agent.md': '---\nname: idle-agent\ndescription: |\n never wired\n---\n',
|
|
'commands/plan.md': 'The idle-agent is dormant and is never spawned.\n',
|
|
'commands/end.md': '<!-- idle-agent -->\nThe research-idle-agents phase.\n',
|
|
};
|
|
const dir = fixture(files);
|
|
try {
|
|
assert.equal(rowById(measure(dir, registry()), 'dormant-agents').open, 1);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('dormant-agents tally: an unmeasurable agents dir is n/a (null), never 0', () => {
|
|
const dir = fixture({ 'STATE.md': '## Open decisions\n', 'commands/r.md': 'x\n' });
|
|
try {
|
|
const result = measure(dir, registry());
|
|
const row = rowById(result, 'dormant-agents');
|
|
assert.equal(row.open, null);
|
|
assert.equal(row.total, null);
|
|
assert.match(row.detail, /not measurable/);
|
|
assert.equal(result.green, false);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('decisions tally: counts open markers inside the section only, and says it counts ticks', () => {
|
|
const state = [
|
|
'# STATE',
|
|
'- [ ] outside the section, ignored',
|
|
'## Open decisions',
|
|
'- [ ] first open',
|
|
'- [x] already decided',
|
|
'- [ ] second open',
|
|
'## Next section',
|
|
'- [ ] also ignored',
|
|
'',
|
|
].join('\n');
|
|
const red = fixture({ 'STATE.md': state, 'agents/a.md': 'x\n', 'commands/r.md': 'Launch `a`\n' });
|
|
const green = fixture({
|
|
'STATE.md': '## Open decisions\n- [x] done\n## Next\n',
|
|
'agents/a.md': 'x\n',
|
|
'commands/r.md': 'Launch `a`\n',
|
|
});
|
|
try {
|
|
let row = rowById(measure(red, registry()), 'decisions');
|
|
assert.equal(row.open, 2);
|
|
assert.equal(row.total, 3);
|
|
assert.match(row.detail, /tick/i, 'the output must say a tick is not verified as a decision');
|
|
row = rowById(measure(green, registry()), 'decisions');
|
|
assert.equal(row.open, 0);
|
|
assert.equal(row.total, 1);
|
|
} finally { cleanup(red, green); }
|
|
});
|
|
|
|
test('decisions tally: a missing STATE.md or section is NOT MEASURABLE (null), never 0', () => {
|
|
const noState = fixture({ 'agents/a.md': 'x\n', 'commands/r.md': 'Launch `a`\n' });
|
|
const noSection = fixture({ 'STATE.md': '# STATE\n- [ ] x\n', 'agents/a.md': 'x\n', 'commands/r.md': 'Launch `a`\n' });
|
|
try {
|
|
for (const dir of [noState, noSection]) {
|
|
const result = measure(dir, registry());
|
|
const row = rowById(result, 'decisions');
|
|
assert.equal(row.open, null);
|
|
assert.ok(row.detail.length > 0, 'the reason is reported');
|
|
assert.equal(result.green, false, 'an unmeasurable tally keeps the gate red');
|
|
}
|
|
} finally { cleanup(noState, noSection); }
|
|
});
|
|
|
|
test('decisions tally: an unmarked list item in the section is format drift → NOT MEASURABLE', () => {
|
|
// A STATE written without the marker convention must not read as "0 open".
|
|
for (const item of ['1. **Some decision?** prose', '- plain bullet decision', '* star bullet']) {
|
|
const dir = fixture({
|
|
'STATE.md': `## Open decisions\n- [ ] marked\n${item}\n## Next\n`,
|
|
'agents/a.md': 'x\n',
|
|
'commands/r.md': 'Launch `a`\n',
|
|
});
|
|
try {
|
|
const row = rowById(measure(dir, registry()), 'decisions');
|
|
assert.equal(row.open, null, `unmarked "${item}" must make the tally unmeasurable`);
|
|
assert.match(row.detail, /unmarked/);
|
|
} finally { cleanup(dir); }
|
|
}
|
|
});
|
|
|
|
test('feat-after-freeze tally: not a git repo → n/a (red), never 0', () => {
|
|
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
|
try {
|
|
const result = measure(dir, registry());
|
|
const row = rowById(result, 'feat-after-freeze');
|
|
assert.equal(row.open, null);
|
|
assert.match(row.detail, /not measured/);
|
|
assert.equal(result.green, false);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('feat-after-freeze tally: no freeze tag yet → n/a (red), and the detail names the tag', () => {
|
|
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
|
try {
|
|
gitInit(dir);
|
|
const row = rowById(measure(dir, registry()), 'feat-after-freeze');
|
|
assert.equal(row.open, null);
|
|
assert.match(row.detail, /end-state-freeze/);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('feat-after-freeze tally: counts feat commits after the tag, 0 when there are none', () => {
|
|
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
|
try {
|
|
gitInit(dir);
|
|
gitCommit(dir, 'feat: before the freeze does not count');
|
|
git(dir, 'tag', 'end-state-freeze');
|
|
let row = rowById(measure(dir, registry()), 'feat-after-freeze');
|
|
assert.equal(row.open, 0);
|
|
assert.equal(row.total, 0);
|
|
gitCommit(dir, 'fix(x): a defect fix is allowed');
|
|
gitCommit(dir, 'feature: not a conventional feat');
|
|
row = rowById(measure(dir, registry()), 'feat-after-freeze');
|
|
assert.equal(row.open, 0);
|
|
assert.equal(row.total, 2);
|
|
gitCommit(dir, 'feat(research): a new capability');
|
|
gitCommit(dir, 'feat!: a breaking capability');
|
|
row = rowById(measure(dir, registry()), 'feat-after-freeze');
|
|
assert.equal(row.open, 2);
|
|
assert.equal(row.total, 4);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
// --- registry integrity (the judged party must not own the ledger) ---------
|
|
|
|
const SAMPLE = registry({
|
|
defects: [
|
|
{ id: 'D-01', summary: 'first defect', probe: 'phrase', check: [{ path: 'a.md', pattern: 'bad', expect: 'match' }] },
|
|
{ id: 'D-02', summary: 'second defect', probe: 'phrase', check: [{ path: 'a.md', pattern: 'worse', expect: 'match' }] },
|
|
],
|
|
experiments: [
|
|
{ id: 'E-01', summary: 'first experiment', probe: 'phrase', check: [{ path: 'a.md', pattern: 'FLAG', expect: 'match' }] },
|
|
{ id: 'E-02', summary: 'second experiment', probe: 'phrase', check: [{ path: 'a.md', pattern: 'OPT', expect: 'match' }] },
|
|
],
|
|
});
|
|
const clone = (v) => JSON.parse(JSON.stringify(v));
|
|
|
|
test('verifyIntegrity: the frozen registry itself is intact; a new entry may be added', () => {
|
|
const frozen = frozenFor(SAMPLE);
|
|
assert.deepEqual(verifyIntegrity(SAMPLE, frozen), { ok: true, violations: [] });
|
|
const grown = clone(SAMPLE);
|
|
grown.defects.push({ id: 'D-03', summary: 'found later', probe: 'phrase', check: [{ path: 'a.md', pattern: 'new', expect: 'match' }] });
|
|
assert.equal(verifyIntegrity(grown, frozen).ok, true);
|
|
});
|
|
|
|
test('verifyIntegrity: every registry attack is a violation that names what moved', () => {
|
|
const frozen = frozenFor(SAMPLE);
|
|
const attacks = {
|
|
'delete a defect': (r) => { r.defects = r.defects.filter((d) => d.id !== 'D-02'); },
|
|
'delete an experiment': (r) => { r.experiments = r.experiments.filter((e) => e.id !== 'E-01'); },
|
|
'weaken a defect check': (r) => { r.defects[0].check[0].pattern = '^NEVER MATCHES$'; },
|
|
'weaken an experiment check': (r) => { r.experiments[1].check[0].expect = 'no-match'; },
|
|
'drop a condition': (r) => { r.defects[0].check = []; },
|
|
'point spawnSites at agents': (r) => { r.agents = { ...r.agents, spawnSites: 'agents/*.md' }; },
|
|
'change the reference marker': (r) => { r.agents = { ...r.agents, referenceMarker: 'DORMANT' }; },
|
|
'move the decisions section': (r) => { r.decisions = { ...r.decisions, section: '## Elsewhere' }; },
|
|
'change the open marker': (r) => { r.decisions = { ...r.decisions, open: '^NEVER' }; },
|
|
'rename the freeze tag': (r) => { r.freeze = { ...r.freeze, tag: 'some-future-tag' }; },
|
|
'duplicate an id': (r) => { r.defects.push(clone(r.defects[0])); },
|
|
};
|
|
for (const [name, attack] of Object.entries(attacks)) {
|
|
const reg = clone(SAMPLE);
|
|
attack(reg);
|
|
const v = verifyIntegrity(reg, frozen);
|
|
assert.equal(v.ok, false, `${name} must be caught`);
|
|
assert.ok(v.violations.length > 0 && v.violations.every((s) => typeof s === 'string' && s.length > 0), name);
|
|
}
|
|
});
|
|
|
|
test('verifyIntegrity: spawnSites may never include agents/, even if the manifest says so', () => {
|
|
const reg = clone(SAMPLE);
|
|
reg.agents = { ...reg.agents, spawnSites: 'agents/*.md' };
|
|
const v = verifyIntegrity(reg, frozenFor(reg));
|
|
assert.equal(v.ok, false);
|
|
assert.ok(v.violations.some((s) => /agents\//.test(s)));
|
|
});
|
|
|
|
test('checkSignature: key order does not matter; every field of a condition does', () => {
|
|
const a = [{ path: 'a.md', pattern: 'x', flags: 'm', expect: 'match' }];
|
|
const reordered = [{ expect: 'match', flags: 'm', pattern: 'x', path: 'a.md' }];
|
|
assert.equal(checkSignature(a), checkSignature(reordered));
|
|
for (const change of [{ path: 'b.md' }, { pattern: 'y' }, { flags: '' }, { expect: 'no-match' }, { section: '## 1.' }]) {
|
|
assert.notEqual(checkSignature([{ ...a[0], ...change }]), checkSignature(a), JSON.stringify(change));
|
|
}
|
|
assert.match(checkSignature(a), /^[0-9a-f]{64}$/);
|
|
});
|
|
|
|
test('measure: a missing frozen manifest makes integrity n/a and keeps the gate red', () => {
|
|
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
|
try {
|
|
const result = measure(dir, registry());
|
|
assert.equal(result.integrity.ok, null);
|
|
assert.match(result.integrity.detail, /end-state-frozen\.json/);
|
|
assert.equal(result.green, false);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
// --- render ----------------------------------------------------------------
|
|
|
|
test('render: the header counts only tallies that ARE 0, never n/a ones', () => {
|
|
const result = {
|
|
green: false,
|
|
integrity: { ok: true, violations: [], detail: '' },
|
|
rows: [
|
|
{ id: 'defects', label: 'l', source: 's', open: 0, total: 1, target: 0, items: [], detail: '' },
|
|
{ id: 'experiments', label: 'l', source: 's', open: null, total: null, target: 0, items: [], detail: 'not measurable: x' },
|
|
{ id: 'dormant-agents', label: 'l', source: 's', open: 0, total: 3, target: 0, items: [], detail: '' },
|
|
],
|
|
};
|
|
const text = render(result);
|
|
assert.match(text, /RED \(2 of 3 tallies at 0\)/);
|
|
assert.match(text, /\| experiments \| n\/a \| n\/a \|/);
|
|
});
|
|
|
|
test('render: open phrase probes are labelled, closed ids are listed, integrity is shown', () => {
|
|
const result = {
|
|
green: false,
|
|
integrity: { ok: false, violations: ['D-02 removed from the registry'], detail: '' },
|
|
rows: [{
|
|
id: 'defects', label: 'open pipeline defects', source: 's', open: 1, total: 2, target: 0, detail: '',
|
|
items: [
|
|
{ id: 'D-01', summary: 'still open', probe: 'phrase', status: 'open', detail: '' },
|
|
{ id: 'D-06', summary: 'fixed', probe: 'byte', status: 'closed', detail: '' },
|
|
],
|
|
}],
|
|
};
|
|
const text = render(result);
|
|
assert.match(text, /\[open · phrase probe\] D-01/);
|
|
assert.match(text, /closed: D-06/);
|
|
assert.match(text, /registry integrity: VIOLATED/);
|
|
assert.match(text, /D-02 removed from the registry/);
|
|
});
|
|
|
|
// --- CLI -------------------------------------------------------------------
|
|
|
|
function writeGateFiles(dir, reg, frozen = frozenFor(reg)) {
|
|
mkdirSync(join(dir, 'scripts'), { recursive: true });
|
|
writeFileSync(join(dir, 'scripts', 'end-state-registry.json'), JSON.stringify(reg, null, 2));
|
|
mkdirSync(join(dir, 'tests', 'fixtures'), { recursive: true });
|
|
writeFileSync(join(dir, 'tests', 'fixtures', 'end-state-frozen.json'), JSON.stringify(frozen, null, 2));
|
|
}
|
|
|
|
function runGate(dir, extra = []) {
|
|
return spawnSync(process.execPath, [GATE, '--root', dir, ...extra], { encoding: 'utf8' });
|
|
}
|
|
|
|
const ALL_IDS = ['defects', 'experiments', 'dormant-agents', 'decisions', 'feat-after-freeze'];
|
|
|
|
test('CLI: all five tallies at 0 with an intact registry → green, exit 0; --json carries the rows', () => {
|
|
const dir = fixture({
|
|
'STATE.md': '## Open decisions\n- [x] done\n',
|
|
'agents/a.md': '---\nname: a\n---\n',
|
|
'commands/r.md': 'Spawn `a`\n',
|
|
});
|
|
try {
|
|
writeGateFiles(dir, registry());
|
|
gitInit(dir);
|
|
git(dir, 'tag', 'end-state-freeze');
|
|
const r = runGate(dir, ['--json']);
|
|
assert.equal(r.status, 0, r.stdout + r.stderr);
|
|
const out = JSON.parse(r.stdout);
|
|
assert.equal(out.green, true);
|
|
assert.equal(out.integrity.ok, true);
|
|
assert.deepEqual(out.rows.map((x) => x.id), ALL_IDS);
|
|
assert.ok(out.rows.every((x) => x.open === 0 && x.target === 0 && typeof x.source === 'string'));
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('CLI: an intact registry with every measurable tally at 0 but no freeze tag → red, exit 1', () => {
|
|
// n/a must never be read as 0 — also when nothing else is wrong.
|
|
const dir = fixture({
|
|
'STATE.md': '## Open decisions\n- [x] done\n',
|
|
'agents/a.md': '---\nname: a\n---\n',
|
|
'commands/r.md': 'Spawn `a`\n',
|
|
});
|
|
try {
|
|
writeGateFiles(dir, registry());
|
|
gitInit(dir);
|
|
const r = runGate(dir, ['--json']);
|
|
assert.equal(r.status, 1, r.stdout + r.stderr);
|
|
const out = JSON.parse(r.stdout);
|
|
assert.equal(out.integrity.ok, true);
|
|
assert.equal(out.green, false);
|
|
assert.equal(out.rows.find((x) => x.id === 'feat-after-freeze').open, null);
|
|
assert.ok(out.rows.filter((x) => x.id !== 'feat-after-freeze').every((x) => x.open === 0));
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('CLI: all tallies at 0 but a tampered registry → red, exit 1 (the combined green attack)', () => {
|
|
const dir = fixture({
|
|
'STATE.md': '## Open decisions\n- [x] ticked\n',
|
|
'agents/a.md': '---\nname: a\n---\n',
|
|
'commands/r.md': 'Spawn `a`\n',
|
|
});
|
|
try {
|
|
const honest = registry({ defects: [{ id: 'D-01', summary: 'a real defect', probe: 'phrase', check: [{ path: 'commands/r.md', pattern: 'Spawn', expect: 'match' }] }] });
|
|
const tampered = registry(); // the open defect simply left the ledger
|
|
writeGateFiles(dir, tampered, frozenFor(honest));
|
|
gitInit(dir);
|
|
git(dir, 'tag', 'end-state-freeze');
|
|
const r = runGate(dir);
|
|
assert.equal(r.status, 1, r.stdout + r.stderr);
|
|
assert.match(r.stdout, /RED/);
|
|
assert.match(r.stdout, /VIOLATED/);
|
|
assert.match(r.stdout, /D-01/);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('CLI: any tally > 0 → red, exit 1; the table names every row', () => {
|
|
const dir = fixture({
|
|
'STATE.md': '## Open decisions\n- [ ] undecided\n',
|
|
'agents/a.md': '---\nname: a\n---\n',
|
|
'commands/r.md': 'Spawn `a`\n',
|
|
});
|
|
try {
|
|
writeGateFiles(dir, registry());
|
|
const r = runGate(dir);
|
|
assert.equal(r.status, 1, r.stderr);
|
|
for (const label of ALL_IDS) assert.match(r.stdout, new RegExp(label));
|
|
assert.match(r.stdout, /RED/);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('CLI: an unreadable registry is a usage error (exit 2), not a verdict', () => {
|
|
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
|
try {
|
|
const r = runGate(dir);
|
|
assert.equal(r.status, 2);
|
|
assert.match(r.stderr, /end-state-registry\.json/);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
// --- the real repo: denominator pinned, state never pinned -----------------
|
|
|
|
test('the frozen denominator: tests/fixtures/end-state-frozen.json is exactly the literal above', () => {
|
|
const file = loadFrozen(ROOT);
|
|
const { why, ...rest } = file;
|
|
assert.ok(typeof why === 'string' && why.length > 20, 'the manifest says why it is frozen');
|
|
assert.deepEqual(rest, FROZEN);
|
|
});
|
|
|
|
test('the frozen denominator: the real registry is intact against it', () => {
|
|
const v = verifyIntegrity(loadRegistry(ROOT), FROZEN);
|
|
assert.deepEqual(v, { ok: true, violations: [] }, v.violations.join('\n'));
|
|
});
|
|
|
|
test('real registry: every counted entry has an id, a summary, a probe kind and a check that RUNS here', () => {
|
|
const reg = loadRegistry(ROOT);
|
|
const entries = [...reg.defects, ...reg.experiments];
|
|
const ids = new Set();
|
|
for (const e of entries) {
|
|
assert.match(e.id, /^[DE]-\d{2}$/, `bad id ${e.id}`);
|
|
assert.ok(!ids.has(e.id), `duplicate id ${e.id}`);
|
|
ids.add(e.id);
|
|
assert.ok(typeof e.summary === 'string' && e.summary.length > 10, `${e.id} needs a summary`);
|
|
assert.ok(Object.keys(PROBE_NOTES).includes(e.probe), `${e.id} must declare a probe kind the gate can explain`);
|
|
// A check that cannot evaluate against this very repo is a typo, not a finding.
|
|
assert.notEqual(evaluateCheck(ROOT, e.check).status, 'not-fellable', `${e.id} check does not run here`);
|
|
}
|
|
});
|
|
|
|
test('real repo: five rows, open is a count or null, exit code agrees with rows + integrity', () => {
|
|
const r = spawnSync(process.execPath, [GATE, '--json'], { encoding: 'utf8', cwd: ROOT });
|
|
assert.ok(r.status === 0 || r.status === 1, r.stderr);
|
|
const out = JSON.parse(r.stdout);
|
|
assert.deepEqual(out.rows.map((x) => x.id), ALL_IDS);
|
|
for (const row of out.rows) {
|
|
assert.ok(row.open === null || (Number.isInteger(row.open) && row.open >= 0), `${row.id}: ${row.open}`);
|
|
}
|
|
const green = out.rows.every((row) => row.open === 0) && out.integrity.ok === true;
|
|
assert.equal(out.green, green);
|
|
assert.equal(r.status, green ? 0 : 1);
|
|
assert.ok(existsSync(join(ROOT, 'scripts', 'end-state-registry.json')));
|
|
assert.ok(readFileSync(join(ROOT, 'tests', 'fixtures', 'end-state-frozen.json'), 'utf8').length > 0);
|
|
});
|
|
|
|
// --- probe labelling -------------------------------------------------------
|
|
//
|
|
// The gate labelled only `phrase` probes, so every other probe kind rendered
|
|
// unlabelled — indistinguishable from an entry that declared nothing. Once
|
|
// D-03/D-04 became behaviour probes (operator decision 2026-09-18) the reader
|
|
// needs to see WHICH kind of evidence each closed entry rests on.
|
|
|
|
test('PROBE_NOTES: every probe kind the registry may declare carries a note', () => {
|
|
assert.deepEqual(Object.keys(PROBE_NOTES).sort(), ['behaviour', 'byte', 'phrase']);
|
|
for (const [kind, note] of Object.entries(PROBE_NOTES)) {
|
|
assert.ok(note.length > 20, `${kind} needs a note that says what the probe can and cannot prove`);
|
|
}
|
|
});
|
|
|
|
test('render: every probe kind is labelled, open and closed — not just phrase', () => {
|
|
const text = render({
|
|
green: false,
|
|
integrity: { ok: true, violations: [], detail: '' },
|
|
rows: [{
|
|
id: 'defects', label: 'open pipeline defects', source: 's', open: 2, total: 3, target: 0, detail: '',
|
|
items: [
|
|
{ id: 'D-01', summary: 'still open', probe: 'phrase', status: 'open', detail: '' },
|
|
{ id: 'D-03', summary: 'still open', probe: 'behaviour', status: 'open', detail: '' },
|
|
{ id: 'D-06', summary: 'fixed', probe: 'byte', status: 'closed', detail: '' },
|
|
],
|
|
}],
|
|
});
|
|
assert.match(text, /\[open · phrase probe\] D-01/);
|
|
assert.match(text, /\[open · behaviour probe\] D-03/);
|
|
assert.match(text, /closed: D-06 \(byte probe\)/);
|
|
});
|
|
|
|
test('the detail line carries one note per probe kind PRESENT, and none for kinds absent', () => {
|
|
const dir = fixture({
|
|
'a.md': 'bad\n',
|
|
'STATE.md': '## Open decisions\n- [x] done\n',
|
|
'agents/a.md': '---\nname: a\n---\n',
|
|
'commands/r.md': 'Spawn `a`\n',
|
|
});
|
|
try {
|
|
writeGateFiles(dir, registry({
|
|
defects: [
|
|
{ id: 'D-01', summary: 'a phrase defect', probe: 'phrase', check: [{ path: 'a.md', pattern: 'bad', expect: 'match' }] },
|
|
{ id: 'D-02', summary: 'a behaviour defect', probe: 'behaviour', check: [{ path: 'a.md', pattern: 'nope', expect: 'match' }] },
|
|
],
|
|
}));
|
|
gitInit(dir);
|
|
const out = JSON.parse(runGate(dir, ['--json']).stdout);
|
|
const detail = rowById(out, 'defects').detail;
|
|
assert.match(detail, /phrase probe closes on rewording/);
|
|
assert.match(detail, /behaviour probe/);
|
|
assert.ok(!/byte probe/.test(detail), 'a kind no entry declares must not be explained');
|
|
assert.equal(rowById(out, 'experiments').detail, '', 'an empty row explains nothing');
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
// --- a probe that RUNS a test, and the M6 mutant ---------------------------
|
|
//
|
|
// Measured 2026-09-18 (PM checkpoint on 6cafb4c): a two-line file
|
|
// export function runPlanVerification() { return { ok: true }; }
|
|
// export function formatCriteriaEvidence() { return ''; }
|
|
// dropped into an otherwise UNFIXED tree closed both D-03 and D-04, and the
|
|
// gate reported "defects 0 of 7, registry intact". A grep condition can only
|
|
// ask whether a string is present, so a behaviour probe built from one is a
|
|
// phrase probe pointed at code. A condition that RUNS a named test and reads
|
|
// its TAP result cannot be closed that way.
|
|
|
|
const M6_STUB = [
|
|
'export function runPlanVerification() { return { ok: true }; }',
|
|
"export function formatCriteriaEvidence() { return ''; }",
|
|
'',
|
|
].join('\n');
|
|
|
|
// A fixture pair: a module, and a test that BINDS it. `green one` passes on
|
|
// the tree and fails against a stub of the module - which is what makes it a
|
|
// behaviour probe rather than a name.
|
|
const MODULE = 'export function answer() { return 42; }\n';
|
|
|
|
const TEST_FILE = [
|
|
"import { test } from 'node:test';",
|
|
"import { strict as assert } from 'node:assert';",
|
|
"import { answer } from '../lib/m.mjs';",
|
|
"test('green one', () => { assert.equal(answer(), 42); });",
|
|
"test('red one', () => { assert.ok(false, 'red on purpose'); });",
|
|
'',
|
|
].join('\n');
|
|
|
|
test('a test condition RUNS the named test and reads its result', () => {
|
|
const dir = fixture({ 'tests/t.test.mjs': TEST_FILE, 'lib/m.mjs': MODULE });
|
|
try {
|
|
const cond = (name, expect) => ({ test: 'tests/t.test.mjs', name, expect, stubs: ['lib/m.mjs'] });
|
|
assert.equal(evaluateCondition(dir, cond('green one', 'passes')), true);
|
|
assert.equal(evaluateCondition(dir, cond('green one', 'fails')), false);
|
|
assert.equal(evaluateCondition(dir, cond('red one', 'passes')), false);
|
|
assert.equal(evaluateCondition(dir, cond('red one', 'fails')), true);
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('a test condition naming a test that does not run is NOT FELLABLE, never closed', () => {
|
|
const dir = fixture({ 'tests/t.test.mjs': TEST_FILE, 'lib/m.mjs': MODULE });
|
|
try {
|
|
for (const cond of [
|
|
{ test: 'tests/t.test.mjs', name: 'no such test', expect: 'fails', stubs: ['lib/m.mjs'] },
|
|
{ test: 'tests/missing.test.mjs', name: 'green one', expect: 'fails', stubs: ['lib/m.mjs'] },
|
|
]) {
|
|
const r = evaluateCheck(dir, [cond]);
|
|
assert.equal(r.status, 'not-fellable', JSON.stringify(cond));
|
|
assert.ok(r.detail.length > 0, 'the gate must say WHY the check could not fire');
|
|
}
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('M6 mutant: a stub exporting both symbols does NOT close D-03 or D-04', () => {
|
|
const dir = fixture({ 'lib/cosmetic/stub.mjs': M6_STUB });
|
|
try {
|
|
const entries = loadRegistry(ROOT).defects.filter((e) => e.id === 'D-03' || e.id === 'D-04');
|
|
assert.equal(entries.length, 2, 'D-03 and D-04 are still in the registry');
|
|
for (const e of entries) {
|
|
assert.notEqual(
|
|
evaluateCheck(dir, e.check).status, 'closed',
|
|
`${e.id} closed on a tree where nothing is fixed: ${JSON.stringify(e.check)}`,
|
|
);
|
|
}
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('the gate states out loud that the wiring is pinned by text, not proven', () => {
|
|
const r = spawnSync(process.execPath, [GATE], { encoding: 'utf8', cwd: ROOT });
|
|
assert.ok(r.status === 0 || r.status === 1, r.stderr);
|
|
assert.match(r.stdout, /wiring: /);
|
|
assert.match(r.stdout, /pinned by TEXT/);
|
|
assert.match(r.stdout, /plugin-eval/);
|
|
});
|
|
|
|
// --- M7: a probe must bind the test's CONTENT, not its name ---------------
|
|
//
|
|
// Measured 2026-09-18 (PM checkpoint on e1e7bdf): the cosmetic close came back
|
|
// one level up. The probe reads the test file from the SAME tree it measures,
|
|
// so a four-line file holding two EMPTY tests with the two named names —
|
|
// test('runPlanVerification: a plan whose success criterion FAILS fells the run', () => {});
|
|
// test('formatCriteriaEvidence: one row per criterion, with command and exit code', () => {});
|
|
// — closed D-03 and D-04 on a tree where `lib/verification/` did not exist at
|
|
// all: "defects 0 of 7, registry intact". An empty test is the new two-line
|
|
// stub. The only form that cannot be closed by a name is one that requires the
|
|
// named test to FELL a mutant: the gate runs it a second time against a tree
|
|
// where the module it binds is replaced by a stub exporting the same names and
|
|
// doing nothing. A test that passes against both is not a behaviour probe.
|
|
|
|
const EMPTY_NAMED_TESTS = [
|
|
"import test from 'node:test';",
|
|
"test('runPlanVerification: a plan whose success criterion FAILS fells the run', () => {});",
|
|
"test('formatCriteriaEvidence: one row per criterion, with command and exit code', () => {});",
|
|
'',
|
|
].join('\n');
|
|
|
|
const REAL_RUNNER = join(ROOT, 'lib', 'verification', 'criteria-runner.mjs');
|
|
const REAL_RUNNER_TEST = join(ROOT, 'tests', 'lib', 'criteria-runner.test.mjs');
|
|
|
|
test('M7 mutant: two EMPTY tests with the right names do NOT close D-03 or D-04', () => {
|
|
// The checkpoint's reproduction: an unfixed tree (no lib/verification/) plus
|
|
// the four-line test file.
|
|
const dir = fixture({ 'tests/lib/criteria-runner.test.mjs': EMPTY_NAMED_TESTS });
|
|
try {
|
|
const entries = loadRegistry(ROOT).defects.filter((e) => e.id === 'D-03' || e.id === 'D-04');
|
|
assert.equal(entries.length, 2, 'D-03 and D-04 are still in the registry');
|
|
for (const e of entries) {
|
|
const r = evaluateCheck(dir, e.check);
|
|
assert.equal(r.status, 'not-fellable', `${e.id} closed on a tree where nothing is fixed`);
|
|
assert.ok(r.detail.length > 0, 'the gate must say WHY the check could not fire');
|
|
}
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('M7 mutant: an empty named test does not close the defect even when the module IS present', () => {
|
|
// The harder case: the real module is there, so the stub CAN be built — and
|
|
// the empty test passes against it, which is exactly the proof that the test
|
|
// binds the name and nothing else.
|
|
const dir = fixture({
|
|
'tests/lib/criteria-runner.test.mjs': EMPTY_NAMED_TESTS,
|
|
'lib/verification/criteria-runner.mjs': readFileSync(REAL_RUNNER, 'utf8'),
|
|
});
|
|
try {
|
|
for (const e of loadRegistry(ROOT).defects.filter((x) => x.id === 'D-03' || x.id === 'D-04')) {
|
|
const r = evaluateCheck(dir, e.check);
|
|
assert.equal(r.status, 'not-fellable', `${e.id} closed on an empty test`);
|
|
assert.match(r.detail, /stub/i, 'the gate must name the mutant the test failed to fell');
|
|
}
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test("D-03/D-04 still CLOSE on the real tree — the named tests do fell a stub", () => {
|
|
// The positive control. Without it, "not closed" everywhere would read as a
|
|
// working probe when the probe is simply broken.
|
|
for (const e of loadRegistry(ROOT).defects.filter((x) => x.id === 'D-03' || x.id === 'D-04')) {
|
|
const r = evaluateCheck(ROOT, e.check);
|
|
assert.equal(r.status, 'closed', `${e.id} is not closed on the real tree: ${r.detail}`);
|
|
}
|
|
});
|
|
|
|
test('a test condition must name the module its test binds, or it cannot fire', () => {
|
|
const dir = fixture({ 'tests/t.test.mjs': TEST_FILE, 'lib/m.mjs': MODULE });
|
|
try {
|
|
for (const cond of [
|
|
{ test: 'tests/t.test.mjs', name: 'green one', expect: 'passes' },
|
|
{ test: 'tests/t.test.mjs', name: 'green one', expect: 'passes', stubs: [] },
|
|
{ test: 'tests/t.test.mjs', name: 'green one', expect: 'passes', stubs: ['lib/gone.mjs'] },
|
|
]) {
|
|
const r = evaluateCheck(dir, [cond]);
|
|
assert.equal(r.status, 'not-fellable', JSON.stringify(cond));
|
|
assert.ok(r.detail.length > 0, 'the gate must say WHY');
|
|
}
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('the stub mutant leaves the measured tree untouched', () => {
|
|
const dir = fixture({ 'tests/t.test.mjs': TEST_FILE, 'lib/m.mjs': MODULE });
|
|
try {
|
|
const before = readFileSync(join(dir, 'lib', 'm.mjs'), 'utf8');
|
|
evaluateCheck(dir, [{ test: 'tests/t.test.mjs', name: 'green one', expect: 'passes', stubs: ['lib/m.mjs'] }]);
|
|
assert.equal(readFileSync(join(dir, 'lib', 'm.mjs'), 'utf8'), before, 'the mutation must happen in a copy, never in place');
|
|
} finally { cleanup(dir); }
|
|
});
|
|
|
|
test('the frozen file claims no authority it does not have', () => {
|
|
// MINOR from the same checkpoint: the tracked frozen file said the second
|
|
// amendment was "operator-authorised by work order". No operator authorised
|
|
// it; it followed from a maintenance decision. A file whose whole job is to
|
|
// be an explicit decision trail may not overclaim in either direction.
|
|
const why = loadFrozen(ROOT).why;
|
|
assert.ok(why.length > 0, 'the frozen manifest must say WHY it changed');
|
|
assert.ok(
|
|
!/operator-authoris|operator-authoriz|by work order|PM checkpoint/i.test(why),
|
|
'the frozen file may not claim operator authorisation, and coordination metadata belongs in the local plan',
|
|
);
|
|
});
|