fix(end-state): the gate can no longer be turned green by editing its own ledger
An independent adversarial review of the end-state gate reproduced a GREEN result, exit 0, with the gate's 18 tests passing. Only the registry and the STATE markers were edited; every other file stayed byte-identical. The judged party owned the denominator, and nothing guarded it. This change closes that finding and three more, and reopens a defect the gate had closed too early. Registry integrity (the blocker): - tests/fixtures/end-state-frozen.json freezes the denominator. It holds the 7 defect ids and 3 experiment ids, each with a sha256 signature of its check (canonical JSON, key order irrelevant), plus the exact agents / decisions / freeze configuration. - The rule: an entry may close; it may never disappear or have its check changed. New entries are allowed. spawnSites may never include agents/. - The test file pins the manifest literally, with a comment that the denominator was frozen on 2026-09-17 and that changing it is a decision. The gate verifies the registry against the manifest on every run. - A violation makes the gate red and names what moved. A missing manifest makes integrity n/a, which is also red. Dormant means "never spawned", not "never mentioned": - An agent counts as spawned only when a command names it in a spawn instruction: a row of a table headed `Agent`, a line that starts with Launch/Spawn, or a **name** block followed by a Prompt: line. - Prose, negations, HTML comments and fenced code never count. Block quotes cannot match either, because every form is anchored at the start of the line. - Re-measured under this definition: still 1 of 20 (synthesis-agent). Per agent, every other spawnable agent has a spawn instruction in at least one command. A fifth tally, feat-after-freeze: - It counts commits after the `end-state-freeze` tag whose subject matches ^feat(\(|!|:). - The row is n/a, and therefore red, when there is no tag yet, the root is not a git work tree, or the root is not the top of its work tree. It is never 0. Honest output: - Every registry entry now declares its probe kind (phrase or byte). - Open phrase probes are labelled as such, and closed ids are listed. The output states that a closed phrase probe is evidence, not proof of behaviour. D-03 and D-04 stay open and labelled; how they are fixed is still to be decided. - The decisions row now says it counts ticks, not verified decisions. D-07 reopened: - Its check now also covers README.md and commands/trekresearch.md, where the "orchestrator runs on opus" claim still lives. README even ships a sed recipe for `model: opus` lines no command has. - Defects are 3 of 7 until the separate docs fix lands. Three surviving review mutants were killed with tests: - an empty glob over an existing dir is silent; - the header counts n/a rows as zero; - a dormant-row error reads as 0. Measured on a clean export of the index: - Mutation harness, adapted from the review's: review code mutants 18 of 18 killed. Four of them were re-targeted at the equivalent new code. M02 survived until a test with an intact registry plus a missing freeze tag was added. - New-logic mutants: 12 of 12 killed. One equivalent mutant (block-quote stripping) was removed together with the dead clause it targeted. - Registry attacks: 7 of 7 killed. - The review's combined green attack (registry + STATE only) is now RED, exit 1: "defects: D-03 is frozen but missing from the registry". - Gate tests 38/38. Suite 1059 -> 1079 (1077/0/2). - Gate: RED, defects 3 of 7, experiments 3 of 3, dormant 1 of 20; decisions and feat-after-freeze are n/a in a clean export. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
e43153c3c4
commit
09feb415b3
4 changed files with 762 additions and 144 deletions
|
|
@ -1,27 +1,32 @@
|
|||
#!/usr/bin/env node
|
||||
// scripts/end-state-gate.mjs
|
||||
// End-state gate — counts the distance to "Voyage is finished" as four tallies:
|
||||
// End-state gate — counts the distance to "Voyage is finished" as five tallies:
|
||||
//
|
||||
// defects open pipeline defects (listed in the registry, each with a runnable check)
|
||||
// experiments unresolved experiments/opt-ins (listed in the registry, each with a runnable check)
|
||||
// dormant-agents spawnable agents no command references (measured structurally from agents/)
|
||||
// dormant-agents spawnable agents no command spawns (measured structurally, see spawnedNames)
|
||||
// decisions open operator decisions (counted in STATE.md by a fixed marker)
|
||||
// feat-after-freeze feat commits after the freeze tag (measured from git)
|
||||
//
|
||||
// "Finished" = all four at 0. Exit 0 = green, 1 = red (any tally > 0 or not measurable),
|
||||
// 2 = usage or registry error. The denominators come from the tracked registry and the
|
||||
// tracked agents/ directory — never from free-text greps over prose.
|
||||
// "Finished" = all five at 0 AND the registry is intact against its frozen denominator
|
||||
// (tests/fixtures/end-state-frozen.json: an entry may close, never disappear or change its
|
||||
// check). Exit 0 = green, 1 = red (a tally > 0, a tally not measurable, or the registry
|
||||
// tampered with), 2 = usage or registry error.
|
||||
//
|
||||
// A registry entry is OPEN while ALL of its conditions hold. An entry whose check is null,
|
||||
// or whose check cannot run (missing file, empty glob, missing section), is NOT FELLABLE
|
||||
// and is counted as open — a check that cannot fire must never read as "fixed".
|
||||
// or whose check cannot run (missing file, empty glob, missing section), is NOT FELLABLE and
|
||||
// is counted as open — a check that cannot fire must never read as "fixed".
|
||||
//
|
||||
// Usage: node scripts/end-state-gate.mjs [--json] [--root <dir>]
|
||||
|
||||
import { readFileSync, existsSync, readdirSync, statSync } from 'node:fs';
|
||||
import { readFileSync, existsSync, readdirSync, statSync, realpathSync } from 'node:fs';
|
||||
import { join, resolve, dirname, relative, sep, basename } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
|
||||
export const REGISTRY_PATH = 'scripts/end-state-registry.json';
|
||||
export const FROZEN_PATH = 'tests/fixtures/end-state-frozen.json';
|
||||
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
|
||||
|
||||
const toPosix = (p) => p.split(sep).join('/');
|
||||
|
|
@ -101,52 +106,166 @@ export function evaluateCheck(root, check) {
|
|||
}
|
||||
}
|
||||
|
||||
// --- the frozen denominator -------------------------------------------------
|
||||
|
||||
// JSON with object keys sorted, so a signature does not depend on key order.
|
||||
function canonical(value) {
|
||||
if (Array.isArray(value)) return `[${value.map(canonical).join(',')}]`;
|
||||
if (value && typeof value === 'object') {
|
||||
return `{${Object.keys(value).sort().map((k) => `${JSON.stringify(k)}:${canonical(value[k])}`).join(',')}}`;
|
||||
}
|
||||
return JSON.stringify(value ?? null);
|
||||
}
|
||||
|
||||
export function checkSignature(check) {
|
||||
return createHash('sha256').update(canonical(check)).digest('hex');
|
||||
}
|
||||
|
||||
export function loadFrozen(root) {
|
||||
return JSON.parse(readFileSync(join(root, FROZEN_PATH), 'utf8'));
|
||||
}
|
||||
|
||||
// The judged party must not own the ledger: every frozen entry must still be present with
|
||||
// the same check, and the counting configuration must not move. New entries are allowed.
|
||||
export function verifyIntegrity(registry, frozen) {
|
||||
const violations = [];
|
||||
for (const kind of ['defects', 'experiments']) {
|
||||
const byId = new Map();
|
||||
for (const e of registry[kind] || []) {
|
||||
if (byId.has(e.id)) violations.push(`${kind}: duplicate id ${e.id}`);
|
||||
byId.set(e.id, e);
|
||||
}
|
||||
for (const [id, sig] of Object.entries(frozen[kind] || {})) {
|
||||
const e = byId.get(id);
|
||||
if (!e) {
|
||||
violations.push(`${kind}: ${id} is frozen but missing from the registry (an entry may close, never disappear)`);
|
||||
} else if (checkSignature(e.check) !== sig) {
|
||||
violations.push(`${kind}: ${id} has a changed check (signature ${checkSignature(e.check).slice(0, 12)}, frozen ${sig.slice(0, 12)})`);
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const key of ['agents', 'decisions', 'freeze']) {
|
||||
if (canonical(registry[key]) !== canonical(frozen[key])) {
|
||||
violations.push(`${key}: configuration differs from the frozen manifest`);
|
||||
}
|
||||
}
|
||||
const sites = [].concat(registry.agents?.spawnSites ?? []);
|
||||
if (sites.some((s) => /(^|\/)agents\//.test(String(s)))) {
|
||||
violations.push('agents: spawnSites may never include agents/ (an agent file would reference itself)');
|
||||
}
|
||||
return { ok: violations.length === 0, violations };
|
||||
}
|
||||
|
||||
// --- tallies ----------------------------------------------------------------
|
||||
|
||||
const notMeasured = (detail) => ({ open: null, total: null, items: [], detail });
|
||||
|
||||
function tallyEntries(root, entries) {
|
||||
const items = entries.map((e) => ({ id: e.id, summary: e.summary, ...evaluateCheck(root, e.check) }));
|
||||
const items = entries.map((e) => ({ id: e.id, summary: e.summary, probe: e.probe, ...evaluateCheck(root, e.check) }));
|
||||
const phrase = items.some((i) => i.probe === 'phrase');
|
||||
return {
|
||||
open: items.filter((i) => i.status !== 'closed').length,
|
||||
total: items.length,
|
||||
items,
|
||||
detail: '',
|
||||
detail: phrase ? 'a phrase probe closes on rewording: a closed phrase probe is evidence, not proof of behaviour' : '',
|
||||
};
|
||||
}
|
||||
|
||||
const escapeRe = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
// Lines that can carry an instruction: HTML comments and fenced code are blanked (line numbers
|
||||
// are kept so "the next line" still means the next line). Block quotes need no stripping: every
|
||||
// spawn form below is anchored at the start of the line, and a quoted line starts with `>`.
|
||||
function instructionLines(markdown) {
|
||||
const uncommented = markdown.replace(/<!--[\s\S]*?-->/g, (m) => m.replace(/[^\n]/g, ''));
|
||||
const out = [];
|
||||
let fenced = false;
|
||||
for (const line of uncommented.split('\n')) {
|
||||
if (/^\s*(```|~~~)/.test(line)) { fenced = !fenced; out.push(''); continue; }
|
||||
out.push(fenced ? '' : line);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const NAME = '[a-z0-9][a-z0-9-]*';
|
||||
const ROSTER_ROW = new RegExp(`^\\s*\\|\\s*\`(${NAME})\`\\s*\\|`);
|
||||
const LAUNCH_LINE = /^\s*(?:[-*]\s+|\d+\.\s+)?(?:\*\*)?(?:launch|spawn)\b(?:\*\*)?(.*)$/i;
|
||||
const LAUNCH_NAME = new RegExp(`\`(?:voyage:)?(${NAME})\`|\\*\\*(${NAME})\\*\\*|voyage:(${NAME})`, 'g');
|
||||
const AGENT_BLOCK = new RegExp(`^\\s*\\*\\*(${NAME})\\*\\*`);
|
||||
|
||||
// The agent names a command markdown file actually spawns. A spawn instruction is one of:
|
||||
// 1. a row of a table whose header's first cell is `Agent` (a roster the step launches);
|
||||
// 2. an imperative line that STARTS with Launch/Spawn and names the agent as `name`,
|
||||
// **name** or voyage:name;
|
||||
// 3. a line starting with **name** whose next non-empty line starts with `Prompt:`.
|
||||
// Prose, negations, HTML comments, fenced code and block quotes never count.
|
||||
export function spawnedNames(markdown) {
|
||||
const lines = instructionLines(markdown);
|
||||
const names = new Set();
|
||||
let inTable = false;
|
||||
let roster = false;
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
if (/^\s*\|/.test(line)) {
|
||||
if (!inTable) {
|
||||
inTable = true;
|
||||
roster = /^\s*\|\s*Agent\s*\|/.test(line);
|
||||
continue;
|
||||
}
|
||||
const row = roster && line.match(ROSTER_ROW);
|
||||
if (row) names.add(row[1]);
|
||||
continue;
|
||||
}
|
||||
inTable = false;
|
||||
const launch = line.match(LAUNCH_LINE);
|
||||
if (launch) {
|
||||
for (const m of launch[1].matchAll(LAUNCH_NAME)) names.add(m[1] || m[2] || m[3]);
|
||||
continue;
|
||||
}
|
||||
const block = line.match(AGENT_BLOCK);
|
||||
if (block) {
|
||||
let j = i + 1;
|
||||
while (j < lines.length && lines[j].trim() === '') j++;
|
||||
if (j < lines.length && /^\s*Prompt:/.test(lines[j])) names.add(block[1]);
|
||||
}
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
function tallyDormantAgents(root, cfg) {
|
||||
try {
|
||||
const agentFiles = expandPath(root, `${cfg.dir}/*.md`);
|
||||
const sites = expandPath(root, cfg.spawnSites).map((f) => readFileSync(f, 'utf8')).join('\n');
|
||||
const spawned = new Set();
|
||||
for (const f of expandPath(root, cfg.spawnSites)) {
|
||||
for (const n of spawnedNames(readFileSync(f, 'utf8'))) spawned.add(n);
|
||||
}
|
||||
const items = [];
|
||||
for (const file of agentFiles) {
|
||||
const text = readFileSync(file, 'utf8');
|
||||
const fm = text.startsWith('---') ? text.split('\n---')[0] : '';
|
||||
if (cfg.referenceMarker && fm.includes(cfg.referenceMarker)) continue;
|
||||
const name = (fm.match(/^name:\s*(\S+)\s*$/m) || [])[1] || basename(file, '.md');
|
||||
const ref = new RegExp(`(?<![A-Za-z0-9-])${escapeRe(name)}(?![A-Za-z0-9-])`);
|
||||
const referenced = ref.test(sites);
|
||||
const isSpawned = spawned.has(name);
|
||||
items.push({
|
||||
id: name,
|
||||
summary: toPosix(relative(root, file)),
|
||||
status: referenced ? 'closed' : 'open',
|
||||
detail: referenced ? '' : `not referenced by any ${cfg.spawnSites}`,
|
||||
status: isSpawned ? 'closed' : 'open',
|
||||
detail: isSpawned ? '' : `no spawn instruction in ${cfg.spawnSites}`,
|
||||
});
|
||||
}
|
||||
return {
|
||||
open: items.filter((i) => i.status === 'open').length,
|
||||
total: items.length,
|
||||
items,
|
||||
detail: 'spawnable = agent files without the reference-document marker; referenced = name appears (word-bounded) in a spawn site',
|
||||
detail: 'spawnable = agent files without the reference-document marker; spawned = named in a spawn instruction (an `Agent` table row, a line starting with Launch/Spawn, or a **name** block followed by Prompt:); prose, comments, fences and quotes do not count',
|
||||
};
|
||||
} catch (err) {
|
||||
return { open: null, total: null, items: [], detail: `not measurable: ${err.message}` };
|
||||
return notMeasured(`not measurable: ${err.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
function tallyDecisions(root, cfg) {
|
||||
const abs = join(root, cfg.file);
|
||||
if (!existsSync(abs)) {
|
||||
return { open: null, total: null, items: [], detail: `not measurable: ${cfg.file} not found (it is local-only)` };
|
||||
return notMeasured(`not measurable: ${cfg.file} not found (it is local-only)`);
|
||||
}
|
||||
try {
|
||||
const section = sectionOf(readFileSync(abs, 'utf8'), cfg.section, cfg.file);
|
||||
|
|
@ -158,33 +277,60 @@ function tallyDecisions(root, cfg) {
|
|||
// Format drift guard: a list item that carries neither marker would silently read as "0 open".
|
||||
const unmarked = lines.filter((l) => /^\s*(\d+\.|[-*]) /.test(l) && !openRe.test(l) && !closedRe.test(l));
|
||||
if (unmarked.length > 0) {
|
||||
return {
|
||||
open: null,
|
||||
total: null,
|
||||
items: [],
|
||||
detail: `not measurable: ${unmarked.length} unmarked list item(s) in ${cfg.section} — mark each with the open/decided marker`,
|
||||
};
|
||||
return notMeasured(`not measurable: ${unmarked.length} unmarked list item(s) in ${cfg.section} — mark each with the open/decided marker`);
|
||||
}
|
||||
return {
|
||||
open: open.length,
|
||||
total: open.length + closed.length,
|
||||
items: open.map((l, i) => ({ id: `#${i + 1}`, summary: l.replace(openRe, '').slice(0, 100), status: 'open', detail: '' })),
|
||||
detail: `marker: open = /${cfg.open}/, decided = /${cfg.closed}/`,
|
||||
detail: `marker: open = /${cfg.open}/, decided = /${cfg.closed}/; this counts ticks — a ticked line is not verified to be an actual decision`,
|
||||
};
|
||||
} catch (err) {
|
||||
return { open: null, total: null, items: [], detail: `not measurable: ${err.message}` };
|
||||
return notMeasured(`not measurable: ${err.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
export function measure(root, registry) {
|
||||
function tallyFeatAfterFreeze(root, cfg) {
|
||||
const git = (...args) => spawnSync('git', ['-C', root, ...args], { encoding: 'utf8' });
|
||||
const top = git('rev-parse', '--show-toplevel');
|
||||
if (top.status !== 0) return notMeasured(`not measured: ${toPosix(root)} is not a git work tree`);
|
||||
let atTop = false;
|
||||
try { atTop = realpathSync(top.stdout.trim()) === realpathSync(root); } catch { atTop = false; }
|
||||
if (!atTop) return notMeasured('not measured: the gate root is not the top of its git work tree');
|
||||
const ref = `refs/tags/${cfg.tag}`;
|
||||
if (git('rev-parse', '-q', '--verify', ref).status !== 0) {
|
||||
return notMeasured(`not measured: no freeze tag "${cfg.tag}" yet — open until the freeze is declared`);
|
||||
}
|
||||
const log = git('log', '--format=%s', `${ref}..HEAD`);
|
||||
if (log.status !== 0) return notMeasured(`not measured: git log failed: ${log.stderr.trim()}`);
|
||||
const subjects = log.stdout.split('\n').filter(Boolean);
|
||||
const featRe = new RegExp(cfg.featPattern);
|
||||
const feats = subjects.filter((s) => featRe.test(s));
|
||||
return {
|
||||
open: feats.length,
|
||||
total: subjects.length,
|
||||
items: feats.map((s, i) => ({ id: `#${i + 1}`, summary: s, status: 'open', detail: '' })),
|
||||
detail: `commits after ${cfg.tag} whose subject matches /${cfg.featPattern}/`,
|
||||
};
|
||||
}
|
||||
|
||||
export function measure(root, registry, frozen) {
|
||||
const d = registry.decisions;
|
||||
const f = registry.freeze;
|
||||
const rows = [
|
||||
{ id: 'defects', label: 'open pipeline defects', source: `${REGISTRY_PATH}#defects`, ...tallyEntries(root, registry.defects) },
|
||||
{ id: 'experiments', label: 'unresolved experiments/opt-ins', source: `${REGISTRY_PATH}#experiments`, ...tallyEntries(root, registry.experiments) },
|
||||
{ id: 'dormant-agents', label: 'dormant agents', source: `${registry.agents.dir}/*.md vs ${registry.agents.spawnSites}`, ...tallyDormantAgents(root, registry.agents) },
|
||||
{ id: 'decisions', label: 'open operator decisions', source: `${d.file} ${d.section.replace(/^#+\s*/, '§ ')}`, ...tallyDecisions(root, d) },
|
||||
{ id: 'feat-after-freeze', label: 'feat commits after the freeze tag', source: `git log ${f.tag}..HEAD`, ...tallyFeatAfterFreeze(root, f) },
|
||||
].map((r) => ({ ...r, target: 0 }));
|
||||
return { green: rows.every((r) => r.open === 0), rows };
|
||||
let integrity;
|
||||
try {
|
||||
integrity = { ...verifyIntegrity(registry, frozen ?? loadFrozen(root)), detail: '' };
|
||||
} catch (err) {
|
||||
integrity = { ok: null, violations: [], detail: `not measurable: cannot read ${FROZEN_PATH}: ${err.message}` };
|
||||
}
|
||||
return { green: rows.every((r) => r.open === 0) && integrity.ok === true, rows, integrity };
|
||||
}
|
||||
|
||||
export function loadRegistry(root) {
|
||||
|
|
@ -198,7 +344,7 @@ export function loadRegistry(root) {
|
|||
for (const key of ['defects', 'experiments']) {
|
||||
if (!Array.isArray(reg[key])) throw new Error(`${REGISTRY_PATH}: "${key}" must be a list`);
|
||||
}
|
||||
for (const key of ['agents', 'decisions']) {
|
||||
for (const key of ['agents', 'decisions', 'freeze']) {
|
||||
if (!reg[key] || typeof reg[key] !== 'object') throw new Error(`${REGISTRY_PATH}: "${key}" must be an object`);
|
||||
}
|
||||
return reg;
|
||||
|
|
@ -208,6 +354,15 @@ export function render(result) {
|
|||
const zero = result.rows.filter((r) => r.open === 0).length;
|
||||
const out = [];
|
||||
out.push(`Voyage end-state gate: ${result.green ? 'GREEN' : 'RED'} (${zero} of ${result.rows.length} tallies at 0)`);
|
||||
const integrity = result.integrity;
|
||||
if (integrity?.ok === true) {
|
||||
out.push(`registry integrity: intact against ${FROZEN_PATH}`);
|
||||
} else if (integrity?.ok === false) {
|
||||
out.push('registry integrity: VIOLATED — the gate cannot be green while the ledger differs from its frozen denominator');
|
||||
for (const v of integrity.violations) out.push(` - ${v}`);
|
||||
} else if (integrity) {
|
||||
out.push(`registry integrity: n/a — ${integrity.detail}`);
|
||||
}
|
||||
out.push('');
|
||||
out.push('| tally | open | of | target | source |');
|
||||
out.push('|---|---|---|---|---|');
|
||||
|
|
@ -221,9 +376,14 @@ export function render(result) {
|
|||
out.push(`${r.id} — ${r.label}${r.detail ? ` (${r.detail})` : ''}`);
|
||||
for (const i of r.items) {
|
||||
if (i.status === 'closed') continue;
|
||||
const tag = i.status === 'not-fellable' ? 'NOT FELLABLE, counted open' : 'open';
|
||||
let tag = i.status === 'not-fellable' ? 'NOT FELLABLE, counted open' : 'open';
|
||||
if (i.probe === 'phrase') tag += ' · phrase probe';
|
||||
out.push(` [${tag}] ${i.id}: ${i.summary}${i.detail ? ` — ${i.detail}` : ''}`);
|
||||
}
|
||||
const closed = r.items.filter((i) => i.status === 'closed' && i.probe);
|
||||
if (closed.length > 0) {
|
||||
out.push(` closed: ${closed.map((i) => (i.probe === 'phrase' ? `${i.id} (phrase probe)` : i.id)).join(', ')}`);
|
||||
}
|
||||
}
|
||||
return out.join('\n');
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,59 +4,117 @@
|
|||
"id": "D-01",
|
||||
"summary": "commands/trekplan.md still references TeamCreate/TeamDelete (allowed-tools and the execute-with-team path); both tools were removed in Claude Code 2.1.178, so that path always falls back to sequential",
|
||||
"closesWhen": "trekplan.md no longer names TeamCreate or TeamDelete",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": "commands/trekplan.md", "pattern": "\\bTeam(Create|Delete)\\b", "expect": "match" }
|
||||
{
|
||||
"path": "commands/trekplan.md",
|
||||
"pattern": "\\bTeam(Create|Delete)\\b",
|
||||
"expect": "match"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "D-02",
|
||||
"summary": "commands/trekresearch.md tells the swarm engine to run the `### Bridge agent` block, but that heading was removed together with gemini-bridge",
|
||||
"closesWhen": "the dangling reference is gone (or the heading exists again)",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": "commands/trekresearch.md", "pattern": "`### Bridge agent`", "expect": "match" },
|
||||
{ "path": "commands/trekresearch.md", "pattern": "^### Bridge agent", "flags": "m", "expect": "no-match" }
|
||||
{
|
||||
"path": "commands/trekresearch.md",
|
||||
"pattern": "`### Bridge agent`",
|
||||
"expect": "match"
|
||||
},
|
||||
{
|
||||
"path": "commands/trekresearch.md",
|
||||
"pattern": "^### Bridge agent",
|
||||
"flags": "m",
|
||||
"expect": "no-match"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "D-03",
|
||||
"summary": "commands/trekexecute.md never runs a trekplan's `## Verification` (where the brief's success criteria land) on the single-session path: Phase 7 says 'Skip for trekplans', and only the multi-session wave path runs master verification",
|
||||
"closesWhen": "Phase 7 no longer skips trekplans (proxy: the fix must also make that path run the plan's Verification; Phase 4's entry-condition skip is legitimate and out of scope)",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": "commands/trekexecute.md", "section": "## Phase 7 —", "pattern": "^\\*\\*Skip for trekplans\\.\\*\\*", "flags": "m", "expect": "match" }
|
||||
{
|
||||
"path": "commands/trekexecute.md",
|
||||
"section": "## Phase 7 —",
|
||||
"pattern": "^\\*\\*Skip for trekplans\\.\\*\\*",
|
||||
"flags": "m",
|
||||
"expect": "match"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "D-04",
|
||||
"summary": "agents/brief-conformance-reviewer.md must judge whether a success criterion's verification command 'exists and passes', but its tools are Read/Glob/Grep, so it cannot run anything",
|
||||
"closesWhen": "the rubric no longer asks it to judge 'passes', or the reviewer can execute the command",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": "agents/brief-conformance-reviewer.md", "pattern": "exists and passes", "expect": "match" },
|
||||
{ "path": "agents/brief-conformance-reviewer.md", "pattern": "^tools:.*\"Bash\"", "flags": "m", "expect": "no-match" }
|
||||
{
|
||||
"path": "agents/brief-conformance-reviewer.md",
|
||||
"pattern": "exists and passes",
|
||||
"expect": "match"
|
||||
},
|
||||
{
|
||||
"path": "agents/brief-conformance-reviewer.md",
|
||||
"pattern": "^tools:.*\"Bash\"",
|
||||
"flags": "m",
|
||||
"expect": "no-match"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "D-05",
|
||||
"summary": "commands/trekplan.md allowed-tools lists TaskCreate/TaskUpdate, which Claude Code 2.1.233 no longer offers on Opus 4.8 / Sonnet 5 / Fable 5 and newer by default (runtime effect not measured)",
|
||||
"closesWhen": "trekplan.md allowed-tools no longer lists TaskCreate or TaskUpdate",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": "commands/trekplan.md", "pattern": "^allowed-tools:.*\\bTask(Create|Update)\\b", "flags": "m", "expect": "match" }
|
||||
{
|
||||
"path": "commands/trekplan.md",
|
||||
"pattern": "^allowed-tools:.*\\bTask(Create|Update)\\b",
|
||||
"flags": "m",
|
||||
"expect": "match"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "D-06",
|
||||
"summary": "lib/review/gold-scorer.mjs contains a literal NUL byte, so git treats the file as binary and hides it from diffs and --numstat",
|
||||
"closesWhen": "the file contains no NUL byte (write the separator as an escape sequence)",
|
||||
"probe": "byte",
|
||||
"check": [
|
||||
{ "path": "lib/review/gold-scorer.mjs", "pattern": "\\u0000", "expect": "match" }
|
||||
{
|
||||
"path": "lib/review/gold-scorer.mjs",
|
||||
"pattern": "\\u0000",
|
||||
"expect": "match"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "D-07",
|
||||
"summary": "CLAUDE.md's command table says every /trek* command runs on opus, but since v5.9.0 no command pins model: and the orchestrator follows the session model",
|
||||
"closesWhen": "the table stops claiming opus per command, or the commands pin it again",
|
||||
"summary": "CLAUDE.md, README.md and commands/trekresearch.md claim the /trek* orchestrators run on opus (README even ships a sed recipe for `model: opus` lines no command has), but since v5.9.0 no command pins model: and the orchestrator follows the session model",
|
||||
"closesWhen": "none of the three files claims an opus orchestrator any more, or the commands pin opus again",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": "CLAUDE.md", "pattern": "^\\| `/trek[a-z]+` \\|.*\\| opus \\|\\s*$", "flags": "m", "expect": "match" },
|
||||
{ "path": "commands/*.md", "pattern": "^model:", "flags": "m", "expect": "no-match" }
|
||||
{
|
||||
"path": [
|
||||
"CLAUDE.md",
|
||||
"README.md",
|
||||
"commands/trekresearch.md"
|
||||
],
|
||||
"pattern": "(^\\| `/trek[a-z]+` \\|.*\\| opus \\|\\s*$)|(\\^model: opus\\$)|(default for `/trekbrief`[\\s\\S]{0,80}?is `opus`)|(orchestrator runs on Opus)",
|
||||
"flags": "m",
|
||||
"expect": "match"
|
||||
},
|
||||
{
|
||||
"path": "commands/*.md",
|
||||
"pattern": "^model:",
|
||||
"flags": "m",
|
||||
"expect": "no-match"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
|
|
@ -65,24 +123,46 @@
|
|||
"id": "E-01",
|
||||
"summary": "STORM dimension discovery + bounded research loop ships default-off behind VOYAGE_STORM_ENABLED; adoption is gated on a pre-registered measurement that has not run",
|
||||
"closesWhen": "the env gate is gone from the pipeline surface: adopted as default, or the loop is removed",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": ["commands/*.md", "agents/*.md", "lib/**/*.mjs", "hooks/**/*.mjs"], "pattern": "VOYAGE_STORM_ENABLED", "expect": "match" }
|
||||
{
|
||||
"path": [
|
||||
"commands/*.md",
|
||||
"agents/*.md",
|
||||
"lib/**/*.mjs",
|
||||
"hooks/**/*.mjs"
|
||||
],
|
||||
"pattern": "VOYAGE_STORM_ENABLED",
|
||||
"expect": "match"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "E-02",
|
||||
"summary": "`/trekresearch --engine deep-research` delegates to Claude Code's /deep-research, which is manual-only since 2.1.218, so the opt-in always falls back to the swarm",
|
||||
"closesWhen": "the --engine opt-in is removed from trekresearch.md",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": "commands/trekresearch.md", "pattern": "--engine\\b", "expect": "match" }
|
||||
{
|
||||
"path": "commands/trekresearch.md",
|
||||
"pattern": "--engine\\b",
|
||||
"expect": "match"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "E-03",
|
||||
"summary": "the delegated-orchestration head-to-head (T1 §5) is designed but has no recorded decision",
|
||||
"closesWhen": "T1 §5 carries a STATUS block (run, or declined)",
|
||||
"probe": "phrase",
|
||||
"check": [
|
||||
{ "path": "docs/T1-cc26-delegated-orchestration.md", "section": "## 5.", "pattern": "^> \\*\\*STATUS:", "flags": "m", "expect": "no-match" }
|
||||
{
|
||||
"path": "docs/T1-cc26-delegated-orchestration.md",
|
||||
"section": "## 5.",
|
||||
"pattern": "^> \\*\\*STATUS:",
|
||||
"flags": "m",
|
||||
"expect": "no-match"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
|
|
@ -96,5 +176,9 @@
|
|||
"section": "## Åpne operatørbeslutninger",
|
||||
"open": "^- \\[ \\] ",
|
||||
"closed": "^- \\[x\\] "
|
||||
},
|
||||
"freeze": {
|
||||
"tag": "end-state-freeze",
|
||||
"featPattern": "^feat(\\(|!|:)"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
32
tests/fixtures/end-state-frozen.json
vendored
Normal file
32
tests/fixtures/end-state-frozen.json
vendored
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
{
|
||||
"why": "Frozen 2026-09-17, when the end-state direction was chosen. These ids and check signatures are the end-state gate's denominator: an entry may CLOSE, it may never disappear or have its check changed. Changing this file is a decision, not a refactor - tests/scripts/end-state-gate.test.mjs pins it literally.",
|
||||
"defects": {
|
||||
"D-01": "48f7f5cca070091e8f9b9ca0c7cdb7a1880733691935c25d07ea1d3e8d508fc9",
|
||||
"D-02": "dfc94dba04a9116ae9be2097a0c5a9d8313b02c7de1ca6b49a472c1823a6d1e1",
|
||||
"D-03": "41dea56c14764494ed6d555ca97311b6001b0542f4ab69fa1dfc405780a57c32",
|
||||
"D-04": "721bdda19c0e98962acbfc78419bfb02790ee739694d1609b72d10481f0d6b62",
|
||||
"D-05": "8fe6df52b43496c3302400507cb98002dcf8d2a12d92085e14bda4c7819bc7ba",
|
||||
"D-06": "72a787c8154d1c18849e9856dcf1a67703d6b5541f88db81d6bab13cab14b418",
|
||||
"D-07": "1bf3e0cd36c621720697475b0a3fd7db78611d66b58a9ce035e7695e57c69f1b"
|
||||
},
|
||||
"experiments": {
|
||||
"E-01": "f7718897cec8ace6879fe79f5f32760ec957c3642e7186f8aa1688ce77b3f4e8",
|
||||
"E-02": "c8ff12c504ba7a9ac36f11bb8f9b713263e9b4eb28c07d984d9e2115980e740d",
|
||||
"E-03": "17153ad8808615a833a49ae0c3523264ce055aa690e5542a75bae1874495e7e8"
|
||||
},
|
||||
"agents": {
|
||||
"dir": "agents",
|
||||
"spawnSites": "commands/*.md",
|
||||
"referenceMarker": "Reference document, not a spawnable capability"
|
||||
},
|
||||
"decisions": {
|
||||
"file": "STATE.md",
|
||||
"section": "## Åpne operatørbeslutninger",
|
||||
"open": "^- \\[ \\] ",
|
||||
"closed": "^- \\[x\\] "
|
||||
},
|
||||
"freeze": {
|
||||
"tag": "end-state-freeze",
|
||||
"featPattern": "^feat(\\(|!|:)"
|
||||
}
|
||||
}
|
||||
|
|
@ -1,16 +1,18 @@
|
|||
// tests/scripts/end-state-gate.test.mjs
|
||||
// The end-state gate counts the distance to "Voyage is finished" as four tallies:
|
||||
// open pipeline defects, unresolved experiments/opt-ins, dormant agents, and open
|
||||
// operator decisions. "Finished" = all four at 0.
|
||||
// The end-state gate counts the distance to "Voyage is finished" as five tallies:
|
||||
// open pipeline defects, unresolved experiments/opt-ins, dormant agents, open operator
|
||||
// decisions, and feat commits after the freeze tag. "Finished" = all five at 0 AND the
|
||||
// registry the gate counts from is intact against its frozen denominator.
|
||||
//
|
||||
// These tests pin the gate's MECHANICS in both directions — every tally can be 0 and
|
||||
// can be > 0 — against throwaway fixture trees. They deliberately do NOT assert that the
|
||||
// real repo is red today: the gate is supposed to go green, and a test that pins the
|
||||
// current distance would fail on the day the work is done.
|
||||
// These tests pin the gate's MECHANICS in both directions — every tally can be 0 and can
|
||||
// be > 0 — against throwaway fixture trees. They deliberately do NOT assert that the real
|
||||
// repo is red today: the gate is supposed to go green, and a test that pins the current
|
||||
// distance would fail on the day the work is done. What they DO pin about the real repo
|
||||
// is the denominator itself (see "the frozen denominator" below).
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, existsSync } from 'node:fs';
|
||||
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, existsSync, readFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, dirname, resolve } from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
|
|
@ -20,11 +22,52 @@ import {
|
|||
evaluateCheck,
|
||||
measure,
|
||||
loadRegistry,
|
||||
loadFrozen,
|
||||
spawnedNames,
|
||||
checkSignature,
|
||||
verifyIntegrity,
|
||||
render,
|
||||
} from '../../scripts/end-state-gate.mjs';
|
||||
|
||||
const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
||||
const GATE = join(ROOT, 'scripts', 'end-state-gate.mjs');
|
||||
|
||||
// --- the frozen denominator ------------------------------------------------
|
||||
// Denominator frozen 2026-09-17 by the end-state work order, the day the end-state
|
||||
// direction was chosen. An entry may CLOSE; it may never be removed or have its check
|
||||
// changed, and the gate's agent/decision/freeze configuration may not move. To change
|
||||
// anything below you must change this test — say in the commit message who decided it.
|
||||
const FROZEN = {
|
||||
defects: {
|
||||
'D-01': '48f7f5cca070091e8f9b9ca0c7cdb7a1880733691935c25d07ea1d3e8d508fc9',
|
||||
'D-02': 'dfc94dba04a9116ae9be2097a0c5a9d8313b02c7de1ca6b49a472c1823a6d1e1',
|
||||
'D-03': '41dea56c14764494ed6d555ca97311b6001b0542f4ab69fa1dfc405780a57c32',
|
||||
'D-04': '721bdda19c0e98962acbfc78419bfb02790ee739694d1609b72d10481f0d6b62',
|
||||
'D-05': '8fe6df52b43496c3302400507cb98002dcf8d2a12d92085e14bda4c7819bc7ba',
|
||||
'D-06': '72a787c8154d1c18849e9856dcf1a67703d6b5541f88db81d6bab13cab14b418',
|
||||
'D-07': '1bf3e0cd36c621720697475b0a3fd7db78611d66b58a9ce035e7695e57c69f1b',
|
||||
},
|
||||
experiments: {
|
||||
'E-01': 'f7718897cec8ace6879fe79f5f32760ec957c3642e7186f8aa1688ce77b3f4e8',
|
||||
'E-02': 'c8ff12c504ba7a9ac36f11bb8f9b713263e9b4eb28c07d984d9e2115980e740d',
|
||||
'E-03': '17153ad8808615a833a49ae0c3523264ce055aa690e5542a75bae1874495e7e8',
|
||||
},
|
||||
agents: {
|
||||
dir: 'agents',
|
||||
spawnSites: 'commands/*.md',
|
||||
referenceMarker: 'Reference document, not a spawnable capability',
|
||||
},
|
||||
decisions: {
|
||||
file: 'STATE.md',
|
||||
section: '## Åpne operatørbeslutninger',
|
||||
open: '^- \\[ \\] ',
|
||||
closed: '^- \\[x\\] ',
|
||||
},
|
||||
freeze: { tag: 'end-state-freeze', featPattern: '^feat(\\(|!|:)' },
|
||||
};
|
||||
|
||||
// --- helpers ---------------------------------------------------------------
|
||||
|
||||
function fixture(files) {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'end-state-gate-'));
|
||||
for (const [rel, body] of Object.entries(files)) {
|
||||
|
|
@ -35,26 +78,40 @@ function fixture(files) {
|
|||
return dir;
|
||||
}
|
||||
|
||||
const DECISIONS = {
|
||||
file: 'STATE.md',
|
||||
section: '## Open decisions',
|
||||
open: '^- \\[ \\] ',
|
||||
closed: '^- \\[x\\] ',
|
||||
};
|
||||
const AGENTS = {
|
||||
dir: 'agents',
|
||||
spawnSites: 'commands/*.md',
|
||||
referenceMarker: 'Reference document, not a spawnable capability',
|
||||
};
|
||||
const cleanup = (...dirs) => { for (const d of dirs) rmSync(d, { recursive: true, force: true }); };
|
||||
|
||||
const DECISIONS = { file: 'STATE.md', section: '## Open decisions', open: '^- \\[ \\] ', closed: '^- \\[x\\] ' };
|
||||
const AGENTS = { dir: 'agents', spawnSites: 'commands/*.md', referenceMarker: 'Reference document, not a spawnable capability' };
|
||||
const FREEZE = { tag: 'end-state-freeze', featPattern: '^feat(\\(|!|:)' };
|
||||
|
||||
function registry(overrides = {}) {
|
||||
return {
|
||||
defects: [],
|
||||
experiments: [],
|
||||
agents: AGENTS,
|
||||
decisions: DECISIONS,
|
||||
...overrides,
|
||||
};
|
||||
return { defects: [], experiments: [], agents: AGENTS, decisions: DECISIONS, freeze: FREEZE, ...overrides };
|
||||
}
|
||||
|
||||
// A frozen manifest that matches `reg` exactly.
|
||||
function frozenFor(reg) {
|
||||
const sigs = (list) => Object.fromEntries(list.map((e) => [e.id, checkSignature(e.check)]));
|
||||
return { defects: sigs(reg.defects), experiments: sigs(reg.experiments), agents: reg.agents, decisions: reg.decisions, freeze: reg.freeze };
|
||||
}
|
||||
|
||||
function git(dir, ...args) {
|
||||
const r = spawnSync('git', [
|
||||
'-c', 'user.name=fixture', '-c', 'user.email=fixture@example.invalid',
|
||||
'-c', 'core.hooksPath=/dev/null', '-c', 'commit.gpgsign=false', '-c', 'tag.gpgsign=false',
|
||||
...args,
|
||||
], { cwd: dir, encoding: 'utf8' });
|
||||
assert.equal(r.status, 0, `git ${args.join(' ')}: ${r.stderr}`);
|
||||
return r.stdout;
|
||||
}
|
||||
|
||||
function gitInit(dir) {
|
||||
git(dir, 'init', '-q');
|
||||
git(dir, 'add', '-A');
|
||||
git(dir, 'commit', '-q', '--allow-empty', '-m', 'chore: fixture baseline');
|
||||
}
|
||||
|
||||
function gitCommit(dir, subject) {
|
||||
git(dir, 'commit', '-q', '--allow-empty', '-m', subject);
|
||||
}
|
||||
|
||||
const rowById = (result, id) => result.rows.find((r) => r.id === id);
|
||||
|
|
@ -67,7 +124,7 @@ test('evaluateCondition: match / no-match on a single file, multiline regex', ()
|
|||
assert.equal(evaluateCondition(dir, { path: 'a.md', pattern: '^### Heading', flags: 'm', expect: 'match' }), true);
|
||||
assert.equal(evaluateCondition(dir, { path: 'a.md', pattern: '^### Other', flags: 'm', expect: 'match' }), false);
|
||||
assert.equal(evaluateCondition(dir, { path: 'a.md', pattern: '^### Other', flags: 'm', expect: 'no-match' }), true);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('evaluateCondition: globs (dir/*.ext, dir/**/*.ext) and path arrays match ANY file', () => {
|
||||
|
|
@ -80,7 +137,7 @@ test('evaluateCondition: globs (dir/*.ext, dir/**/*.ext) and path arrays match A
|
|||
assert.equal(evaluateCondition(dir, { path: 'lib/**/*.mjs', pattern: 'SOME_FLAG', expect: 'match' }), true);
|
||||
assert.equal(evaluateCondition(dir, { path: ['commands/*.md', 'lib/**/*.mjs'], pattern: 'SOME_FLAG', expect: 'match' }), true);
|
||||
assert.equal(evaluateCondition(dir, { path: ['commands/*.md', 'lib/**/*.mjs'], pattern: 'SOME_FLAG', expect: 'no-match' }), false);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('evaluateCondition: section scoping reads only from the heading to the next "## "', () => {
|
||||
|
|
@ -92,7 +149,7 @@ test('evaluateCondition: section scoping reads only from the heading to the next
|
|||
const inSix = { ...inFive, section: '## 6.' };
|
||||
assert.equal(evaluateCondition(dir, inFive), true, 'section 5 has no status block');
|
||||
assert.equal(evaluateCondition(dir, inSix), false, 'section 6 has one');
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('evaluateCondition: a literal NUL byte is detectable', () => {
|
||||
|
|
@ -100,16 +157,25 @@ test('evaluateCondition: a literal NUL byte is detectable', () => {
|
|||
try {
|
||||
assert.equal(evaluateCondition(dir, { path: 'nul.mjs', pattern: '\\u0000', expect: 'match' }), true);
|
||||
assert.equal(evaluateCondition(dir, { path: 'clean.mjs', pattern: '\\u0000', expect: 'match' }), false);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('evaluateCondition: a missing file or empty glob throws (never a silent "no match")', () => {
|
||||
test('evaluateCondition: a missing file, a missing glob base or a missing section throws', () => {
|
||||
const dir = fixture({ 'a.md': 'x\n' });
|
||||
try {
|
||||
assert.throws(() => evaluateCondition(dir, { path: 'missing.md', pattern: 'x', expect: 'no-match' }), /missing\.md/);
|
||||
assert.throws(() => evaluateCondition(dir, { path: 'nodir/*.md', pattern: 'x', expect: 'no-match' }), /nodir/);
|
||||
assert.throws(() => evaluateCondition(dir, { path: 'a.md', section: '## 9.', pattern: 'x', expect: 'no-match' }), /## 9\./);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('evaluateCondition: a glob over an EXISTING dir with no matching files throws (no silent no-match)', () => {
|
||||
// In a no-match condition, "silently nothing matched" would read as the condition holding.
|
||||
const dir = fixture({ 'docs/readme.txt': 'x\n' });
|
||||
try {
|
||||
assert.throws(() => evaluateCondition(dir, { path: 'docs/*.md', pattern: 'x', expect: 'no-match' }), /docs\/\*\.md/);
|
||||
assert.throws(() => evaluateCondition(dir, { path: 'docs/**/*.md', pattern: 'x', expect: 'no-match' }), /docs\/\*\*\/\*\.md/);
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('evaluateCheck: open only when ALL conditions hold; null check is not fellable', () => {
|
||||
|
|
@ -128,21 +194,85 @@ test('evaluateCheck: open only when ALL conditions hold; null check is not fella
|
|||
const broken = evaluateCheck(dir, dangling('gone.md'));
|
||||
assert.equal(broken.status, 'not-fellable');
|
||||
assert.match(broken.detail, /gone\.md/);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
// --- the four tallies, both directions -------------------------------------
|
||||
// --- spawn instructions (what "referenced by a command" means) -------------
|
||||
|
||||
test('spawnedNames: a row of a table headed `Agent` is a spawn instruction; other tables are not', () => {
|
||||
const md = [
|
||||
'Launch the new agents:', '',
|
||||
'| Agent | Purpose |', '|-------|---------|', '| `alpha-agent` | does a |', '| `beta-agent` | does b |', '',
|
||||
'| Name | Purpose |', '|------|---------|', '| `gamma-agent` | just a list |', '',
|
||||
].join('\n');
|
||||
const names = spawnedNames(md);
|
||||
assert.ok(names.has('alpha-agent') && names.has('beta-agent'));
|
||||
assert.ok(!names.has('gamma-agent'), 'a table not headed "Agent" is not a roster');
|
||||
});
|
||||
|
||||
test('spawnedNames: an imperative Launch/Spawn line names the agents it launches', () => {
|
||||
const md = [
|
||||
'Launch the **alpha-agent** agent:',
|
||||
'- Launch `beta-agent` (Agent tool) with the findings.',
|
||||
'3. Spawn `voyage:gamma-agent` in the background.',
|
||||
'**Launch** the `delta-agent` agent now.',
|
||||
'launch `epsilon-agent` too',
|
||||
].join('\n');
|
||||
const names = spawnedNames(md);
|
||||
for (const n of ['alpha-agent', 'beta-agent', 'gamma-agent', 'delta-agent', 'epsilon-agent']) {
|
||||
assert.ok(names.has(n), `${n} should count as spawned`);
|
||||
}
|
||||
});
|
||||
|
||||
test('spawnedNames: an agent block with a Prompt: line is a spawn instruction; a bare bold name is not', () => {
|
||||
const md = [
|
||||
'**alpha-agent** — adversarial review of the plan.',
|
||||
'Prompt: "Review this plan."',
|
||||
'',
|
||||
'**beta-agent** — mentioned for context only.',
|
||||
'It is not launched here.',
|
||||
].join('\n');
|
||||
const names = spawnedNames(md);
|
||||
assert.ok(names.has('alpha-agent'));
|
||||
assert.ok(!names.has('beta-agent'));
|
||||
});
|
||||
|
||||
test('spawnedNames: prose, negation, comments, fences and quotes are never spawn instructions', () => {
|
||||
const md = [
|
||||
'The alpha-agent is dormant and is never spawned.',
|
||||
'Never launch `beta-agent`.',
|
||||
'In quick mode we used to launch `gamma-agent`.',
|
||||
'<!-- Launch the `delta-agent` agent -->',
|
||||
'<!--',
|
||||
'Launch the `epsilon-agent` agent',
|
||||
'-->',
|
||||
'```',
|
||||
'Launch the `zeta-agent` agent',
|
||||
'| Agent |',
|
||||
'|---|',
|
||||
'| `eta-agent` |',
|
||||
'```',
|
||||
'> Launch the `theta-agent` agent',
|
||||
].join('\n');
|
||||
const names = spawnedNames(md);
|
||||
for (const n of ['alpha-agent', 'beta-agent', 'gamma-agent', 'delta-agent', 'epsilon-agent', 'zeta-agent', 'eta-agent', 'theta-agent']) {
|
||||
assert.ok(!names.has(n), `${n} must not count as spawned`);
|
||||
}
|
||||
});
|
||||
|
||||
// --- the tallies, both directions ------------------------------------------
|
||||
|
||||
test('defects tally: > 0 while a check holds, 0 once fixed; not-fellable counts as open', () => {
|
||||
const defect = {
|
||||
id: 'D-X',
|
||||
summary: 'removed tool still referenced',
|
||||
probe: 'phrase',
|
||||
check: [{ path: 'commands/plan.md', pattern: '\\bTeamCreate\\b', expect: 'match' }],
|
||||
};
|
||||
const unfellable = { id: 'D-Y', summary: 'no runnable check yet', check: null };
|
||||
const base = { 'STATE.md': '## Open decisions\n', 'agents/a.md': 'x\n', 'commands/plan.md': 'uses a\n' };
|
||||
const base = { 'STATE.md': '## Open decisions\n', 'agents/a.md': 'x\n', 'commands/plan.md': 'Launch `a`\n' };
|
||||
|
||||
const red = fixture({ ...base, 'commands/plan.md': 'uses a and TeamCreate\n' });
|
||||
const red = fixture({ ...base, 'commands/plan.md': 'Launch `a` and TeamCreate\n' });
|
||||
const green = fixture(base);
|
||||
try {
|
||||
let row = rowById(measure(red, registry({ defects: [defect] })), 'defects');
|
||||
|
|
@ -153,31 +283,26 @@ test('defects tally: > 0 while a check holds, 0 once fixed; not-fellable counts
|
|||
row = rowById(measure(green, registry({ defects: [defect, unfellable] })), 'defects');
|
||||
assert.equal(row.open, 1, 'a defect without a runnable check stays open');
|
||||
assert.equal(row.items.find((i) => i.id === 'D-Y').status, 'not-fellable');
|
||||
} finally {
|
||||
rmSync(red, { recursive: true, force: true });
|
||||
rmSync(green, { recursive: true, force: true });
|
||||
}
|
||||
} finally { cleanup(red, green); }
|
||||
});
|
||||
|
||||
test('experiments tally: an env-gated opt-in is open until its gate is gone', () => {
|
||||
const exp = {
|
||||
id: 'E-X',
|
||||
summary: 'default-off loop',
|
||||
probe: 'phrase',
|
||||
check: [{ path: ['commands/*.md', 'lib/**/*.mjs'], pattern: 'LOOP_ENABLED', expect: 'match' }],
|
||||
};
|
||||
const base = { 'STATE.md': '## Open decisions\n', 'agents/a.md': 'x\n', 'commands/r.md': 'uses a\n' };
|
||||
const base = { 'STATE.md': '## Open decisions\n', 'agents/a.md': 'x\n', 'commands/r.md': 'Launch `a`\n' };
|
||||
const red = fixture({ ...base, 'lib/util/cap.mjs': 'if (process.env.LOOP_ENABLED) {}\n' });
|
||||
const green = fixture({ ...base, 'lib/util/cap.mjs': 'export const x = 1;\n' });
|
||||
try {
|
||||
assert.equal(rowById(measure(red, registry({ experiments: [exp] })), 'experiments').open, 1);
|
||||
assert.equal(rowById(measure(green, registry({ experiments: [exp] })), 'experiments').open, 0);
|
||||
} finally {
|
||||
rmSync(red, { recursive: true, force: true });
|
||||
rmSync(green, { recursive: true, force: true });
|
||||
}
|
||||
} finally { cleanup(red, green); }
|
||||
});
|
||||
|
||||
test('dormant-agents tally: unreferenced spawnable agent counts; reference docs never do', () => {
|
||||
test('dormant-agents tally: an agent no command spawns counts; reference docs never do', () => {
|
||||
const files = {
|
||||
'STATE.md': '## Open decisions\n',
|
||||
'agents/used-agent.md': '---\nname: used-agent\ndescription: |\n does work\n---\n',
|
||||
|
|
@ -186,7 +311,7 @@ test('dormant-agents tally: unreferenced spawnable agent counts; reference docs
|
|||
'commands/run.md': 'Launch the **used-agent** agent.\n',
|
||||
};
|
||||
const red = fixture(files);
|
||||
const green = fixture({ ...files, 'commands/other.md': 'Then spawn `idle-agent`.\n' });
|
||||
const green = fixture({ ...files, 'commands/other.md': 'Spawn `idle-agent` in parallel.\n' });
|
||||
try {
|
||||
let row = rowById(measure(red, registry()), 'dormant-agents');
|
||||
assert.equal(row.open, 1);
|
||||
|
|
@ -194,24 +319,35 @@ test('dormant-agents tally: unreferenced spawnable agent counts; reference docs
|
|||
assert.deepEqual(row.items.filter((i) => i.status === 'open').map((i) => i.id), ['idle-agent']);
|
||||
row = rowById(measure(green, registry()), 'dormant-agents');
|
||||
assert.equal(row.open, 0);
|
||||
} finally {
|
||||
rmSync(red, { recursive: true, force: true });
|
||||
rmSync(green, { recursive: true, force: true });
|
||||
}
|
||||
} finally { cleanup(red, green); }
|
||||
});
|
||||
|
||||
test('dormant-agents tally: a name inside a longer word is not a reference', () => {
|
||||
const dir = fixture({
|
||||
test('dormant-agents tally: a prose sentence or an HTML comment does not wake a dormant agent', () => {
|
||||
const files = {
|
||||
'STATE.md': '## Open decisions\n',
|
||||
'agents/scout.md': '---\nname: scout\ndescription: |\n x\n---\n',
|
||||
'commands/run.md': 'the scouting phase and research-scouts\n',
|
||||
});
|
||||
'agents/idle-agent.md': '---\nname: idle-agent\ndescription: |\n never wired\n---\n',
|
||||
'commands/plan.md': 'The idle-agent is dormant and is never spawned.\n',
|
||||
'commands/end.md': '<!-- idle-agent -->\nThe research-idle-agents phase.\n',
|
||||
};
|
||||
const dir = fixture(files);
|
||||
try {
|
||||
assert.equal(rowById(measure(dir, registry()), 'dormant-agents').open, 1);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('decisions tally: counts open markers inside the section only', () => {
|
||||
test('dormant-agents tally: an unmeasurable agents dir is n/a (null), never 0', () => {
|
||||
const dir = fixture({ 'STATE.md': '## Open decisions\n', 'commands/r.md': 'x\n' });
|
||||
try {
|
||||
const result = measure(dir, registry());
|
||||
const row = rowById(result, 'dormant-agents');
|
||||
assert.equal(row.open, null);
|
||||
assert.equal(row.total, null);
|
||||
assert.match(row.detail, /not measurable/);
|
||||
assert.equal(result.green, false);
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('decisions tally: counts open markers inside the section only, and says it counts ticks', () => {
|
||||
const state = [
|
||||
'# STATE',
|
||||
'- [ ] outside the section, ignored',
|
||||
|
|
@ -223,28 +359,26 @@ test('decisions tally: counts open markers inside the section only', () => {
|
|||
'- [ ] also ignored',
|
||||
'',
|
||||
].join('\n');
|
||||
const red = fixture({ 'STATE.md': state, 'agents/a.md': 'x\n', 'commands/r.md': 'a\n' });
|
||||
const red = fixture({ 'STATE.md': state, 'agents/a.md': 'x\n', 'commands/r.md': 'Launch `a`\n' });
|
||||
const green = fixture({
|
||||
'STATE.md': '## Open decisions\n- [x] done\n## Next\n',
|
||||
'agents/a.md': 'x\n',
|
||||
'commands/r.md': 'a\n',
|
||||
'commands/r.md': 'Launch `a`\n',
|
||||
});
|
||||
try {
|
||||
let row = rowById(measure(red, registry()), 'decisions');
|
||||
assert.equal(row.open, 2);
|
||||
assert.equal(row.total, 3);
|
||||
assert.match(row.detail, /tick/i, 'the output must say a tick is not verified as a decision');
|
||||
row = rowById(measure(green, registry()), 'decisions');
|
||||
assert.equal(row.open, 0);
|
||||
assert.equal(row.total, 1);
|
||||
} finally {
|
||||
rmSync(red, { recursive: true, force: true });
|
||||
rmSync(green, { recursive: true, force: true });
|
||||
}
|
||||
} finally { cleanup(red, green); }
|
||||
});
|
||||
|
||||
test('decisions tally: a missing STATE.md or section is NOT MEASURABLE (null), never 0', () => {
|
||||
const noState = fixture({ 'agents/a.md': 'x\n', 'commands/r.md': 'a\n' });
|
||||
const noSection = fixture({ 'STATE.md': '# STATE\n- [ ] x\n', 'agents/a.md': 'x\n', 'commands/r.md': 'a\n' });
|
||||
const noState = fixture({ 'agents/a.md': 'x\n', 'commands/r.md': 'Launch `a`\n' });
|
||||
const noSection = fixture({ 'STATE.md': '# STATE\n- [ ] x\n', 'agents/a.md': 'x\n', 'commands/r.md': 'Launch `a`\n' });
|
||||
try {
|
||||
for (const dir of [noState, noSection]) {
|
||||
const result = measure(dir, registry());
|
||||
|
|
@ -253,10 +387,7 @@ test('decisions tally: a missing STATE.md or section is NOT MEASURABLE (null), n
|
|||
assert.ok(row.detail.length > 0, 'the reason is reported');
|
||||
assert.equal(result.green, false, 'an unmeasurable tally keeps the gate red');
|
||||
}
|
||||
} finally {
|
||||
rmSync(noState, { recursive: true, force: true });
|
||||
rmSync(noSection, { recursive: true, force: true });
|
||||
}
|
||||
} finally { cleanup(noState, noSection); }
|
||||
});
|
||||
|
||||
test('decisions tally: an unmarked list item in the section is format drift → NOT MEASURABLE', () => {
|
||||
|
|
@ -265,59 +396,257 @@ test('decisions tally: an unmarked list item in the section is format drift →
|
|||
const dir = fixture({
|
||||
'STATE.md': `## Open decisions\n- [ ] marked\n${item}\n## Next\n`,
|
||||
'agents/a.md': 'x\n',
|
||||
'commands/r.md': 'a\n',
|
||||
'commands/r.md': 'Launch `a`\n',
|
||||
});
|
||||
try {
|
||||
const row = rowById(measure(dir, registry()), 'decisions');
|
||||
assert.equal(row.open, null, `unmarked "${item}" must make the tally unmeasurable`);
|
||||
assert.match(row.detail, /unmarked/);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
}
|
||||
});
|
||||
|
||||
// --- verdict + CLI ---------------------------------------------------------
|
||||
test('feat-after-freeze tally: not a git repo → n/a (red), never 0', () => {
|
||||
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
||||
try {
|
||||
const result = measure(dir, registry());
|
||||
const row = rowById(result, 'feat-after-freeze');
|
||||
assert.equal(row.open, null);
|
||||
assert.match(row.detail, /not measured/);
|
||||
assert.equal(result.green, false);
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
function writeRegistry(dir, reg) {
|
||||
test('feat-after-freeze tally: no freeze tag yet → n/a (red), and the detail names the tag', () => {
|
||||
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
||||
try {
|
||||
gitInit(dir);
|
||||
const row = rowById(measure(dir, registry()), 'feat-after-freeze');
|
||||
assert.equal(row.open, null);
|
||||
assert.match(row.detail, /end-state-freeze/);
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('feat-after-freeze tally: counts feat commits after the tag, 0 when there are none', () => {
|
||||
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
||||
try {
|
||||
gitInit(dir);
|
||||
gitCommit(dir, 'feat: before the freeze does not count');
|
||||
git(dir, 'tag', 'end-state-freeze');
|
||||
let row = rowById(measure(dir, registry()), 'feat-after-freeze');
|
||||
assert.equal(row.open, 0);
|
||||
assert.equal(row.total, 0);
|
||||
gitCommit(dir, 'fix(x): a defect fix is allowed');
|
||||
gitCommit(dir, 'feature: not a conventional feat');
|
||||
row = rowById(measure(dir, registry()), 'feat-after-freeze');
|
||||
assert.equal(row.open, 0);
|
||||
assert.equal(row.total, 2);
|
||||
gitCommit(dir, 'feat(research): a new capability');
|
||||
gitCommit(dir, 'feat!: a breaking capability');
|
||||
row = rowById(measure(dir, registry()), 'feat-after-freeze');
|
||||
assert.equal(row.open, 2);
|
||||
assert.equal(row.total, 4);
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
// --- registry integrity (the judged party must not own the ledger) ---------
|
||||
|
||||
const SAMPLE = registry({
|
||||
defects: [
|
||||
{ id: 'D-01', summary: 'first defect', probe: 'phrase', check: [{ path: 'a.md', pattern: 'bad', expect: 'match' }] },
|
||||
{ id: 'D-02', summary: 'second defect', probe: 'phrase', check: [{ path: 'a.md', pattern: 'worse', expect: 'match' }] },
|
||||
],
|
||||
experiments: [
|
||||
{ id: 'E-01', summary: 'first experiment', probe: 'phrase', check: [{ path: 'a.md', pattern: 'FLAG', expect: 'match' }] },
|
||||
{ id: 'E-02', summary: 'second experiment', probe: 'phrase', check: [{ path: 'a.md', pattern: 'OPT', expect: 'match' }] },
|
||||
],
|
||||
});
|
||||
const clone = (v) => JSON.parse(JSON.stringify(v));
|
||||
|
||||
test('verifyIntegrity: the frozen registry itself is intact; a new entry may be added', () => {
|
||||
const frozen = frozenFor(SAMPLE);
|
||||
assert.deepEqual(verifyIntegrity(SAMPLE, frozen), { ok: true, violations: [] });
|
||||
const grown = clone(SAMPLE);
|
||||
grown.defects.push({ id: 'D-03', summary: 'found later', probe: 'phrase', check: [{ path: 'a.md', pattern: 'new', expect: 'match' }] });
|
||||
assert.equal(verifyIntegrity(grown, frozen).ok, true);
|
||||
});
|
||||
|
||||
test('verifyIntegrity: every registry attack is a violation that names what moved', () => {
|
||||
const frozen = frozenFor(SAMPLE);
|
||||
const attacks = {
|
||||
'delete a defect': (r) => { r.defects = r.defects.filter((d) => d.id !== 'D-02'); },
|
||||
'delete an experiment': (r) => { r.experiments = r.experiments.filter((e) => e.id !== 'E-01'); },
|
||||
'weaken a defect check': (r) => { r.defects[0].check[0].pattern = '^NEVER MATCHES$'; },
|
||||
'weaken an experiment check': (r) => { r.experiments[1].check[0].expect = 'no-match'; },
|
||||
'drop a condition': (r) => { r.defects[0].check = []; },
|
||||
'point spawnSites at agents': (r) => { r.agents = { ...r.agents, spawnSites: 'agents/*.md' }; },
|
||||
'change the reference marker': (r) => { r.agents = { ...r.agents, referenceMarker: 'DORMANT' }; },
|
||||
'move the decisions section': (r) => { r.decisions = { ...r.decisions, section: '## Elsewhere' }; },
|
||||
'change the open marker': (r) => { r.decisions = { ...r.decisions, open: '^NEVER' }; },
|
||||
'rename the freeze tag': (r) => { r.freeze = { ...r.freeze, tag: 'some-future-tag' }; },
|
||||
'duplicate an id': (r) => { r.defects.push(clone(r.defects[0])); },
|
||||
};
|
||||
for (const [name, attack] of Object.entries(attacks)) {
|
||||
const reg = clone(SAMPLE);
|
||||
attack(reg);
|
||||
const v = verifyIntegrity(reg, frozen);
|
||||
assert.equal(v.ok, false, `${name} must be caught`);
|
||||
assert.ok(v.violations.length > 0 && v.violations.every((s) => typeof s === 'string' && s.length > 0), name);
|
||||
}
|
||||
});
|
||||
|
||||
test('verifyIntegrity: spawnSites may never include agents/, even if the manifest says so', () => {
|
||||
const reg = clone(SAMPLE);
|
||||
reg.agents = { ...reg.agents, spawnSites: 'agents/*.md' };
|
||||
const v = verifyIntegrity(reg, frozenFor(reg));
|
||||
assert.equal(v.ok, false);
|
||||
assert.ok(v.violations.some((s) => /agents\//.test(s)));
|
||||
});
|
||||
|
||||
test('checkSignature: key order does not matter; every field of a condition does', () => {
|
||||
const a = [{ path: 'a.md', pattern: 'x', flags: 'm', expect: 'match' }];
|
||||
const reordered = [{ expect: 'match', flags: 'm', pattern: 'x', path: 'a.md' }];
|
||||
assert.equal(checkSignature(a), checkSignature(reordered));
|
||||
for (const change of [{ path: 'b.md' }, { pattern: 'y' }, { flags: '' }, { expect: 'no-match' }, { section: '## 1.' }]) {
|
||||
assert.notEqual(checkSignature([{ ...a[0], ...change }]), checkSignature(a), JSON.stringify(change));
|
||||
}
|
||||
assert.match(checkSignature(a), /^[0-9a-f]{64}$/);
|
||||
});
|
||||
|
||||
test('measure: a missing frozen manifest makes integrity n/a and keeps the gate red', () => {
|
||||
const dir = fixture({ 'STATE.md': '## Open decisions\n' });
|
||||
try {
|
||||
const result = measure(dir, registry());
|
||||
assert.equal(result.integrity.ok, null);
|
||||
assert.match(result.integrity.detail, /end-state-frozen\.json/);
|
||||
assert.equal(result.green, false);
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
// --- render ----------------------------------------------------------------
|
||||
|
||||
test('render: the header counts only tallies that ARE 0, never n/a ones', () => {
|
||||
const result = {
|
||||
green: false,
|
||||
integrity: { ok: true, violations: [], detail: '' },
|
||||
rows: [
|
||||
{ id: 'defects', label: 'l', source: 's', open: 0, total: 1, target: 0, items: [], detail: '' },
|
||||
{ id: 'experiments', label: 'l', source: 's', open: null, total: null, target: 0, items: [], detail: 'not measurable: x' },
|
||||
{ id: 'dormant-agents', label: 'l', source: 's', open: 0, total: 3, target: 0, items: [], detail: '' },
|
||||
],
|
||||
};
|
||||
const text = render(result);
|
||||
assert.match(text, /RED \(2 of 3 tallies at 0\)/);
|
||||
assert.match(text, /\| experiments \| n\/a \| n\/a \|/);
|
||||
});
|
||||
|
||||
test('render: open phrase probes are labelled, closed ids are listed, integrity is shown', () => {
|
||||
const result = {
|
||||
green: false,
|
||||
integrity: { ok: false, violations: ['D-02 removed from the registry'], detail: '' },
|
||||
rows: [{
|
||||
id: 'defects', label: 'open pipeline defects', source: 's', open: 1, total: 2, target: 0, detail: '',
|
||||
items: [
|
||||
{ id: 'D-01', summary: 'still open', probe: 'phrase', status: 'open', detail: '' },
|
||||
{ id: 'D-06', summary: 'fixed', probe: 'byte', status: 'closed', detail: '' },
|
||||
],
|
||||
}],
|
||||
};
|
||||
const text = render(result);
|
||||
assert.match(text, /\[open · phrase probe\] D-01/);
|
||||
assert.match(text, /closed: D-06/);
|
||||
assert.match(text, /registry integrity: VIOLATED/);
|
||||
assert.match(text, /D-02 removed from the registry/);
|
||||
});
|
||||
|
||||
// --- CLI -------------------------------------------------------------------
|
||||
|
||||
function writeGateFiles(dir, reg, frozen = frozenFor(reg)) {
|
||||
mkdirSync(join(dir, 'scripts'), { recursive: true });
|
||||
writeFileSync(join(dir, 'scripts', 'end-state-registry.json'), JSON.stringify(reg, null, 2));
|
||||
mkdirSync(join(dir, 'tests', 'fixtures'), { recursive: true });
|
||||
writeFileSync(join(dir, 'tests', 'fixtures', 'end-state-frozen.json'), JSON.stringify(frozen, null, 2));
|
||||
}
|
||||
|
||||
function runGate(dir, extra = []) {
|
||||
return spawnSync(process.execPath, [GATE, '--root', dir, ...extra], { encoding: 'utf8' });
|
||||
}
|
||||
|
||||
test('CLI: all four tallies at 0 → green, exit 0; --json carries the rows', () => {
|
||||
const ALL_IDS = ['defects', 'experiments', 'dormant-agents', 'decisions', 'feat-after-freeze'];
|
||||
|
||||
test('CLI: all five tallies at 0 with an intact registry → green, exit 0; --json carries the rows', () => {
|
||||
const dir = fixture({
|
||||
'STATE.md': '## Open decisions\n- [x] done\n',
|
||||
'agents/a.md': '---\nname: a\n---\n',
|
||||
'commands/r.md': 'spawn `a`\n',
|
||||
'commands/r.md': 'Spawn `a`\n',
|
||||
});
|
||||
try {
|
||||
writeRegistry(dir, registry());
|
||||
writeGateFiles(dir, registry());
|
||||
gitInit(dir);
|
||||
git(dir, 'tag', 'end-state-freeze');
|
||||
const r = runGate(dir, ['--json']);
|
||||
assert.equal(r.status, 0, r.stderr);
|
||||
assert.equal(r.status, 0, r.stdout + r.stderr);
|
||||
const out = JSON.parse(r.stdout);
|
||||
assert.equal(out.green, true);
|
||||
assert.deepEqual(out.rows.map((x) => x.id), ['defects', 'experiments', 'dormant-agents', 'decisions']);
|
||||
assert.equal(out.integrity.ok, true);
|
||||
assert.deepEqual(out.rows.map((x) => x.id), ALL_IDS);
|
||||
assert.ok(out.rows.every((x) => x.open === 0 && x.target === 0 && typeof x.source === 'string'));
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('CLI: an intact registry with every measurable tally at 0 but no freeze tag → red, exit 1', () => {
|
||||
// n/a must never be read as 0 — also when nothing else is wrong.
|
||||
const dir = fixture({
|
||||
'STATE.md': '## Open decisions\n- [x] done\n',
|
||||
'agents/a.md': '---\nname: a\n---\n',
|
||||
'commands/r.md': 'Spawn `a`\n',
|
||||
});
|
||||
try {
|
||||
writeGateFiles(dir, registry());
|
||||
gitInit(dir);
|
||||
const r = runGate(dir, ['--json']);
|
||||
assert.equal(r.status, 1, r.stdout + r.stderr);
|
||||
const out = JSON.parse(r.stdout);
|
||||
assert.equal(out.integrity.ok, true);
|
||||
assert.equal(out.green, false);
|
||||
assert.equal(out.rows.find((x) => x.id === 'feat-after-freeze').open, null);
|
||||
assert.ok(out.rows.filter((x) => x.id !== 'feat-after-freeze').every((x) => x.open === 0));
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('CLI: all tallies at 0 but a tampered registry → red, exit 1 (the combined green attack)', () => {
|
||||
const dir = fixture({
|
||||
'STATE.md': '## Open decisions\n- [x] ticked\n',
|
||||
'agents/a.md': '---\nname: a\n---\n',
|
||||
'commands/r.md': 'Spawn `a`\n',
|
||||
});
|
||||
try {
|
||||
const honest = registry({ defects: [{ id: 'D-01', summary: 'a real defect', probe: 'phrase', check: [{ path: 'commands/r.md', pattern: 'Spawn', expect: 'match' }] }] });
|
||||
const tampered = registry(); // the open defect simply left the ledger
|
||||
writeGateFiles(dir, tampered, frozenFor(honest));
|
||||
gitInit(dir);
|
||||
git(dir, 'tag', 'end-state-freeze');
|
||||
const r = runGate(dir);
|
||||
assert.equal(r.status, 1, r.stdout + r.stderr);
|
||||
assert.match(r.stdout, /RED/);
|
||||
assert.match(r.stdout, /VIOLATED/);
|
||||
assert.match(r.stdout, /D-01/);
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('CLI: any tally > 0 → red, exit 1; the table names every row', () => {
|
||||
const dir = fixture({
|
||||
'STATE.md': '## Open decisions\n- [ ] undecided\n',
|
||||
'agents/a.md': '---\nname: a\n---\n',
|
||||
'commands/r.md': 'spawn `a`\n',
|
||||
'commands/r.md': 'Spawn `a`\n',
|
||||
});
|
||||
try {
|
||||
writeRegistry(dir, registry());
|
||||
writeGateFiles(dir, registry());
|
||||
const r = runGate(dir);
|
||||
assert.equal(r.status, 1, r.stderr);
|
||||
for (const label of ['defects', 'experiments', 'dormant-agents', 'decisions']) {
|
||||
assert.match(r.stdout, new RegExp(label));
|
||||
}
|
||||
for (const label of ALL_IDS) assert.match(r.stdout, new RegExp(label));
|
||||
assert.match(r.stdout, /RED/);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
test('CLI: an unreadable registry is a usage error (exit 2), not a verdict', () => {
|
||||
|
|
@ -326,36 +655,49 @@ test('CLI: an unreadable registry is a usage error (exit 2), not a verdict', ()
|
|||
const r = runGate(dir);
|
||||
assert.equal(r.status, 2);
|
||||
assert.match(r.stderr, /end-state-registry\.json/);
|
||||
} finally { rmSync(dir, { recursive: true, force: true }); }
|
||||
} finally { cleanup(dir); }
|
||||
});
|
||||
|
||||
// --- the real registry: shape, never state ---------------------------------
|
||||
// --- the real repo: denominator pinned, state never pinned -----------------
|
||||
|
||||
test('real registry: every counted entry has an id, a summary and a check that RUNS here', () => {
|
||||
test('the frozen denominator: tests/fixtures/end-state-frozen.json is exactly the literal above', () => {
|
||||
const file = loadFrozen(ROOT);
|
||||
const { why, ...rest } = file;
|
||||
assert.ok(typeof why === 'string' && why.length > 20, 'the manifest says why it is frozen');
|
||||
assert.deepEqual(rest, FROZEN);
|
||||
});
|
||||
|
||||
test('the frozen denominator: the real registry is intact against it', () => {
|
||||
const v = verifyIntegrity(loadRegistry(ROOT), FROZEN);
|
||||
assert.deepEqual(v, { ok: true, violations: [] }, v.violations.join('\n'));
|
||||
});
|
||||
|
||||
test('real registry: every counted entry has an id, a summary, a probe kind and a check that RUNS here', () => {
|
||||
const reg = loadRegistry(ROOT);
|
||||
const entries = [...reg.defects, ...reg.experiments];
|
||||
assert.ok(reg.defects.length >= 1 && reg.experiments.length >= 1);
|
||||
const ids = new Set();
|
||||
for (const e of entries) {
|
||||
assert.match(e.id, /^[DE]-\d{2}$/, `bad id ${e.id}`);
|
||||
assert.ok(!ids.has(e.id), `duplicate id ${e.id}`);
|
||||
ids.add(e.id);
|
||||
assert.ok(typeof e.summary === 'string' && e.summary.length > 10, `${e.id} needs a summary`);
|
||||
assert.ok(['phrase', 'byte'].includes(e.probe), `${e.id} must declare its probe kind`);
|
||||
// A check that cannot evaluate against this very repo is a typo, not a finding.
|
||||
assert.notEqual(evaluateCheck(ROOT, e.check).status, 'not-fellable', `${e.id} check does not run here`);
|
||||
}
|
||||
});
|
||||
|
||||
test('real repo: four rows, open is a count or null, exit code agrees with the rows', () => {
|
||||
test('real repo: five rows, open is a count or null, exit code agrees with rows + integrity', () => {
|
||||
const r = spawnSync(process.execPath, [GATE, '--json'], { encoding: 'utf8', cwd: ROOT });
|
||||
assert.ok(r.status === 0 || r.status === 1, r.stderr);
|
||||
const out = JSON.parse(r.stdout);
|
||||
assert.equal(out.rows.length, 4);
|
||||
assert.deepEqual(out.rows.map((x) => x.id), ALL_IDS);
|
||||
for (const row of out.rows) {
|
||||
assert.ok(row.open === null || (Number.isInteger(row.open) && row.open >= 0), `${row.id}: ${row.open}`);
|
||||
}
|
||||
const allZero = out.rows.every((row) => row.open === 0);
|
||||
assert.equal(out.green, allZero);
|
||||
assert.equal(r.status, allZero ? 0 : 1);
|
||||
const green = out.rows.every((row) => row.open === 0) && out.integrity.ok === true;
|
||||
assert.equal(out.green, green);
|
||||
assert.equal(r.status, green ? 0 : 1);
|
||||
assert.ok(existsSync(join(ROOT, 'scripts', 'end-state-registry.json')));
|
||||
assert.ok(readFileSync(join(ROOT, 'tests', 'fixtures', 'end-state-frozen.json'), 'utf8').length > 0);
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue