feat(gate): stop rule in the plan gate (decision 217, 220)
recordVerdict appends each round's summary to a new optional verdict field `rounds` (carried from the previous verdict; newByClass null for a round counted in full) and stores `stop_rule`; extractVerdict shape-checks both. After a fired stop rule --record exits 1 with PLAN_GATE_STOP_RULE and writes nothing (--extra-round included). The bare gate stops with PLAN_GATE_STOP_RULE and the operator line in place of the open-blocker and open-major reasons; --json and --record output carry the decision. --accept-open after a fired stop rule is allowed with rounds left, covers open blockers and majors (`under: "stop-rule"`), refuses a counting security finding (PLAN_GATE_SECURITY_OPEN); the gate then warns. Known issue fixed: REVIEW_LOG_SECTIONS drops "Adversarial Pass 2 (v5.1.1 high-effort)"; the CHANGELOG known-issue line closes and Handover 4 names the heading list, the new fields and codes. Existing tests updated for the planned rule: fixtures that ran a later round after one with no fewer blocking findings now use a falling count (plan-gate.test round cap, plan-gate-open-major lastRound, diff-mode 5/18/27 resolve two earlier majors in round 2); the diff-mode CLI pipe case now expects PLAN_GATE_STOP_RULE (round 3: 2 blocking) after its clean round 2. Part 1, step 1.2 of the E plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
7e09edb24f
commit
a576391e28
SSH key fingerprint:
SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
7 changed files with 448 additions and 32 deletions
|
|
@ -240,7 +240,6 @@ Fix round 4.5a (decisions 57, 67 and 68): the round's 30 findings, where a Voyag
|
|||
- **The query guard has no caller** (cleanup R-b). `lib/validators/query-privacy-gate.mjs` and its tests stay, but its callers were the removed STORM steps, so no research query passes through it; its header and `skills/researching-sources/` say so. Left for the design (187).
|
||||
- **Eleven agents take the session's effort** (cleanup R-b). Nine agents pin `effort: high`; the other eleven inherit the session level, so a session started below high runs them below high. Left for the design (187).
|
||||
- **A retired flag is dropped without a word** (cleanup R-b). A typed `--profile x` (or `/trekresearch --engine x`) is stripped with its value, and no command says the flag does nothing.
|
||||
- **The plan gate still leaves out a Pass 2 heading** (cleanup R-b). `lib/validators/plan-gate.mjs` keeps its exclusion of the "Adversarial Pass 2 (v5.1.1 high-effort)" heading that `/trekplan`'s high-effort path wrote, so older plans hash the same; no command writes the marker now.
|
||||
- **The lessons apply still skips `original_rule_key`** (cleanup R-b). `lib/review/recurrence.mjs` skips a finding's `original_rule_key`, which only the removed high-effort normalization wrote; harmless, left.
|
||||
- **Part A's end-to-end script no longer runs** (cleanup R-a). `tests/e2e-fix-round-4-5b2-a.sh` stays, but its callers were the removed round scripts, and `npm test` runs only `*.test.mjs`; it is listed for an upgrade (A1).
|
||||
- **The path check has blind spots** (cleanup R-a). `tests/lib/living-paths.test.mjs` skips a line with an `@<sha>` or `MIT` on it or on the two lines above, so a live path next to a credit is not checked, and it reads a backticked span only when the whole span is a path, so `lib/x.mjs --flag` is not checked unless it carries `${CLAUDE_PLUGIN_ROOT}/`. It reads only `.md` files, so a dead path in code (a script's default input, a comment) is not checked either.
|
||||
|
|
|
|||
|
|
@ -250,7 +250,7 @@ prose form (relaxed in S26 so the template's prose emission parses).
|
|||
- at least one step has `End-to-end:` with a backticked command — the "run the thing" step;
|
||||
- the optional step fields `Interface in:`, `Interface out:`, `Review focus:` are never empty when present (consumers tolerate their absence);
|
||||
- `ceremony` is one of the three classes; an `architectural` plan has `## Design` with the subsections Architecture, Data Flow, Interfaces, Data Models, Error Handling, Module Map, Architecture Rules, Requirements Check, plus `## Test Strategy`; the module map (mermaid) has no cycle; every architecture rule says "checked by `<command>`" (with a Proof column naming an existing step) or "advisory — `.claude/rules/<file>.md`".
|
||||
6. **`## Critic Verdict`** (written by `lib/validators/plan-gate.mjs --record`, never by hand): one fenced `json` block, schema `voyage.plan-verdict/1` — `round`, `max_rounds` (3), `extra_rounds`, `plan_sha256`, `verdict` (`BLOCK` | `REVISE` | `PASS`), `counts`, `findings` (deduplicated, each with `severity`). `plan_sha256` hashes the plan minus its review log (the `> **Plan quality:` line and the sections Critic Verdict, Revisions, Plan Quality Score, Adversarial Pass 2), with CRLF and line-end whitespace normalized. The verdict sits inside the plan because Claude Code's plan approval hands a hook the plan file's text and nothing beside it. **Optional rule-A fields** (decision 43; `DIFF_FIELDS` in `lib/review/critic-diff.mjs`): each one is optional, and absent means round-1 behaviour, so an older block gives the same exit codes. Top level: `diff_base` (the sha256 the round was counted against), `diff_note` (one line: why a round from round 2 on was counted fully), `resolved` (the ids of previous findings that left the count). Per finding: `op_line` and `op_line_end` (the cited lines in the operative text), `line_end`, `rule_keys` (the union of the keys dedup merged), `original_severity` (what a reviewer rated, set whenever the counted `severity` differs), `lowered_by` (`rule A: outside the diff since round <P>` or `rule B: <caught_by>`), `caught_by` (the test, `Regression:` command or gate a rule-B lowering names), `lowering_refused` (why a rule-B lowering was refused) and `carried_from` (the round a carried finding first appeared). `severity` is always the counted severity. `recordVerdict` called without `base`, `baseNote` and `planPaths` records as before, plus the rule-B record, and a per-finding `rule_keys` from dedup is always passed through (an optional field).
|
||||
6. **`## Critic Verdict`** (written by `lib/validators/plan-gate.mjs --record`, never by hand): one fenced `json` block, schema `voyage.plan-verdict/1` — `round`, `max_rounds` (3), `extra_rounds`, `plan_sha256`, `verdict` (`BLOCK` | `REVISE` | `PASS`), `counts`, `findings` (deduplicated, each with `severity`). `plan_sha256` hashes the plan minus its review log (the `> **Plan quality:` line and the sections Critic Verdict, Revisions, Plan Quality Score; an old Adversarial Pass 2 section is plan text and hashed), with CRLF and line-end whitespace normalized. The verdict sits inside the plan because Claude Code's plan approval hands a hook the plan file's text and nothing beside it. **Optional rule-A fields** (decision 43; `DIFF_FIELDS` in `lib/review/critic-diff.mjs`): each one is optional, and absent means round-1 behaviour, so an older block gives the same exit codes. Top level: `diff_base` (the sha256 the round was counted against), `diff_note` (one line: why a round from round 2 on was counted fully), `resolved` (the ids of previous findings that left the count). Per finding: `op_line` and `op_line_end` (the cited lines in the operative text), `line_end`, `rule_keys` (the union of the keys dedup merged), `original_severity` (what a reviewer rated, set whenever the counted `severity` differs), `lowered_by` (`rule A: outside the diff since round <P>` or `rule B: <caught_by>`), `caught_by` (the test, `Regression:` command or gate a rule-B lowering names), `lowering_refused` (why a rule-B lowering was refused) and `carried_from` (the round a carried finding first appeared). `severity` is always the counted severity. `recordVerdict` called without `base`, `baseNote` and `planPaths` records as before, plus the rule-B record, and a per-finding `rule_keys` from dedup is always passed through (an optional field). **Optional stop-rule fields** (decisions 217, 220; `lib/review/stop-rule.mjs`): `rounds` (one `{round, blocking, newByClass}` per recorded round, carried from the previous verdict; `newByClass` is `null` for a round counted in full) and `stop_rule` (`{fired, reason: same-class | not-fewer | null, classes, perRound}`); a verdict without them starts its history at the next round. `accepted_open.under` is `"stop-rule"` for an acceptance made after the stop rule fired.
|
||||
7. **Every plan_version, no version bump** (a plan without the field validates unchanged; voyage-lekerepo trial 2, 2026-10-08):
|
||||
- **K10 — `Regression:`** (optional step field): ONE backticked command the criteria runner's allowlist runs, with no pipe and no `&&` (first backticked span, as for `Verify:`). The executor runs it green before the step's checkpoint; a step without it runs nothing new. `Verify:` stays the step's own red-first test. One that can never run is `PLAN_STEP_REGRESSION_NOT_RUNNABLE` (an error; a warning under `--soft`).
|
||||
- **K11 — `On failure:`** opens with one of `revert`, `retry`, `skip`, `escalate` (case-insensitive; a leading `**`, `_`, backtick or `-` is ignored, other leading characters are not; `onFailureAction` in `lib/parsers/plan-schema.mjs`), the words `/trekexecute` reads; any other opening is the warning `PLAN_STEP_ON_FAILURE_UNKNOWN`. A missing `On failure:` is unchanged (the executor defaults to `escalate`).
|
||||
|
|
@ -298,7 +298,7 @@ The strongest validation in the entire pipeline. `/trekplan` Phase 8 **must** ru
|
|||
| `PLAN_STEP_PATH_NOT_A_PATH` | 1.9: a `Files:` entry opens with a backticked span that is no path (a call `parse()`, an option, a URL, a backticked placeholder such as `` `TBD` `` or `` `none` ``, a mixed-case bare name such as `` `Parser` ``; operator decision 22); error, a warning under `--soft` | Put the path first; the span goes in the description; a file named `Parser` is written `./Parser` |
|
||||
| `PLAN_STEP_PATH_UNTICKED` | Below 1.9, warning: the `Files:` reader found a path outside backticks; the plan is read as before (a warning under `--strict` too) | Backtick the path; from 1.9 an unticked path is refused |
|
||||
|
||||
**The plan gate (K3).** `node lib/validators/plan-gate.mjs <plan.md> [--json]` decides whether a plan may go on: **0** = it validates strict, its `## Critic Verdict` matches the current text, no BLOCKER is open, and every open MAJOR was accepted by the operator; **1** = stop, with `reasons` (`PLAN_GATE_INVALID_PLAN`, `PLAN_GATE_VERSION` — below plan_version 1.8, `PLAN_GATE_NO_VERDICT`, `PLAN_GATE_VERDICT_MALFORMED`, `PLAN_GATE_VERDICT_STALE`, `PLAN_GATE_OPEN_BLOCKER`, `PLAN_GATE_ROUNDS_EXHAUSTED`, `PLAN_GATE_OPEN_MAJOR` — rounds left: revise, `PLAN_GATE_OPEN_MAJOR_FINAL` — after the last round: ask the operator) and `warnings` (`PLAN_GATE_OPEN_MAJOR` for majors the operator accepted); **2** = could not read the plan. `--hash <plan.md>` prints the operative hash; Phase 9 takes it before the reviewers read the plan. `--record <plan.md> --reviewed <hash>` reads plan-review-dedup output on stdin (which lists the reviewers that delivered, `sources`) and writes the next round; exit 2 when a reviewer is missing or the plan changed since `--hash`; it refuses round 4 (exit 1, stop and ask) unless `--extra-round` follows the operator's yes, and refuses that flag before the cap is spent. Unrated findings count as blockers. The review log is excluded from the hash only up to its first other heading, so a step placed under it is hashed. `--accept-open <plan.md>` records the operator's "build as is" for the current round's open MAJORs (`accepted_open` in the verdict block, bound to the round and the plan hash; only after the last round; never for a BLOCKER; exit 1 when rounds are left, the verdict is stale, a blocker is open or no major is). Operator decision 2026-10-08: open majors used to be warnings, and a weakness the critics named was built and came back from `/trekreview` as MAJOR. plan-critic and scope-guardian rate on the review catalogue's tiers. The verdict is a trace of the review, not a signature.
|
||||
**The plan gate (K3).** `node lib/validators/plan-gate.mjs <plan.md> [--json]` decides whether a plan may go on: **0** = it validates strict, its `## Critic Verdict` matches the current text, no BLOCKER is open, and every open MAJOR was accepted by the operator; **1** = stop, with `reasons` (`PLAN_GATE_INVALID_PLAN`, `PLAN_GATE_VERSION` — below plan_version 1.8, `PLAN_GATE_NO_VERDICT`, `PLAN_GATE_VERDICT_MALFORMED`, `PLAN_GATE_VERDICT_STALE`, `PLAN_GATE_OPEN_BLOCKER`, `PLAN_GATE_ROUNDS_EXHAUSTED`, `PLAN_GATE_OPEN_MAJOR` — rounds left: revise, `PLAN_GATE_OPEN_MAJOR_FINAL` — after the last round: ask the operator, `PLAN_GATE_STOP_RULE` — the stop rule fired with a blocking finding open and not accepted; it replaces the open-blocker and open-major reasons) and `warnings` (`PLAN_GATE_OPEN_MAJOR` for majors the operator accepted, `PLAN_GATE_OPEN_BLOCKER` for blockers accepted under the stop rule); **2** = could not read the plan. `--hash <plan.md>` prints the operative hash; Phase 9 takes it before the reviewers read the plan. `--record <plan.md> --reviewed <hash>` reads plan-review-dedup output on stdin (which lists the reviewers that delivered, `sources`) and writes the next round; exit 2 when a reviewer is missing or the plan changed since `--hash`; it refuses round 4 (exit 1, stop and ask) unless `--extra-round` follows the operator's yes, and refuses that flag before the cap is spent; after a verdict whose stop rule fired it refuses any new round (exit 1, `PLAN_GATE_STOP_RULE`, `--extra-round` included). The stop rule: no new round after round N ≥ 2 when one finding class has new blocking findings in rounds N−1 and N, or when the blocking findings did not get fewer; a class is the catalogue rule key, else the reviewer's dimension (the `rule_key` before the first `/`). Unrated findings count as blockers. The review log is excluded from the hash only up to its first other heading, so a step placed under it is hashed. `--accept-open <plan.md>` records the operator's "build as is" for the current round's open MAJORs (`accepted_open` in the verdict block, bound to the round and the plan hash; only after the last round; never for a BLOCKER; exit 1 when rounds are left, the verdict is stale, a blocker is open or no major is). After the stop rule fired it is allowed with rounds left and accepts the open BLOCKERs and MAJORs as known issues, except a counting security finding (exit 1, `PLAN_GATE_SECURITY_OPEN`). Operator decision 2026-10-08: open majors used to be warnings, and a weakness the critics named was built and came back from `/trekreview` as MAJOR. plan-critic and scope-guardian rate on the review catalogue's tiers. The verdict is a trace of the review, not a signature.
|
||||
|
||||
**Rule A and the diff (decision 43).** Round 1 is unchanged. From round 2 on, `--record` counts the round against the operative text the previous round reviewed, and both reviewers mark each previous finding by id (`previous`). The gate counts by this rule, not the session:
|
||||
|
||||
|
|
|
|||
|
|
@ -15,8 +15,10 @@
|
|||
//
|
||||
// node plan-gate.mjs --record <plan.md> --reviewed <sha256> [--extra-round]
|
||||
// stdin = plan-review-dedup.mjs output. Writes this round's verdict into
|
||||
// the plan. 0 = written; 1 = refused: the round cap is spent (stop and ask
|
||||
// the operator; `--extra-round` only after their yes, and only then);
|
||||
// the plan. 0 = written; 1 = refused: the previous verdict's stop rule
|
||||
// fired (`PLAN_GATE_STOP_RULE`, below; `--extra-round` cannot open it) or
|
||||
// the round cap is spent (stop and ask the operator; `--extra-round` only
|
||||
// after their yes, and only then);
|
||||
// 2 = could not run: bad stdin, a reviewer missing from the dedup input,
|
||||
// the plan changed since --hash (the verdict would bless unreviewed text),
|
||||
// an unreadable plan or a failed write. Nothing is written on 1 or 2.
|
||||
|
|
@ -26,7 +28,9 @@
|
|||
// 0 it keeps this round's operative text as a snapshot (tmp + rename)
|
||||
// before the plan is written. Output: one JSON line, which adds
|
||||
// `diff_base`, `diff_note`, `lowered` (a number) and `snapshot` ("kept: ..."
|
||||
// or "not kept: <reason>"; a failed snapshot write does not fail the round).
|
||||
// or "not kept: <reason>"; a failed snapshot write does not fail the round),
|
||||
// `stop_rule` (this round's decision, or on a refusal the one that fired)
|
||||
// and `reason` (`PLAN_GATE_STOP_RULE` on that refusal, else null).
|
||||
//
|
||||
// node plan-gate.mjs --diff <plan.md>
|
||||
// Prints what the next critic round reviews: `diff-base: <sha256> (round N)`
|
||||
|
|
@ -45,7 +49,10 @@
|
|||
// MAJORs of the current round as accepted (`accepted_open` in the verdict
|
||||
// block, bound to the round and the plan hash). 0 = written; 1 = refused
|
||||
// (rounds left, a BLOCKER is open, no MAJOR is open, or the verdict is
|
||||
// stale); 2 = could not run.
|
||||
// stale); 2 = could not run. After the stop rule fired (decision 220) it
|
||||
// is allowed with rounds left and covers the open BLOCKERs and MAJORs as
|
||||
// known issues (`under: "stop-rule"`), except a counting security finding
|
||||
// (refused, `PLAN_GATE_SECURITY_OPEN`); the gate then warns on each.
|
||||
//
|
||||
// Open MAJORs (operator decision 2026-10-08, voyage-lekerepo trial point 6):
|
||||
// they used to pass as warnings, and after the last round there was no way to
|
||||
|
|
@ -62,9 +69,10 @@
|
|||
// What the verdict is bound to: sha256 of the plan's operative text — the plan
|
||||
// minus its review log, with CRLF and line-end whitespace normalized. The
|
||||
// review log is the `> **Plan quality:` header line and the level-2 sections
|
||||
// titled exactly Critic Verdict, Revisions, Plan Quality Score and
|
||||
// "Adversarial Pass 2 (v5.1.1 high-effort)" — each only up to its first
|
||||
// other heading, so a step or section placed under a log heading is hashed.
|
||||
// titled exactly Critic Verdict, Revisions and Plan Quality Score — each only
|
||||
// up to its first other heading, so a step or section placed under a log
|
||||
// heading is hashed. (The old "Adversarial Pass 2 (v5.1.1 high-effort)"
|
||||
// section is plan text now: no command writes it.)
|
||||
// Edit a step, the design or the verification after the critic ran, and the
|
||||
// verdict is stale until the next round.
|
||||
//
|
||||
|
|
@ -89,6 +97,20 @@
|
|||
// the same exits. Trust limit: anything that can write the plan can also
|
||||
// rewrite the previous verdict's plan_sha256 and plant a matching base; rule A
|
||||
// raises what such a forgery buys, and an unchanged base lowers nothing.
|
||||
//
|
||||
// The stop rule (decisions 217, 220; lib/review/stop-rule.mjs): each round
|
||||
// appends its summary (blocking findings, and the new blocking findings per
|
||||
// class unless the round was counted in full) to the optional verdict field
|
||||
// `rounds`, carried from the previous verdict, and stores the decision as
|
||||
// `stop_rule`. Once it fired — one class with new blocking findings two
|
||||
// rounds in a row, or blocking findings that did not get fewer — there is no
|
||||
// new round: --record refuses (exit 1, PLAN_GATE_STOP_RULE, `--extra-round`
|
||||
// included), and the bare gate stops with PLAN_GATE_STOP_RULE and the
|
||||
// operator line (the class, the counts per round, "known issue" or "new
|
||||
// mechanism") in place of the open-blocker and open-major reasons, until the
|
||||
// operator accepts the open findings (--accept-open) or the direction
|
||||
// changes. A verdict without `rounds` starts its history at the next round,
|
||||
// so an older block never fires on its first new round.
|
||||
|
||||
import {
|
||||
readFileSync, existsSync, writeFileSync, renameSync, rmSync, statSync, mkdirSync, realpathSync,
|
||||
|
|
@ -97,8 +119,9 @@ import { createHash } from 'node:crypto';
|
|||
import { dirname, join, resolve } from 'node:path';
|
||||
import { validatePlanContent } from './plan-validator.mjs';
|
||||
import { splitSections, fenceTracker, extractPlanVersion, versionAtLeast } from '../parsers/plan-schema.mjs';
|
||||
import { normalizeSeverity } from '../review/plan-review-dedup.mjs';
|
||||
import { normalizeSeverity, isSecurity } from '../review/plan-review-dedup.mjs';
|
||||
import { countRound, lineDiff, oneLine } from '../review/critic-diff.mjs';
|
||||
import { roundSummary, decideStop, describeStop, STOP_REASONS } from '../review/stop-rule.mjs';
|
||||
import { resolveRepo } from '../verification/repo-tools.mjs';
|
||||
import { isMain } from '../util/is-main.mjs';
|
||||
|
||||
|
|
@ -107,9 +130,7 @@ export const VERDICT_SCHEMA = 'voyage.plan-verdict/1';
|
|||
export const VERDICT_HEADING = '## Critic Verdict';
|
||||
export const REQUIRED_REVIEWERS = Object.freeze(['plan-critic', 'scope-guardian']);
|
||||
// Level-2 sections that are review log, not plan: excluded from the hash.
|
||||
export const REVIEW_LOG_SECTIONS = Object.freeze([
|
||||
'Critic Verdict', 'Revisions', 'Plan Quality Score', 'Adversarial Pass 2 (v5.1.1 high-effort)',
|
||||
]);
|
||||
export const REVIEW_LOG_SECTIONS = Object.freeze(['Critic Verdict', 'Revisions', 'Plan Quality Score']);
|
||||
// The quality line alone; a line that also carries `plan_version`/`ceremony` stays hashed.
|
||||
const QUALITY_LINE = /^> \*\*Plan quality:[^`]*$/;
|
||||
|
||||
|
|
@ -210,13 +231,32 @@ export function extractVerdict(text) {
|
|||
Number.isInteger(v.max_rounds) && v.max_rounds >= 1 && v.max_rounds <= MAX_ROUNDS &&
|
||||
Number.isInteger(v.extra_rounds ?? 0) && (v.extra_rounds ?? 0) >= 0 &&
|
||||
typeof v.plan_sha256 === 'string' && Array.isArray(v.findings) &&
|
||||
v.findings.every(f => f && typeof f === 'object' && typeof f.severity === 'string');
|
||||
v.findings.every(f => f && typeof f === 'object' && typeof f.severity === 'string') &&
|
||||
okRounds(v.rounds) && okStopRule(v.stop_rule);
|
||||
if (!okShape) {
|
||||
return { ...none, found: true, error: `verdict block is not ${VERDICT_SCHEMA} (schema, round, max_rounds <= ${MAX_ROUNDS}, extra_rounds, plan_sha256, findings with severity)` };
|
||||
return { ...none, found: true, error: `verdict block is not ${VERDICT_SCHEMA} (schema, round, max_rounds <= ${MAX_ROUNDS}, extra_rounds, plan_sha256, findings with severity, optional rounds and stop_rule)` };
|
||||
}
|
||||
return { found: true, placeholder: false, verdict: v, error: null };
|
||||
}
|
||||
|
||||
const isCount = n => Number.isInteger(n) && n >= 0;
|
||||
const isPlainObject = o => o !== null && typeof o === 'object' && !Array.isArray(o);
|
||||
|
||||
/** Optional `rounds`: a list of {round >= 1, blocking >= 0, newByClass: null | {class: count}}. */
|
||||
function okRounds(r) {
|
||||
if (r === undefined) return true;
|
||||
return Array.isArray(r) && r.every(x => isPlainObject(x) && Number.isInteger(x.round) && x.round >= 1 &&
|
||||
isCount(x.blocking) && (x.newByClass === null || (isPlainObject(x.newByClass) && Object.values(x.newByClass).every(isCount))));
|
||||
}
|
||||
|
||||
/** Optional `stop_rule`: {fired: boolean, reason: one of STOP_REASONS when fired else null, classes: string[], perRound: list}. */
|
||||
function okStopRule(s) {
|
||||
if (s === undefined) return true;
|
||||
return isPlainObject(s) && typeof s.fired === 'boolean' &&
|
||||
(s.fired ? STOP_REASONS.includes(s.reason) : s.reason === null) &&
|
||||
Array.isArray(s.classes) && s.classes.every(c => typeof c === 'string') && okRounds(s.perRound);
|
||||
}
|
||||
|
||||
function verdictFor(counts) {
|
||||
if (counts.blocker > 0) return 'BLOCK';
|
||||
if (counts.major > 0) return 'REVISE';
|
||||
|
|
@ -292,6 +332,13 @@ export function recordVerdict(text, dedup, opts = {}) {
|
|||
}
|
||||
const prev = extractVerdict(text);
|
||||
if (prev.error && !prev.placeholder) return { code: 2, message: `existing verdict unreadable: ${prev.error}` };
|
||||
// The stop rule (decision 217): once it fired there is no new round, whatever the cap or --extra-round says.
|
||||
if (prev.verdict?.stop_rule?.fired === true) {
|
||||
return {
|
||||
code: 1, reason: 'PLAN_GATE_STOP_RULE', stop_rule: prev.verdict.stop_rule,
|
||||
message: `PLAN_GATE_STOP_RULE: no round ${prev.verdict.round + 1}. ${describeStop(prev.verdict.stop_rule)}`,
|
||||
};
|
||||
}
|
||||
const maxRounds = Math.min(prev.verdict?.max_rounds ?? opts.maxRounds ?? MAX_ROUNDS, MAX_ROUNDS);
|
||||
const prevExtra = prev.verdict?.extra_rounds ?? 0;
|
||||
const round = (prev.verdict?.round ?? 0) + 1;
|
||||
|
|
@ -335,9 +382,16 @@ export function recordVerdict(text, dedup, opts = {}) {
|
|||
verdict.counts = counts;
|
||||
verdict.findings = findings;
|
||||
if (c.resolved !== null) verdict.resolved = c.resolved;
|
||||
// The stop rule's history: this round's summary after the carried ones. A
|
||||
// round with a diff_note (or, outside diff mode, any round from 2 on) was
|
||||
// counted in full, so "new" is not known: newByClass null.
|
||||
const history = Array.isArray(prev.verdict?.rounds) ? prev.verdict.rounds : [];
|
||||
verdict.rounds = [...history, { round, ...roundSummary(findings, { fullCount: c.diffNote !== null }) }];
|
||||
verdict.stop_rule = decideStop(verdict.rounds);
|
||||
return {
|
||||
code: 0, text: placeSection(text, renderSection(verdict)), verdict, lowered: c.lowered,
|
||||
message: `round ${round} recorded: ${verdict.verdict}`,
|
||||
message: `round ${round} recorded: ${verdict.verdict}` +
|
||||
(verdict.stop_rule.fired ? `; PLAN_GATE_STOP_RULE: ${describeStop(verdict.stop_rule)}` : ''),
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -354,7 +408,10 @@ function acceptanceCovers(v, majors) {
|
|||
|
||||
/**
|
||||
* Record the operator's "build as is" for the current round's open MAJORs.
|
||||
* Only after their explicit yes; never for a BLOCKER.
|
||||
* Only after their explicit yes; never for a BLOCKER — except after the stop
|
||||
* rule fired (decision 220): then the open BLOCKERs and MAJORs are accepted as
|
||||
* known issues, with rounds left too, unless one is a counting security
|
||||
* finding (never accepted).
|
||||
* @returns {{code: 0|1|2, text?: string, message: string}}
|
||||
*/
|
||||
export function acceptOpen(rawText, opts = {}) {
|
||||
|
|
@ -362,7 +419,33 @@ export function acceptOpen(rawText, opts = {}) {
|
|||
const ex = extractVerdict(text);
|
||||
if (!ex.found || ex.error) return { code: 2, message: `no readable verdict: ${ex.error ?? 'no ## Critic Verdict'}` };
|
||||
const v = ex.verdict;
|
||||
if (v.plan_sha256 !== planHash(text)) return { code: 1, message: 'the verdict is stale: the plan changed after the last round; run the next round instead' };
|
||||
if (v.plan_sha256 !== planHash(text)) {
|
||||
return {
|
||||
code: 1,
|
||||
message: v.stop_rule?.fired === true
|
||||
? 'the verdict is stale: the plan changed after the last round, and the stop rule fired, so there is no next round; restore the reviewed text to accept, or stop and take the direction to the operator'
|
||||
: 'the verdict is stale: the plan changed after the last round; run the next round instead',
|
||||
};
|
||||
}
|
||||
if (v.stop_rule?.fired === true) {
|
||||
const open = v.findings.filter(f => sevOf(f) !== 'minor');
|
||||
if (open.length === 0) return { code: 1, message: 'no BLOCKER or MAJOR is open; nothing to accept' };
|
||||
const security = open.filter(isSecurity);
|
||||
if (security.length > 0) {
|
||||
return { code: 1, message: `PLAN_GATE_SECURITY_OPEN: ${security.length} open security finding(s) count and are never accepted: ${security.map(s => s.text).join(' | ')}` };
|
||||
}
|
||||
const accepted = {
|
||||
...v,
|
||||
accepted_open: {
|
||||
round: v.round,
|
||||
plan_sha256: v.plan_sha256,
|
||||
accepted_at: opts.now ?? new Date().toISOString(),
|
||||
under: 'stop-rule',
|
||||
findings: open.map(m => ({ id: m.id ?? null, text: m.text ?? '' })),
|
||||
},
|
||||
};
|
||||
return { code: 0, text: placeSection(text, renderSection(accepted)), message: `${open.length} open finding(s) of round ${v.round} accepted as known issues under the stop rule` };
|
||||
}
|
||||
if (v.findings.some(f => sevOf(f) !== 'major' && sevOf(f) !== 'minor')) return { code: 1, message: 'a BLOCKER is open; a blocker is never accepted' };
|
||||
const majors = v.findings.filter(f => sevOf(f) === 'major');
|
||||
if (majors.length === 0) return { code: 1, message: 'no MAJOR is open; nothing to accept' };
|
||||
|
|
@ -384,7 +467,7 @@ export function acceptOpen(rawText, opts = {}) {
|
|||
|
||||
/**
|
||||
* K3 — decide whether the plan may go on.
|
||||
* @returns {{code: 0|1, status: 'pass'|'stop', reasons: object[], warnings: object[], verdict: object|null, validation: object}}
|
||||
* @returns {{code: 0|1, status: 'pass'|'stop', reasons: object[], warnings: object[], verdict: object|null, stop_rule: object|null, validation: object}}
|
||||
*/
|
||||
export function gatePlan(rawText) {
|
||||
const text = norm(rawText);
|
||||
|
|
@ -411,13 +494,32 @@ export function gatePlan(rawText) {
|
|||
const sev = f => normalizeSeverity(f.severity);
|
||||
const blockers = v.findings.filter(f => sev(f) !== 'major' && sev(f) !== 'minor');
|
||||
const majors = v.findings.filter(f => sev(f) === 'major');
|
||||
if (blockers.length > 0) {
|
||||
const stop = v.stop_rule?.fired === true ? v.stop_rule : null;
|
||||
if (stop) {
|
||||
// The stop rule replaces the open-blocker and open-major reasons: no new
|
||||
// round, the operator decides (decision 220). Covered = accepted under
|
||||
// the stop rule for this round and text, with no security finding open.
|
||||
const open = [...blockers, ...majors];
|
||||
const a = v.accepted_open;
|
||||
const covered = open.length > 0 && a?.under === 'stop-rule' && acceptanceCovers(v, open) &&
|
||||
v.plan_sha256 === planHash(text) && !open.some(isSecurity);
|
||||
if (open.length > 0 && !covered) {
|
||||
reasons.push({ code: 'PLAN_GATE_STOP_RULE', message: describeStop(stop) });
|
||||
} else if (covered) {
|
||||
if (blockers.length > 0) {
|
||||
warnings.push({ code: 'PLAN_GATE_OPEN_BLOCKER', message: `${blockers.length} BLOCKER(s) open in round ${v.round}, accepted by the operator as known issues after the stop rule (${a.accepted_at}): ${blockers.map(b => b.text).join(' | ')}` });
|
||||
}
|
||||
if (majors.length > 0) {
|
||||
warnings.push({ code: 'PLAN_GATE_OPEN_MAJOR', message: `${majors.length} MAJOR finding(s) open in round ${v.round}, accepted by the operator as known issues after the stop rule (${a.accepted_at}): ${majors.map(m => m.text).join(' | ')}` });
|
||||
}
|
||||
}
|
||||
} else if (blockers.length > 0) {
|
||||
reasons.push({ code: 'PLAN_GATE_OPEN_BLOCKER', message: `${blockers.length} open BLOCKER(s) in critic round ${v.round}: ${blockers.map(b => b.text).join(' | ')}` });
|
||||
if (v.round >= v.max_rounds + (v.extra_rounds ?? 0)) {
|
||||
reasons.push({ code: 'PLAN_GATE_ROUNDS_EXHAUSTED', message: `All ${v.round} critic rounds are spent with a BLOCKER open: stop and ask the operator.` });
|
||||
}
|
||||
}
|
||||
if (majors.length > 0) {
|
||||
if (!stop && majors.length > 0) {
|
||||
const exhausted = v.round >= v.max_rounds + (v.extra_rounds ?? 0);
|
||||
const list = majors.map(m => m.text).join(' | ');
|
||||
if (acceptanceCovers(v, majors) && v.plan_sha256 === planHash(text)) {
|
||||
|
|
@ -433,7 +535,10 @@ export function gatePlan(rawText) {
|
|||
}
|
||||
}
|
||||
}
|
||||
return { code: reasons.length === 0 ? 0 : 1, status: reasons.length === 0 ? 'pass' : 'stop', reasons, warnings, verdict: v ?? null, validation };
|
||||
return {
|
||||
code: reasons.length === 0 ? 0 : 1, status: reasons.length === 0 ? 'pass' : 'stop', reasons, warnings,
|
||||
verdict: v ?? null, stop_rule: v?.stop_rule ?? null, validation,
|
||||
};
|
||||
}
|
||||
|
||||
// ---- CLI --------------------------------------------------------------------
|
||||
|
|
@ -624,12 +729,13 @@ if (isMain(import.meta.url)) {
|
|||
verdict: r.verdict?.verdict ?? null, counts: r.verdict?.counts ?? null,
|
||||
diff_base: r.verdict?.diff_base ?? null, diff_note: r.verdict?.diff_note ?? null,
|
||||
lowered: r.code === 0 ? r.lowered : null, snapshot,
|
||||
stop_rule: r.verdict?.stop_rule ?? r.stop_rule ?? null, reason: r.reason ?? null,
|
||||
});
|
||||
process.exit(r.code);
|
||||
}
|
||||
const g = gatePlan(text);
|
||||
if (args.includes('--json')) {
|
||||
process.stdout.write(JSON.stringify({ code: g.code, status: g.status, plan: planPath, reasons: g.reasons, warnings: g.warnings, verdict: g.verdict }, null, 2) + '\n');
|
||||
process.stdout.write(JSON.stringify({ code: g.code, status: g.status, plan: planPath, reasons: g.reasons, warnings: g.warnings, stop_rule: g.stop_rule, verdict: g.verdict }, null, 2) + '\n');
|
||||
} else {
|
||||
process.stdout.write(
|
||||
`plan-gate: ${g.status.toUpperCase()} ${planPath}\n` +
|
||||
|
|
|
|||
|
|
@ -108,6 +108,17 @@ function assertGuardLowered(v, round, text = GUARD) {
|
|||
/** Round 1 on buildPlan() with the given payloads; returns its result. */
|
||||
const round1 = (payload = {}) => rec(buildPlan(), dd(payload));
|
||||
|
||||
// The stop rule (lib/review/stop-rule.mjs, decision 217) gives no round after
|
||||
// one whose blocking findings did not get fewer. A case about a later round's
|
||||
// counting opens with `EARLIER`: two round-1 majors of their own class, marked
|
||||
// resolved in round 2 by `earlierGone`, so the blocking count falls.
|
||||
const EARLIER = [
|
||||
['Earlier point: the manifest omits the generated fixture folder entirely', S1_VERIFY],
|
||||
['Earlier point: the checkpoint message pattern never matches a scoped commit', S1_ROLLBACK],
|
||||
];
|
||||
const earlier = t => EARLIER.map(([text, line]) => f('major', at(t, line), text, { rule_key: 'earlier' }));
|
||||
const earlierGone = v => EARLIER.map(([text]) => ({ id: byText(v, text).id, resolved: true }));
|
||||
|
||||
// ---- exports ---------------------------------------------------------------------
|
||||
|
||||
test('critic-diff exports the canonical rule sentences, the new verdict fields and the security check', () => {
|
||||
|
|
@ -214,9 +225,10 @@ test('4: a new security major outside the diff stays major (catalogue key in any
|
|||
// ---- 5 -------------------------------------------------------------------------
|
||||
|
||||
test('5: a new blocker outside the diff stays blocker (BLOCK, PLAN_GATE_OPEN_BLOCKER)', () => {
|
||||
const r1 = round1();
|
||||
const r1 = round1({ pc: earlier(buildPlan()) });
|
||||
const t2 = editA(r1.text);
|
||||
const r2 = rec(t2, dd({ pc: [guard(t2), f('blocker', at(t2, S1_CHANGES), 'Step 1 reads a config value that no step ever writes')] }), { base: baseOf(r1.text) });
|
||||
const r2 = rec(t2, dd({ pc: [guard(t2), f('blocker', at(t2, S1_CHANGES), 'Step 1 reads a config value that no step ever writes')], pcPrev: earlierGone(r1.verdict) }), { base: baseOf(r1.text) });
|
||||
assert.equal(r2.verdict.stop_rule?.fired, false, 'fewer blocking findings: the stop rule is quiet');
|
||||
const v = r2.verdict;
|
||||
assertGuardLowered(v, 1);
|
||||
assert.equal(byText(v, 'Step 1 reads a config value that no step ever writes').severity, 'blocker');
|
||||
|
|
@ -630,11 +642,13 @@ test('18: carried positions — another file keeps its line; an edited own line
|
|||
{ file: 'src/mod1.mjs', line: 12, rule_key: 'step-quality', severity: 'major', text: 'Source module already exports a conflicting run helper' },
|
||||
f('major', ownOld, 'Second step change names no return value for run2'),
|
||||
f('major', shiftedOld, 'Step 1 never says what run1 returns on an empty input file'),
|
||||
...earlier(p0),
|
||||
],
|
||||
});
|
||||
const ids = Object.fromEntries(r1.verdict.findings.map(x => [x.text, x.id]));
|
||||
const t2 = editA(editC(r1.text));
|
||||
const r2 = rec(t2, dd({ pc: [guard(t2)] }), { base: baseOf(r1.text) });
|
||||
const r2 = rec(t2, dd({ pc: [guard(t2)], pcPrev: earlierGone(r1.verdict) }), { base: baseOf(r1.text) });
|
||||
assert.equal(r2.verdict.stop_rule?.fired, false, 'fewer blocking findings: the stop rule is quiet');
|
||||
const v2 = r2.verdict;
|
||||
assertGuardLowered(v2, 1);
|
||||
|
||||
|
|
@ -985,7 +999,9 @@ test('CLI pipe: dedup --stdin into --record counts round 2 by rule A, and a refu
|
|||
assert.match(r3.out, /"blocker":2/, `round 3 counts two blockers: ${r3.out}`);
|
||||
const g3 = gate(dir, [p]);
|
||||
assert.equal(g3.code, 1, g3.out + g3.err);
|
||||
assert.match(g3.out + g3.err, /PLAN_GATE_OPEN_BLOCKER/);
|
||||
// Two blockers after a clean round 2: the stop rule ("not fewer") fired, and
|
||||
// its STOP line counts both as blocking in place of PLAN_GATE_OPEN_BLOCKER.
|
||||
assert.match(g3.out + g3.err, /PLAN_GATE_STOP_RULE\] .*round 3: 2 blocking/);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -1038,7 +1054,7 @@ test('26: an unrelated major at an old finding\'s place, from the other reviewer
|
|||
test('27: a new file-level major with an old rule-B-lowered finding\'s id counts at major with its own text and id, and survives the old one\'s resolution', () => {
|
||||
const OLD = 'The plan names no owner for the generated fixtures';
|
||||
const r1 = round1({
|
||||
pc: [{ file: 'plan.md', line: 0, rule_key: 'gap', severity: 'minor', lowered_from: 'major', caught_by: 'npm test', text: OLD }],
|
||||
pc: [{ file: 'plan.md', line: 0, rule_key: 'gap', severity: 'minor', lowered_from: 'major', caught_by: 'npm test', text: OLD }, ...earlier(buildPlan())],
|
||||
});
|
||||
const o1 = byText(r1.verdict, OLD);
|
||||
assert.equal(o1.severity, 'minor');
|
||||
|
|
@ -1049,7 +1065,9 @@ test('27: a new file-level major with an old rule-B-lowered finding\'s id counts
|
|||
const NEW2 = 'The rollout order of the two migrations is never stated';
|
||||
const r2 = rec(t2, dd({
|
||||
pc: [{ file: 'plan.md', line: 0, rule_key: 'gap', severity: 'major', text: NEW2 }, guard(t2)],
|
||||
pcPrev: earlierGone(r1.verdict),
|
||||
}), { base: baseOf(r1.text) });
|
||||
assert.equal(r2.verdict.stop_rule?.fired, false, 'fewer blocking findings: the stop rule is quiet');
|
||||
const v2 = r2.verdict;
|
||||
const n2 = byText(v2, NEW2);
|
||||
assert.equal(n2.severity, 'major');
|
||||
|
|
|
|||
|
|
@ -29,12 +29,19 @@ const dedup = (...f) => ({ findings: f, sources: BOTH });
|
|||
const rec = (text, d, opts = {}) => recordVerdict(text, d, { reviewed: planHash(text), ...opts });
|
||||
const codes = (list) => list.map(x => x.code);
|
||||
|
||||
// The earlier rounds hold more open majors than the last (a falling count), so
|
||||
// the stop rule (lib/review/stop-rule.mjs, "not fewer") does not fire and the
|
||||
// last round is judged by the open-major rules alone.
|
||||
function lastRound(...f) {
|
||||
let text = buildPlan();
|
||||
for (let i = 1; i < MAX_ROUNDS; i++) text = rec(text, dedup()).text;
|
||||
for (let i = 1; i < MAX_ROUNDS; i++) {
|
||||
const earlier = Array.from({ length: MAX_ROUNDS - i + 5 }, (_, k) => finding('major', `earlier round ${i} open point ${'x'.repeat(k)}`));
|
||||
text = rec(text, dedup(...earlier)).text;
|
||||
}
|
||||
const r = rec(text, dedup(...f));
|
||||
assert.equal(r.code, 0, r.message);
|
||||
assert.equal(r.verdict.round, MAX_ROUNDS);
|
||||
assert.equal(r.verdict.stop_rule?.fired, false, 'the stop rule does not fire on a falling count');
|
||||
return r.text;
|
||||
}
|
||||
|
||||
|
|
@ -76,7 +83,8 @@ test('--accept-open never accepts a BLOCKER, and refuses when no MAJOR is open',
|
|||
});
|
||||
|
||||
test('an acceptance does not carry over to a later round', () => {
|
||||
const accepted = acceptOpen(lastRound(finding('major', WEAK))).text;
|
||||
// Two majors accepted, one in the extra round: fewer, so the stop rule stays quiet.
|
||||
const accepted = acceptOpen(lastRound(finding('major', WEAK), finding('major', 'Step 3 verify line never names the failing assertion'))).text;
|
||||
const revised = accepted.replace('Add `run1()`', 'Add `run1()` and its error path');
|
||||
const next = rec(revised, dedup(finding('major', 'Step 1 error path untested')), { extraRound: true });
|
||||
assert.equal(next.code, 0, next.message);
|
||||
|
|
|
|||
281
tests/validators/plan-gate-stop-rule.test.mjs
Normal file
281
tests/validators/plan-gate-stop-rule.test.mjs
Normal file
|
|
@ -0,0 +1,281 @@
|
|||
// tests/validators/plan-gate-stop-rule.test.mjs
|
||||
// Part 1, step 1.2 of the E plan (decisions 217, 220): the stop rule in the
|
||||
// plan gate (lib/validators/plan-gate.mjs, rules in lib/review/stop-rule.mjs).
|
||||
//
|
||||
// - recordVerdict appends each round's summary to the optional verdict field
|
||||
// `rounds` (carried from the previous verdict) and stores `stop_rule`;
|
||||
// extractVerdict shape-checks both.
|
||||
// - --record after a verdict whose stop rule fired: exit 1,
|
||||
// PLAN_GATE_STOP_RULE, nothing written; --extra-round cannot open it.
|
||||
// - The bare gate: the stop rule fired and a blocking finding open and not
|
||||
// accepted -> STOP [PLAN_GATE_STOP_RULE], replacing the open-blocker and
|
||||
// open-major reasons.
|
||||
// - --accept-open (220): allowed with rounds left once the stop rule fired,
|
||||
// covers blockers and majors, refuses a counting security finding
|
||||
// (PLAN_GATE_SECURITY_OPEN), recorded `under: "stop-rule"`; the gate turns
|
||||
// each covered blocker into a WARN.
|
||||
// - Known issue fixed: "Adversarial Pass 2 (v5.1.1 high-effort)" is no longer
|
||||
// review log, so a plan holding it hashes differently.
|
||||
//
|
||||
// The record cases follow tests/validators/plan-gate-diff-mode.test.mjs test 1
|
||||
// (round 1 on buildPlan(), an edit, round 2 against the base); the CLI case
|
||||
// runs the gate as a child in its own temp `git init` repo.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { resolve, join } from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { mkdtempSync, writeFileSync, readFileSync, rmSync, realpathSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
gatePlan, recordVerdict, planHash, operativeText, acceptOpen, extractVerdict, REVIEW_LOG_SECTIONS,
|
||||
} from '../../lib/validators/plan-gate.mjs';
|
||||
import { dedupFindings } from '../../lib/review/plan-review-dedup.mjs';
|
||||
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
|
||||
import { childEnv } from '../helpers/child-env.mjs';
|
||||
|
||||
const NOW = '2026-11-08T00:00:00.000Z';
|
||||
const PLAN = resolve('plan.md');
|
||||
const PATHS = [PLAN];
|
||||
|
||||
function at(text, s) {
|
||||
const n = text.split('\n').findIndex(l => l.includes(s)) + 1;
|
||||
assert.ok(n > 0, `the plan holds ${JSON.stringify(s)}`);
|
||||
return n;
|
||||
}
|
||||
const baseOf = t => ({ sha256: planHash(t), text: operativeText(t) });
|
||||
function rec(text, d, opts = {}) {
|
||||
const r = recordVerdict(text, d, { reviewed: planHash(text), planPaths: PATHS, now: NOW, ...opts });
|
||||
assert.equal(r.code, 0, r.message);
|
||||
return r;
|
||||
}
|
||||
function dd({ pc = [], sg = [], pcPrev, sgPrev } = {}) {
|
||||
return dedupFindings([
|
||||
{ agent: 'plan-critic', payload: { findings: pc, ...(pcPrev ? { previous: pcPrev } : {}) } },
|
||||
{ agent: 'scope-guardian', payload: { findings: sg, ...(sgPrev ? { previous: sgPrev } : {}) } },
|
||||
]);
|
||||
}
|
||||
const f = (severity, line, text, rule_key, extra = {}) => ({ file: 'plan.md', line, rule_key, severity, text, ...extra });
|
||||
|
||||
const R2_CHANGES = '- **Changes:** Add `run2()` to `src/mod2.mjs`. (new file)';
|
||||
const R2_EDITED = '- **Changes:** Add `run2()` and its error path to `src/mod2.mjs`. (new file)\n - `run2()` throws on empty input.';
|
||||
const R1_CHANGES = 'Add `run1()`';
|
||||
const editA = t => { assert.ok(t.includes(R2_CHANGES)); return t.replace(R2_CHANGES, R2_EDITED); };
|
||||
|
||||
const B1 = 'The dependency on the parser module is never declared in the manifest list';
|
||||
const B2 = 'A second import chain reaches the CLI without passing through its argument checker';
|
||||
const M2 = 'Rollback wording names a file the step never touches anywhere in this plan';
|
||||
|
||||
/** Round 1: one blocker of class `dep`; round 2 (after editA): the round-1 blocker resolved, one new `dep` blocker. */
|
||||
function sameClassTwice() {
|
||||
const t1 = buildPlan();
|
||||
const r1 = rec(t1, dd({ pc: [f('blocker', at(t1, R1_CHANGES), B1, 'dep/manifest')] }));
|
||||
const id = r1.verdict.findings[0].id;
|
||||
const t2 = editA(r1.text);
|
||||
const r2 = rec(t2, dd({
|
||||
pc: [f('blocker', at(t2, 'its error path'), B2, 'Dep/import-chain')],
|
||||
pcPrev: [{ id, resolved: true }],
|
||||
}), { base: baseOf(r1.text) });
|
||||
return { r1, r2 };
|
||||
}
|
||||
|
||||
// ---- recordVerdict: rounds and stop_rule ---------------------------------------
|
||||
|
||||
test('record: each round appends its summary to `rounds` and stores `stop_rule`', () => {
|
||||
const { r1, r2 } = sameClassTwice();
|
||||
assert.deepEqual(r1.verdict.rounds, [{ round: 1, blocking: 1, newByClass: { dep: 1 } }]);
|
||||
assert.equal(r1.verdict.stop_rule.fired, false);
|
||||
assert.deepEqual(r2.verdict.rounds, [
|
||||
{ round: 1, blocking: 1, newByClass: { dep: 1 } },
|
||||
{ round: 2, blocking: 1, newByClass: { dep: 1 } },
|
||||
]);
|
||||
assert.equal(r2.verdict.stop_rule.fired, true);
|
||||
assert.equal(r2.verdict.stop_rule.reason, 'same-class');
|
||||
assert.deepEqual(r2.verdict.stop_rule.classes, ['dep']);
|
||||
// The written block reads back with both fields.
|
||||
const ex = extractVerdict(r2.text);
|
||||
assert.equal(ex.error, null);
|
||||
assert.deepEqual(ex.verdict.rounds, r2.verdict.rounds);
|
||||
});
|
||||
|
||||
test('record: a round counted in full (a diff_note) has newByClass null; a falling count does not fire', () => {
|
||||
const t1 = buildPlan();
|
||||
const r1 = rec(t1, dd({ pc: [
|
||||
f('blocker', at(t1, R1_CHANGES), B1, 'dep/manifest'),
|
||||
f('major', at(t1, R1_CHANGES), M2, 'rollback/wording'),
|
||||
] }));
|
||||
const t2 = editA(r1.text);
|
||||
const r2 = rec(t2, dd({ pc: [f('blocker', at(t2, 'its error path'), B2, 'dep/import-chain')] }),
|
||||
{ baseNote: 'no snapshot of round 1; full counting' });
|
||||
assert.ok(typeof r2.verdict.diff_note === 'string');
|
||||
assert.ok(Array.isArray(r2.verdict.rounds), 'rounds is recorded');
|
||||
const last = r2.verdict.rounds.at(-1);
|
||||
assert.equal(last.round, 2);
|
||||
assert.equal(last.newByClass, null);
|
||||
assert.equal(last.blocking, 1);
|
||||
assert.equal(r2.verdict.stop_rule.fired, false);
|
||||
});
|
||||
|
||||
test('record: an old verdict without history starts one and does not fire', () => {
|
||||
const { r1 } = sameClassTwice();
|
||||
const ex = extractVerdict(r1.text);
|
||||
const old = { ...ex.verdict };
|
||||
delete old.rounds;
|
||||
delete old.stop_rule;
|
||||
const oldText = r1.text.replace(/```json\n[\s\S]*?\n```/, '```json\n' + JSON.stringify(old, null, 2) + '\n```');
|
||||
assert.equal(extractVerdict(oldText).error, null);
|
||||
const t2 = editA(oldText);
|
||||
const r2 = rec(t2, dd({ pc: [f('blocker', at(t2, 'its error path'), B2, 'dep/import-chain')] }), { base: baseOf(oldText) });
|
||||
assert.equal(r2.verdict.rounds?.length, 1);
|
||||
assert.equal(r2.verdict.rounds[0].round, 2);
|
||||
assert.equal(r2.verdict.stop_rule.fired, false);
|
||||
});
|
||||
|
||||
test('record: after a fired stop rule --record exits 1 with PLAN_GATE_STOP_RULE and writes nothing, --extra-round too', () => {
|
||||
const { r2 } = sameClassTwice();
|
||||
const t3 = r2.text.replace('Why: the brief asks for a parser CLI.', 'Why: the brief asks for a parser CLI with clear errors.');
|
||||
for (const extraRound of [false, true]) {
|
||||
const r3 = recordVerdict(t3, dd(), { reviewed: planHash(t3), planPaths: PATHS, now: NOW, base: baseOf(r2.text), extraRound });
|
||||
assert.equal(r3.code, 1, r3.message);
|
||||
assert.equal(r3.reason, 'PLAN_GATE_STOP_RULE');
|
||||
assert.match(r3.message, /PLAN_GATE_STOP_RULE/);
|
||||
assert.match(r3.message, /dep/);
|
||||
assert.equal(r3.text, undefined, 'nothing is written');
|
||||
}
|
||||
});
|
||||
|
||||
test('extractVerdict shape-checks rounds and stop_rule', () => {
|
||||
const { r2 } = sameClassTwice();
|
||||
const v = extractVerdict(r2.text).verdict;
|
||||
const put = obj => r2.text.replace(/```json\n[\s\S]*?\n```/, '```json\n' + JSON.stringify(obj, null, 2) + '\n```');
|
||||
for (const bad of [
|
||||
{ ...v, rounds: 'x' },
|
||||
{ ...v, rounds: [{ round: 0, blocking: 1, newByClass: null }] },
|
||||
{ ...v, rounds: [{ round: 1, blocking: -1, newByClass: null }] },
|
||||
{ ...v, rounds: [{ round: 1, blocking: 1, newByClass: { dep: 'one' } }] },
|
||||
{ ...v, stop_rule: { fired: 'yes' } },
|
||||
{ ...v, stop_rule: 'fired' },
|
||||
]) {
|
||||
const ex = extractVerdict(put(bad));
|
||||
assert.ok(ex.error, `malformed: ${JSON.stringify({ rounds: bad.rounds, stop_rule: bad.stop_rule })}`);
|
||||
}
|
||||
});
|
||||
|
||||
// ---- the bare gate ------------------------------------------------------------------
|
||||
|
||||
test('gate: a fired stop rule with an open blocker stops with PLAN_GATE_STOP_RULE in place of the open-finding reasons', () => {
|
||||
const { r2 } = sameClassTwice();
|
||||
const g = gatePlan(r2.text);
|
||||
assert.equal(g.code, 1);
|
||||
const codes = g.reasons.map(x => x.code);
|
||||
assert.deepEqual(codes, ['PLAN_GATE_STOP_RULE'], JSON.stringify(g.reasons));
|
||||
assert.match(g.reasons[0].message, /class: dep/);
|
||||
assert.match(g.reasons[0].message, /round 1: 1 blocking/);
|
||||
assert.match(g.reasons[0].message, /round 2: 1 blocking/);
|
||||
assert.doesNotMatch(g.reasons[0].message, /one more round/i);
|
||||
assert.equal(g.stop_rule.fired, true);
|
||||
});
|
||||
|
||||
// ---- --accept-open under the stop rule (220) ----------------------------------------
|
||||
|
||||
test('accept-open: after a fired stop rule it is allowed with rounds left, covers the blocker, and the gate warns', () => {
|
||||
const { r2 } = sameClassTwice();
|
||||
assert.ok(r2.verdict.max_rounds - r2.verdict.round > 0, 'rounds are left');
|
||||
const a = acceptOpen(r2.text, { now: NOW });
|
||||
assert.equal(a.code, 0, a.message);
|
||||
const v = extractVerdict(a.text).verdict;
|
||||
assert.equal(v.accepted_open.under, 'stop-rule');
|
||||
assert.deepEqual(v.accepted_open.findings.map(x => x.text), [B2]);
|
||||
const g = gatePlan(a.text);
|
||||
assert.equal(g.code, 0, JSON.stringify(g.reasons));
|
||||
assert.ok(g.warnings.some(w => w.code === 'PLAN_GATE_OPEN_BLOCKER' && w.message.includes(B2)), JSON.stringify(g.warnings));
|
||||
});
|
||||
|
||||
test('accept-open: a counting security finding is refused with PLAN_GATE_SECURITY_OPEN', () => {
|
||||
const t1 = buildPlan();
|
||||
const r1 = rec(t1, dd({ pc: [f('blocker', at(t1, R1_CHANGES), B1, 'SECURITY_INJECTION')] }));
|
||||
const id = r1.verdict.findings[0].id;
|
||||
const t2 = editA(r1.text);
|
||||
const r2 = rec(t2, dd({
|
||||
pc: [f('blocker', at(t2, 'its error path'), B2, 'SECURITY_INJECTION')],
|
||||
pcPrev: [{ id, resolved: true }],
|
||||
}), { base: baseOf(r1.text) });
|
||||
assert.equal(r2.verdict.stop_rule?.fired, true);
|
||||
assert.deepEqual(r2.verdict.stop_rule.classes, ['SECURITY_INJECTION']);
|
||||
const a = acceptOpen(r2.text, { now: NOW });
|
||||
assert.equal(a.code, 1);
|
||||
assert.match(a.message, /PLAN_GATE_SECURITY_OPEN/);
|
||||
assert.equal(a.text, undefined);
|
||||
});
|
||||
|
||||
test('accept-open: without the stop rule it behaves as before (rounds left -> refused, a blocker -> refused)', () => {
|
||||
const t1 = buildPlan();
|
||||
const r1 = rec(t1, dd({ pc: [f('major', at(t1, R1_CHANGES), M2, 'rollback/wording')] }));
|
||||
assert.equal(r1.verdict.stop_rule?.fired, false);
|
||||
const a = acceptOpen(r1.text, { now: NOW });
|
||||
assert.equal(a.code, 1);
|
||||
assert.match(a.message, /round\(s\) left/);
|
||||
});
|
||||
|
||||
// ---- known issue: the Pass 2 heading is plan text now ---------------------------------
|
||||
|
||||
test('the "Adversarial Pass 2 (v5.1.1 high-effort)" heading is no longer review log: a plan holding it hashes differently', () => {
|
||||
assert.ok(!REVIEW_LOG_SECTIONS.includes('Adversarial Pass 2 (v5.1.1 high-effort)'));
|
||||
const t = buildPlan();
|
||||
const withPass2 = `${t.replace(/\s*$/, '\n')}\n## Adversarial Pass 2 (v5.1.1 high-effort)\n\nSkipped.\n`;
|
||||
assert.notEqual(planHash(withPass2), planHash(t));
|
||||
});
|
||||
|
||||
// ---- CLI ------------------------------------------------------------------------------
|
||||
|
||||
const ROOT = join(fileURLToPath(new URL('.', import.meta.url)), '..', '..');
|
||||
const GATE = join(ROOT, 'lib', 'validators', 'plan-gate.mjs');
|
||||
const GIT_CLEAN = { GIT_DIR: undefined, GIT_WORK_TREE: undefined, GIT_INDEX_FILE: undefined };
|
||||
const run = (cwd, args, input) => {
|
||||
const r = spawnSync(process.execPath, [GATE, ...args], { cwd, input, encoding: 'utf-8', env: childEnv(GIT_CLEAN) });
|
||||
return { code: r.status, out: r.stdout, err: r.stderr };
|
||||
};
|
||||
const lastJson = out => JSON.parse(out.split('\n').filter(l => l !== '').at(-1));
|
||||
|
||||
test('CLI: two rounds with a new dep blocker each -> the gate stops with PLAN_GATE_STOP_RULE and a third --record is refused', () => {
|
||||
const dir = realpathSync(mkdtempSync(join(tmpdir(), 'plan-gate-stop-')));
|
||||
try {
|
||||
assert.equal(spawnSync('git', ['init', '-q'], { cwd: dir, env: childEnv(GIT_CLEAN) }).status, 0);
|
||||
const p = join(dir, 'plan.md');
|
||||
writeFileSync(p, buildPlan());
|
||||
const round = (payload, extra = []) => run(dir, ['--record', p, '--reviewed', planHash(readFileSync(p, 'utf-8')), ...extra], JSON.stringify(dd(payload)));
|
||||
const t1 = readFileSync(p, 'utf-8');
|
||||
const c1 = round({ pc: [{ ...f('blocker', at(t1, R1_CHANGES), B1, 'dep/manifest'), file: p }] });
|
||||
assert.equal(c1.code, 0, c1.out + c1.err);
|
||||
const id = lastJson(c1.out).stop_rule === undefined ? null : extractVerdict(readFileSync(p, 'utf-8')).verdict.findings[0].id;
|
||||
assert.ok(id, 'the --record output carries stop_rule');
|
||||
writeFileSync(p, editA(readFileSync(p, 'utf-8')));
|
||||
const t2 = readFileSync(p, 'utf-8');
|
||||
const c2 = round({ pc: [{ ...f('blocker', at(t2, 'its error path'), B2, 'dep/import-chain'), file: p }], pcPrev: [{ id, resolved: true }] });
|
||||
assert.equal(c2.code, 0, c2.out + c2.err);
|
||||
const j2 = lastJson(c2.out);
|
||||
assert.equal(j2.stop_rule.fired, true);
|
||||
assert.equal(j2.stop_rule.reason, 'same-class');
|
||||
|
||||
const g = run(dir, [p]);
|
||||
assert.equal(g.code, 1, g.out);
|
||||
assert.match(g.out, /STOP {2}\[PLAN_GATE_STOP_RULE\] .*class: dep.*round 1: 1 blocking.*round 2: 1 blocking/);
|
||||
assert.doesNotMatch(g.out, /PLAN_GATE_OPEN_BLOCKER/);
|
||||
const gj = run(dir, [p, '--json']);
|
||||
assert.equal(gj.code, 1);
|
||||
assert.equal(JSON.parse(gj.out).stop_rule.fired, true);
|
||||
|
||||
const before = readFileSync(p, 'utf-8');
|
||||
for (const extra of [[], ['--extra-round']]) {
|
||||
const c3 = round({}, extra);
|
||||
assert.equal(c3.code, 1, c3.out + c3.err);
|
||||
const j3 = lastJson(c3.out);
|
||||
assert.equal(j3.recorded, false);
|
||||
assert.equal(j3.reason, 'PLAN_GATE_STOP_RULE');
|
||||
assert.equal(readFileSync(p, 'utf-8'), before, 'nothing written');
|
||||
}
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
|
@ -172,8 +172,12 @@ test(`--record counts rounds; round ${MAX_ROUNDS + 1} needs the operator's extra
|
|||
assert.equal(MAX_ROUNDS, 3);
|
||||
let text = buildPlan();
|
||||
assert.equal(rec(text, dedupOut(), { extraRound: true }).code, 2, '--extra-round within the cap is refused, not banked');
|
||||
// A falling count of blockers (3, 2, 1), so the stop rule (lib/review/stop-rule.mjs,
|
||||
// "not fewer") never fires and the cap is what refuses round 4.
|
||||
for (let i = 1; i <= 3; i++) {
|
||||
const r = rec(text, dedupOut(finding('blocker', `still broken ${i}`)));
|
||||
const blockers = Array.from({ length: 4 - i }, (_, k) => finding('blocker', `still broken ${i}.${k}`));
|
||||
const r = rec(text, dedupOut(...blockers));
|
||||
assert.equal(r.verdict?.stop_rule?.fired, false, `round ${i}: the stop rule does not fire on a falling count`);
|
||||
assert.equal(r.code, 0, r.message);
|
||||
assert.equal(r.verdict.round, i);
|
||||
text = r.text;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue