open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

merge: plan 1.9 trial-4 delivery and its three review remediation rounds

Browse source 
Review WARN (2 MAJOR against decision 41, 4 MINOR); operator decision 45 = merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-14 19:55:55 +02:00
commit cd3225b630
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
72 changed files with 5782 additions and 330 deletions
Download patch file Download diff file Expand all files Collapse all files

243
CHANGELOG.md
View file

@ -4,6 +4,249 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## Unreleased — plan_version 1.9, the trial-4 findings, session-clean tests
A new plan version, 1.9, in which every path in a step's `Files:` and on its
`Test first:` `File:` line stands first, in backticks, so the reader no longer
guesses paths from prose (decision 16). The same delivery closes the six
findings from the test repo's trial 4 and stops `npm test` from pointing a live
session's stop guard at a temp-repo lock. Every fix began with a test that
failed on the code before it. Suite 2517 (2515 pass, 0 fail, 2 skipped),
measured by `npm test` on the third remediation's Step 2 commit
(`829da48`), the commit before this section.
Nothing here asks anything of users. Older plans warn, they do not fail: a plan
below 1.9 is read as in 6.3.1, apart from the fixes under Fixed, and gets one
warning per step whose `Files:` names a path that is not backticked.
### Doors
- two-way: plan_version 1.9 — a 1.9 plan read by 6.3.1 gets PLAN_VERSION_MISMATCH as a warning and the 6.3.1 reader, so a revert rewrites no plan.
- two-way: Files reader (three 6.3.0 answers restored) — a revert brings back 6.3.1's answers; no file format changes.
- two-way: the test files on the Test first File line are the backticked paths right after File: (operator decision 20) — a revert reads every path-shaped span again.
- two-way: red proof checks --tests against the plan — a revert accepts any list again.
- two-way: brief-conflict mark on review findings — an older verdict gate ignores the field.
- two-way: review Pass 1 ranks by catalogue tier and a dropped duplicate never lowers its group — a revert restores the asserted-severity ranking.
- two-way: the workflow review puts back every engine finding the coordinator did not return (operator decision 23) — opt-in path only; a revert restores the old floor.
- two-way: approval commits require a Co-Authored-By trailer — only Voyage's own commands call the CLI, and they pass it.
- two-way: explorer prompts, stamp order, test env — prose and test-side only.
- two-way: a 1.9 wrapped line keeps its paths only after a comma (operator decision 29), and a File: line with nothing after it takes its first wrapped line — a revert joins every wrapped line again.
- two-way: below 1.9 a test file after a note in brackets, and, plus or + is read when it is clearly a test (operator decision 28 and operator decision 31) — a revert reads only the run right after File: again.
- two-way: red proof reads the step's test files from the plan, prints them, and refuses a plan it cannot read (operator decision 30) — the older prose that passes --tests still runs against it.
- two-way: original_severity names what the kept finding asserted — only tests read the field.
- two-way: the workflow review fails closed when its coordinator returns nothing, and names every reviewer that did not run in allow_blocked_by — opt-in path only.
- two-way: below 1.8, when the plan names no test file, the helper and the executor give red proof the files the helper wrote (operator decision 41) — a revert leaves such a step unproven and unlocked.
### Added
- **plan_version 1.9: the file first on every line.** Each entry of `Files:`
and each `Test first:` `File:` line opens with its backticked path or paths;
everything from the first other text on is description and is never read.
A 1.9 plan that breaks this gets `PLAN_STEP_PATH_NOT_FIRST` or
`PLAN_STEP_PATH_NOT_A_PATH` (errors under `--strict`, warnings without). The
parser, the validator, the tests-only check, the plan-file sweep (K6) and red
proof read a 1.9 plan by this grammar. 1.9 is the current target: Handover
4, the plan template, `/trekplan`, the planning orchestrator and CLAUDE.md
say so, and the rules introduced at 1.8 say "1.8 or later".
- **A warning for older plans.** Below 1.9, a path in `Files:` that is not
backticked gets the warning `PLAN_STEP_PATH_UNTICKED`, under `--strict` too.
- **The brief-conflict mark.** A `/code-review` finding at the MAJOR tier whose
recommended action goes against a brief Non-Goal or another stated choice
drops to MINOR, marked `contradicts_brief` and `brief_lowered_from: MAJOR`;
any other engine finding only gets the mark, and a BLOCKER is never lowered.
`/trekreview` passes the brief's choices to the coordinator, which writes the
mark into `review.md`; the opt-in workflow review applies the same rule and
floors an illegal lowering. On an ALLOW or WARN review the verdict gate
answers exit 2 for a finding lowered or marked outside that one shape:
`brief_lowered_from` on any other finding, a mark on a reviewer finding, a
mark or an engine finding below its catalogue tier, or a mark with a missing
or unknown rule key.
### Fixed
- **The suite no longer hands the session id to the lock CLI.** `npm test`
preloads a helper that removes `CLAUDE_CODE_SESSION_ID`, and every test that
starts a child process passes an environment without it. A live session's
stop guard is no longer pointed at a temp-repo lock.
- **The three `Files:` regressions from 6.3.1's Known issues** give 6.3.0's
answer again: a source file after a described test is kept, a path that a
description only mentions is not read as a file, and a backticked bare name
such as `LICENSE` alone on a following line is read as a file.
- **The test files on the `Test first:` `File:` line** are the backticked
paths that stand right after `File:`, separated only by commas or spaces,
and below plan_version 1.9 also each further path after a note in brackets
(`(new)`, `*(new)*`, `(existing …)`) or after `and`, `plus` or `+`, as long
as it is clearly a test (operator decision 28, 13.10, and
operator decision 31, 14.10; review `bd2520b0`). Every one of them is read,
locked and copied for red proof, not only the first. A path in the
description after them is no test file, at every plan_version
(operator decision 20, 12.10; review `6c31212c`), so a step whose `File:`
line names its source in the description is no longer read as tests-only.
- **A 1.9 `Test first:` names its test only on a line that opens with
`File:`**, the way the executor, K6 and red proof read it. Below 1.9 the
check reads as before.
- **Engine provenance in the review.** Ingest takes `engine` from the payload,
never from a field a reviewer wrote on its own finding, and dedup ranks by
catalogue tier, so a reviewer-agent finding is kept over an engine finding,
and the kept finding carries the group's highest severity, in the
coordinator's Pass 1 and in the workflow dedup. A brief-conflict mark can no
longer lower a reviewer's MAJOR (operator decision 2026-10-12). On the
workflow path, floor (b) and the put-back of every engine finding the
coordinator did not return (both below) hold the same line after the
coordinator answers.
- **Explorers and reviewers stay in the repo.** `/trekplan` tells its
brief-reviewer, the eight Phase 5 explorers, the Phase 6 deep-dives, the
plan-critic and the scope-guardian to read local files only inside the repo
under exploration plus the paths their prompt gives, and every explorer gets
the brief.
- **The brief review stamp covers only text the reviewer saw.** For a `small`
brief, the reviewer's gaps are written after the stamp, so the stamp reads
stale and `/trekplan` reviews the brief once more.
- **Approval commits carry the caller's Co-Authored-By trailer.**
`artifact-commit.mjs` takes `--trailer` and refuses (exit 1, before anything
is staged or committed) a missing, multi-line or malformed trailer, an unfilled `{model}`, and
one that names a session link. `/trekbrief`, `/trekplan` and `/trekexecute`
pass it; `/trekexecute` prints the reason and goes on when the approval is
logged but not committed.
- **A list joiner no longer ends the read of a plain path below 1.9.** In
`Files:`, a `+`, `&`, `and` or `or` between backticked paths is no word of
description, so a plain path after them is read as a file again, as 6.3.1
read it, and a step of two tests joined by `+` and a plain source is not
tests-only (review `182ceb93`).
- **A 1.9 entry keeps the paths of a line wrapped after a comma, and a
numbered `File:` line names its test.** A line that is no list item keeps
its paths only after a comma, when the entry so far ends with one
(operator decision 29, 14.10; reviews `d21ff81e`, `eaeb398e`), so a
`Files:` list wrapped after a comma names every file: the step is not
tests-only, and K6 plans the second file too. Otherwise that line, and
every later wrapped line of the entry, is description and never read. A
`File:` line with nothing after it takes its first wrapped line. The
`Test first:` `File:` line may open with `-`, `*`, `+`, `1.` or `1)`, and
any list item ends it (reviews `b4fa1927`, `4b6460e5`).
- **A 1.9 plan refuses placeholders and mixed-case bare names as files**
(operator decision 22, 13.10; review `33f37f53`). A leading backticked span
that is a placeholder (`none`, `TBD`, `TODO`, `TBA`, `N/A`, `NA`, `...`, in
any case) or a bare name that is neither in capitals nor a build file, such
as the mixed-case `Parser`, is `PLAN_STEP_PATH_NOT_A_PATH`, and its hint
says to write `./Name` for such a file. `LICENSE`, `README` and `Makefile`
stay files; below 1.9 nothing changes.
- **The engine of a payload is read from an own property.** Ingest no longer
gives the findings of a payload whose reviewer is named `toString`,
`constructor`, `__proto__` or another name `Object.prototype` carries an
`engine` (it gave them a function or an object), and `toReviewerPayload`
refuses an engine named `toString` with its "engine must be one of" error
(review `3799ea6c`).
- **Pass 1 ranks by catalogue tier, and a duplicate never lowers its group.**
Of findings with the same `(file, line, rule_key)`, the coordinator's Pass 1
and `dedupByTriplet` keep the one with the higher catalogue tier, then a
reviewer-agent finding over an engine finding, then the higher asserted
severity, then the `brief-conformance-reviewer` finding, then the first
seen, and write the kept finding at the highest severity its group asserted.
Before, a marked code-review MAJOR beat a reviewer's MINOR on the same
triplet and the review came out ALLOW; now it is WARN (review `bc7f4e23`;
the coordinator's prose, operator decision 24).
- **The workflow review puts back every engine finding the coordinator did not
return** (operator decision 23, 13.10; reviews `ab1b5d9f`, `bc7f4e23`). On
the opt-in workflow path, dedup ranks as Pass 1 does, and every engine
finding with no returned finding at its exact `(file, line, rule_key)` is
put back at its own severity, one per triplet (the higher-severity copy when
two share it), so dropping, moving or relabelling an engine finding no
longer lowers the verdict. Beside it, floor (b): a returned finding is never
below the lower of its reviewer counterpart's severity and its catalogue
tier (for a key outside the catalogue, the counterpart's severity). The
workflow path is now stricter than the default path (see Known issues).
- **An approval trailer with a blank or padded name is refused.**
`artifact-commit.mjs` refuses a trailer whose name is empty or only spaces,
or that holds two spaces in a row, so an empty `{model}` ("Claude" and two
spaces) fails as the module header promises. A name with a parenthesis,
such as "Claude Opus 5.5 (1M context)", is still accepted (review
`8420915b`).
- **The plan validator parses each step once.** `validatePlanContent` reads a
step's fields once and every check, `--steps-from` included, reads them from
there; its errors and warnings are unchanged (review `93382464`).
- **Red proof reads the step's test files from the plan and prints them, and
the executor defines them once** (review `cf241526`;
operator decision 30, 14.10; reviews `793fc1aa`, `09ae46d8`, `d01df508`,
`11ee78c5`). With `--plan` and `--step`, `red-proof.mjs` reads the step's
test files from the plan as decision 20 and, below 1.9, decision 31 set
them, and prints the list it ran as `tests` (a `tests:` line in text mode).
The helper and the executor pass no `--tests`, and the lock's `--files` is
that printed list. A `--tests` that names another set is exit 2 with a
reason that starts `tests-mismatch` and the plan's list in `plan_tests`; a
plan or step it cannot read is exit 2 with a reason that starts
`plan-unreadable`, and the executor corrects its own call; a step whose
plan names no test file reads as before. `commands/trekexecute.md` defines
the step's test files in one sentence the other places point to.
- **The `/trekbrief` pass branch names no `## Brief Quality`, and the stats
rows span their blocks** (operator decision 21, 12.10). That section belongs
to the exhausted and force-stopped branches only. In
`tests/fixtures/jsonl-schemas.md` the `line_ref` of the trekbrief, trekplan
and trekreview stats rows runs from the phase heading to the closing fence of
its stats-append block, and a test pins all four command rows (trekexecute's
too) by exact equality.
- **`original_severity` names what the kept finding asserted** (review
`08deae68`). When Pass 1 raises a finding to its duplicate group's
severity, Pass 3 writes `original_severity` from the finding's own
severity, and only when that differs from its catalogue tier; the carrier
field `pass1_raised_from` never leaves the contract.
- **The workflow review fails closed when its coordinator returns nothing**
(review `edf2ec56`). A coordinator that returns no verdict and findings
counts as a missing reviewer: the merged findings are read at their
catalogue tier or higher (never lowered), the verdict starts at WARN (a
BLOCK stays BLOCK), and every reviewer, engine and coordinator that did not
deliver is named in `allow_blocked_by` as `missing-reviewer:<name>`, so the
verdict gate exits 2 on such a WARN.
- **K6 keeps its own normaliser and `LIST_ITEM` is one literal again**
(operator decision 27, 13.10; review `7961a700`). The two `/simplify` edits
that crossed the first remediation's fence around K6 are reverted.
- **The control-run test leaves headroom over a slow start** (review
`703d87e1`). Under full-suite load the start-point run of red proof's
control-run test was killed at its 2 s timeout. The test now gives red proof
a 15 s timeout and starts every run but the control slowly on purpose, so a
timeout without headroom fails on demand; the numbers live in
`tests/helpers/timing.mjs`.
- **Below 1.8 a plan that names no test file gets the files the helper
wrote** (review `7b3b59ed`; operator decision 41, 14.10). A `File:` line
below 1.8 may name its test without backticks, so red proof reads no test
file from the plan and answers `no test files given`. The helper then runs
the same red proof again with `--tests` naming the files it wrote, and the
executor gives red proof that same list, so such a step is proven and locked.
### Known issues
- The session split reads a 1.9 plan as an older plan (operator answer 3).
- Marked MINOR findings still feed `recurrence` / project-memory proposals.
- The default review path's verdict gate does not compare `review.md` with the
engine payloads, so only the opt-in workflow path puts an engine finding the
coordinator did not return back (operator decision 23). The operator made
that check a separate delivery later (operator decision 25, 13.10).
- Below 1.9 the plan-file sweep (K6) keeps its own wider reader of the `File:`
line (operator 2026-10-11): it plans every backticked path on that line, also
one in the description that decision 20 no longer counts as a test file.
- Red proof accepts a `--tests` that matches the plan's list and prints it in
the spelling it was given, not the plan's (review `2d624b54`; deferred by
operator decision 42, 14.10).
- `ingest` keeps a `pass1_raised_from` and an `original_severity` that a
payload wrote on its own finding (reviews `c06d5385`, `e031a480`; deferred
by operator decision 42, 14.10).
- When its coordinator returns nothing, the workflow review raises the merged
findings in place; this has no effect on its output today (review
`6ff82e75`; deferred by operator decision 42, 14.10).
- A whitespace-only line ends a 1.9 `Test first:` `File:` entry but not a
`Files:` entry, to be skipped in both by operator decision 40 (review
`4b6460e5`; deferred by operator decision 42, 14.10). The same id stands
in a Fixed bullet above, which fixed the list-item case of that review; the
whitespace-only case came back open.
- A 1.9 wrapped line read as description is dropped without a warning, even
when it opens with a backticked path, to be warned on by
operator decision 39 (review `4e1607cb`; deferred by
operator decision 42, 14.10).
- Handover 4's `Files:` bullet words its empty-entry rule as for a `File:`
line (review `4f22b101`; deferred by operator decision 42, 14.10).
- Accepted deviation: the 1.9 `File:` selector repeats the list-marker set of
`LIST_ITEM` (review `4dce7c74`; operator decision 38, 14.10, decision 27
stands).
## v6.3.1 — 2026-10-11 — Fixes from the release review of 6.3.0, and two verdict-gate holes closed
Rounds 7 to 10 of the release review of 6.3.0. Round 7 is the WARN 6.3.0 was

2
CLAUDE.md
View file

@ -32,7 +32,7 @@ Voyage — a contract-driven Claude Code pipeline: brief, research, plan, execut
|---------|-------------|--------------------|
| `/trekbrief` | Brief — one question round sized by class (`ceremony: small\|bounded\|architectural`, heaviest in doubt, said aloud, only moved up) produces a task brief with explicit research plan; ends with the operator explicitly approving `## Intent` + `## Goal` («høres bra ut» is not approval), logged in the target repo's hash-chained approval log and committed (the brief too when `artifact_store: git`) (`--approve <dir>` does only the approval, then writes the trekbrief record); optionally orchestrates the pipeline | session |
| `/trekresearch` | Research — deep local + external research, produces structured research brief. Opt-in `--engine {swarm\|deep-research}` delegates the external phase to Claude Code's built-in `/deep-research` workflow (swarm default). The external agents preload the `researching-sources` skill and are told it by name at spawn | session |
| `/trekplan` | Plan — brief-reviewer, explore, plan, review. Requires `--brief` or `--project`; halts unless the brief's intent is approved (hash-bound, stale on edit). Auto-discovers `architecture/overview.md` if present. Plan 1.8: `Test first:` per step + one `End-to-end:` step; a design part for `ceremony: architectural` (absent = architectural); critic loop of max 3 rounds, recorded in the plan, and `plan-gate.mjs` (0 go / 1 stop / 2 could not run) decides; open minors after it are listed for the operator to pick | session |
| `/trekplan` | Plan — brief-reviewer, explore, plan, review. Requires `--brief` or `--project`; halts unless the brief's intent is approved (hash-bound, stale on edit). Auto-discovers `architecture/overview.md` if present. Plan 1.9: the file first on every line — every `Files:` entry and `Test first:` `File:` line opens with its backticked path(s). From 1.8: `Test first:` per step + one `End-to-end:` step; a design part for `ceremony: architectural` (absent = architectural); critic loop of max 3 rounds, recorded in the plan, and `plan-gate.mjs` (0 go / 1 stop / 2 could not run) decides; open minors after it are listed for the operator to pick | session |
| `/trekexecute` | Execute — disciplined plan/session-spec executor: refuses a plan whose logged approval does not cover its current sha256 (K2); one fresh `step-implementer` helper per plan step, one controller that reads the diff (spec + quality verdicts), proof of red / floor guard / step check (K9) / `Regression:` (K10) / done line as commands, fix rounds that go up (max 5: helper → plan → operator), architecture rules (K5) + plan-file sweep (K6) after the last step, then its own review (`/simplify` → `/trekreview` → verdict gate, K7; BLOCK keeps the merge into main closed), closed stop list + decision log in the final report | session |
| `/trekreview` | Review — a gate: Claude Code's `/code-review` + `/security-review` as engine, then Voyage's brief control (which stops); writes `review.md` with a required verdict that `lib/review/verdict-gate.mjs` decides on (0 ALLOW/WARN · 1 BLOCK · 2 never green); proposes recurring finding classes as project memory, line by line (Handover 6) | session |
| `/trekcontinue` | Continue — resumes the next session of a multi-session voyage project. Reads `.session-state.local.json` (Handover 7) and immediately begins executing | session |

8
README.md
View file

@ -48,7 +48,7 @@ Every artifact lives in one project directory: `.claude/projects/{YYYY-MM-DD}-{s
|---|---|---|
| `/trekbrief` | **Capture intent** — intent, goal, non-goals, success criteria, and a research plan with explicit topics. Interactive only. | `brief.md` (task brief) |
| `/trekresearch` | **Gather context** — code state, external docs, community, risk. Makes NO build decisions. | `research/NN-slug.md` (research brief) |
| `/trekplan` | **Transform intent into an executable contract** — per-step YAML manifest, regex-validated checkpoints, verifiable paths. Plan-critic is a hard gate. Auto-discovers `architecture/overview.md` as priors when an opt-in upstream architect plugin (not bundled) is installed. | `plan.md` with Manifest blocks + `plan_version: 1.7` |
| `/trekplan` | **Transform intent into an executable contract** — per-step YAML manifest, regex-validated checkpoints, verifiable paths. Plan-critic is a hard gate. Auto-discovers `architecture/overview.md` as priors when an opt-in upstream architect plugin (not bundled) is installed. | `plan.md` with Manifest blocks + `plan_version: 1.9` |
| `/trekexecute` | **Execute the contract disciplined** — fresh verification, independent manifest audit, honest reporting. Does NOT compensate for weak plans — escalates. | `progress.json` + structured report + manifest-audit status |
| `/trekreview` | **Close the loop** — independent post-hoc reviewer reads `brief.md` and the diff produced by execute, runs brief-conformance + code-correctness reviewers in parallel, dedups via Judge Agent. Severity-tagged findings (Critical/High/Medium/Low/Info) feed back into planning via Handover 6. | `review.md` (`type: trekreview`) with stable 40-char hex finding-IDs |
@ -318,7 +318,7 @@ Every plan includes:
- **Critic Verdict** — the last critic round's machine-readable verdict, written by `plan-gate.mjs --record`
Every implementation step includes:
- **Test first:** — the test the step makes pass (required from `plan_version 1.8`)
- **Test first:** — the test the step makes pass (required from `plan_version 1.8`); from `plan_version 1.9` each `Files:` entry and `Test first:` `File:` line opens with its backticked path(s), the file first on every line
- **Interface in:** / **Interface out:** / **Review focus:** — optional: what the step consumes and produces, and the untested inputs it pins to a test
- **Regression:** — optional: one runnable suite command (`npm test`, no pipe, no `&&`) run green before the checkpoint, while `Verify:` stays the step's own red-first test; the validator refuses one that can never run
- **On failure:** — what to do when verification fails; it opens with `revert`, `retry`, `skip` or `escalate`, or the validator warns
@ -895,13 +895,13 @@ Validators (zero npm deps, hand-rolled YAML subset):
|---|---|
| `lib/validators/brief-validator.mjs` | brief.md frontmatter + state machine (research_topics + status coherence) + body sections |
| `lib/validators/research-validator.mjs` | research-brief frontmatter (confidence ∈ [0,1], dimensions ≥ 1) + body sections; `--dir` mode validates a whole `research/` folder |
| `lib/validators/plan-validator.mjs` | wraps plan-schema + manifest-yaml; enforces v1.7 step heading, manifest count match, and forbidden-narrative-form denylist (`### Fase/Phase/Stage/Steg N`) — replaces the Phase 5.5 grep checks |
| `lib/validators/plan-validator.mjs` | wraps plan-schema + manifest-yaml; enforces v1.9 step heading, manifest count match, and forbidden-narrative-form denylist (`### Fase/Phase/Stage/Steg N`) — replaces the Phase 5.5 grep checks |
| `lib/validators/progress-validator.mjs` | progress.json shape (schema_version, status enum, current_step in range) + resume-readiness check |
| `lib/validators/architecture-discovery.mjs` | EXTERNAL CONTRACT — drift-WARN, never drift-FAIL. Discovers `architecture/overview.md` (owned by an opt-in upstream architect plugin, not bundled) and tolerates non-canonical filenames with warnings. |
Each module exposes a CLI: `node lib/validators/<name>.mjs --json <path>` returns structured `{valid, errors, warnings, parsed}`. Commands invoke the CLI as their schema check.
A doc-consistency test (`tests/lib/doc-consistency.test.mjs`) pins prose-vs-source invariants — the agent table in `CLAUDE.md` must match the `agents/*.md` file count, every command's frontmatter `name:` must match its filename, and `templates/plan-template.md` must declare `plan_version: 1.7`.
A doc-consistency test (`tests/lib/doc-consistency.test.mjs`) pins prose-vs-source invariants — the agent table in `CLAUDE.md` must match the `agents/*.md` file count, every command's frontmatter `name:` must match its filename, and `templates/plan-template.md` must declare `plan_version: 1.9`.
Borrowed pattern from `llm-security` (commit `97c5c9d`); extending the plugin should preserve the invariants the test pins.

2
agents/plan-critic.md
View file

@ -134,7 +134,7 @@ questions, which defeats the purpose.
- Are there assertions about behavior that have no corresponding test?
- Do the verification steps cover error paths, not just happy paths?
- Are the verification commands correct and runnable?
- `plan_version: 1.8`: does every step's `Test first:` name a test that would
- `plan_version` 1.8 or later: does every step's `Test first:` name a test that would
FAIL before the step and pass after it (not a test that already passes)?
- Does any step consist of only tests — every file in its `Files:` a test —
for behaviour an earlier step delivered? Its tests pass at its start point,

11
agents/planning-orchestrator.md
View file

@ -215,14 +215,19 @@ Write a comprehensive implementation plan including:
criterion must be an executable command or observable condition
- **Estimated Scope** — file counts and complexity
**Plan-version header:** Include `plan_version: 1.8` and the brief's
**Plan-version header:** Include `plan_version: 1.9` and the brief's
`ceremony:` in the metadata line below the title. This signals to trekexecute
that the plan includes per-step verification manifests and enables strict
audit mode. Plans without this marker are treated as legacy v1.6 with
synthesized minimal manifests. 1.8 adds `Test first:` on every step, one
`End-to-end:` step, the optional `Interface in:` / `Interface out:` /
`Review focus:` fields and, for an architectural plan, `## Design`
(`commands/trekplan.md` Phase 8 is the full text). 1.7 plans stay valid.
(`commands/trekplan.md` Phase 8 is the full text). 1.9 adds the file first on
every line: each `Files:` entry and each `Test first:` `File:` line opens with
its backticked path(s), and everything after the first other text is
description, so every path stands before any annotation
(`` `src/a.mjs`, `src/b.mjs` (new) ``, never `` `src/a.mjs` (new), `src/b.mjs` ``).
1.8 and 1.7 plans stay valid.
### Mandatory step format — copy this exactly
@ -335,7 +340,7 @@ Internally the validator enforces (same checks as before, now in one place):
- Step count == manifest count
- Zero forbidden narrative headings (`### Fase N`, `### Phase N`, `### Stage N`,
`### Steg N`)
- `plan_version` 1.7 or 1.8 declared (warning only if older / missing)
- `plan_version` 1.7, 1.8 or 1.9 declared (warning only if older / missing)
- From 1.8: `Test first:` per step, an `End-to-end:` step, non-empty optional
fields, a valid `ceremony`, and the design part for an architectural plan

87
agents/review-coordinator.md
View file

@ -66,10 +66,15 @@ You will receive a prompt containing:
the /trekreview triage gate.
- **Brief metadata** — `task`, `slug`, `project_dir`, `brief_path` from
the brief frontmatter.
- **Brief choices** — the brief's `## Non-Goals`, `## Constraints` and
`## Preferences` sections and the operator decisions in
`## Open Questions / Assumptions`, verbatim. Read only by the
brief-conflict rule (Pass 3).
- **Scope SHA range** — `scope_sha_start`, `scope_sha_end`,
`reviewed_files_count`.
- **Mode** — `default` or `quick`. In `quick` mode, skip Pass 3
(reasonableness filter); Passes 1, 2, 4 still run.
(reasonableness filter) except the brief-conflict rule, which applies in every mode;
Passes 1, 2, 4 still run.
- **Rule catalogue** — `lib/review/rule-catalogue.mjs`. Findings whose
`rule_key` is not in this set are dropped by Pass 3.
@ -83,10 +88,21 @@ re-reading, and fresh finding generation are all forbidden.
Two findings collide when their `(file, line, rule_key)` triplets are
identical. When findings collide:
- Keep the finding with the highest catalogue severity (BLOCKER >
MAJOR > MINOR > SUGGESTION).
- If the severity tie, prefer the finding from
`brief-conformance-reviewer` (its findings are anchored to the brief).
- Rank the colliding findings and keep the first:
1. the higher catalogue tier: the severity `lib/review/rule-catalogue.mjs`
gives the `rule_key`, or the finding's own severity when the key is
not in the catalogue (BLOCKER > MAJOR > MINOR > SUGGESTION);
2. if the catalogue tier ties (for a catalogue key it always does,
since the triplet shares the `rule_key`), prefer a reviewer-agent
finding over an engine finding (one from `claude-code-review` /
`claude-security-review`);
3. among findings of one kind, the higher asserted severity;
4. among reviewer-agent findings, prefer the finding from
`brief-conformance-reviewer` (its findings are anchored to the brief);
5. then the first seen.
- Write the kept finding with the highest severity any finding in its
group asserted, so dropping a duplicate never lowers the group and an
engine's brief-conflict mark never lowers what a reviewer agent asserted.
- Concatenate the kept finding's `detail` with a one-line note: "Also
flagged by {other reviewer}: {their title}." This preserves
attribution without duplicating the row.
@ -124,6 +140,7 @@ over-long `detail` sent a finding to `unverified`, and S118's most serious
finding fell out of the Handover 6 list that way. Length is packaging, never
a claim: only small findings are trimmed, and the trims are counted
(`trimmed` in the JSON block). The trim decision reads the **catalogue** tier,
except a lowerable brief conflict, which is judged as MINOR (decide the mark before trimming),
so a mislabelled MAJOR is never cut. Borrowed: "report at most five nits …
summarize the rest as a count" — cap only the nits (ai-native-sdlc
`assets/REVIEW.md:7-23`, MIT).
@ -157,8 +174,32 @@ Remove findings that fail ANY of these tests:
catalogue tier (e.g., `MISSING_TEST` emitted as MINOR). Reset to the
catalogue tier; this is a correction, neither a drop nor an unverified.
**The brief-conflict rule (one fixed rule).** Read the **Brief choices** input.
An engine finding (`engine: code-review` or `security-review`) whose recommended action
contradicts a passed Non-Goal or stated choice gets `contradicts_brief: "<verbatim>"`
(that Non-Goal or choice, word for word) and is kept, never removed;
only a `/code-review` finding at the MAJOR tier and at severity MAJOR then drops to MINOR, with `brief_lowered_from: MAJOR`
(in place of the catalogue reset above);
a BLOCKER-tier, a MINOR-tier and any `/security-review` finding only get the mark,
and no engine finding is otherwise written below its catalogue tier.
The rule reads engine findings alone, so only an engine finding is ever marked:
`engine` is written only on a finding whose kept Pass 1 survivor itself came from an engine payload
(`claude-code-review` → `code-review`, `claude-security-review` → `security-review`),
and a reviewer-agent finding (no `engine`) and a finding whose `rule_key` is not in the catalogue are never marked.
Remove any `brief_lowered_from` an incoming finding carries, and any mark on a finding
without an `engine`: only this rule writes them.
The rule is about the finding's recommended action, not the rule key `NON_GOAL_VIOLATED`
(a reviewer's claim that the delivered code breaks a Non-Goal is judged like any other finding).
The mark is the brief-conflict reason (`contradicts_brief`), a field, never a `suppressed_reason`:
the finding stays kept and counted at the severity written, never `unverified`.
`lib/review/coordinator-contract.mjs` (`lowerableBriefConflict`, `illegalBriefLowering`)
holds the same rule, and on an ALLOW or WARN review
the verdict gate refuses (exit 2) every other marked or lowered shape.
In `quick` mode, skip this pass entirely. Note the skip in the
Executive Summary so the reader knows reasonableness was not applied.
The brief-conflict rule above still applies in quick mode (the mark and the
one lowering) and changes nothing else.
**High-effort normalization (v5.1.1):** When the review is invoked
under high-effort mode (`phase_signals[review].effort: high`), Pass 3
@ -170,7 +211,8 @@ category in the catalogue). The original rule_key is
preserved in the finding's `original_rule_key` field for diagnostic
purposes. This normalization happens BEFORE writing review.md,
ensuring all `rule_key` values in the final review match the
catalogue.
catalogue. The brief-conflict rule (Pass 3) still applies under high-effort
mode (the mark and the one lowering) and changes nothing else.
### Suppression is two-valued (fail-closed)
@ -310,6 +352,9 @@ followed by these fields:
- `- title: {short imperative title}`
- `- detail: {what is wrong, with citation}`
- `- recommended_action: {one imperative step}`
- `- engine: {code-review | security-review, when the finding came from an engine}`
- `- contradicts_brief: {verbatim Non-Goal or choice, only when set}`
- `- brief_lowered_from: {MAJOR, only when lowered for the brief}`
### Trailing JSON block
@ -333,6 +378,20 @@ The LAST fenced block in the file is a `json` block:
"title": "...",
"detail": "...",
"recommended_action": "..."
},
{
"id": "<40-char-hex>",
"severity": "MINOR",
"rule_key": "CODE_REVIEW_DEFECT",
"file": "lib/http.mjs",
"line": 42,
"brief_ref": "Non-Goal 2",
"title": "...",
"detail": "...",
"recommended_action": "...",
"engine": "code-review",
"contradicts_brief": "No retry layer around the HTTP client",
"brief_lowered_from": "MAJOR"
}
]
}
@ -346,15 +405,20 @@ for the ID list.
## Hard rules
- **Bounded operations only.** You do NOT read the diff. You do NOT
re-evaluate findings against the brief. You do NOT generate new
re-evaluate findings against the brief, with one named exception: the
brief-conflict rule (Pass 3), which marks an engine finding against the
**Brief choices** and lowers only the one legal shape. You do NOT generate new
findings. The reviewers' outputs are your sole input. Synthesis-level
inference (e.g., "these 3 findings together suggest a pattern") is
forbidden in v1.0.
- **Verdict is mechanical.** No "ALLOW with caveats" or other custom
verdicts. Only BLOCK / WARN / ALLOW per the threshold table.
- **Severity floor is the catalogue.** Pass 3 corrects mismatches by
resetting to the catalogue tier — never by dropping. Pass 1's severity
tiebreak uses the catalogue tier, not the reviewer's emitted value.
- **Severity floor is the catalogue.** It holds for every finding
except a lowerable brief conflict, which is MINOR (Pass 3, `brief_lowered_from: MAJOR`).
Pass 3 corrects mismatches by
resetting to the catalogue tier — never by dropping. Pass 1 ranks by
the catalogue tier first, in the order Pass 1 gives, and the kept finding
carries the highest severity its group asserted.
- **Block-style YAML for findings list.** The frontmatter parser
(`lib/util/frontmatter.mjs`) does not support flow-style arrays.
- **IDs are computed by code, not by you.** You have no shell, so you
@ -380,6 +444,7 @@ for the ID list.
reviewer outputs. Never escalate a finding's severity beyond what the
catalogue specifies.
- **Quick mode is documented.** When mode is `quick`, the Executive
Summary says so, and Pass 3 is skipped — no other changes.
Summary says so, and Pass 3 is skipped — no other changes,
except the brief-conflict rule, which applies in every mode.
- **Honesty in fallback paths.** If `scope_sha_start` is null (mtime
fallback), the Executive Summary names this limitation explicitly.

35
agents/step-implementer.md
View file

@ -27,18 +27,31 @@ and never touch a file the step does not list.
- `start_sha` — the commit the step starts from. Everything you change is read
later as `git diff {start_sha}`, so the diff is your real report.
- `report_path` — where your report goes.
- `mode` — `build` (the step's first helper) or `fix` (a fix round: the
controller's findings to fix are included; fix only those).
- `mode` — `build` (the step's first helper, or, in the round `build (restart)`,
a fresh one after the executor corrected its own red-proof call) or `fix` (a
fix round: the controller's findings to fix are included; fix only those).
- `voyage_root` — the plugin root, for the CLIs below.
## Mode `build`
1. **Test first.** Read the step's `Test first:` and write that test (and only
that test) first. Follow the `Pattern:` file's style when one is named.
In the round `build (restart)` the tree holds what the first helper left
and nothing is reverted: keep one copy of each test case you write (add no
case the test file already holds).
The step's test files are every file on the `Test first:` `File:` line:
the backticked paths that stand right after `File:`, separated only by
commas or spaces, and below plan_version 1.9 also each further path after
a note in brackets such as `(new)` or after `and`, `plus` or `+`, as long
as it is clearly a test (the first path that is not ends the list; operator
decision 31); a path in the description after them is not a test file
(every plan_version; operator decision 20, 2026-10-12).
Then register every new file you created so the executor's diff sees it:
`git add --intent-to-add -- <path>` (this stages nothing).
2. **Prove it red before you build.** Run, from the repository root:
`node {voyage_root}/lib/verification/red-proof.mjs --red-only --base {start_sha} --cmd "<the step's Verify command>" --tests <test files as ONE comma-separated argument: a,b> --plan "{plan}" --step {step}`
`node {voyage_root}/lib/verification/red-proof.mjs --red-only --base {start_sha} --cmd "<the step's Verify command>" --plan "{plan}" --step {step}`
(Red proof reads the step's test files from the plan and prints the list
it ran on a `tests:` line; give it no `--tests` (operator decision 30), except in the one case below.)
Exit 0 = the test fails at the start point: continue. Exit 1 = the test
already passes at the start point, so it proves nothing: rewrite the test
until it fails for the reason the step exists, and run the check again.
@ -47,10 +60,13 @@ and never touch a file the step does not list.
`Files:` alone decides this, not what you have changed: the behaviour
already exists at the start point, and no test can be red here. Do not rewrite the test and do not build; say so in
the report and stop. The executor sends the step up to the plan.
Exit 2 = it could not run (the Verify command is not one test-runner
invocation, or a file is missing): on `plan_version` ≥ 1.8 do not build;
say so in the report and stop. Below 1.8, write "red proof not run:
{reason}" in the report and go on to step 4 without a lock.
Exit 2 = it could not run. When its reason starts `plan-unreadable`, the
plan path or the step number you were given is wrong: do not build; say
so in the report and stop (the executor corrects its call). When its reason starts `no test files given` and the plan_version is below 1.8 (a `File:` line there may name its test without backticks, so the plan names no test file), run the same command again with `--tests <the test files you wrote, comma-joined>` added (operator decision 41, 2026-10-14) and read that answer as above; it prints the files you wrote on its `tests:` line. From 1.8 the rules below apply unchanged. Exit 2 otherwise (the
Verify command is not one test-runner invocation, or a file is missing):
on `plan_version` ≥ 1.8 do not build; say so in the report and stop.
Below 1.8, write "red proof not run: {reason}" in the report and go on to
step 4 without a lock.
**A step with no `Test first:`** (allowed only below plan_version 1.8):
skip steps 1–3, write "no Test first: no test written, no proof of red" in
@ -58,7 +74,8 @@ and never touch a file the step does not list.
3. **Lock the tests** (when the CLI exists):
`node {voyage_root}/lib/util/test-lock.mjs lock --files <test files> --step {step} --cmd "{Verify command}"`
(the lock stores the step's test command; only that command can later
mark the step green).
mark the step green). `--files` is the list red proof printed on its
`tests:` line; copy that line into the report.
If the file does not exist, write "not-locked (test-lock.mjs missing)" in the
report. Never silent. Once locked you cannot edit the tests, and that is the
point: the code changes to meet the test, never the reverse.
@ -101,7 +118,7 @@ earlier section kept word for word. Never remove or rewrite what is there.
Use the round label the executor gave you in the heading.
```markdown
## Step {N} - {build | fix round R}
## Step {N} - {build | build (restart) | fix round R}
### Ran
- `{command}` -> exit {code}

13
commands/trekbrief.md
View file

@ -177,7 +177,7 @@ Put the task in one of three size classes. In doubt, take the heaviest that fits
| Class | Typical task | What the pipeline does |
|---|---|---|
| `small` | one change in one place, no new dependency, no interface change, easy to undo | one round, no follow-up; brief-reviewer runs once and its gaps become written assumptions, not questions; Phase 3.5 is skipped |
| `small` | one change in one place, no new dependency, no interface change, easy to undo | one round, no follow-up; brief-reviewer runs once and its gaps become written assumptions, not questions, written after the review stamp (Step 4e, passed gate, step 4); Phase 3.5 is skipped |
| `bounded` | several files or one module behind a clear boundary, no architecture decision | the round + at most one follow-up; at most one brief-review follow-up; Phase 3.5 asks one question for all four phases |
| `architectural` | a new module, interface, dependency, data model or migration; anything hard to undo | follow-up rounds while answers open new branches; the full brief-review loop; Phase 3.5 asks per phase |
@ -590,8 +590,8 @@ Read the brief template first:
### Loop bound
**The cap follows the size class:** `small` 1 review iteration (no
follow-up question: what fails is written into `## Open Questions /
Assumptions` and `## Brief Quality`), `bounded` 2 (one follow-up),
follow-up question: its gaps become assumptions after the stamp on a passed gate
(Step 4e, step 4), into `## Brief Quality` on an exhausted loop), `bounded` 2 (one follow-up),
`architectural` 3 (two rounds of targeted follow-ups). A follow-up question
counts against the class budget; when the reviewer shows the task is bigger
than its class, raise the class first (Step 3b) and use the new cap.
@ -723,7 +723,8 @@ stumble.)
Any later edit to the body makes the stamp stale and `/trekplan` reviews
again. Exit 1 (not stamped) is said in one line; it only means the review
runs twice. Only a passed gate is stamped.
4. Break the loop and proceed to Step 4g.
4. Class `small` only: now write the reviewer's gaps into `## Open Questions / Assumptions`. They come after the stamp on purpose: the reviewer never saw them, so the stamp reads stale and `/trekplan` reviews the brief once more.
5. Break the loop and proceed to Step 4g.
**If gate fails AND iteration count < the class cap:**
1. Identify the weakest dimension (lowest score; tie broken by priority:
@ -944,10 +945,10 @@ node ${CLAUDE_PLUGIN_ROOT}/lib/util/approval-log.mjs append --kind intent --subj
6. **Commit in the target repo.** `{ARTIFACT_STORE}` is the brief's
`artifact_store` (`local` when absent, e.g. in `--approve` mode on another
producer's brief); `{slug}` is the brief's `slug`:
producer's brief); `{slug}` is the brief's `slug`; `{model}` is this session's model name (e.g. `Opus 5.5`):
```bash
node ${CLAUDE_PLUGIN_ROOT}/lib/util/artifact-commit.mjs --store {ARTIFACT_STORE} --artifact "{BRIEF_PATH}" --message "docs(voyage): approve intent {slug}"
node ${CLAUDE_PLUGIN_ROOT}/lib/util/artifact-commit.mjs --store {ARTIFACT_STORE} --artifact "{BRIEF_PATH}" --message "docs(voyage): approve intent {slug}" --trailer "Co-Authored-By: Claude {model} <noreply@anthropic.com>"
```
`git`: the brief and the log in one commit. `local`: only the log; the

34
commands/trekexecute.md
View file

@ -186,7 +186,7 @@ Extract every `### Step N: {description}` heading (in order). For each step, ext
- **Reuses** — existing code to leverage (informational)
- **Test first** — the test the step writes before it builds: its `File:`
(existing | new), what it `Verifies:`, an optional `Pattern:` (required from
plan_version ≥ 1.8, see below)
plan_version ≥ 1.8, see below). Its files are the step's test files, as Sub-step B defines them.
- **Verify** — command to run after implementation: the first backticked
span of the field. It is also the step's **test command**: proof of red
(Sub-step B) and the done line (Sub-step G) run it, so it must be one
@ -1451,7 +1451,7 @@ is not proven red at its start point and green at its result does not pass.
skill is the advice; the gate below, `red-proof.mjs`, is the guarantee, and it
stays a command whatever the skill says.
**The step's test files** are the `File:` paths under its `Test first:`. The
**The step's test files** are every file on the `Test first:` `File:` line: the backticked paths that stand right after `File:`, separated only by commas or spaces, and below plan_version 1.9 also each further path after a note in brackets such as `(new)` or after `and`, `plus` or `+`, as long as it is clearly a test (the first path that is not ends the list; operator decision 31); a path in the description after them is not a test file (every plan_version; operator decision 20, 2026-10-12). `red-proof.mjs --plan --step` reads that list from the plan, so neither the helper nor the executor passes `--tests` (operator decision 30, 2026-10-14), except in the one case below; it prints the list it ran (`tests`), refuses a `--tests` that differs (exit 2, `tests-mismatch`, the plan's list in `plan_tests`) and a plan or step it cannot read (exit 2, `plan-unreadable`). Below plan_version 1.8 only (a `File:` line there may name its test without backticks), a plan that names no test file gets `no test files given` from red proof; then the helper runs it again with `--tests` naming the test files it wrote, and the executor runs its own check again with `--tests` naming the files on the `tests:` line the helper copied into its report (operator decision 41, 2026-10-14). The lock's `--files` is the list red proof printed as `tests`. The
plan template lists them apart from `Files:`, so everywhere below **the step's
scope = its `Files:` + its test files**: both are built, both are committed at
the checkpoint, and neither is a scope finding.
@ -1465,16 +1465,16 @@ already rejects such a step (`PLAN_STEP_VERIFY_NOT_RUNNABLE`).
1. **The helper proves red before it builds** (Sub-step C): it writes the test
from `Test first:`, runs the gate
`red-proof.mjs --red-only --base {start_sha} --cmd "{Verify}" --tests {test files, ONE comma-separated argument} --plan "{plan_path}" --step {N}`
`red-proof.mjs --red-only --base {start_sha} --cmd "{Verify}" --plan "{plan_path}" --step {N}`
(the test must FAIL at the start point), locks the test files, and only then
implements. A `tests_only: true` answer stops the helper before it builds
implements. A `tests_only: true` or `plan-unreadable` answer stops the helper before it builds
(it does not rewrite the test); the executor's own run below then sends the
step up to the plan. The plan's `Files:` alone decides `tests_only`.
step up to the plan, or (`plan-unreadable`) acts as the exit-2 row says. The plan's `Files:` alone decides `tests_only`. Red proof reads the step's test files, as Sub-step B defines them, from the plan.
2. **The executor proves it both ways after the helper returns:**
```bash
node ${CLAUDE_PLUGIN_ROOT}/lib/verification/red-proof.mjs --base {start_sha} --cmd "{Verify command}" --tests {test files, ONE comma-separated argument} --plan "{plan_path}" --step {N} --json
node ${CLAUDE_PLUGIN_ROOT}/lib/verification/red-proof.mjs --base {start_sha} --cmd "{Verify command}" --plan "{plan_path}" --step {N} --json
```
The test must FAIL against the start point and PASS against the result
Red proof reads the step's test files, as Sub-step B defines them, from the plan. The test must FAIL against the start point and PASS against the result
(the working tree), and a **control run** must PASS: the start-point copy
with the step's whole change laid on top. The start point is a temp
`git clone --shared` checked out at `start_sha` (a real git checkout), with
@ -1487,17 +1487,17 @@ already rejects such a step (`PLAN_STEP_VERIFY_NOT_RUNNABLE`).
|------|-----------------------|----------|
| 0 | `"proven"` | continue |
| 1, `not-red-at-base`, `tests_only: true` | `"rejected"` | every file in the step's `Files:` is a test (the plan validator's `PLAN_STEP_TESTS_ONLY`, the same reading). The plan's `Files:` alone decides: the step's diff is not read, and a path changed outside `Files:` is the controller's scope finding (Sub-step C). The behaviour its tests cover already exists at the start point (an earlier step built it, or it was there before the plan), so the step can never be red and no rewrite of the test can fix that. The plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4), with no fix round — the planner moves the test into the step that builds the behaviour, or makes a test meant to pass from the start the `Regression:` of the step that changes the code |
| 1, `not-red-at-base`, otherwise | `"rejected"` | the test passes without the change, so the TEST is wrong: a finding for Sub-step F, and that fix round is the one exception to the lock — the executor releases the step's test lock for it, the fix helper rewrites the test, passes the `--red-only` gate and locks again. `tests_only: null` (the plan or the step could not be read) counts here too |
| 1, `not-red-at-base`, otherwise | `"rejected"` | the test passes without the change, so the TEST is wrong: a finding for Sub-step F, and that fix round is the one exception to the lock — the executor releases the step's test lock for it, the fix helper rewrites the test, passes the `--red-only` gate and locks again |
| 1, `not-green-at-result` | `"rejected"` | the change does not make the test pass: a finding for Sub-step F |
| 2 | `"not-run"` | never a pass and never proven. **plan_version ≥ 1.8:** when the reason is the command (the `Verify:` is not one test-runner invocation), the plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4); any other reason (a missing test file, an unresolvable start point, a control run that stays red) is a finding for Sub-step F. **Below 1.8:** a step with no `Test first:`, or whose `Verify:` is not a test-runner command, is tolerated as `"not-run"` — counted in the final report among the steps without proof of red, never called proven; any other reason is a finding |
| 2 | `"not-run"` | never a pass and never proven. **`plan-unreadable` first:** a reason that starts `plan-unreadable` means the executor's own `--plan` or `--step` is wrong (a path, a step number), which no helper can fix: the executor corrects `{plan_path}` to the absolute path of the plan file it loaded in Phase 1 and `{N}` to the number in the step's own `### Step N:` heading. When the helper's report shows it built (proven and locked), the executor reruns only its own red proof with the corrected call. When the helper stopped on the same answer before it built, the executor records the restart (progress `red_proof: "not-run"`, the reason in the step's `error`, and a decision-log entry with `cost_if_wrong` "the step's first helper run"), and starts the step again with a fresh build helper (Sub-step C, round label `build (restart)`; `attempts` +1 and `iterations_remaining` −1 as for any launch; the fresh helper starts from the tree as the first helper left it, nothing is reverted, and it keeps one copy of each test case it writes), once, outside the fix rounds (`fix_rounds` does not change); a step whose `error` already holds a `plan-unreadable` restart, on a `--resume` too, does not restart again; a second `plan-unreadable` goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4), and the upward file's question is red proof's reason (the plan path or step heading it could not read), not a step assumption. A `tests-mismatch` (a `--tests` that differs from the plan's list, in `plan_tests`) cannot come from the documented call, which passes no `--tests`; when it does, it is a finding for Sub-step F. **plan_version ≥ 1.8:** when the reason is the command (the `Verify:` is not one test-runner invocation), the plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4); any other reason (a missing test file, an unresolvable start point, a control run that stays red) is a finding for Sub-step F. **Below 1.8:** a step with no `Test first:`, or whose `Verify:` is not a test-runner command, is tolerated as `"not-run"` — counted in the final report among the steps without proof of red, never called proven; `no test files given` on a step that has `Test first:` is not a finding: when the helper's report has a `tests:` line, the executor reruns its check with `--tests` naming the files on the `tests:` line in the helper's report, as Sub-step B says (operator decision 41); without such a line, or on a second `no test files given`, the step is tolerated as `"not-run"` like a step with no `Test first:`; any other reason is a finding |
The exit code is the verdict, the output text is not. `base.tap` (when the
runner prints TAP) says what the red was: `fail 1 of 1` on a node test file is
usually a load error, not a named failing test — say so in the step report.
**The test lock (K8).** After its red gate the helper runs
`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs lock --files {test files} --step {N} --cmd "{Verify command}"`;
from then on the guard refuses Write/Edit/NotebookEdit on those files, so the
`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs lock --files {test files} --step {N} --cmd "{Verify command}"`.
`--files` names the step's test files, as Sub-step B defines them (the list red proof printed as `tests`). From then on the guard refuses Write/Edit/NotebookEdit on those files, so the
code moves to meet the test and never the reverse. The executor reads
`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs status` and records
`steps.{N}.test_lock = "locked"` when it exits 0 and names the step's files.
@ -1516,7 +1516,7 @@ Launch `subagent_type: "voyage:step-implementer"` with a short `description`
summary: the plan path, `plan_version`, the step number, the step's text
verbatim (every field, including `Interface in:`, `Interface out:` and
`Review focus:` when present), `start_sha`, `report_path`, `mode: build`, the
round label (`build`, or `fix round R`), `voyage_root: ${CLAUDE_PLUGIN_ROOT}`,
round label (`build`, `build (restart)`, or `fix round R`), `voyage_root: ${CLAUDE_PLUGIN_ROOT}`,
and the skill to work by: `skill: test-first` for a build round,
`skill: debugging` for a fix round (S2; Sub-steps B and F).
@ -1836,7 +1836,7 @@ session's effort, Sub-step E).
| `fix_rounds` | Goes to | What happens |
|---|---|---|
| 1, 2, 3 | a fresh helper | First `node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs pending` (the step is no longer seen green; D1 marks it again). Launch `voyage:step-implementer` again (no `name`, no `isolation`), `mode: fix`, with the findings (file, line, problem, fix) and the mechanical failures. It fixes only those; the tests stay locked (except in a `not-red-at-base` round, Sub-step B). Then B (both ways), D1, D2, D3, D4 run again and a fresh controller re-controls only what was fixed (Sub-step E). `attempts` +1; `iterations_remaining` −1 |
| 4 | the plan | **After three failed fix rounds the question is the architecture, not a fourth fix.** The step goes back to planning (below). Sub-step B sends two reds here directly, with no fix round: a `"not-run"` caused by the `Verify:` command, and a `not-red-at-base` with `tests_only: true` (a step of only tests). Stop class 4 |
| 4 | the plan | **After three failed fix rounds the question is the architecture, not a fourth fix.** The step goes back to planning (below). Sub-step B sends three answers here directly, with no fix round: a `"not-run"` caused by the `Verify:` command, a second `plan-unreadable` (the executor's own call still wrong after one correction), and a `not-red-at-base` with `tests_only: true` (a step of only tests). Stop class 4 |
| 5 | the operator | A step already sent to the plan (`fix_rounds` 4) that fails its control again after re-planning goes to the operator. The executor never builds this step again: on a later `--resume` it only runs the checks (B, D1, D2, D3, D4, E) on whatever the operator changed. If they fail again the step stays at 5 — there is no sixth round, `fix_rounds` never exceeds 5 — and the run stops again. Stop class 4 |
**Back to planning** (`fix_rounds = 4`): save `git diff {start_sha}` to
@ -1848,7 +1848,7 @@ operator to re-plan the step (`/trekplan` revises it) and run
`/trekexecute --resume`, which launches a fresh `build` helper for the revised
step (`attempts` +1, `iterations_remaining` −1) and keeps `fix_rounds` at 4. A step whose proof of red cannot run on a
≥ 1.8 plan because its `Verify:` is not one test-runner invocation goes here
directly (Sub-step B).
directly (Sub-step B), and so does a second `plan-unreadable` after one corrected restart, with red proof's reason as the upward file's question.
**The plan's `On failure:` clause** decides what happens around the ladder:
@ -2266,9 +2266,9 @@ the run is `stopped`, and the merge stays closed.
**Approve** / **Not yet**. Check the answer with
`approval-log.mjs answer` (stdin) exactly as `/trekplan` Phase 11 does;
exit 0 → `approval-log.mjs append --kind plan --subject "{plan_path}" --sha256-of "{plan_path}"`
and `artifact-commit.mjs --store {artifact_store} --artifact "{plan_path}" --message "docs(voyage): approve plan {slug}"`.
This is an approval of new plan content, not an execution decision
(Hard rule 1), and not a merge: no guard asks it.
and `artifact-commit.mjs --store {artifact_store} --artifact "{plan_path}" --message "docs(voyage): approve plan {slug}" --trailer "Co-Authored-By: Claude {model} <noreply@anthropic.com>"`,
`{model}` being this session's model name (e.g. `Opus 5.5`). On exit 1 or 2, print the `reason` in one line and continue: the approval is logged but not committed.
This is an approval of new plan content, not an execution decision (Hard rule 1), and not a merge: no guard asks it.
5. **Approve** → add the new steps to progress (`steps.{M}` pending,
`total_steps` raised), run Phase 6 for them (the same loop: helper, checks,
controller, fix rounds), then Phase 7, then this phase again with

74
commands/trekplan.md
View file

@ -446,7 +446,8 @@ below (operator decision 2026-10-08).
Launch the **brief-reviewer** agent:
Prompt: "Review this task brief for quality: {brief_path}. Check completeness,
consistency, testability, scope clarity, and research-plan validity."
consistency, testability, scope clarity, and research-plan validity.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
Handle the verdict:
- **PROCEED** — continue to Phase 5.
@ -502,40 +503,50 @@ medium: default, large: default) instead of dropping agents.
| `research-scout` | Conditional | Conditional | Conditional | External docs (only when unfamiliar tech detected AND no research brief covers it) |
| `convention-scanner` | No | Yes | Yes | Coding conventions, naming, style, test patterns |
research-scout's web search is not a local read; the line above bounds its local reads only. The session decomposer (`--decompose`) is not an exploring or reviewing agent and does not carry the line.
### Always launch (all codebase sizes):
**architecture-mapper** — full codebase structure, tech stack, patterns, anti-patterns.
Prompt: "Analyze the architecture of this codebase. The task being planned is: {task}"
Prompt: "Analyze the architecture of this codebase. The task being planned is: {task}.
Brief file: {brief_path} — read it first.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
**dependency-tracer** — module connections, data flow, side effects for task-relevant code.
Prompt: "Trace dependencies and data flow relevant to this task: {task}. Focus on modules
that will be affected by the implementation."
that will be affected by the implementation. Brief file: {brief_path} — read it first.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
**risk-assessor** — risks, edge cases, failure modes, technical debt near task area.
Prompt: "Assess risks and failure modes for implementing this task: {task}. Check for
complexity hotspots, security boundaries, and technical debt in the relevant code."
complexity hotspots, security boundaries, and technical debt in the relevant code.
Brief file: {brief_path} — read it first.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
**task-finder** — all files, functions, types, and interfaces directly related to the task.
Prompt: "Find all code relevant to this task: {task}. Include existing implementations
that solve similar problems, API boundaries, database models, configuration files.
Report file paths and line numbers for every finding."
Report file paths and line numbers for every finding. Brief file: {brief_path} — read it first.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
**test-strategist** — existing test patterns, coverage gaps, test strategy.
Prompt: "Analyze the test infrastructure and design a test strategy for this task: {task}.
Discover existing patterns and identify coverage gaps."
Discover existing patterns and identify coverage gaps. Brief file: {brief_path} — read it first.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
**git-historian** — recent changes, code ownership, hot files, active branches.
Prompt: "Analyze git history relevant to this task: {task}. Brief file:
{brief_path} — read its Constraints and Non-Goals for files it keeps unchanged,
and never recommend changing one. Report recent changes, ownership, hot files,
and active branches that may affect planning. Name the remote as `git remote -v`
shows it."
shows it.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
### Launch for medium+ codebases (50+ files):
**Convention Scanner** — use the `convention-scanner` plugin agent (model: "opus")
for medium+ codebases only.
Provide concrete examples from the codebase, not generic advice."
Prompt: "Discover the coding conventions relevant to this task: {task}. Brief file: {brief_path} — read it first. Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names). Provide concrete examples from the codebase, not generic advice."
### Conditional: External research
@ -553,7 +564,8 @@ If yes (and not covered by attached briefs): launch **research-scout** in
parallel with the other agents.
Prompt: "Research the following technologies for this task: {task}.
Specific questions: {list specific questions about the technology}.
Technologies to research: {list}."
Technologies to research: {list}. Brief file: {brief_path} — read it first.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
If no external technology is involved or all topics are covered by briefs:
skip research-scout and note:
@ -571,7 +583,10 @@ Common reasons for deep-dives:
- A risk was flagged but the actual impact needs verification
For each significant gap, spawn a targeted deep-dive agent (model: "opus",
subagent_type: "Explore") with a narrow, specific brief.
subagent_type: "Explore") with a narrow, specific brief. Each deep-dive prompt
carries these two lines:
Brief file: {brief_path} — read it first.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names).
Launch up to 3 deep-dive agents in parallel. If no gaps exist, skip this phase
and note: "Initial exploration was sufficient — no deep-dives needed."
@ -653,12 +668,12 @@ Do NOT write this synthesis to disk. It is internal working context only.
> ```
> ````
>
> **plan_version 1.8 additions** (the validator enforces them; 1.7 plans stay
> **plan_version 1.8 or later additions** (the validator enforces them; 1.7 plans stay
> valid): `Test first:` with `File:` and `Verifies:` on EVERY step; exactly the
> optional fields `Interface in:`, `Interface out:`, `Review focus:` (never
> empty when present), plus `Regression:` below; at least one step with `End-to-end:` and a backticked
> command — the "run the thing" step; the header line carries
> `` `plan_version: 1.8` · `ceremony: <class>` ``; an architectural plan carries
> `` `plan_version: 1.9` · `ceremony: <class>` ``; an architectural plan carries
> `## Design` (see "Design part" below); and every check the run executes can
> run — each `## Verification` criterion, the first backticked span of each
> `Verify:` (the step's test command) and each `End-to-end:` command pass the
@ -667,6 +682,31 @@ Do NOT write this synthesis to disk. It is internal working context only.
> `PLAN_STEP_VERIFY_NOT_RUNNABLE`, `PLAN_END_TO_END_NOT_RUNNABLE`,
> `PLAN_END_TO_END_NOT_VERIFIED`).
>
> **plan_version 1.9 additions** (the validator enforces them on a 1.9 plan;
> 1.8 and 1.7 plans are read as before, with the warning
> `PLAN_STEP_PATH_UNTICKED` for a path outside backticks): the file first on
> every line. Each `Files:` entry (the field's first line and each list item)
> and each `Test first:` `File:` line opens with its backticked path(s),
> separated only by commas or spaces; everything from the first other text on
> is description and is never read. So every path stands before any annotation:
> `` `src/a.mjs`, `src/b.mjs` (new) ``, never `` `src/a.mjs` (new), `src/b.mjs` ``
> (the second path would be description); two test files share one line,
> `` File: `a.test.mjs`, `b.test.mjs` (new) ``. A line that is no list item
> continues the entry above it, so a list wrapped after a comma keeps its paths (`` `src/a.mjs`, `` with
> `` `src/b.mjs` `` on the next line names both) and a wrapped description stays description: once the
> entry so far does not end with a comma, a wrapped line is description even when it opens with a backticked
> span (operator decision 29), and a `File:` line with nothing after it takes its first wrapped line. The
> `File:` line may open with `-`, `*`, `+`, `1.` or `1)`, and any list item (the `Verifies:` line too) ends it.
> A 1.9 step always names a file: an entry that does not open with a backticked path, the bare word none
> (written without backticks), or an empty
> `Files:` is `PLAN_STEP_PATH_NOT_FIRST`; a leading span that is no path (a
> call, an option, a URL) is `PLAN_STEP_PATH_NOT_A_PATH`.
> So is a backticked placeholder (`` `none` ``, `` `TBD` ``, `` `TODO` ``,
> `` `TBA` ``, `` `N/A` ``, `` `NA` ``, `` `...` ``) and a mixed-case bare name such as
> `` `Parser` `` (no `.` or `/`, neither in capitals like `LICENSE` nor a build
> file like `Makefile`), by operator decision 22: write `./Name` for such a
> file. The canonical step above already opens each line with its path.
>
> **Two step rules at every plan_version** (no version bump; a plan without
> the field is unchanged): the optional `Regression:` field (K10) holds ONE
> backticked command the criteria runner allows, with no pipe and no `&&` —
@ -905,7 +945,8 @@ wrong ordering, fragile assumptions, missing error handling, scope creep,
underspecified steps. Rate each finding as blocker, major, or minor.
End your response with the REQUIRED machine-readable `json` findings block
(schema in `agents/plan-critic.md`) so the orchestrator can pipe it to the
dedup helper. You have no Write tool — return the block inline, do not write a file."
dedup helper. You have no Write tool — return the block inline, do not write a file.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
**scope-guardian** — scope alignment check.
Prompt: "Check this implementation plan against the brief.
@ -915,7 +956,8 @@ Find scope creep (plan does more than the brief requires) and scope gaps
exist. Verify that every Success Criterion in the brief is covered by the
plan's Verification section. End your response with the REQUIRED machine-readable
`json` findings block (schema in `agents/scope-guardian.md`). You have no Write
tool — return the block inline, do not write a file."
tool — return the block inline, do not write a file.
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
After both complete, **extract each reviewer's trailing `json` findings block**
and pipe both into the dedup helper via **stdin** — the reviewers are read-only
@ -1160,11 +1202,11 @@ ANSWER
answer): say so in one line and treat it as **Not yet**.
3. On exit 0, log the approval and commit it in the target repo
(`{artifact_store}` from Phase 1; `local` keeps `plan.md` out of git and
commits only the log):
commits only the log; `{model}` is this session's model name, e.g. `Opus 5.5`):
```bash
node ${CLAUDE_PLUGIN_ROOT}/lib/util/approval-log.mjs append --kind plan --subject "{plan_path}" --sha256-of "{plan_path}"
node ${CLAUDE_PLUGIN_ROOT}/lib/util/artifact-commit.mjs --store {artifact_store} --artifact "{plan_path}" --message "docs(voyage): approve plan {slug}"
node ${CLAUDE_PLUGIN_ROOT}/lib/util/artifact-commit.mjs --store {artifact_store} --artifact "{plan_path}" --message "docs(voyage): approve plan {slug}" --trailer "Co-Authored-By: Claude {model} <noreply@anthropic.com>"
```
Print one line each: `Plan approval: record {seq}` and `Committed: {short

29
commands/trekreview.md
View file

@ -480,6 +480,15 @@ from Phase 5 AND the Phase 4.7 engine payloads, plus the triage map, brief
metadata, SHA range, the expected reviewer set (engines included), and each
not-run engine with its reason.
**Brief choices.** Next to the brief metadata, pass the brief's choices
verbatim: extract the `## Non-Goals`, `## Constraints` and `## Preferences`
sections and the operator decisions in `## Open Questions / Assumptions` from
the brief, word for word (a section the brief does not carry is passed as
absent, never written for it). The coordinator reads them for the
brief-conflict rule only (`agents/review-coordinator.md` Pass 3): an engine
finding whose recommended action contradicts one of them is kept and marked
`contradicts_brief`, never removed.
The coordinator runs the 4-pass process documented in
`agents/review-coordinator.md`:
@ -510,8 +519,9 @@ block. Do NOT re-run the reviewers based on the coordinator's output.
## Phase 5–6 via the Workflow substrate (opt-in `--workflow`)
Runs **only** when `workflow_substrate == true`. This is the **NW2 port**: it expresses
the SAME Phase 5–6 pipeline (parallel reviewers → triplet-dedup → coordinator verdict) as
a single Workflow, reusing the NW1 findings schema. The S10 bake-off found it
the same Phase 5–6 pipeline shape (parallel reviewers → triplet-dedup → coordinator verdict)
as a single Workflow, reusing the NW1 findings schema; its verdict floor is stricter than
the prose path's (decision 23, below). The S10 bake-off found it
**fidelity-equivalent** to the prose path — see `docs/T2-bakeoff-results.md` (verdict
**POSITIVE**: verdict-match 1.0, issue-coverage 100%, `(file,rule_key)` jaccard ≥
within-arm, tokens +4.4%). It stays **opt-in**, not the default, because the Workflow tool
@ -534,8 +544,13 @@ Workflow({
})
```
A reviewer that returns nothing, or an engine listed in `engineNotRun`,
forbids `ALLOW` on this path too (`missing_reviewers` in the result).
A reviewer that returns nothing, an engine listed in `engineNotRun`, or a coordinator that returns no verdict and findings forbids `ALLOW` on this path too (`missing_reviewers` in the result), and the result's `allow_blocked_by` names each one (`missing-reviewer:<name>`, the coordinator as `missing-reviewer:review-coordinator`);
Phase 7 writes that list as the trailing JSON block's `allow_blocked_by`, so the verdict gate reads such a WARN review as not performed in full (exit 2; a BLOCK stays exit 1).
On this path dedup ranks as Pass 1 does (both paths), every engine finding the
coordinator did not return at its exact `(file, line, rule_key)` is put back at its own
severity (operator decision 23), and a returned finding is never below the lower of its
reviewer counterpart's severity and its catalogue tier (floor (b)), so the workflow path
is stricter than the prose path.
Contract (verified in S10 part B — follow exactly):
- **Pass `args` as a JSON object.** The script defensively re-parses a JSON *string*, but
@ -552,7 +567,11 @@ Contract (verified in S10 part B — follow exactly):
Then continue at **Phase 7** exactly as the prose path does — Phase 7 rendering, Phase 8
validation, and the operator gate are **shared** and substrate-independent (both paths
return the same `{verdict, findings}` shape).
return the same `{verdict, findings}` shape, and the workflow's `allow_blocked_by` goes into the trailing JSON block). The workflow's returned findings keep
`engine` (set by the script from the payload a finding came from, never from a label),
`contradicts_brief` and `brief_lowered_from`, and Phase 7 writes them into each
finding's field list and the trailing JSON block of review.md exactly as the prose path
does.
**Known limitation (per bake-off §Posture, surfaced not hidden).** Classifier interference
was measured **0** at 9-agent concurrency in the session's default permission mode; an

47
docs/HANDOVER-CONTRACTS.md
View file

@ -12,7 +12,7 @@ Each artifact carries an explicit version field. Schema bumps are coordinated:
|---|---|---|
| `brief.md` | `brief_version` (frontmatter) | `2.2` |
| `research/*.md` | (implicit; tracked via `type: trekresearch-brief`) | unversioned |
| `plan.md` | `plan_version` (plan header line) | `1.8` (1.7 still valid) |
| `plan.md` | `plan_version` (plan header line) | `1.9` (1.8 and 1.7 still valid) |
| `progress.json` | `schema_version` (top-level) | `"1"` |
| `review.md` | `review_version` (frontmatter) | `1.1` |
| `.session-state.local.json` | `schema_version` (top-level) | `1` (number) |
@ -112,7 +112,7 @@ Optional but standard sections: `## Non-Goals`, `## Constraints`, `## Preference
**Intent approval gate (veikart steg 1) — additive schema, consumer-side requirement.** The two `intent_approved_*` fields are additive and optional in the schema: `brief-validator.mjs` does not require them, `/trekresearch` does not read them, and no `brief_version` bump is involved. The requirement lives in `/trekplan` Phase 1 (`### Read the brief`), which runs `intent-approval.mjs --check` and **halts** on `BRIEF_INTENT_NOT_APPROVED` (no marker), `BRIEF_INTENT_APPROVAL_STALE` (`## Intent` or `## Goal` changed after approval) or `BRIEF_INTENT_APPROVAL_INVALID` (malformed marker, or a section missing, empty or present more than once; a `## ` line inside a code block or HTML comment is not a heading). It is version-independent, so a brief from ANY producer, of any `brief_version`, stops at `/trekplan` until approved. No producer has to emit anything new: the operator approves through `/trekbrief --approve <project-dir | brief-file>`, a Voyage-side step open to every producer's brief, whether it is planned with `--project` (`<dir>/brief.md`) or `--brief <file>`, so no producer is privileged. `trekreview` briefs are exempt. The exemption is read from the brief's own frontmatter, so relabelling a brief `type: trekreview` walks past the gate, the same trust class as self-stamping. **What the marker does not prove:** same user, same machine. Any session can run `--stamp`. The marker is a trace that the approval step ran against this exact text, not a signature (see the header of `lib/validators/intent-approval.mjs`). `--stamp` also emits `brief-approved` to `trekexecute-stats.jsonl` under `CLAUDE_PLUGIN_DATA`, falling back to the plugin data dir the yardstick reads, on both the manual and the auto path.
**Approval log (K2) — consumer-side requirement, no schema change.** After the stamp, `/trekbrief` appends the approval to the target repo's log with `lib/util/approval-log.mjs append --kind intent --subject <brief> --sha256 <intent hash>` and commits it with `lib/util/artifact-commit.mjs` (the log always; the brief too when `artifact_store: git`). The log (`.claude/voyage/approvals.jsonl`, read and written only by that CLI) is append-only and hash-chained: each record carries `seq`, `ts`, `kind` (`intent | plan | review | merge | release`), `subject`, the content's `sha256`, an optional `expires`, `prev` and its own `hash`. `/trekplan` Phase 1 runs `approval-log.mjs verify --kind intent --subject <brief> --sha256 <intent_approved_hash>` after the marker check and halts on `broken` (a record edited, removed or reordered), `diverged` (the committed log is no longer a prefix of the file), `not-logged` or `mismatch`; exit 2 (could not run) halts too. A brief approved before the log existed is `not-logged`: the remedy is the same `/trekbrief --approve`, open to every producer's brief. Only an explicit "Approve" counts: `approval-log.mjs answer` refuses «høres bra ut», «gjør som du vil» and an empty answer. Trust limits are those of the marker (same user, same machine; a trace, not a signature); what the log adds is that each approval is a commit in `git log`, and that a committed approval cannot be rewritten without `diverged`. Adapted from bashebr/ai-native-sdlc@11fb8d7 (MIT), corrected so the approval is bound to content and cannot be written unseen.
**Approval log (K2) — consumer-side requirement, no schema change.** After the stamp, `/trekbrief` appends the approval to the target repo's log with `lib/util/approval-log.mjs append --kind intent --subject <brief> --sha256 <intent hash>` and commits it with `lib/util/artifact-commit.mjs` (the log always; the brief too when `artifact_store: git`). The commit carries the caller's trailer: every call passes `--trailer "Co-Authored-By: Claude {model} <noreply@anthropic.com>"` with `{model}` filled in, and the CLI refuses (exit 1, nothing committed) a missing trailer, a multi-line one, one that does not read `Co-Authored-By: <name> <<email>>` (an unfilled `{model}` included) and one holding `Claude-Session:`; a bad `--store` stays exit 2 and is checked first. The log (`.claude/voyage/approvals.jsonl`, read and written only by that CLI) is append-only and hash-chained: each record carries `seq`, `ts`, `kind` (`intent | plan | review | merge | release`), `subject`, the content's `sha256`, an optional `expires`, `prev` and its own `hash`. `/trekplan` Phase 1 runs `approval-log.mjs verify --kind intent --subject <brief> --sha256 <intent_approved_hash>` after the marker check and halts on `broken` (a record edited, removed or reordered), `diverged` (the committed log is no longer a prefix of the file), `not-logged` or `mismatch`; exit 2 (could not run) halts too. A brief approved before the log existed is `not-logged`: the remedy is the same `/trekbrief --approve`, open to every producer's brief. Only an explicit "Approve" counts: `approval-log.mjs answer` refuses «høres bra ut», «gjør som du vil» and an empty answer. Trust limits are those of the marker (same user, same machine; a trace, not a signature); what the log adds is that each approval is a commit in `git log`, and that a committed approval cannot be rewritten without `diverged`. Adapted from bashebr/ai-native-sdlc@11fb8d7 (MIT), corrected so the approval is bound to content and cannot be written unseen.
**Failure modes:**
- `BRIEF_INTENT_NOT_APPROVED` / `BRIEF_INTENT_APPROVAL_STALE` / `BRIEF_INTENT_APPROVAL_INVALID` → `/trekplan` halts (intent-approval gate, above); remedy `/trekbrief --approve <project-dir | brief-file>`
@ -215,7 +215,7 @@ The validator (`lib/validators/architecture-discovery.mjs`) is intentionally min
**Failure modes:** none. Discovery always succeeds (returns `found: false` if absent). The handover is additive.
**Not the design part.** A plan's own design (`## Design`, plan_version 1.8, architectural plans only) is part of Handover 4, written by `/trekplan` itself. It does not read or write this external slot, and this slot stays drift-WARN.
**Not the design part.** A plan's own design (`## Design`, plan_version 1.8 or later, architectural plans only) is part of Handover 4, written by `/trekplan` itself. It does not read or write this external slot, and this slot stays drift-WARN.
---
@ -237,7 +237,7 @@ Handover 6. Brief-generated plans carry no frontmatter at all.
| Field | Type | Required | Location | Allowed |
|---|---|---|---|---|
| `plan_version` | string | yes | plan header line, not frontmatter | `"1.8"` (current), `"1.7"` (valid) |
| `plan_version` | string | yes | plan header line, not frontmatter | `"1.9"` (current), `"1.8"` (valid), `"1.7"` (valid) |
| `ceremony` | string | no | plan header line, after `plan_version` | `small` \| `bounded` \| `architectural`; absent = `architectural` |
`plan_version` is emitted in the plan's "Generated by" header metadata line
@ -260,7 +260,7 @@ prose form (relaxed in S26 so the template's prose emission parses).
- Indented fenced YAML: ` ```yaml\n manifest:\n ...\n ``` `
- Required keys: `expected_paths` (list), `min_file_count` (number), `commit_message_pattern` (string compilable to RegExp), `bash_syntax_check` (list), `forbidden_paths` (list), `must_contain` (list of `{path, pattern}` dicts or empty list)
4. Step count == manifest count
5. **plan_version 1.8 only** (1.7 plans are not held to these):
5. **plan_version 1.8 or later** (1.7 plans are not held to these):
- every step has `Test first:` with `File:` and `Verifies:` sub-bullets;
- at least one step has `End-to-end:` with a backticked command — the "run the thing" step;
- the optional step fields `Interface in:`, `Interface out:`, `Review focus:` are never empty when present (consumers tolerate their absence);
@ -269,7 +269,14 @@ prose form (relaxed in S26 so the template's prose emission parses).
7. **Every plan_version, no version bump** (a plan without the field validates unchanged; voyage-lekerepo trial 2, 2026-10-08):
- **K10 — `Regression:`** (optional step field): ONE backticked command the criteria runner's allowlist runs, with no pipe and no `&&` (first backticked span, as for `Verify:`). The executor runs it green before the step's checkpoint; a step without it runs nothing new. `Verify:` stays the step's own red-first test. One that can never run is `PLAN_STEP_REGRESSION_NOT_RUNNABLE` (an error; a warning under `--soft`).
- **K11 — `On failure:`** opens with one of `revert`, `retry`, `skip`, `escalate` (case-insensitive; a leading `**`, `_`, backtick or `-` is ignored, other leading characters are not; `onFailureAction` in `lib/parsers/plan-schema.mjs`), the words `/trekexecute` reads; any other opening is the warning `PLAN_STEP_ON_FAILURE_UNKNOWN`. A missing `On failure:` is unchanged (the executor defaults to `escalate`).
- **A step of only tests** (order 10, voyage-lekerepo trial 3, 2026-10-09): when every file in a step's `Files:` is a test (its `Test first:` file, or a path `lib/util/test-path.mjs` reads as a test: under `tests/`, `test/`, `__tests__/`, `spec/`, `specs/`, or named `*.test.*`, `*.spec.*`, `*_test.*`, `*_spec.*`, `test_*.py`, but never a document such as `docs/specs/api.md`; `stepTestsOnly` in `lib/parsers/plan-schema.mjs`), the warning is `PLAN_STEP_TESTS_ONLY`: the behaviour its tests cover exists at its start point, so it can never be proven red. A step without `Files:` is not judged. At run time `red-proof.mjs --plan <plan.md> --step <N>` puts the same reading on a `not-red-at-base` as `tests_only`: the plan's `Files:` alone decides, and the step's diff is not read (operator 2026-10-11), and `/trekexecute` sends a `tests_only: true` step straight up to the plan.
- **A step of only tests** (order 10, voyage-lekerepo trial 3, 2026-10-09): when every file in a step's `Files:` is a test (a file on its `Test first:` `File:` line, or a path `lib/util/test-path.mjs` reads as a test: under `tests/`, `test/`, `__tests__/`, `spec/`, `specs/`, or named `*.test.*`, `*.spec.*`, `*_test.*`, `*_spec.*`, `test_*.py`, but never a document such as `docs/specs/api.md`; `stepTestsOnly` in `lib/parsers/plan-schema.mjs`), the warning is `PLAN_STEP_TESTS_ONLY`: the behaviour its tests cover exists at its start point, so it can never be proven red. A step without `Files:` is not judged. At run time `red-proof.mjs --plan <plan.md> --step <N>` puts the same reading on a `not-red-at-base` as `tests_only`: the plan's `Files:` alone decides, and the step's diff is not read (operator 2026-10-11), and `/trekexecute` sends a `tests_only: true` step straight up to the plan.
- **The `Test first:` files** (operator decision 20, 2026-10-12): a step's test files are the backticked paths that stand right after `File:` on its `Test first:` `File:` line, separated only by commas or whitespace; a path in the description after them (`` covers `parse()` in `src/parser.mjs` ``) is no test file. This holds at every plan_version (`testFirstFiles` in `lib/parsers/plan-schema.mjs`; from 1.9 the line is read by the grammar of invariant 8). Below plan_version 1.9 the run also goes on past a note in brackets (`(new)`, `*(new)*`, `(existing …)`) and the words `and`, `plus` and `+`, to a further path only when it is clearly a test (as `lib/util/test-path.mjs` and the tests-only check read it); a path that is not ends the run (operator decision 28, 2026-10-13, and operator decision 31, 2026-10-14). `red-proof.mjs --plan <plan.md> --step <N>` reads them from the plan, and the documented call passes no `--tests` (operator decision 30, 2026-10-14): it prints the list it ran (`tests`); a given `--tests` that differs is `not-run` with reason `tests-mismatch` and the plan's list in `plan_tests`; a plan or step it cannot read is `not-run` with reason `plan-unreadable`; below plan_version 1.8 only (a `File:` line there may name its test without backticks), a plan that names no test file gets `no test files given`, and then the helper runs it again with `--tests` naming the test files it wrote, and the executor with the files on the `tests:` line in the helper's report (operator decision 41, 2026-10-14).
8. **plan_version 1.9 only** (1.8 and 1.7 plans are read as before, with one warning, `PLAN_STEP_PATH_UNTICKED`, for a path their reader found outside backticks; decision 16, operator 2026-10-11) — the file first on every line:
- the entries of a step's `Files:` field are its first non-empty line and each list-item line (`-`, `*`, `+`, `1.`, `1)`); a line that is no list item continues the entry above it, when the entry so far ends with a comma, its text joins that entry with one space before the entry is read, so a list wrapped after a comma keeps its paths; otherwise it is description, never read, and so is every later wrapped line of that entry (operator decision 29, 2026-10-14); an entry with nothing on its own line (a `File:` line with nothing after it) takes its first wrapped line;
- each entry opens with its backticked path(s), separated only by commas or whitespace; everything from the first other text on is description and is never read (`` `a.test.mjs` - covers `src/parse.mjs` `` names one file), so every path stands before any annotation (`` `src/a.mjs`, `src/b.mjs` (new) ``, never `` `src/a.mjs` (new), `src/b.mjs` ``);
- each `Test first:` `File:` line is one entry under the same grammar (several test files share one line: `` File: `a.test.mjs`, `b.test.mjs` (new) ``). A line counts only when `File:` opens it (after a list marker `-`, `*`, `+`, `1.` or `1)`, or `**`); a line right after it that is no list item continues it (a wrapped line, read as above), and any list item (a `Verifies:` or `Pattern:` line too) or a blank line ends it; a step whose `Test first:` has no such line with a backticked path names no test and is `PLAN_STEP_NO_TEST_FIRST` (`File:` mid-line is description, as the executor reads it);
- an entry that does not open with a backticked span, the bare word none (written without backticks), and an empty or absent `Files:` are `PLAN_STEP_PATH_NOT_FIRST` (a 1.9 step always names a file); a leading span that is no path is `PLAN_STEP_PATH_NOT_A_PATH`: a call, an option, a URL, and (operator decision 22, 2026-10-13) a backticked placeholder (`` `none` ``, `` `TBD` ``, `` `TODO` ``, `` `TBA` ``, `` `N/A` ``, `` `NA` ``, `` `...` ``, in any case) or a bare name (no `.` or `/`) that is neither in capitals (`` `LICENSE` ``) nor a build file (`` `Makefile` ``), such as the mixed-case `` `Parser` ``; `` `./Parser` `` names such a file (`./Name`; `isPathShape` in `lib/parsers/plan-schema.mjs`). Both are errors under `--strict`, warnings under `--soft`;
- K6 (`lib/verification/plan-scope-sweep.mjs`) follows the grammar from 1.9: it reads a 1.9 plan's `Files:` and `Test first:` `File:` line by it (plan-schema `entryPaths` / `testFirstFiles`), and so does the red proof's `tests_only` reading (`lib/verification/red-proof.mjs`); below 1.9 both keep their earlier readers.
**Validation strategy:**
@ -284,10 +291,10 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
| `MANIFEST_MISSING` | Specific step has no manifest YAML | Add Manifest block |
| `MANIFEST_MISSING_KEY` | Manifest is missing a required key | Add the key |
| `MANIFEST_PATTERN_INVALID` | `commit_message_pattern` does not compile | Check escaping (`\\(` not `\(` in YAML double-quoted strings) |
| `PLAN_VERSION_MISMATCH` | `plan_version` other than 1.7 / 1.8 | Warning only; planner should bump |
| `PLAN_VERSION_MISMATCH` | `plan_version` other than 1.7 / 1.8 / 1.9 | Warning only; planner should bump |
| `PLAN_NO_VERSION` | No `plan_version` found | Warning only; add the header line |
| `PLAN_NOT_FOUND` / `PLAN_READ_ERROR` | The plan file is missing or unreadable | Check the path |
| `PLAN_STEP_NO_TEST_FIRST` | 1.8: a step lacks `Test first:` with File + Verifies | Add the test the step makes pass |
| `PLAN_STEP_NO_TEST_FIRST` | 1.8: a step lacks `Test first:` with File + Verifies; 1.9: also when no line opens with `File:` and its backticked path | Add the test the step makes pass |
| `PLAN_NO_END_TO_END` | 1.8: no step carries `End-to-end:` with a command | Add the "run the thing" step |
| `PLAN_STEP_FIELD_EMPTY` | 1.8: `Interface in/out:` or `Review focus:` present but empty | Fill or remove it |
| `PLAN_VERIFICATION_NOT_RUNNABLE` | 1.8: a `## Verification` criterion has no command, or one the criteria runner's allowlist never runs | Name a test runner, or wrap the check as `bash tests/<script>.sh` |
@ -302,12 +309,15 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
| `PLAN_STEP_REGRESSION_NOT_RUNNABLE` | K10, any version: a step's `Regression:` is not one command the allowlist runs (a pipe, `&&`, no backticks, an unknown runner) | One backticked runner call (`npm test`); a wider check goes in `bash tests/<script>.sh` |
| `PLAN_STEP_ON_FAILURE_UNKNOWN` | K11, any version, warning: `On failure:` does not open with revert / retry / skip / escalate | Open with the word, then the specifics |
| `PLAN_STEP_TESTS_ONLY` | Any version, warning: every file in the step's `Files:` is a test, so the step can never be proven red | Move the test into the step that builds the behaviour |
| `PLAN_STEP_PATH_NOT_FIRST` | 1.9: a `Files:` entry does not open with a backticked path (also the bare word none, and an empty or absent `Files:`: the step names no file); error, a warning under `--soft` | Open each entry with its backticked path(s), then the description: `` `src/a.mjs` - the parser `` |
| `PLAN_STEP_PATH_NOT_A_PATH` | 1.9: a `Files:` entry opens with a backticked span that is no path (a call `parse()`, an option, a URL, a backticked placeholder such as `` `TBD` `` or `` `none` ``, a mixed-case bare name such as `` `Parser` ``; operator decision 22); error, a warning under `--soft` | Put the path first; the span goes in the description; a file named `Parser` is written `./Parser` |
| `PLAN_STEP_PATH_UNTICKED` | Below 1.9, warning: the `Files:` reader found a path outside backticks; the plan is read as before (a warning under `--strict` too) | Backtick the path; from 1.9 an unticked path is refused |
**The plan gate (K3).** `node lib/validators/plan-gate.mjs <plan.md> [--json]` decides whether a plan may go on: **0** = it validates strict, its `## Critic Verdict` matches the current text, no BLOCKER is open, and every open MAJOR was accepted by the operator; **1** = stop, with `reasons` (`PLAN_GATE_INVALID_PLAN`, `PLAN_GATE_VERSION` — below plan_version 1.8, `PLAN_GATE_NO_VERDICT`, `PLAN_GATE_VERDICT_MALFORMED`, `PLAN_GATE_VERDICT_STALE`, `PLAN_GATE_OPEN_BLOCKER`, `PLAN_GATE_ROUNDS_EXHAUSTED`, `PLAN_GATE_OPEN_MAJOR` — rounds left: revise, `PLAN_GATE_OPEN_MAJOR_FINAL` — after the last round: ask the operator) and `warnings` (`PLAN_GATE_OPEN_MAJOR` for majors the operator accepted); **2** = could not read the plan. `--hash <plan.md>` prints the operative hash; Phase 9 takes it before the reviewers read the plan. `--record <plan.md> --reviewed <hash>` reads plan-review-dedup output on stdin (which lists the reviewers that delivered, `sources`) and writes the next round; exit 2 when a reviewer is missing or the plan changed since `--hash`; it refuses round 4 (exit 1, stop and ask) unless `--extra-round` follows the operator's yes, and refuses that flag before the cap is spent. Unrated findings count as blockers. The review log is excluded from the hash only up to its first other heading, so a step placed under it is hashed. `--accept-open <plan.md>` records the operator's "build as is" for the current round's open MAJORs (`accepted_open` in the verdict block, bound to the round and the plan hash; only after the last round; never for a BLOCKER; exit 1 when rounds are left, the verdict is stale, a blocker is open or no major is). Operator decision 2026-10-08: open majors used to be warnings, and a weakness the critics named was built and came back from `/trekreview` as MAJOR. plan-critic and scope-guardian rate on the review catalogue's tiers. The verdict is a trace of the review, not a signature.
**Verification CLIs that read a plan** (0/1/2, JSON on stdout, 2 is never green): `lib/verification/arch-rules.mjs --repo <dir>` runs the target repo's own dependency-cruiser or import-linter (installs nothing; `--prove --rule <name> --plant <file>=<dest>` shows pass → fail on that rule → pass, refusing a destination that exists or resolves outside the repo); `lib/verification/plan-scope-sweep.mjs --plan <plan.md> --base <ref>` lists changed files no step names (`Files:`, `Test first:` File, manifest `expected_paths`) and no process path covers.
**Versioning:** v1.7 has been stable since v1.8.0 of the plugin (when literal-template + Phase 5.5 self-check were added to fix Opus 4.7 schema drift). v1.6 → v1.7 added the Manifest block (mandatory). v1.7 → v1.8 makes `Test first:` and the "run the thing" step required, adds the optional step fields and the design part, and was written validator-first with tests for both shapes (`tests/validators/plan-validator-v18.test.mjs`); 1.7 plans validate unchanged, so readers need no change (the executor already treats `plan_version ≥ 1.7` as strict).
**Versioning:** v1.7 has been stable since v1.8.0 of the plugin (when literal-template + Phase 5.5 self-check were added to fix Opus 4.7 schema drift). v1.6 → v1.7 added the Manifest block (mandatory). v1.7 → v1.8 makes `Test first:` and the "run the thing" step required, adds the optional step fields and the design part, and was written validator-first with tests for both shapes (`tests/validators/plan-validator-v18.test.mjs`); 1.7 plans validate unchanged, so readers need no change (the executor already treats `plan_version ≥ 1.7` as strict). v1.8 → v1.9 gives `Files:` and the `Test first:` `File:` line a grammar, the file first on every line (invariant 8), so no reader has to tell a path from a description in free text; it was written validator-first (`tests/validators/plan-version-1-9.test.mjs`). Only a 1.9 plan is refused for it; 1.8 and 1.7 plans validate as before, with the warning `PLAN_STEP_PATH_UNTICKED`, and the 1.8 rules (invariant 5) bind 1.8 and 1.9 plans alike.
**Failure modes:** strict mode is the default for both producer and consumer. There is no soft mode here — a malformed plan is a hard failure for execute.
@ -452,6 +462,25 @@ Since 1.1 (ordre 5):
- `## Declined to Judge` — required at `review_version` ≥ 1.1 (strict error, soft warning; 1.0 reviews exempt): every finding set aside without a ruling, with its reason (including engine findings outside the review scope, `engine:out-of-scope`), plus `Trimmed: N`. **No finding is struck for length**: only MINOR/SUGGESTION text is trimmed, and counted (`trimmed` in the JSON block); a long BLOCKER/MAJOR keeps every character.
- Findings from Claude Code's own `/code-review` and `/security-review` enter under the catalogue keys `CODE_REVIEW_DEFECT` (MAJOR), `CODE_REVIEW_IMPROVEMENT` (MINOR: a `/code-review` finding whose category is simplification, efficiency, reuse, style or readability) and `SECURITY_REVIEW_VULNERABILITY` (BLOCKER), or a narrower key, with `reviewer` `claude-code-review` / `claude-security-review`.
**The brief-conflict mark (fields `engine`, `contradicts_brief`, `brief_lowered_from`).**
Each finding may carry three more fields, in its body field list and in the trailing JSON block:
`engine` (`code-review` | `security-review`, on a finding that came from an engine),
`contradicts_brief` (the brief's Non-Goal or stated choice the finding's recommended action contradicts, verbatim)
and `brief_lowered_from` (`MAJOR`, written only by the one lowering).
The mark is a field on a kept finding, never a `suppressed_reason`.
Only an engine finding is ever marked; a reviewer-agent finding (no `engine`) and a finding whose `rule_key` is not in the catalogue never are.
The one legal lowering: a `/code-review` MAJOR-tier finding at severity MAJOR drops to MINOR, with `brief_lowered_from: MAJOR`;
a BLOCKER-tier, a MINOR-tier and any `/security-review` finding keep their severity and only carry the mark.
On an ALLOW or WARN review the verdict gate exits 2 on every finding `illegalBriefLowering` (`lib/review/coordinator-contract.mjs`) names:
any other `brief_lowered_from`, a mark on a reviewer-agent finding, a marked finding below its tier, a marked finding with an unknown key,
and an engine finding below its tier, marked or not (a BLOCK review exits 1 as before).
The gate cannot check an `engine` label on the prose path: there the coordinator writes the label, so a finding wrongly labelled `engine` is judged as an engine finding;
the workflow path (`--workflow`) takes `engine` from the payload source and writes it into the returned finding.
`runContract` (`lib/review/coordinator-contract.mjs`) likewise takes `engine` from the payload a finding arrived in, never from the finding's own field,
and in dedup the survivor is ranked by catalogue tier (for a catalogue key it always ties inside one triplet), so a reviewer-agent finding survives over an engine finding and carries the highest severity the group asserted; an engine's mark never lowers what a reviewer agent asserted.
On the workflow path dedup ranks as Pass 1 does (both paths), every engine finding the coordinator did not return at its exact `(file, line, rule_key)` is put back at its own severity (operator decision 23), and a returned finding is never below the lower of its reviewer counterpart's severity and its catalogue tier (floor (b)), so the workflow path is stricter than the prose path; a coordinator that returns no verdict and findings counts as a missing reviewer there (`missing-reviewer:review-coordinator` in `allow_blocked_by`), and its findings are read at their catalogue tier or higher, so the review is at least WARN and the verdict gate exits 2 on such a WARN (a BLOCK stays exit 1).
A plan made from the review skips a lowered finding: it is MINOR, and `/trekplan` takes BLOCKER and MAJOR findings only.
**Validation strategy:**
| Layer | When | What |

400
lib/parsers/plan-schema.mjs
View file

@ -8,7 +8,9 @@
// plan_version 1.8 adds per-step field parsing (`parseStepFields`), the
// ceremony class the plan was made under (`small | bounded | architectural`,
// absent = architectural) and the design-part helpers (`splitSections`,
// `findModuleCycle`). The rules that use them live in plan-validator.mjs.
// `findModuleCycle`). plan_version 1.9 (the current target) adds the grammar
// of the `Files:` field and the `Test first:` `File:` line, the file first on
// every line (`entryPaths`). The rules that use them live in plan-validator.mjs.
import { ok, fail, issue } from '../util/result.mjs';
import { parseDocument } from '../util/frontmatter.mjs';
@ -135,9 +137,10 @@ export function validatePlanHeadings(text, opts = {}) {
// ---- plan_version 1.8 ------------------------------------------------------
export const PLAN_VERSION_CURRENT = '1.8';
// 1.7 plans stay valid: the 1.8 rules apply only from 1.8 on.
export const PLAN_VERSIONS_SUPPORTED = Object.freeze(['1.7', '1.8']);
export const PLAN_VERSION_CURRENT = '1.9';
// 1.7 plans stay valid: the 1.8 rules apply only from 1.8 on, the 1.9 `Files:`
// grammar only from 1.9 on (see "plan_version 1.9" below).
export const PLAN_VERSIONS_SUPPORTED = Object.freeze(['1.7', '1.8', '1.9']);
/** True when dotted version `v` is >= `min` (numeric per part; null → false). */
export function versionAtLeast(v, min) {
@ -152,6 +155,9 @@ export function versionAtLeast(v, min) {
return true;
}
/** True when plan_version `v` reads `Files:` and the `Test first:` `File:` line by the 1.9 grammar. */
export const usesFilesGrammar = (v) => versionAtLeast(v, '1.9');
// ---- ceremony (the brief's size class) -------------------------------------
export const CEREMONY_VALUES = Object.freeze(['small', 'bounded', 'architectural']);
@ -332,6 +338,26 @@ export function onFailureAction(text) {
*
* A backslash is read as a slash, a leading `./` is dropped. Release review of
* 6.3.0, rounds 1-10.
*
* Three answers 6.3.0 gave and 6.3.1 lost are given back, each in its narrow
* form, and the round 7-10 neighbours of each keep 6.3.1's answer:
*
* - 6.4 / regression A: inside a description, a comma entry that opens with a
* plain file with a slash, or with one followed by a colon or a dash, starts
* a new entry ("`a.test.mjs` - new test, src/a.mjs - the fix" names both).
* That is the new-line test without its "alone on the line" branch
* (opensChunk), so "— mocks fs, child_process.spawnSync and path.join" stays
* description (round 9).
* - 6.4 / regression B: in an entry opened by a backticked path, a plain path
* with a slash after a word of description ("`a.test.mjs` covers the parser
* in src/parse.mjs") counts only where a list starts (after "and", "or", "+"
* or "&"). Right after the backticked path ("`a.test.mjs` src/b.mjs"), and in
* an entry not opened by one (", the CLI entry src/cli.mjs", round 7), it is
* read as before. A list joiner is no word of description: "`a.mjs` +
* `b.mjs` src/c.mjs" reads all three.
* - 6.4 / regression C: a new line that is no list item and holds only a
* backticked capitalised name ("`LICENSE`") ends the description, and the
* name is a file. On the description's own line it stays no file (round 10).
*/
const SEG = '[\\w@.*?()-]';
const PATH_WORD = new RegExp(`^(?:${SEG}+\\/)+${SEG}*$|^[\\w@*?-][\\w@.*?-]*\\.[A-Za-z*][A-Za-z0-9*]*$|^\\.[\\w-][\\w.-]*$`);
@ -349,7 +375,9 @@ const DESCRIPTION_DASH = /^(?:\u2014|\u2013|-|--)$/;
const LIST_JOINER = /^(?:and|or|\+|&)$/i;
const LIST_ITEM = /^\s*(?:[-*+]|\d+[.)])\s+/;
const lastSegment = (p) => p.replace(/\/$/, '').split('/').pop();
const norm = (p) => p.replace(/\\/g, '/').replace(/^\.\//, '');
/** A plan path as the readers hold it: `/` separators, no leading `./`. */
export const normPlanPath = (p) => p.replace(/\\/g, '/').replace(/^\.\//, '');
const norm = normPlanPath;
// A backticked span that is a path, not a URL, an option or a call (`parse()`, `p.parse()`).
const tickedPath = (s) => /[./\\]/.test(s) && !/^https?:/.test(s) && !s.startsWith('-') && !/^[\w$.]+\(.*\)$/.test(s);
// A plain word without the brackets and punctuation around it, `\` read as `/`.
@ -361,10 +389,14 @@ const isFileWord = (w) => BUILD_FILE.test(w) || (isPathWord(w) && (FILE_OR_DIR.t
// plain file with a slash, or a plain file alone on the line or followed by a
// colon or a dash ("src/cli.mjs", "package.json", "Makefile — a target"), not
// "child_process.spawnSync, path.join" or "Node.js 22 only" (round 10).
const opensEntry = (tokens) => {
// A comma entry inside a description that opens a new entry: opensEntry's test
// without "alone on the line", so "path.join" after a comma stays description
// (6.4 / regression A).
const opensChunk = (tokens) => {
const w = plainWord(tokens[0]);
return isFileWord(w) && (w.includes('/') || tokens.length === 1 || tokens[0].endsWith(':') || DESCRIPTION_DASH.test(tokens[1]));
return isFileWord(w) && (w.includes('/') || tokens[0].endsWith(':') || DESCRIPTION_DASH.test(tokens[1]));
};
const opensEntry = (tokens) => opensChunk(tokens) || (tokens.length === 1 && isFileWord(plainWord(tokens[0])));
export function fieldPaths(text) {
const spans = [];
@ -382,12 +414,21 @@ export function fieldPaths(text) {
const item = LIST_ITEM.exec(line);
const body = item ? line.slice(item[0].length) : line;
if (item || (described && body.trim() !== '' && opensEntry(body.trim().split(/\s+/)))) described = false;
// A new line that holds only a backticked capitalised name ends the
// description (6.4 / regression C).
const alone = /^\u0000(\d+)\u0000$/.exec(body.trim());
if (alone && BARE_NAME.test(spans[Number(alone[1])])) described = false;
for (const raw of body.split(',')) {
const entry = raw.trim().replace(/^[-*]\s+/, '');
if (entry === '') continue;
const tokens = entry.split(/\s+/);
const first = /^\u0000(\d+)\u0000/.exec(entry);
if (first && isTicked[Number(first[1])]) described = false;
// An entry opened by a backticked path: after a word of description a
// plain path with a slash counts only where a list starts (6.4 / regression B).
const tickedEntry = Boolean(first && isTicked[Number(first[1])]);
if (tickedEntry) described = false;
if (described && opensChunk(tokens)) described = false;
let prose = false;
// A backticked capitalised name alone in its entry, before any
// description, is a file (rounds 6, 10).
if (first && first[0] === entry && !described && BARE_NAME.test(spans[Number(first[1])])) {
@ -410,8 +451,10 @@ export function fieldPaths(text) {
}
if (DESCRIPTION_DASH.test(t)) described = true;
const w = plainWord(t);
if (described || !isFileWord(w)) continue;
if (w.includes('/') || i === 0 || LIST_JOINER.test(tokens[i - 1])) out.push(w);
const file = isFileWord(w);
if (!file && !LIST_JOINER.test(t)) prose = true;
if (described || !file) continue;
if ((w.includes('/') && !(tickedEntry && prose)) || i === 0 || LIST_JOINER.test(tokens[i - 1])) out.push(w);
if (t.endsWith(':')) described = true;
}
continue;
@ -425,6 +468,167 @@ export function fieldPaths(text) {
return out.filter(Boolean).map(norm);
}
// ---- plan_version 1.9 ------------------------------------------------------
//
// From plan_version 1.9 a step's `Files:` field has a grammar, so no reader has
// to tell a path from a description in free text (decision 16, operator
// 2026-10-11: only a 1.9 plan is refused; an older plan is read by fieldPaths
// as before and gets one warning, PLAN_STEP_PATH_UNTICKED):
//
// - The entries are the field's first non-empty line and each list-item line
// (`-`, `*`, `+`, `1.`, `1)`). Any other non-empty line continues the entry
// above it: when the entry so far ends with a comma it is appended with one
// space before the entry is read, so a list wrapped after a comma
// ("`tests/a.test.mjs`," and "`src/a.mjs`" on the next line) names both
// files; otherwise it and every later wrapped line of that entry are
// description and never read (operator decision 29, 14.10; reviews
// `d21ff81e`, `eaeb398e`). An entry with nothing on its own line (a `File:`
// line with nothing after it) takes its first wrapped line. An empty first
// line before a nested list is no entry. An issue carries the line number
// of the entry's first line (review `b4fa1927`).
// - An entry's files are the backticked spans it opens with, separated only by
// commas or whitespace. Everything from the first other text on is
// description and is never read ("`a.test.mjs` - covers `src/parse.mjs`"
// names one file).
// - An entry that does not open with a backticked span is refused
// (PLAN_STEP_PATH_NOT_FIRST); so are `none` and a field that names no file.
// A leading span that fails isPathShape (`parse()`, an option, a URL) is
// refused as no path (PLAN_STEP_PATH_NOT_A_PATH).
//
// - A `Test first:` `File:` line is a line that opens with `File:` (or
// `**File:**`) after an optional list marker of the same set (`-`, `*`, `+`,
// `1.`, `1)`); its rest is one entry under this grammar. Any other
// non-empty line right after it continues it by the same rule: when the
// entry so far ends with a comma it is appended with one space, so a list
// wrapped after a comma names both files; otherwise it and every later
// wrapped line of that entry are description and never read (operator
// decision 29, 14.10; reviews `d21ff81e`, `eaeb398e`). A `File:` line with
// nothing after it takes its first wrapped line. Any list item (a
// `Verifies:` or `Pattern:` line too) ends it, and a non-list line after
// such an item belongs to that item, never to the `File:` entry. A line that
// opens with `File:` always starts a new entry; a `File:` mid-line names no
// test (review `4b6460e5`).
//
// isPathShape is the 1.9-only test on a span in the leading position of an
// entry (operator decision 22, 13.10): a placeholder (none, TBD, TODO, TBA,
// N/A, NA, `...`) is no file; a path, a build file or an all-capitals name
// such as `LICENSE` or `README` is; a mixed-case bare name (`Parser`) is not,
// and `./Name` names such a file. The readers below 1.9 keep BARE_NAME.
const LEADING_SPAN = /^`([^`\n]+)`/;
// Operator decision 22: these words are the input a 1.9 plan refuses, not unfinished code.
const PLACEHOLDER = /^(?:none|n\/?a|tbd|tba|todo|\.{3})$/i;
const CAPS_NAME = /^[A-Z][A-Z0-9_-]*$/;
/** A backticked span in the leading position of an entry that names a file. */
export function isPathShape(s) {
const span = String(s ?? '').trim();
if (PLACEHOLDER.test(span)) return false;
return tickedPath(span) || BUILD_FILE.test(span) || CAPS_NAME.test(span);
}
/**
* The trimmed contents of the backticked spans `text` opens with, each
* separated from the next only by commas or whitespace; empty when `text` does
* not open with a backticked span. The caller judges each span.
* @param {string} text
* @returns {string[]}
*/
function leadingSpans(text) {
const out = [];
let rest = String(text ?? '');
let m = LEADING_SPAN.exec(rest);
while (m) {
out.push(m[1].trim());
rest = rest.slice(m[0].length);
const sep = /^[\s,]+/.exec(rest);
m = sep ? LEADING_SPAN.exec(rest.slice(sep[0].length)) : null;
if (m) rest = rest.slice(sep[0].length);
}
return out;
}
/**
* Whether a wrapped line (a non-empty line that is no list item) joins the
* 1.9 entry above it (operator decision 29, 14.10; reviews `d21ff81e`,
* `eaeb398e`). An entry with nothing on its own line (a `File:` line with
* nothing after it) takes its first wrapped line; an entry whose text so far
* ends with a comma gets the line appended with one space, so a list wrapped
* after a comma names both files. Otherwise the line is description: null,
* and the caller closes the entry, so every later wrapped line of it is
* description too.
* @param {string} text - the entry so far
* @param {string} line - the wrapped line
* @returns {string|null} the entry's new text, or null when the line is description
*/
function wrapLine(text, line) {
if (text === '') return line.trim();
return text.endsWith(',') ? `${text} ${line.trim()}` : null;
}
/**
* The files a 1.9 `Files:` field names, by the grammar above.
* @param {string} text - the field's text (parseStepFields(body).get('Files').text)
* @returns {{files: string[], issues: Array<{line: number, entry: string,
* code: 'PLAN_STEP_PATH_NOT_FIRST'|'PLAN_STEP_PATH_NOT_A_PATH'}>}}
* files normalised, in order, deduplicated; `line` is 1-based in `text`.
*/
export function entryPaths(text) {
const files = [];
const issues = [];
const entries = [];
// Whether the last entry still takes wrapped lines (operator decision 29).
let open = false;
String(text ?? '').split('\n').forEach((line, i) => {
const item = LIST_ITEM.exec(line);
const rest = line.trim();
if (item) { entries.push({ line: i + 1, entry: line.slice(item[0].length).trim() }); open = true; }
else if (rest === '') return;
else if (entries.length === 0) { entries.push({ line: i + 1, entry: rest }); open = true; }
else {
// A wrapped line joins the entry above it only after a comma, or when
// the entry is empty (wrapLine); otherwise it closes the entry, and it
// and every later wrapped line of that entry are description, never read
// (operator decision 29).
if (!open) return;
const last = entries[entries.length - 1];
const joined = wrapLine(last.entry, rest);
if (joined === null) open = false;
else last.entry = joined;
}
});
if (entries.length === 0) {
issues.push({ line: 1, entry: '', code: 'PLAN_STEP_PATH_NOT_FIRST' });
return { files, issues };
}
for (const { line, entry } of entries) {
const spans = leadingSpans(entry);
if (spans.length === 0) {
issues.push({ line, entry, code: 'PLAN_STEP_PATH_NOT_FIRST' });
continue;
}
for (const span of spans) {
if (isPathShape(span)) {
const p = norm(span);
if (!files.includes(p)) files.push(p);
} else {
issues.push({ line, entry, code: 'PLAN_STEP_PATH_NOT_A_PATH' });
}
}
}
return { files, issues };
}
/**
* The paths fieldPaths reads from `text` that do not stand in backticks: the
* paths below plan_version 1.9 that a 1.9 plan would refuse.
* @returns {string[]}
*/
export function untickedPaths(text) {
const ticked = new Set([...String(text ?? '').matchAll(/`([^`\n]+)`/g)].map((m) => norm(m[1].trim())));
return fieldPaths(text).filter((p) => !ticked.has(p));
}
// A document or a data file outside tests/ is never the test here:
// `docs/specs/api.md` and `api/specs/openapi.yaml` sit under a `specs/` folder,
// which lib/util/test-path.mjs reads as tests for the floor guard. Under tests/
@ -433,6 +637,11 @@ const DOC_RE = /\.(md|markdown|mdx|txt|rst|adoc)$/i;
const DATA_RE = /\.(ya?ml|json|toml|xml|csv)$/i;
const UNDER_TESTS_RE = /(^|\/)(tests?|__tests__)\//;
/** A path that is clearly a test: a test by its path, and under tests/ or neither a document nor a data file. */
function clearlyTest(p) {
return isTestPath(p) && (UNDER_TESTS_RE.test(p) || !(DOC_RE.test(p) || DATA_RE.test(p)));
}
/**
* Whether every file in a step's `Files:` is a test: the step's own `Test
* first:` file, or a path that is clearly a test (lib/util/test-path.mjs). Such
@ -442,31 +651,174 @@ const UNDER_TESTS_RE = /(^|\/)(tests?|__tests__)\//;
* trial 3, 2026-10-09: `not-red-at-base`, stop class 4). The plan validator
* warns on it (PLAN_STEP_TESTS_ONLY) and red-proof.mjs reports it
* (`tests_only`), so the executor sends the step up to the plan. A step
* without `Files:` is not such a step.
* without `Files:` is not such a step. From plan_version 1.9 `Files:` is read
* by its grammar (entryPaths); below 1.9 (or without a version) by fieldPaths.
* @param {string} body - the step's section (sliceSteps)
* @param {string|null} [version] - the plan's plan_version
* @returns {{testsOnly: boolean, files: string[]}}
*/
export function stepTestsOnly(body) {
return stepTestsOnlyFromFields(parseStepFields(body));
export function stepTestsOnly(body, version) {
return stepTestsOnlyFromFields(parseStepFields(body), version);
}
// A line of the `Test first:` field that names its test files (6.3.1's
// selector); global, so read it with matchAll only: one match per `File:`.
const TEST_FIRST_FILE_LINE = /\bFile:\**[ \t]*(?=`)/gi;
// From plan_version 1.9: a line that opens with `File:` (after an optional
// list marker of the set LIST_ITEM accepts, and `**`), backticked or not.
const TEST_FIRST_FILE_LINE_19 = /^\s*(?:(?:[-*+]|\d+[.)])\s+)?(?:\*\*)?File:(?:\*\*)?/i;
// Below plan_version 1.9: one glue token at the start of the rest of a `File:`
// run (operator decisions 28, 13.10, and 31, 14.10): whitespace or a comma
// (group 1); a note in brackets holding no bracket, backtick or newline,
// optionally in one `*` or `_` (`(new)`, `*(new)*`, `(existing, the parser)`);
// the whole word `and` or `plus`, any case (`android` is no glue); or `+`.
const OLDER_GLUE = /^(?:(\s+|,)|[*_]?\([^()`\n]*\)[*_]?|(?:and|plus)(?!\w)|\+)/i;
/**
* The files of one below-1.9 `File:` run, read from just after the marker:
* the leading backticked span, then each span after one or more glue tokens
* (OLDER_GLUE). While every glue token so far was whitespace or a comma, a
* span with the shape of a path is kept and any other span is skipped
* (operator decision 20). Once a glue token was a note or a word, a span is
* kept only when it has the shape of a path and is clearly a test, and the
* first span that is not ends the run (operator decisions 28 and 31). Any text
* that is neither a span nor glue ends the run.
* @param {string} text - the line after one `File:` marker
* @returns {string[]} the kept spans, trimmed, not normalised
*/
function olderFileRun(text) {
const out = [];
let rest = String(text ?? '');
let loose = false;
let m = LEADING_SPAN.exec(rest);
while (m) {
const s = m[1].trim();
if (!loose) {
if (tickedPath(s) || BUILD_FILE.test(s)) out.push(s);
} else if (tickedPath(s) && clearlyTest(norm(s))) {
out.push(s);
} else {
break;
}
rest = rest.slice(m[0].length);
let glued = false;
let g = OLDER_GLUE.exec(rest);
while (g) {
glued = true;
if (g[1] === undefined) loose = true;
rest = rest.slice(g[0].length);
g = OLDER_GLUE.exec(rest);
}
m = glued ? LEADING_SPAN.exec(rest) : null;
}
return out;
}
/**
* Whether a path counts as a test for this step: its own `Test first:` file, or
* a path that is clearly a test.
* plan_version 1.9: the rest of each `Test first:` `File:` line (after `File:`
* and any `**`), one entry per `File:` line, in order. Any other non-empty
* line right after it continues the entry above it: when the entry so far
* ends with a comma it is appended with one space, so a list wrapped after a
* comma names both files; otherwise it and every later wrapped line of that
* entry are description and never read (operator decision 29, 14.10; reviews
* `d21ff81e`, `eaeb398e`). An entry with nothing on its own line (a `File:`
* line with nothing after it) takes its first wrapped line. Any list item ends
* the entry; a line that opens with `File:` starts a new one. The one
* selector both testFirstFiles and the plan validator's grammar check read.
* @param {Map<string, {text: string}>} fields - parseStepFields(body)
* @returns {string[]}
*/
export function testFirstFileEntries(fields) {
if (!fields || !fields.has('Test first')) return [];
const out = [];
// Whether the line above is the current `File:` line or its continuation.
let open = false;
for (const line of String(fields.get('Test first').text ?? '').split('\n')) {
const at = TEST_FIRST_FILE_LINE_19.exec(line);
if (at) {
out.push(line.slice(at[0].length).trim());
open = true;
} else if (LIST_ITEM.test(line) || line.trim() === '') {
open = false;
} else if (open) {
const joined = wrapLine(out[out.length - 1], line);
if (joined === null) open = false;
else out[out.length - 1] = joined;
}
}
return out;
}
/**
* The step's test files: every file on the `Test first:` `File:` line.
*
* Below plan_version 1.9 (or without a version) a line counts as that line
* exactly when TEST_FIRST_FILE_LINE matches it. Each `File:` marker on the
* line opens a run: the backticked spans that stand right after it, separated
* only by commas or whitespace (olderFileRun), and every span in that run with
* the shape of a path is a test file (a call `parse()`, an option, a URL is
* skipped and the run goes on). A path in the description after the run is no
* test file (operator decision 20, 12.10; review `6c31212c`: "covers `parse()`
* in `src/parser.mjs`" made the source a test and the step tests-only). The
* markers are matched globally because 6.3.1 read the first span after each
* `File:`, so a line with two markers keeps at least 6.3.1's files. Below 1.9
* the run also goes on past a note in brackets (`(new)`, `*(new)*`,
* `(existing ...)`) and the words `and`, `plus` and `+`, to a further path
* only when it is clearly a test, and a source path there ends it
* (olderFileRun; operator decisions 28 and 31, 13.10 and 14.10; review
* `bd2520b0`; review `6c31212c` for the guard).
*
* From 1.9 a line counts when it opens with `File:` (testFirstFileEntries),
* backticked or not, after an optional list marker (`1.` and `1)` included);
* its rest is one entry read by entryPaths: the leading backticked paths are
* the test files, the rest is description. Any other non-empty line right
* after it continues the entry: when the entry so far ends with a comma it is
* appended with one space, so a list wrapped after a comma names both files;
* otherwise it and every later wrapped line of that entry are description and
* never read (operator decision 29, 14.10; reviews `d21ff81e`, `eaeb398e`). A
* `File:` line with nothing after it takes its first wrapped line.
*
* Normalised, in order, deduplicated. A `Verifies:` line without a `File:` is
* never read.
* @param {Map<string, {text: string}>} fields - parseStepFields(body)
* @param {string|null} [version] - the plan's plan_version
* @returns {string[]}
*/
export function testFirstFiles(fields, version) {
if (!fields || !fields.has('Test first')) return [];
const out = [];
if (usesFilesGrammar(version)) {
for (const entry of testFirstFileEntries(fields)) {
for (const p of entryPaths(entry).files) if (!out.includes(p)) out.push(p);
}
return out;
}
for (const line of String(fields.get('Test first').text ?? '').split('\n')) {
for (const at of line.matchAll(TEST_FIRST_FILE_LINE)) {
for (const s of olderFileRun(line.slice(at.index + at[0].length))) {
const p = norm(s);
if (!out.includes(p)) out.push(p);
}
}
}
return out;
}
/**
* Whether a path counts as a test for this step: its own `Test first:` files
* (testFirstFiles), or a path that is clearly a test.
* @returns {(path: string) => boolean}
*/
function stepTestFile(f) {
const testFirst = f.has('Test first')
? [...f.get('Test first').text.matchAll(/\bFile:\**[ \t]*`([^`\n]+)`/gi)].map((m) => norm(m[1].trim()))
: [];
return (p) => testFirst.includes(p) ||
(isTestPath(p) && (UNDER_TESTS_RE.test(p) || !(DOC_RE.test(p) || DATA_RE.test(p))));
function stepTestFile(f, version) {
const testFirst = testFirstFiles(f, version);
return (p) => testFirst.includes(p) || clearlyTest(p);
}
/** `stepTestsOnly` for a caller that already holds the step's parsed fields. */
export function stepTestsOnlyFromFields(f) {
const files = f.has('Files') ? fieldPaths(f.get('Files').text) : [];
return { testsOnly: files.length > 0 && files.every(stepTestFile(f)), files };
export function stepTestsOnlyFromFields(f, version) {
const text = f.has('Files') ? f.get('Files').text : null;
const files = text === null ? [] : usesFilesGrammar(version) ? entryPaths(text).files : fieldPaths(text);
return { testsOnly: files.length > 0 && files.every(stepTestFile(f, version)), files };
}
// ---- sections (fence-aware) ------------------------------------------------

210
lib/review/coordinator-contract.mjs
View file

@ -16,8 +16,14 @@
// placeholders and lib/review/fill-finding-ids.mjs computes them (the
// old doc named a 4-tuple with `title` the code never computed).
//
// What IS implemented, purely: Pass 1 (triplet dedup → highest-severity-wins
// survivor + conformance tiebreak + detail concat + raised_by provenance),
// What IS implemented, purely: Pass 1 (triplet dedup → survivor ranked by
// catalogue tier, then reviewer over engine, asserted severity, conformance,
// first seen; the survivor carries its group's highest asserted severity
// (review bc7f4e23) + detail concat + raised_by provenance; when that raises
// it, `pass1_raised_from` keeps what the survivor itself asserted, read by
// Pass 3 and never returned: `original_severity` is what the kept finding
// itself asserted, before Pass 1 raised it to its group's severity (review
// 08deae68)),
// Pass 2 succinctness (trims MINOR/SUGGESTION text, never removes) +
// actionability-presence, Pass 3 reasonableness
// (citation / unknown-rule_key suppression, severity-mismatch correction),
@ -28,12 +34,13 @@
// Reuses: SEVERITY_VALUES / RULE_KEYS / getRule (rule-catalogue.mjs),
// computeFindingId (finding-id.mjs, triplet), validateFindings
// (findings-schema.mjs). Triplet key format mirrors
// scripts/bakeoff-armA-merge.mjs:33; raised_by provenance mirrors
// scripts/bakeoff-armA-merge.mjs `dedupByTriplet`; raised_by provenance mirrors
// lib/review/plan-review-dedup.mjs.
import { SEVERITY_VALUES, RULE_KEYS, getRule } from './rule-catalogue.mjs';
import { computeFindingId } from '../parsers/finding-id.mjs';
import { validateFindings } from './findings-schema.mjs';
import { ENGINES } from './engine-findings.mjs';
import { isMain } from '../util/is-main.mjs';
export const JUDGE_TITLE_MAX = 100;
@ -80,6 +87,85 @@ export const UNVERIFIED_REASONS = Object.freeze([
'unknown-rule_key',
'file-existence:indeterminate',
]);
// The brief-conflict mark (`contradicts_brief`) is in neither set on purpose:
// it is a field on a KEPT finding, never a `suppressed_reason`. As a reason it
// would fail closed through `classifySuppression`, land the finding in
// `unverified` and forbid ALLOW — the opposite of what the mark is for.
// ---- The brief-conflict rule ---------------------------------------------------
//
// The one place that decides when a finding marked as contradicting the brief
// (`contradicts_brief: "<the Non-Goal or stated choice, verbatim>"`) may drop
// to MINOR. Only an engine finding (`engine` `code-review` or `security-review`)
// is ever marked; only a `/code-review` finding at the MAJOR tier ever drops,
// to MINOR, writing `brief_lowered_from: 'MAJOR'`. A BLOCKER and any
// `/security-review` finding keep their severity and only carry the mark.
// The verdict gate (verdict-gate.mjs) imports `illegalBriefLowering`; the
// opt-in workflow review mirrors it. `original_severity` (what the kept finding
// itself asserted before Pass 1 raised it, review 08deae68) is never read here.
// The payload `reviewer` names lib/review/engine-findings.mjs writes, and the
// engine each one stands for.
const ENGINE_OF_PAYLOAD = Object.freeze(Object.fromEntries(
Object.entries(ENGINES).map(([engine, spec]) => [spec.reviewer, engine])));
const isEngineFinding = (f) => Object.hasOwn(ENGINES, f.engine ?? '');
function carries(f, field) {
return f?.[field] !== undefined && f?.[field] !== null;
}
function isMark(v) {
return typeof v === 'string' && v.trim().length > 0;
}
/**
* True when a finding, before the lowering, may drop to MINOR: a non-empty
* string mark, `engine: 'code-review'`, a known `rule_key` whose catalogue tier
* is MAJOR, and severity MAJOR. Anything else (missing engine, unknown key,
* non-string mark, a BLOCKER-tier or MINOR-tier key) → false.
* @param {object} f
* @returns {boolean}
*/
export function lowerableBriefConflict(f) {
return markedMajorCodeReview(f) && f.severity === 'MAJOR';
}
function markedMajorCodeReview(f) {
return isMark(f?.contradicts_brief) &&
f.engine === 'code-review' &&
getRule(f.rule_key)?.severity === 'MAJOR';
}
function isLegalBriefLowering(f) {
return markedMajorCodeReview(f) &&
f.severity === 'MINOR' &&
f.brief_lowered_from === 'MAJOR';
}
/**
* True when what the coordinator wrote is not the one legal shape and (a)
* carries `brief_lowered_from`, or (b) carries a mark without an engine, or
* (c) carries a mark below its catalogue tier, or (d) carries a mark with a
* missing or unknown `rule_key` (fail-closed), or (e) is an engine finding
* below its catalogue tier, marked or not. An unmarked reviewer finding (no
* `engine`) without `brief_lowered_from` is never judged by this rule.
* @param {object} f
* @returns {boolean}
*/
export function illegalBriefLowering(f) {
if (f === null || typeof f !== 'object') return false;
if (isLegalBriefLowering(f)) return false;
const marked = carries(f, 'contradicts_brief');
const engine = isEngineFinding(f);
const rule = getRule(f.rule_key);
const belowTier = rule !== null && severityRank(f.severity) > severityRank(rule.severity);
if (carries(f, 'brief_lowered_from')) return true; // (a)
if (marked && !engine) return true; // (b)
if (marked && rule === null) return true; // (d)
if (engine && belowTier) return true; // (c) and (e): a marked finding is an engine one past (b)
return false;
}
/**
* Classify a suppression reason. Anything not declared refuting is
@ -141,18 +227,34 @@ export function ingest(reviewerPayloads) {
skipped.push({ reviewer: payload?.reviewer ?? null, error_codes: r.errors.map((e) => e.code) });
continue;
}
// "Came from an engine" is read from the payload, never from an `engine`
// field a reviewer wrote on its own finding (decision 13, operator 12.10):
// an engine payload's findings get `engine` from its `reviewer`, every other
// payload's findings lose any `engine` they carry.
const eng = Object.hasOwn(ENGINE_OF_PAYLOAD, payload.reviewer) ? ENGINE_OF_PAYLOAD[payload.reviewer] : null;
for (const f of payload.findings) {
findings.push({ ...f, reviewer: f.reviewer ?? payload.reviewer ?? f.owner_reviewer ?? null });
const { engine: _claimed, ...rest } = f;
findings.push({
...rest,
...(eng ? { engine: eng } : {}),
reviewer: f.reviewer ?? payload.reviewer ?? f.owner_reviewer ?? null,
});
}
}
return { findings, skipped };
}
/**
* Pass 1 — dedup by (file, line, rule_key) triplet. Survivor = highest
* catalogue severity; severity tie → prefer the conformance reviewer; carries
* raised_by provenance, concatenates other reviewers' attribution into detail,
* and recomputes the id over the triplet.
* Pass 1 — dedup by (file, line, rule_key) triplet. The survivor is ranked by
* (1) the catalogue tier (`getRule(rule_key)?.severity`, else the finding's own
* severity; lower `severityRank` wins), (2) a reviewer-agent finding over an
* engine finding, (3) among findings of one kind the higher asserted severity,
* (4) the conformance reviewer, (5) first seen. It is written with the highest
* severity any finding in its group asserted, so dropping a duplicate never
* lowers the group (review bc7f4e23; plan 2026-10-11-files-backticks-trial4,
* ADR "Pass 1 survivor order and group severity"). Carries raised_by
* provenance, concatenates other reviewers' attribution into detail, and
* recomputes the id over the triplet.
* @param {object[]} findings
* @returns {object[]}
*/
@ -165,13 +267,36 @@ export function dedupByTriplet(findings) {
}
const out = [];
for (const group of groups.values()) {
// Ranked as the JSDoc above says. A brief-conflict mark is not carried over
// from a group member to the survivor: when the survivor is unmarked, the
// group stays unmarked and is judged at its full severity. For a catalogue
// key the tier always ties inside one triplet (the rule_key is shared).
const tier = (f) => severityRank(getRule(f.rule_key)?.severity ?? f.severity);
let survivor = group[0];
for (const f of group.slice(1)) {
const higher = severityRank(f.severity) < severityRank(survivor.severity);
const tieToConformance =
severityRank(f.severity) === severityRank(survivor.severity) &&
isConformance(f.reviewer) && !isConformance(survivor.reviewer);
if (higher || tieToConformance) survivor = f;
const tf = tier(f);
const ts = tier(survivor);
if (tf !== ts) {
if (tf < ts) survivor = f;
continue;
}
const ef = isEngineFinding(f);
const es = isEngineFinding(survivor);
if (ef !== es) {
if (!ef) survivor = f;
continue;
}
const rf = severityRank(f.severity);
const rs = severityRank(survivor.severity);
if (rf !== rs) {
if (rf < rs) survivor = f;
continue;
}
if (isConformance(f.reviewer) && !isConformance(survivor.reviewer)) survivor = f;
}
let severity = survivor.severity;
for (const f of group) {
if (severityRank(f.severity) < severityRank(severity)) severity = f.severity;
}
const raised_by = [...new Set(group.map((f) => f.reviewer).filter(Boolean))];
const others = group.filter((f) => f !== survivor);
@ -183,7 +308,14 @@ export function dedupByTriplet(findings) {
}
}
const id = computeFindingId(survivor.file, survivor.line, survivor.rule_key);
out.push({ ...survivor, id, ...(detail !== undefined ? { detail } : {}), raised_by });
// When the group raised the survivor, Pass 3 still needs what the survivor
// itself asserted for `original_severity` (review 08deae68).
const raised = severityRank(severity) < severityRank(survivor.severity);
out.push({
...survivor, ...(severity !== undefined ? { severity } : {}), id,
...(detail !== undefined ? { detail } : {}), raised_by,
...(raised ? { pass1_raised_from: survivor.severity } : {}),
});
}
return out;
}
@ -202,11 +334,13 @@ export const TRIM_MARKER = ' […trimmed]';
* The tier a finding will be judged at: the catalogue tier when the rule_key is
* known (Pass 3 resets to it anyway), else the reviewer-asserted severity. The
* trim decision reads this, so a reviewer cannot get a MAJOR trimmed by
* mislabelling it MINOR.
* mislabelling it MINOR. A finding the brief rule will lower (Pass 3) is
* judged at MINOR, the tier it will be counted at.
* @param {object} f
* @returns {string}
*/
function effectiveSeverity(f) {
if (lowerableBriefConflict(f)) return 'MINOR';
return getRule(f.rule_key)?.severity ?? f.severity;
}
@ -287,6 +421,10 @@ export function judgeFilter(findings) {
* checkable claim) and is dropped. `unknown-rule_key` does not: an ad-hoc key
* is a real defect wearing the wrong label — v5.1.1 high-effort mode already
* KEEPS these, normalised to PLAN_EXECUTE_DRIFT — so it routes to `unverified`.
*
* The brief rule runs here too: a finding `lowerableBriefConflict` accepts is
* set to MINOR with `brief_lowered_from: 'MAJOR'`; every other finding gets
* the catalogue reset above.
* @param {object[]} findings
* @returns {{ kept: object[], dropped: object[], unverified: object[] }}
*/
@ -294,7 +432,20 @@ export function reasonablenessFilter(findings) {
const kept = [];
const dropped = [];
const unverified = [];
for (const f of findings) {
for (const incoming of findings) {
// The brief rule, first: only this pass writes `brief_lowered_from`, so an
// incoming one is removed; only an engine finding is ever marked, so a mark
// on any other finding is removed. Every finding written here then makes
// `illegalBriefLowering` false.
// `pass1_raised_from` (Pass 1) names what the finding itself asserted
// before its group raised it; it is read here and never written out.
const { brief_lowered_from: _lowered, pass1_raised_from: raisedFrom, ...rest } = incoming;
const own = raisedFrom ?? rest.severity;
let f = rest;
if (!isEngineFinding(f) && 'contradicts_brief' in f) {
const { contradicts_brief: _mark, ...unmarked } = f;
f = unmarked;
}
if (typeof f.file !== 'string' || f.file.length === 0 ||
(typeof f.line === 'number' && f.line < 0)) {
suppress(f, 'no-citation', dropped, unverified);
@ -304,11 +455,20 @@ export function reasonablenessFilter(findings) {
suppress(f, 'unknown-rule_key', dropped, unverified);
continue;
}
const rule = getRule(f.rule_key);
if (rule && f.severity !== rule.severity) {
kept.push({ ...f, severity: rule.severity, original_severity: f.severity });
const rule = getRule(f.rule_key); // never null: RULE_KEYS checked above
if (lowerableBriefConflict(f)) {
// The one legal lowering: a marked /code-review MAJOR on a MAJOR-tier key.
kept.push({ ...f, severity: 'MINOR', brief_lowered_from: 'MAJOR' });
} else {
kept.push(f);
// Today's catalogue reset. A marked engine finding arrives at its tier
// (engine-findings.mjs), so it lands here only when its severity was
// changed on the way, and the reset puts it back with its mark kept.
// `original_severity` names what the kept finding itself asserted
// (`own`), never the severity Pass 1 lent it (review 08deae68).
kept.push({
...f, severity: rule.severity,
...(own !== rule.severity ? { original_severity: own } : {}),
});
}
}
return { kept, dropped, unverified };
@ -384,6 +544,12 @@ export function runContract(reviewerPayloads, options = {}) {
const deduped = dedupByTriplet(ingested);
const judged = judgeFilter(deduped);
const reasoned = reasonablenessFilter(judged.kept);
// Pass 2's drops and unverified come straight from Pass 1: its
// `pass1_raised_from` bookkeeping is removed there too, so no returned
// finding carries it (Pass 3 removes it from everything it sees).
const withoutPass1 = ({ pass1_raised_from: _raised, ...f }) => f;
const judgedDropped = judged.dropped.map(withoutPass1);
const judgedUnverified = judged.unverified.map(withoutPass1);
// Engine findings on files outside the review's SHA range (engine-findings.mjs
// `out_of_scope`). /security-review diffs against origin's default branch,
// wider than this review: a real claim nobody here judged, so it forbids
@ -398,7 +564,7 @@ export function runContract(reviewerPayloads, options = {}) {
const minorOnly = (f) => f.engine === 'code-review' && (f.severity === 'MINOR' || f.severity === 'SUGGESTION');
const outOfScopeBlocking = outOfScope.filter((f) => !minorOnly(f));
const outOfScopeNoted = outOfScope.filter(minorOnly);
const unverified = [...judged.unverified, ...reasoned.unverified, ...outOfScopeBlocking];
const unverified = [...judgedUnverified, ...reasoned.unverified, ...outOfScopeBlocking];
// What the coordinator set aside without ruling on it, one line each, so a
// reader sees every finding that did not reach a verdict tier. Borrowed:
// "Declined to judge … nothing you set aside is dropped silently"
@ -446,7 +612,7 @@ export function runContract(reviewerPayloads, options = {}) {
verdict,
counts,
findings: reasoned.kept,
suppressed: [...judged.dropped, ...judged.unverified, ...reasoned.dropped, ...reasoned.unverified],
suppressed: [...judgedDropped, ...judgedUnverified, ...reasoned.dropped, ...reasoned.unverified],
unverified,
skipped,
missing_reviewers,

2
lib/review/engine-findings.mjs
View file

@ -90,7 +90,7 @@ export const IMPROVEMENT_CATEGORIES = Object.freeze(new Set(['simplification', '
*/
export function toReviewerPayload(input, opts = {}) {
const errors = [];
const spec = ENGINES[input?.engine];
const spec = Object.hasOwn(ENGINES, input?.engine ?? '') ? ENGINES[input.engine] : null;
if (!spec) return { ok: false, errors: [`engine must be one of ${Object.keys(ENGINES).join('|')}, got ${JSON.stringify(input?.engine)}`] };
if (!Array.isArray(input.findings)) return { ok: false, errors: ['findings must be an array'] };
if (opts.scopeFiles && opts.scopeFiles.length === 0) {

12
lib/review/verdict-gate.mjs
View file

@ -8,7 +8,9 @@
// 0 = ALLOW or WARN
// 1 = BLOCK
// 2 = could not decide: missing/unreadable file, not a review, no verdict,
// an unknown verdict, a file that contradicts its own verdict, or a
// an unknown verdict, a file that contradicts its own verdict, a
// finding lowered or marked against the brief outside the one legal
// shape (coordinator-contract `illegalBriefLowering`), or a
// WARN because a reviewer or engine did not run (`missing-reviewer:` /
// `unattributable-payload` in allow_blocked_by — operator decision
// 2026-10-06: a review not performed in full stops a merge).
@ -29,6 +31,7 @@
import { readFileSync, existsSync } from 'node:fs';
import { validateReviewContent, VERDICT_VALUES } from '../validators/review-validator.mjs';
import { fencedBlocks, isJsonBlock } from './findings-schema.mjs';
import { illegalBriefLowering } from './coordinator-contract.mjs';
import { isMain } from '../util/is-main.mjs';
export const EXIT = Object.freeze({ PASS: 0, BLOCK: 1, UNDECIDED: 2 });
@ -100,6 +103,13 @@ export function decideVerdict(text) {
if (blockerFindings > 0 || blockerCount > 0 || blockerSection > 0) {
return undecided(`verdict ${verdict} over BLOCKER finding(s) (json: ${blockerFindings}, counts: ${blockerCount}, ## Findings (BLOCKER): ${blockerSection})`, verdict);
}
// The brief-conflict rule (coordinator-contract `illegalBriefLowering`, the
// one place it lives): a finding lowered or marked outside the one legal
// shape means the verdict may be greener than the findings — never green.
const illegal = payload.findings.filter((f) => illegalBriefLowering(f)).map((f) => f.id);
if (illegal.length > 0) {
return undecided(`verdict ${verdict} over a brief-conflict lowering outside the one legal shape (finding ${illegal.join(', ')})`, verdict);
}
const fmIds = new Set(Array.isArray(fm.findings) ? fm.findings : []);
const jsonIds = new Set(payload.findings.map((f) => f.id));
if (fmIds.size !== jsonIds.size || [...fmIds].some((id) => !jsonIds.has(id))) {

44
lib/util/artifact-commit.mjs
View file

@ -10,13 +10,24 @@
// operator has staged stays staged and uncommitted. A path the repo ignores is
// never forced in; the refusal says so. The repo's own git hooks run as usual.
//
// The commit carries the caller's trailer: `--trailer "<line>"`, one line
// matching `Co-Authored-By: <name> <<email>>` (TRAILER_RE), written as the
// message's last paragraph (`git commit -m <message> -m <trailer>`). A missing
// trailer, a multi-line one, one that does not match (an unfilled `{model}`
// placeholder, no `<email>`, another key) or one holding `Claude-Session:` is
// refused with exit 1. Order: the `--store` check first (exit 2), then the
// trailer (exit 1), both before commitArtifacts makes any git call (the CLI
// only finds the repo root first), so "nothing to commit" cannot mask a bad
// trailer.
//
// CLI:
// node lib/util/artifact-commit.mjs --store <git|local> [--artifact <path>]...
// [--message <msg>] [--repo <dir>]
// --trailer "Co-Authored-By: <name> <<email>>" [--message <msg>] [--repo <dir>]
// prints {committed, commit?, paths, excluded, warnings, reason?}
// exit 0 = committed, or nothing new to commit; 1 = refused (ignored path, path
// outside the repo, git add/commit failed); 2 = could not run (not a git repo,
// bad arguments, git timed out). 2 is never green.
// exit 0 = committed, or nothing new to commit; 1 = refused (missing or bad
// trailer, ignored path, path outside the repo, git add/commit failed);
// 2 = could not run (not a git repo, bad arguments, git timed out). 2 is never
// green.
import { spawnSync } from 'node:child_process';
import { appendFileSync, existsSync, readFileSync, realpathSync } from 'node:fs';
@ -26,6 +37,24 @@ import { isMain } from './is-main.mjs';
export const ARTIFACT_STORES = Object.freeze(['git', 'local']);
const DEFAULT_MESSAGE = 'docs(voyage): record approval';
// One `Co-Authored-By:` line; `{`/`}` are refused so an unfilled `{model}` fails.
// The name starts and ends with a non-space, and trailerProblem refuses two
// spaces in a row, so an empty `{model}` ("Claude" + two spaces), a blank name
// and a padded one fail too (review 8420915b).
const TRAILER_RE = /^Co-Authored-By: [^\s<>{}](?:[^<>{}\n]*[^\s<>{}])? <[^\s<>@]+@[^\s<>]+>$/;
/** @returns {string|null} why `trailer` is refused, or null when it is valid */
function trailerProblem(trailer) {
if (typeof trailer !== 'string' || trailer === '') {
return '--trailer "Co-Authored-By: <name> <<email>>" is required — nothing committed';
}
if (/[\r\n]/.test(trailer)) return '--trailer must be one line — nothing committed';
if (trailer.includes('Claude-Session:')) return '--trailer must not hold a Claude-Session: line — nothing committed';
if (!TRAILER_RE.test(trailer) || trailer.includes(' ')) {
return `--trailer must read "Co-Authored-By: <name> <<email>>" with every placeholder filled; got "${trailer}" — nothing committed`;
}
return null;
}
function git(repo, args, timeout = 60_000) {
return spawnSync('git', ['-C', repo, ...args], { encoding: 'utf8', timeout });
@ -48,11 +77,13 @@ function excludeLocally(repo, rel) {
* @returns {{ exit: number, committed: boolean, commit?: string, paths: string[],
* excluded: string[], warnings: string[], reason?: string }}
*/
export function commitArtifacts({ repo, store, artifacts = [], message = DEFAULT_MESSAGE }) {
export function commitArtifacts({ repo, store, artifacts = [], message = DEFAULT_MESSAGE, trailer }) {
const res = { committed: false, paths: [], excluded: [], warnings: [] };
if (!ARTIFACT_STORES.includes(store)) {
return { ...res, exit: 2, reason: `--store must be one of [${ARTIFACT_STORES.join(', ')}]` };
}
const bad = trailerProblem(trailer);
if (bad) return { ...res, exit: 1, reason: bad };
if (existsSync(logPath(repo))) res.paths.push(LOG_RELPATH);
for (const a of artifacts) {
const rel = normalizeSubject(repo, a);
@ -79,7 +110,7 @@ export function commitArtifacts({ repo, store, artifacts = [], message = DEFAULT
if (git(repo, ['diff', '--cached', '--quiet', '--', ...res.paths]).status === 0) {
return { ...res, exit: 0, reason: 'nothing to commit' };
}
const c = git(repo, ['commit', '-q', '--only', '-m', message, '--', ...res.paths]);
const c = git(repo, ['commit', '-q', '--only', '-m', message, '-m', trailer, '--', ...res.paths]);
if (c.error) return { ...res, exit: 2, reason: `git commit did not finish: ${c.error.message}` };
if (c.status !== 0) return { ...res, exit: 1, reason: `git commit failed: ${(c.stderr || c.stdout).trim()}` };
return { ...res, exit: 0, committed: true, commit: git(repo, ['rev-parse', 'HEAD']).stdout.trim() };
@ -98,6 +129,7 @@ if (isMain(import.meta.url)) {
} else {
r = commitArtifacts({
repo: realpathSync(top.stdout.trim()), store: opt('--store'), artifacts, message: opt('--message') || DEFAULT_MESSAGE,
trailer: opt('--trailer'),
});
}
const { exit, ...rest } = r;

125
lib/validators/plan-validator.mjs
View file

@ -32,14 +32,30 @@
// /trekexecute reads; any other opening is a warning.
// - A step whose `Files:` are all tests (plan-schema `stepTestsOnly`) is the
// warning PLAN_STEP_TESTS_ONLY: its tests pass at its start point, so it can
// never be proven red (voyage-lekerepo trial 3, 2026-10-09).
// never be proven red (voyage-lekerepo trial 3, 2026-10-09). The check reads
// `Files:` by the plan's version (the 1.9 grammar from 1.9 on).
//
// plan_version 1.9 rules (decision 16, operator 2026-10-11: an unticked path in
// `Files:` is refused only from 1.9; older plans are read as before):
// - every `Files:` entry opens with its backticked path(s); the rest of the
// entry is description (plan-schema `entryPaths`). An entry that opens with
// anything else, `none`, and an empty or absent `Files:` are
// PLAN_STEP_PATH_NOT_FIRST; a leading span that is no path (`parse()`, an
// option, a URL) is PLAN_STEP_PATH_NOT_A_PATH. Errors under strict,
// warnings under --soft (checkPathGrammar). Each `Test first:` `File:`
// line (a line that opens with `File:`, backticked or not) is one entry
// under the same grammar and raises the same two codes.
// - below 1.9, a step whose `Files:` reader (plan-schema `fieldPaths`) finds a
// path outside backticks gets the warning PLAN_STEP_PATH_UNTICKED, under
// --strict too; the plan is read and gated as before.
import { readFileSync, existsSync } from 'node:fs';
import {
sliceSteps, validatePlanHeadings, extractPlanVersion, extractPlanCeremony,
parseStepFields, findSection, splitSections, findModuleCycle, parseMermaid, fencedBlocks,
versionAtLeast, PLAN_VERSION_CURRENT, PLAN_VERSIONS_SUPPORTED, CEREMONY_VALUES,
ON_FAILURE_ACTIONS, onFailureAction, stepTestsOnlyFromFields,
versionAtLeast, usesFilesGrammar, PLAN_VERSION_CURRENT, PLAN_VERSIONS_SUPPORTED, CEREMONY_VALUES,
ON_FAILURE_ACTIONS, onFailureAction, stepTestsOnlyFromFields, entryPaths, untickedPaths,
testFirstFileEntries, testFirstFiles,
} from '../parsers/plan-schema.mjs';
import { validateAllManifests } from '../parsers/manifest-yaml.mjs';
import { parseDocument } from '../util/frontmatter.mjs';
@ -54,18 +70,35 @@ export const DESIGN_SUBSECTIONS = Object.freeze([
]);
const BACKTICKED = /`[^`\n]+`/;
function checkSteps(sections) {
// Each step's fields are parsed once, here; every check reads `s.fields`
// (review finding 93382464).
function withFields(sections) {
return sections.map((s) => ({ ...s, fields: parseStepFields(s.body) }));
}
// From 1.9 the step names its test the way the executor reads it
// (testFirstFiles): only through a line that opens with `File:` and its
// backticked path, so a `File:` mid-line names no test. Below 1.9 the
// unanchored test stands (decision 16: older plans are read as before).
function checkSteps(sections, planVersion) {
const out = [];
let endToEnd = 0;
const grammar = usesFilesGrammar(planVersion);
for (const s of sections) {
const f = parseStepFields(s.body);
const f = s.fields;
const tf = f.get('Test first');
const names = tf && /\bFile:\**[ \t]*`[^`\n]+`/i.test(tf.text) && /\bVerifies:\**[ \t]*\S/i.test(tf.text);
const file = grammar
? testFirstFiles(f, planVersion).length > 0
: /\bFile:\**[ \t]*`[^`\n]+`/i.test(tf?.text ?? '');
const names = tf && file && /\bVerifies:\**[ \t]*\S/i.test(tf.text);
if (!names) {
out.push(issue(
'PLAN_STEP_NO_TEST_FIRST',
`Step ${s.n}: ${tf ? '`Test first:` does not name a File (backticked path) and what it Verifies' : 'no `Test first:` field'}`,
'plan_version 1.8 requires `Test first:` on every step, with `File:` and `Verifies:` sub-bullets.',
grammar
? 'plan_version 1.9 requires `Test first:` on every step, with a line that opens with `File:` and its ' +
'backticked path, and a `Verifies:` line.'
: 'plan_version 1.8 requires `Test first:` on every step, with `File:` and `Verifies:` sub-bullets.',
));
}
for (const name of OPTIONAL_STEP_FIELDS) {
@ -111,7 +144,7 @@ function checkRunnable(text, sections, opts) {
}
const verified = new Set(criteria.filter(c => !refusal(c)).map(c => c.command.trim().replace(/\s+/g, ' ')));
for (const s of sections) {
const f = parseStepFields(s.body);
const f = s.fields;
if (f.has('Verify')) {
const v = firstCommand(f.get('Verify').text);
const r = runnableCheck(v, opts);
@ -146,11 +179,11 @@ function checkRunnable(text, sections, opts) {
* checkpoint" in prose that nothing enforced (voyage-lekerepo trial 2,
* 2026-10-08). `Regression:` is that command, checked like `Verify:`.
*/
function checkRecovery(sections, opts) {
function checkRecovery(sections, opts, planVersion) {
const errors = [];
const warnings = [];
for (const s of sections) {
const f = parseStepFields(s.body);
const f = s.fields;
if (f.has('Regression')) {
const r = runnableCheck(firstCommand(f.get('Regression').text), opts);
if (!r.runnable) {
@ -165,7 +198,7 @@ function checkRecovery(sections, opts) {
`Step ${s.n}: \`On failure:\` does not open with a recovery word the executor reads`,
`Open with one of ${ON_FAILURE_ACTIONS.join(' | ')}, then the specifics (\`retry once, then escalate\`).`));
}
if (stepTestsOnlyFromFields(f).testsOnly) {
if (stepTestsOnlyFromFields(f, planVersion).testsOnly) {
// The run-the-thing script alone is such a step too: `End-to-end:` rides on
// the last step that builds behaviour (release review of 6.3.0, 2026-10-10).
const e2e = f.has('End-to-end')
@ -184,6 +217,50 @@ function checkRecovery(sections, opts) {
return { errors, warnings };
}
/**
* plan_version 1.9: every `Files:` entry opens with its backticked path(s)
* (plan-schema `entryPaths`). An absent `Files:` is read as empty text, so it
* names no file. Each `Test first:` `File:` line (plan-schema
* `testFirstFileEntries`) is one entry under the same grammar.
*/
// How checkPathGrammar words an issue in each of the two places the grammar applies.
const FILES_WORDING = {
subject: 'a `Files:` entry', none: '`Files:` names no file', spans: 'an entry are its files',
opens: 'each `Files:` entry opens with its backticked path(s)',
example: '`` `src/a.mjs` - the parser ``', tail: ' A step that changes no file names none.',
};
const TEST_FIRST_WORDING = {
subject: 'a `Test first:` `File:` line', none: 'a `Test first:` `File:` line names no file',
spans: 'the `File:` line are the test files', opens: 'the `File:` line opens with its backticked test file(s)',
example: '`` File: `tests/a.test.mjs` (new) ``', tail: '',
};
function pathGrammarIssue(n, code, shown, w) {
if (code === 'PLAN_STEP_PATH_NOT_A_PATH') {
return issue(code, `Step ${n}: ${w.subject} opens with a backticked span that is no path: ${shown}`,
`plan_version 1.9: the leading backticked spans of ${w.spans}; a call (\`parse()\`), an option ` +
'or a URL is description and goes after the path. A placeholder (`none`, `TBD`, `N/A`) names no ' +
'file, and neither does a mixed-case bare name (`Parser`): write `./Name` for such a file.');
}
return issue('PLAN_STEP_PATH_NOT_FIRST',
shown === '' ? `Step ${n}: ${w.none}` : `Step ${n}: ${w.subject} does not open with a backticked path: ${shown}`,
`plan_version 1.9: ${w.opens}, separated by commas or spaces; the description follows (${w.example}).${w.tail}`);
}
function checkPathGrammar(sections) {
const out = [];
for (const s of sections) {
const f = s.fields;
for (const i of entryPaths(f.has('Files') ? f.get('Files').text : '').issues) {
out.push(pathGrammarIssue(s.n, i.code, i.entry, FILES_WORDING));
}
for (const entry of testFirstFileEntries(f)) {
for (const i of entryPaths(entry).issues) out.push(pathGrammarIssue(s.n, i.code, entry, TEST_FIRST_WORDING));
}
}
return out;
}
/** Table body rows (skips the header and the |---| separator). */
function tableRows(text) {
const rows = text.split('\n').filter(l => l.trim().startsWith('|'));
@ -270,7 +347,7 @@ export function validatePlanContent(rawText, opts = {}) {
const warnings = [...headRes.warnings];
const steps = headRes.parsed?.steps || [];
const sections = sliceSteps(text);
const sections = withFields(sliceSteps(text));
const manRes = validateAllManifests(sections);
errors.push(...manRes.errors);
warnings.push(...manRes.warnings);
@ -289,11 +366,29 @@ export function validatePlanContent(rawText, opts = {}) {
warnings.push(issue('PLAN_VERSION_MISMATCH', `plan_version=${planVersion}, current target is ${PLAN_VERSION_CURRENT}`));
}
const recovery = checkRecovery(sections, { cwd: opts.cwd });
const recovery = checkRecovery(sections, { cwd: opts.cwd }, planVersion);
if (strict) errors.push(...recovery.errors);
else warnings.push(...recovery.errors);
warnings.push(...recovery.warnings);
// The `Files:` grammar (decision 16): refused from 1.9, a warning below it.
if (usesFilesGrammar(planVersion)) {
const grammar = checkPathGrammar(sections);
if (strict) errors.push(...grammar);
else warnings.push(...grammar);
} else {
for (const s of sections) {
const f = s.fields;
const unticked = f.has('Files') ? [...new Set(untickedPaths(f.get('Files').text))] : [];
if (unticked.length > 0) {
warnings.push(issue('PLAN_STEP_PATH_UNTICKED',
`Step ${s.n}: \`Files:\` names ${unticked.join(', ')} outside backticks; read as before`,
'From plan_version 1.9 an unticked path in `Files:` is refused: each entry opens with its backticked ' +
'path(s), and the description follows.'));
}
}
}
// Steps a run appended (the review loop's remediation steps, /trekexecute
// Phase 7.2) must be runnable steps whatever the plan's version: below 1.8
// the step rules never ran, and a review step whose Verify is "re-run
@ -309,7 +404,7 @@ export function validatePlanContent(rawText, opts = {}) {
...checkRunnable(text, appended, { cwd: opts.cwd, stepsOnly: true }),
];
for (const s of appended) {
if (!parseStepFields(s.body).has('Verify')) {
if (!s.fields.has('Verify')) {
stepIssues.push(issue('PLAN_STEP_VERIFY_NOT_RUNNABLE', `Step ${s.n}: no \`Verify:\` field`,
'An appended step needs one test command: proof of red and the done line run it.'));
}
@ -320,7 +415,7 @@ export function validatePlanContent(rawText, opts = {}) {
const ceremony = extractPlanCeremony(text);
if (versionAtLeast(planVersion, '1.8')) {
const v18 = [...checkSteps(sections), ...checkRunnable(text, sections, { cwd: opts.cwd })];
const v18 = [...checkSteps(sections, planVersion), ...checkRunnable(text, sections, { cwd: opts.cwd })];
if (!ceremony.valid) {
v18.push(issue('PLAN_CEREMONY_INVALID', `ceremony=${ceremony.raw}`,
`Allowed: ${CEREMONY_VALUES.join(' | ')} (absent = architectural).`));

54
lib/verification/plan-scope-sweep.mjs
View file

@ -10,7 +10,13 @@
// changed = `git diff --name-only --no-renames <merge-base>` (committed AND
// uncommitted tracked changes) + untracked, non-ignored files
// planned = every step's `Files:` paths, its `Test first:` File, and its
// manifest `expected_paths`
// manifest `expected_paths`. From plan_version 1.9 `Files:` and the
// `Test first:` `File:` line are read by the 1.9 grammar
// (plan-schema entryPaths / testFirstFiles): the leading backticked
// paths of each entry, never a path its description mentions.
// Below 1.9 (or without a version) K6 keeps its own reader,
// unchanged (operator 2026-10-11). It never reads `Files:` through
// the tests-only reader (plan-schema fieldPaths).
// process = PROCESS_FILES (the plan itself, its project directory, Voyage's
// own planning areas) + any `--process` entries
// outside = changed − planned − process. It must be empty.
@ -32,7 +38,7 @@
import { readFileSync, existsSync, realpathSync } from 'node:fs';
import { spawnSync } from 'node:child_process';
import { resolve, relative, dirname, isAbsolute, sep } from 'node:path';
import { sliceSteps, parseStepFields } from '../parsers/plan-schema.mjs';
import { sliceSteps, parseStepFields, entryPaths, testFirstFiles, extractPlanVersion, usesFilesGrammar } from '../parsers/plan-schema.mjs';
import { parseManifest } from '../parsers/manifest-yaml.mjs';
import { isMain } from '../util/is-main.mjs';
@ -59,27 +65,35 @@ const norm = p => p.replace(/\\/g, '/').replace(/^\.\//, '');
/** Every path the plan names: Files, Test first File, manifest expected_paths. */
export function plannedPaths(planText) {
const out = new Set();
const version = extractPlanVersion(planText);
const grammar = usesFilesGrammar(version);
for (const s of sliceSteps(planText)) {
const f = parseStepFields(s.body);
const files = f.get('Files');
// v6.2.0's own reading, kept as it was (operator 2026-10-11): every
// backticked span that looks like a path, and plain words only when the
// field has none. A field that names its paths in backticks plans those,
// never a path its description mentions. Release review of 6.3.0, rounds
// 3-5: reading Files: through the tests-only check's reader
// (plan-schema.mjs fieldPaths) dropped paths v6.2.0 planned and planned
// directories a description named, so K6 has its own reader again; the
// two answer different questions (what may change; is every named file a test).
if (files) {
const ticked = pathsIn(files.text);
// Older plans write bare paths: `- Files: a.ts, b.ts`.
const bare = ticked.length ? [] : files.text.split(/[,\s]+/).filter(looksLikePath);
for (const p of [...ticked, ...bare]) out.add(norm(p));
}
const tf = f.get('Test first');
if (tf) {
for (const line of tf.text.split('\n')) {
if (/\bFile:/i.test(line)) for (const p of pathsIn(line)) out.add(norm(p));
if (grammar) {
// plan_version 1.9: the grammar decides what each entry names.
if (files) for (const p of entryPaths(files.text).files) out.add(norm(p));
for (const p of testFirstFiles(f, version)) out.add(norm(p));
} else {
// v6.2.0's own reading, kept as it was (operator 2026-10-11): every
// backticked span that looks like a path, and plain words only when the
// field has none. A field that names its paths in backticks plans those,
// never a path its description mentions. Release review of 6.3.0, rounds
// 3-5: reading Files: through the tests-only check's reader
// (plan-schema.mjs fieldPaths) dropped paths v6.2.0 planned and planned
// directories a description named, so K6 has its own reader again; the
// two answer different questions (what may change; is every named file a test).
if (files) {
const ticked = pathsIn(files.text);
// Older plans write bare paths: `- Files: a.ts, b.ts`.
const bare = ticked.length ? [] : files.text.split(/[,\s]+/).filter(looksLikePath);
for (const p of [...ticked, ...bare]) out.add(norm(p));
}
const tf = f.get('Test first');
if (tf) {
for (const line of tf.text.split('\n')) {
if (/\bFile:/i.test(line)) for (const p of pathsIn(line)) out.add(norm(p));
}
}
}
const man = parseManifest(s.body);

117
lib/verification/red-proof.mjs
View file

@ -19,10 +19,24 @@
// red and goes up to the plan. The plan's `Files:` alone decides (see
// `testsOnly` below).
//
// With `--plan <plan.md> --step <N>` the test list is also checked against the
// step's `Test first:` File line (plan-schema `testFirstFiles`: the leading
// backticked paths, never a path in the description - operator decision 20).
// No `--tests`: the plan's list is used, and that is the documented call. A
// `--tests` whose set differs from the plan's is `not-run` with reason
// `tests-mismatch` and the plan's list in `plan_tests` (review cf241526). A plan
// or step that cannot be read is `not-run` with a reason that starts
// `plan-unreadable`, whether or not `--tests` is given (operator decision 30,
// 2026-10-14). An empty plan list (a 1.7 step without `Test first:`, or below
// 1.8 a `File:` line without backticks) changes nothing here: the caller gives
// `--tests` (operator decision 41). Every result from the point the test list
// is settled on carries it in `tests` (the list it ran); the text output prints it on a ` tests:` line, and
// `plan_tests` on a ` plan_tests:` line, comma-joined as `--tests` takes it.
//
// Exit 0 = proven, 1 = rejected (`not-red-at-base` or `not-green-at-result`),
// 2 = could not run (`not-run`: no test files, an unknown start point, a command
// outside the test-runner allowlist, a command that cannot start, a control run
// that stays red). 2 is never a pass and never "proven".
// 2 = could not run (`not-run`: no test files, a plan or step that cannot be
// read, an unknown start point, a command outside the test-runner allowlist, a
// command that cannot start, a control run that stays red). 2 is never a pass and never "proven".
//
// The start point is a temp `git clone --shared` of the repo checked out at
// start_sha - a real git checkout, so a git-reading command behaves as it does
@ -61,7 +75,7 @@ import { tmpdir } from 'node:os';
import { dirname, isAbsolute, join, resolve } from 'node:path';
import { DEFAULT_TIMEOUT_MS, allowedCommand, runCommand, screenCommand, stoppedBy } from './criteria-runner.mjs';
import { commandEnv } from '../util/command-env.mjs';
import { parseStepFields, sliceSteps, stepTestsOnlyFromFields } from '../parsers/plan-schema.mjs';
import { extractPlanVersion, normPlanPath, parseStepFields, sliceSteps, stepTestsOnlyFromFields, testFirstFiles } from '../parsers/plan-schema.mjs';
import { isMain } from '../util/is-main.mjs';
// The one timeout, defined in criteria-runner.mjs (release review of 6.3.0, round 7).
@ -111,8 +125,9 @@ const notRun = (reason, extra = {}) => ({ status: 'not-run', reason, ...extra })
// PLAN_STEP_TESTS_ONLY). Such a step can never be red: the behaviour its tests
// cover already exists at its start point. Rewriting the test cannot fix that, so
// the executor sends the step up to the plan instead of into fix rounds
// (voyage-lekerepo trial 3, 2026-10-09). `null` = the plan or the step could
// not be read; without `--plan` the field is absent and nothing new happens.
// (voyage-lekerepo trial 3, 2026-10-09). Without `--plan` the field is absent
// and nothing new happens; a plan or step that cannot be read never reaches here
// (`proveRed` returns `plan-unreadable` first).
//
// The plan's `Files:` alone decides (operator 2026-10-11): the step's diff is
// not read. In the documented flow the helper writes only the test before this
@ -121,14 +136,22 @@ const notRun = (reason, extra = {}) => ({ status: 'not-run', reason, ...extra })
// made a true tests-only step look like it built something (release review of
// 6.3.0, round 7). A path the step changed outside `Files:` is the
// controller's scope finding (trekexecute Sub-step C, Hard rule 2), not this check's.
function testsOnly(o) {
if (!o.plan) return {};
function testsOnly(step) {
if (!step) return {};
return { tests_only: stepTestsOnlyFromFields(step.fields, step.version).testsOnly };
}
// With `--plan` and `--step`: the step's parsed fields and the plan's version,
// read once for both the test list and `testsOnly`. null without `--plan`;
// `{error}` when the plan or the step could not be read.
function planStep(o) {
if (!o.plan) return null;
let text;
try { text = readFileSync(o.plan, 'utf8'); }
catch (e) { return { tests_only: null, tests_only_note: `cannot read the plan: ${e.message}` }; }
catch (e) { return { error: `cannot read the plan: ${e.message}` }; }
const step = sliceSteps(text).find((s) => Number(s.n) === Number(o.step));
if (!step) return { tests_only: null, tests_only_note: `no "### Step ${o.step}:" in the plan` };
return { tests_only: stepTestsOnlyFromFields(parseStepFields(step.body)).testsOnly };
if (!step) return { error: `no "### Step ${o.step}:" in the plan` };
return { fields: parseStepFields(step.body), version: extractPlanVersion(text) };
}
// A program that could not start (a spawn error: not installed), or one this
@ -185,28 +208,54 @@ function applyResult(root, tree, base) {
}
/**
* @param {{cwd: string, base: string, cmd: string, tests: string[], link?: string[], redOnly?: boolean, timeoutMs?: number, plan?: string, step?: number}} o
* @returns {{status: 'proven'|'rejected'|'not-run', reason: string, base?: object, result?: object, control?: object}}
* With `plan` and `step`: a plan or step that cannot be read is `not-run` with a
* reason that starts `plan-unreadable`; no `tests` means the step's `Test first:`
* File line is the list; a `tests` that is another set is `tests-mismatch` (the
* plan's list in `plan_tests`); an empty plan list (a 1.7 step without
* `Test first:`) changes nothing. Every result once the list is settled on
* carries it in `tests` (the list it ran).
* @param {{cwd: string, base: string, cmd: string, tests?: string[], link?: string[], redOnly?: boolean, timeoutMs?: number, plan?: string, step?: number}} o
* @returns {{status: 'proven'|'rejected'|'not-run', reason: string, tests?: string[], plan_tests?: string[], base?: object, result?: object, control?: object}}
*/
export function proveRed(o) {
const given = resolve(o.cwd ?? process.cwd());
const timeoutMs = o.timeoutMs ?? DEFAULT_TIMEOUT_MS;
const tests = (o.tests ?? []).filter(Boolean);
let tests = (o.tests ?? []).filter(Boolean);
if (!existsSync(given)) return notRun(`--cwd ${given} does not exist`);
const cwd = repoRoot(given);
if (!cwd) return notRun(`--cwd ${given} is not inside a git repository`);
if (tests.length === 0) return notRun('no test files given: a step without its test cannot prove red');
for (const t of tests) {
if (!safeRel(t)) return notRun(`test path must be repo-relative: ${t}`);
if (!existsSync(join(cwd, t))) return notRun(`test file not found in the working tree: ${t}`);
// The step's own test list (review cf241526): the step's `Test first:` File
// line as testFirstFiles reads it (operator decision 20), taken when none is
// given; a given list that is another set is refused. A plan or step that
// cannot be read is refused with or without a given list (operator decision
// 30); an empty plan list changes nothing.
const step = planStep(o);
if (step?.error) return notRun(`plan-unreadable: ${step.error}`);
const fromPlan = step?.fields ? testFirstFiles(step.fields, step.version) : [];
if (fromPlan.length > 0) {
if (tests.length === 0) tests = fromPlan;
else {
const a = new Set(tests.map((t) => normPlanPath(String(t))));
const b = new Set(fromPlan);
if (a.size !== b.size || [...a].some((t) => !b.has(t))) {
return notRun(`tests-mismatch: --tests names ${tests.join(',')}; the plan's Test first File line names ${fromPlan.join(',')}`, { plan_tests: fromPlan });
}
}
}
for (const l of o.link ?? []) if (!safeRel(l)) return notRun(`--link must be repo-relative: ${l}`);
if (tests.length === 0) return notRun('no test files given: a step without its test cannot prove red');
// From here on every result names the list it ran (`tests`, operator decision 30).
const ran = (reason, extra = {}) => notRun(reason, { tests, ...extra });
for (const t of tests) {
if (!safeRel(t)) return ran(`test path must be repo-relative: ${t}`);
if (!existsSync(join(cwd, t))) return ran(`test file not found in the working tree: ${t}`);
}
for (const l of o.link ?? []) if (!safeRel(l)) return ran(`--link must be repo-relative: ${l}`);
const allowed = allowedCommand(o.cmd, { cwd });
if (!allowed.allowed) return notRun(`command not run: ${allowed.reason}`);
if (!allowed.allowed) return ran(`command not run: ${allowed.reason}`);
const screen = screenCommand(o.cmd);
if (!screen.allowed) return notRun(`command blocked: ${screen.rule}`);
if (!screen.allowed) return ran(`command blocked: ${screen.rule}`);
const sha = git(cwd, ['rev-parse', '--verify', '--quiet', `${o.base}^{commit}`]);
if (sha.status !== 0) return notRun(`cannot resolve the start point ${o.base} in ${cwd}`);
if (sha.status !== 0) return ran(`cannot resolve the start point ${o.base} in ${cwd}`);
const baseSha = sha.stdout.trim();
// The result first: it is the working tree as it stands. `redOnly` is the
@ -215,25 +264,25 @@ export function proveRed(o) {
if (!o.redOnly) {
result = run(o.cmd, cwd, timeoutMs);
const resultBroken = unrunnable(result);
if (resultBroken) return notRun(`at the result: ${resultBroken}`, { result });
if (resultBroken) return ran(`at the result: ${resultBroken}`, { result });
}
const tmp = mkdtempSync(join(tmpdir(), 'voyage-red-proof-'));
try {
const tree = join(tmp, 'tree');
const c = git(tmp, ['clone', '--quiet', '--shared', '--no-checkout', cwd, tree]);
if (c.status !== 0) return notRun(`could not clone the start point: ${String(c.stderr).trim()}`, { result });
if (c.status !== 0) return ran(`could not clone the start point: ${String(c.stderr).trim()}`, { result });
const co = git(tree, ['checkout', '--quiet', '--detach', baseSha]);
if (co.status !== 0) return notRun(`could not check out ${baseSha}: ${String(co.stderr).trim()}`, { result });
if (co.status !== 0) return ran(`could not check out ${baseSha}: ${String(co.stderr).trim()}`, { result });
for (const t of tests) copyIn(cwd, tree, t);
const linked = linkDependencies(cwd, tree, o.link ?? []);
const base = run(o.cmd, tree, timeoutMs);
const baseBroken = unrunnable(base);
if (baseBroken) return notRun(`at the start point: ${baseBroken}`, { base, result, linked });
const out = { base_sha: baseSha, linked, base, result };
if (base.exit === 0) return { status: 'rejected', reason: 'not-red-at-base', ...testsOnly(o), ...out };
if (o.redOnly) return { status: 'proven', reason: 'red at the start point (--red-only: the result was not run)', red_kind: redKind(base), base_sha: baseSha, linked, base };
if (baseBroken) return ran(`at the start point: ${baseBroken}`, { base, result, linked });
const out = { tests, base_sha: baseSha, linked, base, result };
if (base.exit === 0) return { status: 'rejected', reason: 'not-red-at-base', ...testsOnly(step), ...out };
if (o.redOnly) return { status: 'proven', reason: 'red at the start point (--red-only: the result was not run)', red_kind: redKind(base), tests, base_sha: baseSha, linked, base };
if (result.exit !== 0) return { status: 'rejected', reason: 'not-green-at-result', ...out };
// Control: the same copy with the whole change laid on. Red here means the
@ -241,9 +290,9 @@ export function proveRed(o) {
applyResult(cwd, tree, baseSha);
const control = run(o.cmd, tree, timeoutMs);
const controlBroken = unrunnable(control);
if (controlBroken) return notRun(`in the control run: ${controlBroken}`, { ...out, control });
if (controlBroken) return ran(`in the control run: ${controlBroken}`, { ...out, control });
if (control.exit !== 0) {
return notRun('the control run (start-point copy + the change) stayed red: the start point failed for an environment reason, not for the missing change', { ...out, control });
return ran('the control run (start-point copy + the change) stayed red: the start point failed for an environment reason, not for the missing change', { ...out, control });
}
return { status: 'proven', reason: 'red at the start point, green at the result, green in the control', red_kind: redKind(base), ...out, control };
} finally {
@ -254,7 +303,8 @@ export function proveRed(o) {
// --- CLI ------------------------------------------------------------------------
const USAGE =
'usage: red-proof.mjs --base <start_sha> --cmd "<test command>" --tests <a,b> [--plan <plan.md> --step <N>] [--red-only] [--cwd <dir>] [--link <dir,...>] [--timeout <ms>] [--json]';
'usage: red-proof.mjs --base <start_sha> --cmd "<test command>" [--tests <a,b>] [--plan <plan.md> --step <N>] [--red-only] [--cwd <dir>] [--link <dir,...>] [--timeout <ms>] [--json]\n' +
' with --plan/--step the step\'s Test first File line gives the test list and the documented call passes no --tests; a --tests given with them must name the same set; a plan or step that cannot be read is not-run (plan-unreadable). Without --plan, or for a step whose Test first File line lists no test, --tests is required';
const list = (v) => String(v).split(',').map((s) => s.trim()).filter(Boolean);
@ -286,6 +336,9 @@ export function main(argv) {
else {
const counts = (x) => (x?.tap ? ` (tap: ${x.tap.fail} of ${x.tap.tests} failed)` : '');
process.stdout.write(`red proof: ${r.status.toUpperCase()} - ${r.reason}${r.red_kind ? ` (red: ${r.red_kind})` : ''}\n`);
// The exact form --files and --tests take: one comma-joined list.
if (r.tests) process.stdout.write(` tests: ${r.tests.join(',')}\n`);
if (r.plan_tests) process.stdout.write(` plan_tests: ${r.plan_tests.join(',')}\n`);
if (r.tests_only === true) process.stdout.write(' tests_only: true - every file in the step\'s Files: is a test, so it can never be red: up to the plan\n');
if (r.base) process.stdout.write(` start point ${r.base_sha ?? o.base}: exit ${r.base.exit}${counts(r.base)}\n`);
if (r.result) process.stdout.write(` result: exit ${r.result.exit}${counts(r.result)}\n`);

2
package.json
View file

@ -8,7 +8,7 @@
"node": ">=18"
},
"scripts": {
"test": "node --test 'tests/**/*.test.mjs'",
"test": "node --import ./tests/helpers/no-session-id.mjs --test 'tests/**/*.test.mjs'",
"annotate": "node scripts/annotate.mjs",
"verify": "bash verify.sh"
},

15
scripts/bakeoff-armA-merge.mjs
View file

@ -10,9 +10,10 @@
// for each reviewer raw-output file:
// validateReviewerOutput() (NW1 — extract last json fence, parse, schema-check)
// collect findings from VALID reviewers
// triplet-dedup by (file,line,rule_key) [SAME logic as Arm B's dedupByTriplet,
// so the two arms dedup identically —
// triplet-only, no jaccard pass-2]
// triplet-dedup by (file,line,rule_key) [same key as Arm B's dedupByTriplet,
// first-wins here; Arm B ranks by tier
// and keeps the reviewer finding on a
// tie — triplet-only, no jaccard pass-2]
// → emit { merged, validation, raw_finding_count, deduped_count }
//
// The merged findings are then handed to the review-coordinator (spawned by main),
@ -28,9 +29,11 @@ import { readFileSync, existsSync } from 'node:fs';
import { validateReviewerOutput } from '../lib/review/findings-schema.mjs';
import { isMain } from '../lib/util/is-main.mjs';
// Triplet dedup — byte-for-byte the same key + first-wins policy as
// scripts/trekreview-armB.workflow.mjs `dedupByTriplet`, so neither arm gets a
// dedup advantage. Triplet-only (file,line,rule_key); NO jaccard pass-2.
// Triplet dedup — the same (file,line,rule_key) key as
// scripts/trekreview-armB.workflow.mjs `dedupByTriplet`, but not the same
// policy: Arm A keeps the first-seen finding, while Arm B ranks by catalogue
// tier and keeps the reviewer finding on a tie (review bc7f4e23). Triplet-only;
// NO jaccard pass-2.
export function dedupByTriplet(findings) {
const seen = new Map();
for (const f of findings) {

192
scripts/trekreview-armB.workflow.mjs
View file

@ -21,7 +21,9 @@
// Ordre 5: the Claude Code engine payloads (Phase 4.7, run in prose before this
// script) are merged with the reviewers' findings; a reviewer or engine that
// returned nothing forbids ALLOW (fail-closed, as on the prose path); and no
// finding is struck for length.
// finding is struck for length. A coordinator that returns no verdict and
// findings counts as a missing reviewer, and every reviewer, engine and
// coordinator that did not deliver is named in `allow_blocked_by` (review edf2ec56).
//
// Run via the Workflow tool: Workflow({ scriptPath: this file, args }).
// The structured result is logged as a single JSON line (prefix RESULT_JSON:)
@ -36,13 +38,24 @@ export const meta = {
],
};
const RULE_KEYS = [
'MISSING_BRIEF_REF', 'UNIMPLEMENTED_CRITERION', 'SCOPE_CREEP_BUILT', 'NON_GOAL_VIOLATED',
'MISSING_TEST', 'SECURITY_INJECTION', 'PLACEHOLDER_IN_CODE', 'MISSING_ERROR_HANDLING',
'UNDECLARED_DEPENDENCY', 'PLAN_EXECUTE_DRIFT', 'BROKEN_SUCCESS_CRITERION', 'COVERAGE_SILENT_SKIP',
'CODE_REVIEW_DEFECT', 'CODE_REVIEW_IMPROVEMENT', 'SECURITY_REVIEW_VULNERABILITY',
];
const SEVERITIES = ['BLOCKER', 'MAJOR', 'MINOR', 'SUGGESTION'];
// Every catalogue rule_key -> its tier (lib/review/rule-catalogue.mjs; scripts
// cannot import, so it is inlined and pinned equal by the workflow test).
const RULE_TIERS = {
MISSING_BRIEF_REF: 'MAJOR', UNIMPLEMENTED_CRITERION: 'BLOCKER', SCOPE_CREEP_BUILT: 'MAJOR',
NON_GOAL_VIOLATED: 'BLOCKER', MISSING_TEST: 'MAJOR', SECURITY_INJECTION: 'BLOCKER',
PLACEHOLDER_IN_CODE: 'MAJOR', MISSING_ERROR_HANDLING: 'MINOR', UNDECLARED_DEPENDENCY: 'MAJOR',
PLAN_EXECUTE_DRIFT: 'MAJOR', BROKEN_SUCCESS_CRITERION: 'BLOCKER', COVERAGE_SILENT_SKIP: 'MAJOR',
CODE_REVIEW_DEFECT: 'MAJOR', CODE_REVIEW_IMPROVEMENT: 'MINOR', SECURITY_REVIEW_VULNERABILITY: 'BLOCKER',
};
const RULE_KEYS = Object.keys(RULE_TIERS);
// Lower number = higher severity; an unknown severity ranks last (as
// lib/review/coordinator-contract.mjs severityRank does).
const sevRank = (s) => (SEVERITIES.includes(s) ? SEVERITIES.indexOf(s) : SEVERITIES.length);
// The catalogue tier of a rule_key, or null when the key is missing or unknown.
const tierOf = (key) => (typeof key === 'string' && Object.prototype.hasOwnProperty.call(RULE_TIERS, key) ? RULE_TIERS[key] : null);
// The (file,line,rule_key) triplet a finding is deduped and matched on.
const tripletOf = (f) => `${f.file}\u0000${f.line}\u0000${f.rule_key}`;
// Findings schema — mirrors lib/review/findings-schema.mjs (NW1). The enum on
// rule_key enforces the catalogue at the StructuredOutput layer (stronger than
@ -94,6 +107,9 @@ const VERDICT_SCHEMA = {
line: { type: 'integer', minimum: 0 },
title: { type: 'string' },
detail: { type: 'string' },
engine: { type: 'string', enum: ['code-review', 'security-review'] },
contradicts_brief: { type: 'string' },
brief_lowered_from: { type: 'string', enum: ['MAJOR'] },
},
},
},
@ -102,14 +118,47 @@ const VERDICT_SCHEMA = {
// Inlined dedup by (file,line,rule_key) — the JS-side replacement for the
// coordinator's pass-1 triplet dedup (T2 §5 / lib/review/plan-review-dedup.mjs
// pattern; scripts cannot import, so the triplet key is inlined).
// pattern; scripts cannot import, so the triplet key is inlined). The survivor
// order is coordinator-contract Pass 1's (review bc7f4e23): the catalogue tier
// first (for a catalogue key it always ties inside one triplet, since the
// rule_key is shared), then a reviewer finding (no `engine`) over an engine
// finding (decision 13, operator 12.10), then the higher asserted severity;
// the workflow has no conformance rule, so otherwise the first-seen one stays.
// The survivor carries the highest severity any finding in its group asserted,
// so an engine's mark never lowers what a reviewer agent asserted.
function dedupByTriplet(findings) {
const seen = new Map();
const groups = new Map();
for (const f of findings) {
const key = `${f.file}\u0000${f.line}\u0000${f.rule_key}`;
if (!seen.has(key)) seen.set(key, f);
const key = tripletOf(f);
if (!groups.has(key)) groups.set(key, []);
groups.get(key).push(f);
}
return [...seen.values()];
const tier = (f) => sevRank(tierOf(f.rule_key) ?? f.severity);
const out = [];
for (const group of groups.values()) {
let survivor = group[0];
for (const f of group.slice(1)) {
const tf = tier(f);
const ts = tier(survivor);
if (tf !== ts) {
if (tf < ts) survivor = f;
continue;
}
const ef = Boolean(f.engine);
const es = Boolean(survivor.engine);
if (ef !== es) {
if (!ef) survivor = f;
continue;
}
if (sevRank(f.severity) < sevRank(survivor.severity)) survivor = f;
}
let severity = survivor.severity;
for (const f of group) {
if (sevRank(f.severity) < sevRank(severity)) severity = f.severity;
}
out.push(severity === survivor.severity ? survivor : { ...survivor, severity });
}
return out;
}
function reviewerPrompt(kind, a) {
@ -137,6 +186,16 @@ function coordinatorPrompt(merged, a) {
'Return {verdict, findings} via StructuredOutput. Synthesis across files is forbidden.',
`The brief (the contract) is at: ${a.briefPath} — Read it if needed for context.`,
'',
'=== BRIEF CHOICES (the brief-conflict rule) ===',
`Read the brief's choices at ${a.briefPath}: \`## Non-Goals\`, \`## Constraints\`, \`## Preferences\`,`,
'and the operator decisions in `## Open Questions / Assumptions`.',
'Only an engine finding (`engine`: code-review or security-review) whose recommended action',
'contradicts one of those choices is marked, with `contradicts_brief: "<the Non-Goal or choice, verbatim>"`.',
'Only a /code-review finding at the MAJOR tier and at severity MAJOR then drops to MINOR, with',
"`brief_lowered_from: 'MAJOR'`. Every other marked finding keeps its severity. A reviewer-agent finding",
'(no `engine`) and a finding whose rule_key is not in the catalogue are never marked. A brief-conflicting',
'finding is kept and marked, never removed. Each returned finding keeps its `engine` and `rule_key`.',
'',
'=== PRE-DEDUPED FINDINGS (JSON) ===',
JSON.stringify(merged, null, 2),
].join('\n');
@ -160,9 +219,12 @@ const reviewerThunks = [
agentType: 'voyage:code-correctness-reviewer', schema: FINDINGS_SCHEMA, label: 'correctness', phase: 'Review',
}),
];
// The reviewer agent names in run order, so a reviewer that returned nothing is named.
const reviewerNames = a.quick ? ['code-correctness-reviewer'] : ['brief-conformance-reviewer', 'code-correctness-reviewer'];
const toRun = a.quick ? [reviewerThunks[1]] : reviewerThunks;
const rawResults = await parallel(toRun);
const reviewerResults = rawResults.filter(Boolean);
const missingReviewerNames = reviewerNames.filter((_, i) => !rawResults[i]);
// A reviewer that returned nothing is an absent one, not a clean one. The
// engines that were due are counted from what actually arrived, never from
// what the caller remembered to list as not run.
@ -170,9 +232,26 @@ const enginePayloads = Array.isArray(a.enginePayloads) ? a.enginePayloads.filter
const expectedEngines = a.quick ? ['claude-code-review'] : ['claude-code-review', 'claude-security-review'];
const arrivedEngines = new Set(enginePayloads.map((p) => p.reviewer));
const missingEngines = expectedEngines.filter((name) => !arrivedEngines.has(name));
const missingReviewers = (rawResults.length - reviewerResults.length) + missingEngines.length;
const allFindings = [...reviewerResults, ...enginePayloads]
.flatMap((r) => (Array.isArray(r?.findings) ? r.findings : []));
let missingReviewers = (rawResults.length - reviewerResults.length) + missingEngines.length;
// Provenance, before dedup: "came from an engine" is read from the source,
// never from an `engine` field. A reviewer finding loses any `engine` it
// carries; an engine payload's findings get `engine` from its `reviewer`
// (the names lib/review/engine-findings.mjs writes); any other payload gives
// its findings no `engine`. So an `engine` in `merged` means an engine payload.
// Inlined (scripts cannot import) and pinned equal to ENGINES by the workflow test.
const ENGINE_OF_PAYLOAD = {
'claude-code-review': 'code-review', 'claude-security-review': 'security-review',
};
const engineOf = (reviewer) => (Object.prototype.hasOwnProperty.call(ENGINE_OF_PAYLOAD, reviewer) ? ENGINE_OF_PAYLOAD[reviewer] : null);
const withEngine = (f, eng) => {
if (!f || typeof f !== 'object') return f;
const { engine: _dropped, ...rest } = f;
return eng ? { ...rest, engine: eng } : rest;
};
const allFindings = [
...reviewerResults.flatMap((r) => (Array.isArray(r?.findings) ? r.findings : []).map((f) => withEngine(f, null))),
...enginePayloads.flatMap((p) => (Array.isArray(p?.findings) ? p.findings : []).map((f) => withEngine(f, engineOf(p?.reviewer)))),
];
const merged = dedupByTriplet(allFindings);
log(`Arm B: ${reviewerResults.length} reviewer(s); ${allFindings.length} raw → ${merged.length} after triplet-dedup`);
@ -186,20 +265,99 @@ if (merged.length === 0) {
agentType: 'voyage:review-coordinator', schema: VERDICT_SCHEMA, label: 'coordinator', phase: 'Coordinate',
});
}
// A coordinator that returns no verdict and findings is a missing reviewer, not
// a clean one (the zero-findings skip above always sets a valid verdict).
const coordinatorMissing = merged.length > 0 && !(coord && ['BLOCK', 'WARN', 'ALLOW'].includes(coord.verdict) && Array.isArray(coord.findings));
if (coordinatorMissing) missingReviewers += 1;
// The verdict is mechanical: recompute its floor from the findings the
// coordinator returned, so an LLM-written ALLOW over a BLOCKER cannot pass.
const returned = Array.isArray(coord?.findings) ? coord.findings : merged;
const returned = coordinatorMissing ? merged : coord.findings;
// The floor reads two inputs, never the labels the coordinator wrote:
// - `merged` (the coordinator's input, same file/line/rule_key) for each
// returned finding's counterpart: (a) an engine counterpart raises it to the
// engine's severity, unless the one legal lowering; (b) a reviewer
// counterpart, labelled or not, raises it to the lower of the counterpart's
// asserted severity and its catalogue tier (the coordinator may reset a
// finding to its tier, never below; for an unknown key the bound is the
// counterpart's severity), and a labelled finding is raised as before.
// - `allFindings` (before dedup, so an engine finding a reviewer finding
// shadowed in its triplet counts too) for the put-back: (c) every engine
// finding with no returned finding at its exact triplet is appended at its
// own severity, one copy per triplet (the higher severity, else the first
// seen) (operator decision 23, review ab1b5d9f). So the coordinator's drop,
// move or relabel of an engine finding no longer lowers the verdict.
// The raises and put-backs are written into `returned`, the marks kept, so the
// rendered review.md carries them.
// A coordinator that failed (no verdict and findings) is not trusted with a
// partial list: `returned` is then every merged finding, each first raised to
// its catalogue tier when the key is known (never lowered), as the default
// path's Pass 3 would reset it; that raise stands after `raiseTo` is declared.
const counterparts = new Map(merged
.filter((m) => m && typeof m === 'object')
.map((m) => [tripletOf(m), m]));
const present = (f, field) => f[field] !== undefined && f[field] !== null;
const raiseTo = (f, sev) => { if (sevRank(sev) < sevRank(f.severity)) f.severity = sev; };
const lowerOf = (s1, s2) => (sevRank(s1) >= sevRank(s2) ? s1 : s2);
if (coordinatorMissing) {
for (const f of returned) {
if (!f || typeof f !== 'object') continue;
const tier = tierOf(f.rule_key);
if (tier !== null) raiseTo(f, tier);
}
}
for (const f of returned) {
if (!f || typeof f !== 'object') continue;
const c = counterparts.get(tripletOf(f));
const labelled = present(f, 'engine') || present(f, 'contradicts_brief') || present(f, 'brief_lowered_from');
if (c && c.engine) {
// (a) an engine counterpart: raised to its severity, unless the one legal lowering.
const legal = c.engine === 'code-review' && c.severity === 'MAJOR' && tierOf(c.rule_key) === 'MAJOR' &&
f.severity === 'MINOR' && typeof f.contradicts_brief === 'string' && f.contradicts_brief.trim().length > 0 &&
f.brief_lowered_from === 'MAJOR';
if (!legal) raiseTo(f, c.severity);
f.engine = c.engine;
continue;
}
if (c) {
// (b) a reviewer counterpart: never below the lower of its asserted severity and the tier.
const tier = tierOf(c.rule_key);
raiseTo(f, tier === null ? c.severity : lowerOf(c.severity, tier));
}
if (labelled) {
// A labelled reviewer finding, or one with no counterpart: an unknown key raises to BLOCKER.
raiseTo(f, tierOf(f.rule_key) ?? 'BLOCKER');
if (c) raiseTo(f, c.severity);
delete f.engine;
}
}
// (c) the put-back (decision 23).
const returnedTriplets = new Set(returned.filter((f) => f && typeof f === 'object').map(tripletOf));
// Engine findings only, so dedupByTriplet keeps the higher severity, else the first seen.
const putBack = dedupByTriplet(allFindings.filter((f) =>
f && typeof f === 'object' && f.engine && !returnedTriplets.has(tripletOf(f))));
for (const f of putBack) {
const { brief_lowered_from: _lowered, ...copy } = f;
returned.push(copy);
}
const has = (sev) => returned.some((f) => f && f.severity === sev);
let verdict = coord?.verdict ?? null;
let verdict = coordinatorMissing ? (coord?.verdict === 'BLOCK' ? 'BLOCK' : 'WARN') : coord.verdict;
if (has('BLOCKER')) verdict = 'BLOCK';
else if (has('MAJOR') && verdict !== 'BLOCK') verdict = 'WARN';
if (verdict === 'ALLOW' && missingReviewers > 0) verdict = 'WARN';
// Every reviewer, engine and coordinator that did not deliver, in the form the
// default path writes and lib/review/verdict-gate.mjs counts as not run.
const allowBlockedBy = [
...missingReviewerNames.map((name) => `missing-reviewer:${name}`),
...missingEngines.map((name) => `missing-reviewer:${name}`),
...(coordinatorMissing ? ['missing-reviewer:review-coordinator'] : []),
];
const result = {
verdict,
missing_reviewers: missingReviewers,
missing_engines: missingEngines,
allow_blocked_by: allowBlockedBy,
findings: returned,
raw_reviewer_count: reviewerResults.length,
raw_finding_count: allFindings.length,

5
templates/design-template.md
View file

@ -1,5 +1,5 @@
<!--
The design part of a plan (plan_version 1.8). /trekplan Phase 8 writes it as
The design part of a plan (plan_version 1.8 or later). /trekplan Phase 8 writes it as
the plan's `## Design` section, between `## Context` and
`## Implementation Plan`, ONLY when the brief's `ceremony:` is
`architectural` (an absent field means architectural: heaviest on doubt).
@ -8,7 +8,8 @@ the plan's `## Design` section, between `## Context` and
The design lives in the PLAN, never in the brief: the brief format is a public
contract (Handover 1), the plan contract is Voyage's own (Handover 4).
`plan-validator --strict` checks, for an architectural 1.8 plan: the eight
`plan-validator --strict` checks, for an architectural plan at plan_version
1.8 or later: the eight
`###` subsections below exist, the module map is a mermaid graph with no
cycle, every architecture rule names its enforcer, a checked rule names the
step that proves it, and an advisory rule names its `.claude/rules/` file.

25
templates/plan-template.md
View file

@ -18,7 +18,7 @@ source_findings:
> **Plan quality: {grade}** ({score}/100) — {APPROVE | APPROVE_WITH_NOTES | REVISE | REPLAN}
>
> Generated by trekplan v{version} on {YYYY-MM-DD} — `plan_version: 1.8` · `ceremony: {small | bounded | architectural}`
> Generated by trekplan v{version} on {YYYY-MM-DD} — `plan_version: 1.9` · `ceremony: {small | bounded | architectural}`
## Context
@ -71,7 +71,7 @@ their relationships, and the data flow between them.*
Each step targets 1–2 files and one focused change, and is a vertical slice:
a narrow but complete path through every layer it touches, verifiable on its
own. Preparatory refactoring comes first, as its own steps. Every step has
`Test first:` (plan_version 1.8 requires it), and one step runs the thing
`Test first:` (plan_version 1.8 or later requires it), and one step runs the thing
end to end (`End-to-end:`). The test for a behaviour belongs in the step that
builds it: a step of only tests for behaviour an earlier step delivered passes
at its start point and can never be proven red (the validator warns:
@ -89,6 +89,23 @@ runs green before the checkpoint, on a step that can break existing tests;
never empty. `On failure:` opens with `revert`, `retry`, `skip` or `escalate`,
then the specifics.
From plan_version 1.9 the rule is the file first on every line: each `Files:`
entry and each `Test first:` `File:` line opens with its backticked path(s),
separated only by commas or spaces, and everything after the first other text
is description that is never read. A 1.9 step always names a file
(the bare word none or an empty `Files:` is refused). A line that is no list item
continues the entry above it (a list wrapped after a comma keeps its paths;
once the entry so far does not end with a comma, the wrapped line is description;
a `File:` line with nothing after it takes its first wrapped line),
the `File:` line may open with `1.` or `1)` and any list item ends it, and
a backticked placeholder (`TBD`, `TODO`, `TBA`, `N/A`, `NA`, `none`, `...`) or a
mixed-case bare name (`Parser`) is no path, so write `./Name` for such a file. In `Files:`,
every path stands before any annotation: `` `src/a.mjs`, `src/b.mjs` (new) ``,
never `` `src/a.mjs` (new), `src/b.mjs` `` (in 1.9 that second path is
description).
Several test files share one `File:` line the same way:
`` File: `a.test.mjs`, `b.test.mjs` *(new)* ``.
### Step 1: {description}
- **Files:** `path/to/file.ts`
@ -146,8 +163,8 @@ then the specifics.
*A project without tests still gets `Test first:` on every step: the first step
adds the test runner (or a script under `tests/`) the later steps use. A plan
without an `End-to-end:` step is invalid at plan_version 1.8 — without it the
result can only be "not verified end to end", never green.*
without an `End-to-end:` step is invalid at plan_version 1.8 or later — without
it the result can only be "not verified end to end", never green.*
### Manifest — objective completion predicate

3
tests/commands/trekexecute-parallel-portability.test.mjs
View file

@ -25,6 +25,7 @@ import { tmpdir } from 'node:os';
import { realpathSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
@ -137,7 +138,7 @@ function makeRepo(opts = {}) {
}
function runBlock(script, { cwd, env }) {
return spawnSync('bash', ['-c', script], { cwd, env: { ...process.env, ...env }, encoding: 'utf8' });
return spawnSync('bash', ['-c', script], { cwd, env: childEnv(env), encoding: 'utf8' });
}
function copyBlockEnv(repo, pathPrefixDir) {

99
tests/commands/trekplan-explorer-prompts.test.mjs Normal file
View file

@ -0,0 +1,99 @@
// tests/commands/trekplan-explorer-prompts.test.mjs
// Prose pins: every agent /trekplan sends to explore or review is told to read
// local files only inside the repo under exploration (plus the paths its prompt
// gives it), and every exploring agent gets the brief path.
// Blocks are found by a named list; a block that cannot be found fails, so an
// empty scan is never green.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { findSection } from '../../lib/parsers/plan-schema.mjs';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const PLAN_CMD = readFileSync(join(ROOT, 'commands', 'trekplan.md'), 'utf8');
const STAY_LINE =
'Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names).';
/** The text of a level-2 section, up to the next `## ` heading outside a fence. */
function section(heading) {
const s = findSection(PLAN_CMD, heading.replace(/^## /, ''), 2);
assert.ok(s, `missing ${heading}`);
return s.text;
}
/** Collapse line wraps so a sentence wrapped across lines still matches. */
const flat = (s) => s.replace(/\s+/g, ' ');
/**
* The quoted prompt that follows `marker` inside `text`: from `Prompt: "` to
* the next closing `"`. Fails when the marker or the prompt is missing, or
* when the first `Prompt: "` after the marker sits in a later paragraph.
*/
function promptAfter(text, marker, label) {
const at = text.indexOf(marker);
assert.ok(at !== -1, `${label}: marker ${JSON.stringify(marker)} not found`);
const open = text.indexOf('Prompt: "', at);
assert.ok(open !== -1, `${label}: no Prompt: after the marker`);
assert.ok(!text.slice(at, open).includes('\n\n'),
`${label}: the next Prompt: lies in another paragraph, not in this block`);
const start = open + 'Prompt: "'.length;
const close = text.indexOf('"', start);
assert.ok(close !== -1, `${label}: prompt never closes`);
return text.slice(start, close);
}
const P4B = section('## Phase 4b — Brief review');
const P5 = section('## Phase 5 — Parallel exploration (specialized agents + research)');
const P6 = section('## Phase 6 — Targeted deep-dives');
const P9 = section('## Phase 9 — Adversarial review (bounded critic loop)');
function phase6Rule() {
const start = P6.indexOf('spawn a targeted deep-dive agent');
const end = P6.indexOf('Launch up to 3 deep-dive agents');
assert.ok(start !== -1 && end > start, 'Phase 6 deep-dive rule not found');
return P6.slice(start, end);
}
// [label, block text getter, carries {brief_path}?]
const BLOCKS = [
['architecture-mapper', () => promptAfter(P5, '**architecture-mapper**', 'architecture-mapper'), true],
['dependency-tracer', () => promptAfter(P5, '**dependency-tracer**', 'dependency-tracer'), true],
['risk-assessor', () => promptAfter(P5, '**risk-assessor**', 'risk-assessor'), true],
['task-finder', () => promptAfter(P5, '**task-finder**', 'task-finder'), true],
['test-strategist', () => promptAfter(P5, '**test-strategist**', 'test-strategist'), true],
['git-historian', () => promptAfter(P5, '**git-historian**', 'git-historian'), true],
['convention-scanner', () => promptAfter(P5, '**Convention Scanner**', 'convention-scanner'), true],
['research-scout', () => promptAfter(P5, 'launch **research-scout**', 'research-scout'), true],
['Phase 6 deep-dive rule', phase6Rule, true],
['brief-reviewer', () => promptAfter(P4B, '**brief-reviewer**', 'brief-reviewer'), false],
['plan-critic', () => promptAfter(P9, '**plan-critic**', 'plan-critic'), false],
['scope-guardian', () => promptAfter(P9, '**scope-guardian**', 'scope-guardian'), false],
];
test('all 12 named blocks are found (an empty scan fails)', () => {
assert.equal(BLOCKS.length, 12);
let found = 0;
for (const [label, get] of BLOCKS) {
const text = get();
assert.ok(text.trim().length > 0, `${label}: empty block`);
found += 1;
}
assert.equal(found, 12);
});
for (const [label, get] of BLOCKS) {
test(`${label} carries the stay-in-repo line`, () => {
assert.ok(flat(get()).includes(STAY_LINE), `${label} does not carry: ${STAY_LINE}`);
});
}
for (const [label, get, needsBrief] of BLOCKS) {
if (!needsBrief) continue;
test(`${label} is given the brief path`, () => {
assert.ok(get().includes('{brief_path}'), `${label} is not given {brief_path}`);
});
}

286
tests/e2e-files-trial4.sh Normal file
View file

@ -0,0 +1,286 @@
#!/usr/bin/env bash
# tests/e2e-files-trial4.sh
# End to end for the trial-4 plan (steps 11 and 13): the four CLIs run the
# way a command runs them.
# 1. lib/validators/plan-validator.mjs --strict --json on three plans built
# with tests/helpers/plan-v18-builder.mjs: a valid 1.9 plan (exit 0), a
# 1.9 plan with an unticked Files entry (exit 1, PLAN_STEP_PATH_NOT_FIRST),
# a 1.8 plan with an unticked path (exit 0, PLAN_STEP_PATH_UNTICKED).
# 2. lib/util/artifact-commit.mjs in a temp git repo: with a trailer (exit 0,
# the trailer is the message's last line), without (exit 1, HEAD unchanged).
# 3. lib/review/verdict-gate.mjs on the legal (exit 0) and the illegal
# (exit 2) brief-conflict review fixtures.
# 4. plan-validator on a 1.8 plan whose Test first File line names a source
# path in its description (exit 0, no PLAN_STEP_TESTS_ONLY: decision 20).
# 5. plan-validator on a 1.9 plan whose Files is a backticked None (exit 1,
# PLAN_STEP_PATH_NOT_A_PATH).
# 6. lib/verification/red-proof.mjs with --plan/--step and a --tests that
# differs from the plan's list (exit 2, tests-mismatch).
# 7. artifact-commit with an empty-model trailer (exit 1, HEAD unchanged).
# 8. plan-validator on a 1.9 plan whose Files entry wraps onto a line that
# opens with a backticked flag (exit 0, no PLAN_STEP_PATH_NOT_A_PATH).
# 9. plan-validator on a 1.9 plan whose Test first File line wraps onto a
# line that opens with the source path (exit 0, no PLAN_STEP_TESTS_ONLY).
# 10. red-proof --red-only with --plan/--step and no --tests in a second temp
# repo (exit 0; its JSON tests field is the plan's one test file).
# 11. the same with a plan path that does not exist (exit 2, plan-unreadable).
# 12. the same as 10 in text form with a --tests that differs (exit 2, the
# line " plan_tests: tests/parser.test.mjs").
# 13. red-proof --red-only on a 1.8 plan whose File line names two tests,
# each with a (new) note (exit 0; tests is both, in order; decision 31).
# 14. red-proof --red-only on a 1.7 plan whose File line names its test
# without backticks: no --tests (exit 2, no test files given, no tests
# line), then --tests with the file the helper wrote (exit 0, the line
# " tests: tests/parser.test.mjs"; decision 41).
# Prints "e2e: ok" and exits 0, or names the failed check and exits 1.
# bash 3.2 compatible, ASCII only.
set -u
cd "$(dirname "$0")/.." || { echo "e2e: FAIL - cannot cd to the repo root" >&2; exit 1; }
ROOT="$(pwd)"
unset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE
TMP="$(mktemp -d "${TMPDIR:-/tmp}/e2e-files-trial4.XXXXXX")" || { echo "e2e: FAIL - mktemp" >&2; exit 1; }
trap 'rm -rf "$TMP"' EXIT
fail() {
echo "e2e: FAIL - $1" >&2
exit 1
}
TRAILER='Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>'
# ---- 1. the plan validator on three plans ----
node --input-type=module -e '
import { writeFileSync } from "node:fs";
import { buildPlan } from "./tests/helpers/plan-v18-builder.mjs";
const dir = process.argv[1];
const e2e = { endToEnd: "bash tests/e2e.sh" };
writeFileSync(dir + "/plan-1-9-valid.md", buildPlan({ version: "1.9" }));
writeFileSync(dir + "/plan-1-9-unticked.md",
buildPlan({ version: "1.9", steps: [{ filesRaw: "src/mod1.mjs - the parser" }, e2e] }));
writeFileSync(dir + "/plan-1-8-unticked.md",
buildPlan({ steps: [{ filesRaw: "`tests/mod1.test.mjs`, src/mod1.mjs" }, e2e] }));
' "$TMP" || fail "could not write the three plans with tests/helpers/plan-v18-builder.mjs"
out="$(node lib/validators/plan-validator.mjs --strict --json "$TMP/plan-1-9-valid.md" 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || fail "valid 1.9 plan: plan-validator exit $rc, expected 0: $out"
out="$(node lib/validators/plan-validator.mjs --strict --json "$TMP/plan-1-9-unticked.md" 2>&1)"
rc=$?
[ "$rc" -eq 1 ] || fail "1.9 plan with an unticked Files entry: plan-validator exit $rc, expected 1: $out"
echo "$out" | grep -q 'PLAN_STEP_PATH_NOT_FIRST' \
|| fail "1.9 plan with an unticked Files entry: no PLAN_STEP_PATH_NOT_FIRST in: $out"
out="$(node lib/validators/plan-validator.mjs --strict --json "$TMP/plan-1-8-unticked.md" 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || fail "1.8 plan with an unticked path: plan-validator exit $rc, expected 0: $out"
echo "$out" | grep -q 'PLAN_STEP_PATH_UNTICKED' \
|| fail "1.8 plan with an unticked path: no PLAN_STEP_PATH_UNTICKED in: $out"
# ---- 2. the approval commit in a temp git repo ----
# make_repo <dir>: a temp git repo with one commit, no hooks, no signing.
make_repo() {
mkdir -p "$1" "$TMP/hooks-none" || fail "mkdir $1"
git -C "$1" init -q || fail "git init $1"
git -C "$1" config user.name 'E2E Test'
git -C "$1" config user.email 'test@example.invalid'
git -C "$1" config commit.gpgsign false
git -C "$1" config core.hooksPath "$TMP/hooks-none"
echo 'x' > "$1/README.md"
git -C "$1" add README.md && git -C "$1" commit -q -m 'init' || fail "initial commit in $1"
}
REPO="$TMP/repo"
make_repo "$REPO"
echo '# brief' > "$REPO/brief.md"
H1='cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc'
H2='dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd'
(cd "$REPO" && node "$ROOT/lib/util/approval-log.mjs" append --kind intent --subject brief.md --sha256 "$H1" >/dev/null) \
|| fail "approval-log append (first record)"
[ -f "$REPO/.claude/voyage/approvals.jsonl" ] || fail "no .claude/voyage/approvals.jsonl in the temp repo"
out="$(node lib/util/artifact-commit.mjs --repo "$REPO" --store local --message 'docs(voyage): approve intent e2e' --trailer "$TRAILER" 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || fail "artifact-commit with a trailer: exit $rc, expected 0: $out"
last="$(git -C "$REPO" log -1 --format=%B | sed '/^$/d' | tail -n 1)"
[ "$last" = "$TRAILER" ] || fail "artifact-commit with a trailer: the last message line is '$last', expected '$TRAILER'"
(cd "$REPO" && node "$ROOT/lib/util/approval-log.mjs" append --kind intent --subject brief.md --sha256 "$H2" >/dev/null) \
|| fail "approval-log append (second record)"
before="$(git -C "$REPO" rev-parse HEAD)"
out="$(node lib/util/artifact-commit.mjs --repo "$REPO" --store local --message 'docs(voyage): approve intent e2e' 2>&1)"
rc=$?
[ "$rc" -eq 1 ] || fail "artifact-commit without a trailer: exit $rc, expected 1: $out"
after="$(git -C "$REPO" rev-parse HEAD)"
[ "$before" = "$after" ] || fail "artifact-commit without a trailer: HEAD moved from $before to $after"
# ---- 3. the verdict gate on the two brief-conflict fixtures ----
out="$(node lib/review/verdict-gate.mjs tests/fixtures/review-brief-conflict-legal.md 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || fail "verdict-gate on review-brief-conflict-legal.md: exit $rc, expected 0: $out"
out="$(node lib/review/verdict-gate.mjs tests/fixtures/review-brief-conflict-illegal.md 2>&1)"
rc=$?
[ "$rc" -eq 2 ] || fail "verdict-gate on review-brief-conflict-illegal.md: exit $rc, expected 2: $out"
# ---- 4. and 5. the plan validator on decision 20 and on a placeholder ----
node --input-type=module -e '
import { writeFileSync } from "node:fs";
import { buildPlan } from "./tests/helpers/plan-v18-builder.mjs";
const dir = process.argv[1];
const e2e = { endToEnd: "bash tests/e2e.sh" };
writeFileSync(dir + "/plan-1-8-description.md", buildPlan({ steps: [{
file: "src/parser.mjs", test: "tests/parser.test.mjs",
filesRaw: "`src/parser.mjs`, `tests/parser.test.mjs`",
testFirstRaw: " - File: `tests/parser.test.mjs` (new) - covers `parse()` in `src/parser.mjs`",
}, e2e] }));
writeFileSync(dir + "/plan-1-9-none.md",
buildPlan({ version: "1.9", steps: [{ filesRaw: "`None`" }, e2e] }));
' "$TMP" || fail "could not write the plans for checks 4 and 5"
out="$(node lib/validators/plan-validator.mjs --strict --json "$TMP/plan-1-8-description.md" 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || fail "1.8 plan with a source path in the File line description: plan-validator exit $rc, expected 0: $out"
if echo "$out" | grep -q 'PLAN_STEP_TESTS_ONLY'; then
fail "1.8 plan with a source path in the File line description: PLAN_STEP_TESTS_ONLY in: $out"
fi
out="$(node lib/validators/plan-validator.mjs --strict --json "$TMP/plan-1-9-none.md" 2>&1)"
rc=$?
[ "$rc" -eq 1 ] || fail "1.9 plan whose Files is a backticked None: plan-validator exit $rc, expected 1: $out"
echo "$out" | grep -q 'PLAN_STEP_PATH_NOT_A_PATH' \
|| fail "1.9 plan whose Files is a backticked None: no PLAN_STEP_PATH_NOT_A_PATH in: $out"
# ---- 6. red proof refuses a --tests that differs from the plan ----
out="$(node lib/verification/red-proof.mjs --red-only --base HEAD --cwd "$REPO" --cmd 'node --test tests/parser.test.mjs' --tests tests/other.test.mjs --plan "$TMP/plan-1-8-description.md" --step 1 --json 2>&1)"
rc=$?
[ "$rc" -eq 2 ] || fail "red-proof with a --tests that differs from the plan: exit $rc, expected 2: $out"
echo "$out" | grep -q 'tests-mismatch' \
|| fail "red-proof with a --tests that differs from the plan: no tests-mismatch in: $out"
# ---- 7. the approval commit refuses an empty-model trailer ----
before="$(git -C "$REPO" rev-parse HEAD)"
out="$(node lib/util/artifact-commit.mjs --repo "$REPO" --store local --message 'docs(voyage): approve intent e2e' --trailer 'Co-Authored-By: Claude <noreply@anthropic.com>' 2>&1)"
rc=$?
[ "$rc" -eq 1 ] || fail "artifact-commit with an empty-model trailer: exit $rc, expected 1: $out"
after="$(git -C "$REPO" rev-parse HEAD)"
[ "$before" = "$after" ] || fail "artifact-commit with an empty-model trailer: HEAD moved from $before to $after"
# ---- 8. to 13. decisions 29 to 31: wrapped 1.9 lines, red proof without --tests ----
node --input-type=module -e '
import { writeFileSync } from "node:fs";
import { buildPlan } from "./tests/helpers/plan-v18-builder.mjs";
const dir = process.argv[1];
const e2e = { endToEnd: "bash tests/e2e.sh" };
writeFileSync(dir + "/p8.md", buildPlan({ version: "1.9", steps: [{
filesRaw: "`src/a.mjs`\n `--strict` flag added",
}, e2e] }));
writeFileSync(dir + "/p9.md", buildPlan({ version: "1.9", steps: [{
file: "src/parser.mjs", test: "tests/parser.test.mjs",
filesRaw: "`src/parser.mjs`, `tests/parser.test.mjs`",
testFirstRaw: " - File: `tests/parser.test.mjs`\n `src/parser.mjs` is the module it drives",
}, e2e] }));
writeFileSync(dir + "/p13.md", buildPlan({ steps: [{
file: "src/parser.mjs",
filesRaw: "`src/parser.mjs`, `tests/a.test.mjs`, `tests/b.test.mjs`",
testFirstRaw: " - File: `tests/a.test.mjs` (new), `tests/b.test.mjs` (new)",
}, e2e] }));
writeFileSync(dir + "/p14.md", buildPlan({ version: "1.7", steps: [{
file: "src/parser.mjs",
filesRaw: "`src/parser.mjs`, `tests/parser.test.mjs`",
testFirstRaw: " - File: tests/parser.test.mjs (new)",
}, e2e] }));
' "$TMP" || fail "could not write the plans for checks 8 to 14"
out="$(node lib/validators/plan-validator.mjs --strict --json "$TMP/p8.md" 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || fail "8: 1.9 plan with a wrapped Files entry: plan-validator exit $rc, expected 0: $out"
if echo "$out" | grep -q 'PLAN_STEP_PATH_NOT_A_PATH'; then
fail "8: 1.9 plan with a wrapped Files entry: PLAN_STEP_PATH_NOT_A_PATH in: $out"
fi
out="$(node lib/validators/plan-validator.mjs --strict --json "$TMP/p9.md" 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || fail "9: 1.9 plan with a wrapped Test first File line: plan-validator exit $rc, expected 0: $out"
if echo "$out" | grep -q 'PLAN_STEP_TESTS_ONLY'; then
fail "9: 1.9 plan with a wrapped Test first File line: PLAN_STEP_TESTS_ONLY in: $out"
fi
R="$TMP/repo-red"
make_repo "$R"
mkdir -p "$R/src" "$R/tests" || fail "mkdir $R"
echo 'export const parse = (s) => Number(s);' > "$R/src/parser.mjs"
for t in parser a b; do
cat > "$R/tests/$t.test.mjs" <<'EOT'
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { parse } from '../src/parser.mjs';
test('parse reads a number', () => { assert.equal(parse('2'), 2); });
EOT
done
# tests_equal <json file> <expected JSON array>: exit 0 when the JSON's tests deep-equals it.
tests_equal() {
node -e '
const assert = require("node:assert");
const fs = require("node:fs");
const r = JSON.parse(fs.readFileSync(process.argv[1], "utf8"));
try { assert.deepStrictEqual(r.tests, JSON.parse(process.argv[2])); } catch (e) { process.exit(1); }
' "$1" "$2"
}
node lib/verification/red-proof.mjs --red-only --cwd "$R" --base HEAD --cmd 'node --test tests/parser.test.mjs' --plan "$TMP/p9.md" --step 1 --json >"$TMP/rp10.json" 2>"$TMP/rp10.err"
rc=$?
[ "$rc" -eq 0 ] || fail "10: red-proof with --plan/--step and no --tests: exit $rc, expected 0: $(cat "$TMP/rp10.json" "$TMP/rp10.err")"
tests_equal "$TMP/rp10.json" '["tests/parser.test.mjs"]' \
|| fail "10: red-proof tests is not [\"tests/parser.test.mjs\"]: $(cat "$TMP/rp10.json")"
out="$(node lib/verification/red-proof.mjs --red-only --cwd "$R" --base HEAD --cmd 'node --test tests/parser.test.mjs' --plan "$TMP/nope.md" --step 1 --json 2>&1)"
rc=$?
[ "$rc" -eq 2 ] || fail "11: red-proof with a plan that does not exist: exit $rc, expected 2: $out"
echo "$out" | grep -q 'plan-unreadable' \
|| fail "11: red-proof with a plan that does not exist: no plan-unreadable in: $out"
out="$(node lib/verification/red-proof.mjs --red-only --cwd "$R" --base HEAD --cmd 'node --test tests/parser.test.mjs' --tests tests/a.test.mjs --plan "$TMP/p9.md" --step 1 2>&1)"
rc=$?
[ "$rc" -eq 2 ] || fail "12: red-proof with a --tests that differs (text): exit $rc, expected 2: $out"
echo "$out" | grep -Fqx ' plan_tests: tests/parser.test.mjs' \
|| fail "12: red-proof with a --tests that differs (text): no line ' plan_tests: tests/parser.test.mjs' in: $out"
node lib/verification/red-proof.mjs --red-only --cwd "$R" --base HEAD --cmd 'node --test tests/a.test.mjs tests/b.test.mjs' --plan "$TMP/p13.md" --step 1 --json >"$TMP/rp13.json" 2>"$TMP/rp13.err"
rc=$?
[ "$rc" -eq 0 ] || fail "13: red-proof on a 1.8 plan with two (new) tests: exit $rc, expected 0: $(cat "$TMP/rp13.json" "$TMP/rp13.err")"
tests_equal "$TMP/rp13.json" '["tests/a.test.mjs","tests/b.test.mjs"]' \
|| fail "13: red-proof tests is not the two test paths in order: $(cat "$TMP/rp13.json")"
# ---- 14. decision 41: a 1.7 plan that names no test file, then --tests ----
out="$(node lib/verification/red-proof.mjs --red-only --cwd "$R" --base HEAD --cmd 'node --test tests/parser.test.mjs' --plan "$TMP/p14.md" --step 1 2>&1)"
rc=$?
[ "$rc" -eq 2 ] || fail "14: red-proof on a 1.7 plan that names no test file: exit $rc, expected 2: $out"
echo "$out" | grep -q 'no test files given' \
|| fail "14: red-proof on a 1.7 plan that names no test file: no 'no test files given' in: $out"
if echo "$out" | grep -Eq '^ tests: [^ ]'; then
fail "14: red-proof on a 1.7 plan that names no test file: a tests line in: $out"
fi
out="$(node lib/verification/red-proof.mjs --red-only --cwd "$R" --base HEAD --cmd 'node --test tests/parser.test.mjs' --plan "$TMP/p14.md" --step 1 --tests tests/parser.test.mjs 2>&1)"
rc=$?
[ "$rc" -eq 0 ] || fail "14: red-proof on a 1.7 plan with --tests the helper wrote: exit $rc, expected 0: $out"
echo "$out" | grep -Fqx ' tests: tests/parser.test.mjs' \
|| fail "14: red-proof on a 1.7 plan with --tests the helper wrote: no line ' tests: tests/parser.test.mjs' in: $out"
echo 'e2e: ok'
exit 0

6
tests/fixtures/jsonl-schemas.md vendored
View file

@ -19,13 +19,13 @@
| schema_id | fields | writer_path | line_ref | v4.1 additive | PII |
|-----------|--------|-------------|----------|---------------|-----|
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:1130-1145 | profile, phase_models, profile_source | none |
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:1137-1161 | profile, phase_models, profile_source | none |
| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | lib/stats/stats-append.mjs (trekresearch.md Stats tracking) | trekresearch.md:782-808 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | lib/stats/stats-append.mjs (trekplan.md Phase 12) | trekplan.md:1115-1135 | profile, phase_models, parallel_agents, profile_source | none |
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | lib/stats/stats-append.mjs (trekplan.md Phase 12) | trekplan.md:1234-1262 | profile, phase_models, parallel_agents, profile_source | none |
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2892-2914 | profile, phase_models, profile_source | none |
| trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:78-99 | payload.profile, payload.phase_models, payload.profile_source | none |
| trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | lib/stats/stats-append.mjs (trekreview.md Phase 8) | trekreview.md:628-632 | profile, phase_models, profile_source | none |
| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | lib/stats/stats-append.mjs (trekreview.md Phase 8) | trekreview.md:623-673 | profile, phase_models, profile_source | none |
| trekcontinue-stats | ts, project, next_session_label, status | lib/stats/stats-append.mjs (trekcontinue.md Phase 5) | trekcontinue.md:293-297 | profile, profile_source | none |
| token-usage | ts, session_id, scope, model, tokens_input, tokens_output, tokens_cache_creation, tokens_cache_read, cost_usd, is_estimate, price_table_version | lib/stats/token-usage.mjs `captureTokenUsage` (via hooks/scripts/otel-export.mjs Stop hook, opt-in `VOYAGE_TOKEN_METER`) | token-usage.mjs:buildRecord | n/a (SKAL-2 schema) | session_id (stripped at export) |

55
tests/fixtures/review-brief-conflict-illegal.md vendored Normal file
View file

@ -0,0 +1,55 @@
---
type: trekreview
review_version: "1.1"
task: "t"
slug: s
project_dir: .claude/projects/2026-10-06-s/
brief_path: .claude/projects/2026-10-06-s/brief.md
scope_sha_end: abc
reviewed_files_count: 1
verdict: ALLOW
findings:
- 8530de651325215c87cc8ba418c42ba3e8a66ed2
---
# Review: t
## Executive Summary
One reviewer-agent finding (no engine) carries a brief-conflict mark at MINOR: only an engine finding is ever marked, so the gate refuses it.
## Coverage
| File | Treatment | Reason |
|------|-----------|--------|
| lib/b.mjs | summary-only | default |
## Remediation Summary
x
```json
{
"verdict": "ALLOW",
"counts": {
"BLOCKER": 0,
"MAJOR": 0,
"MINOR": 1,
"SUGGESTION": 0
},
"allow_blocked_by": [],
"findings": [
{
"id": "8530de651325215c87cc8ba418c42ba3e8a66ed2",
"severity": "MINOR",
"rule_key": "MISSING_TEST",
"file": "lib/b.mjs",
"line": 7,
"title": "No test for the retry path",
"detail": "The retry branch has no test.",
"recommended_action": "Add a test for the retry branch.",
"contradicts_brief": "No retry layer around the HTTP client (Non-Goal 2)"
}
]
}
```

57
tests/fixtures/review-brief-conflict-legal.md vendored Normal file
View file

@ -0,0 +1,57 @@
---
type: trekreview
review_version: "1.1"
task: "t"
slug: s
project_dir: .claude/projects/2026-10-06-s/
brief_path: .claude/projects/2026-10-06-s/brief.md
scope_sha_end: abc
reviewed_files_count: 1
verdict: ALLOW
findings:
- d3c22f4b1309d6001a0f3948e377a8e80a4b9c55
---
# Review: t
## Executive Summary
One /code-review finding contradicts a brief Non-Goal; it is kept, marked and lowered from MAJOR to MINOR (the one legal shape).
## Coverage
| File | Treatment | Reason |
|------|-----------|--------|
| lib/a.mjs | summary-only | default |
## Remediation Summary
x
```json
{
"verdict": "ALLOW",
"counts": {
"BLOCKER": 0,
"MAJOR": 0,
"MINOR": 1,
"SUGGESTION": 0
},
"allow_blocked_by": [],
"findings": [
{
"id": "d3c22f4b1309d6001a0f3948e377a8e80a4b9c55",
"severity": "MINOR",
"rule_key": "CODE_REVIEW_DEFECT",
"file": "lib/a.mjs",
"line": 12,
"title": "No retry on transient failure",
"detail": "A 503 fails the call.",
"recommended_action": "Wrap the call in a retry with backoff.",
"engine": "code-review",
"contradicts_brief": "No retry layer around the HTTP client (Non-Goal 2)",
"brief_lowered_from": "MAJOR"
}
]
}
```

41
tests/helpers/artifact-repo.mjs Normal file
View file

@ -0,0 +1,41 @@
// tests/helpers/artifact-repo.mjs
// A throwaway target repo for the artifact-commit tests
// (tests/lib/artifact-commit.test.mjs, tests/lib/artifact-commit-trailer.test.mjs):
// one commit, no git hooks, a project folder holding brief.md and, unless
// `withLog` is false, an approval-log entry for it.
import { spawnSync } from 'node:child_process';
import { mkdirSync, mkdtempSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { childEnv } from './child-env.mjs';
const LOG = join(dirname(fileURLToPath(import.meta.url)), '..', '..', 'lib', 'util', 'approval-log.mjs');
const H1 = 'c'.repeat(64);
export function git(cwd, ...args) {
return spawnSync('git', args, { cwd, encoding: 'utf8', env: childEnv() });
}
/** @returns {{ d: string, brief: string }} the repo and its brief.md */
export function artifactRepo({ prefix = 'artifact-commit-', project = '2026-10-06-toggle', withLog = true } = {}) {
const d = mkdtempSync(join(tmpdir(), prefix));
git(d, 'init', '-q');
git(d, 'config', 'user.name', 'Test');
git(d, 'config', 'user.email', 'test@example.invalid');
git(d, 'config', 'commit.gpgsign', 'false');
mkdirSync(join(d, '.hooks-none'));
git(d, 'config', 'core.hooksPath', '.hooks-none');
writeFileSync(join(d, 'README.md'), 'x\n');
git(d, 'add', 'README.md');
git(d, 'commit', '-q', '-m', 'init');
const proj = join(d, '.claude', 'projects', project);
mkdirSync(proj, { recursive: true });
writeFileSync(join(proj, 'brief.md'), '# brief\n');
if (withLog) {
spawnSync(process.execPath, [LOG, 'append', '--kind', 'intent', '--subject', join(proj, 'brief.md'), '--sha256', H1],
{ cwd: d, env: childEnv() });
}
return { d, brief: join(proj, 'brief.md') };
}

20
tests/helpers/child-env.mjs Normal file
View file

@ -0,0 +1,20 @@
// tests/helpers/child-env.mjs
// The environment a test gives a child process. The lock CLI
// (lib/util/test-lock.mjs) writes a session pointer keyed by
// CLAUDE_CODE_SESSION_ID; a child that inherits the id of the session running
// the tests writes a pointer for that live session (S133). So the id never
// reaches a child unless the test sets one itself. Same shape as
// lib/util/command-env.mjs. tests/lib/session-id-isolation.test.mjs checks that
// no other file under tests/ hands the process environment to a child.
/**
* A copy of process.env without CLAUDE_CODE_SESSION_ID, with `overrides`
* applied last (a test that sets its own id keeps it).
*
* @param {Record<string, string|undefined>} [overrides] - vars to set or replace
* @returns {Record<string, string|undefined>} the child's environment
*/
export function childEnv(overrides = {}) {
const { CLAUDE_CODE_SESSION_ID, ...rest } = process.env;
return { ...rest, ...overrides };
}

3
tests/helpers/hook-helper.mjs
View file

@ -6,6 +6,7 @@
// Provenance: borrowed within the same marketplace (same author, MIT).
import { execFile } from 'node:child_process';
import { childEnv } from './child-env.mjs';
// How long a hook may run before the helper kills it. The tests check a
// hook's answer, not its speed (SC #14 times itself, tests/helpers/timing.mjs),
@ -37,7 +38,7 @@ export function runHook(scriptPath, input) {
*/
export function runHookWithEnv(scriptPath, input, envOverrides, opts = {}) {
return new Promise((resolve) => {
const env = { ...process.env, ...envOverrides };
const env = childEnv(envOverrides);
const child = execFile(
'node',
[scriptPath],

6
tests/helpers/no-session-id.mjs Normal file
View file

@ -0,0 +1,6 @@
// tests/helpers/no-session-id.mjs
// Preloaded by `npm test` (node --import). The lock CLI (lib/util/test-lock.mjs)
// writes a session pointer keyed by CLAUDE_CODE_SESSION_ID, and the stop guard
// reads it; a suite run inside a Claude Code session must not write one for
// that live session (S133). The id is removed before any test file starts.
delete process.env.CLAUDE_CODE_SESSION_ID;

10
tests/helpers/plan-v18-builder.mjs
View file

@ -2,6 +2,12 @@
// Builds plan_version 1.8 plans for the plan-contract tests (K3 gate, K4 step
// fields, the design part, the scope sweep). One builder, many variants, so a
// test states only the difference it is about.
//
// Raw options on step() write a field the generated form cannot (the
// plan_version 1.9 grammar tests): `filesRaw` is the `Files:` value written
// verbatim after `- **Files:** `; `testFirstRaw` is the whole `Test first:`
// `File:` line written verbatim in place of ` - File: `<test>` (new)`. Without
// them the output is unchanged.
function manifest(paths) {
return [
@ -25,7 +31,7 @@ export function step(n, opts = {}) {
const lines = [
`### Step ${n}: ${opts.title ?? `Build module ${n}`}`,
'',
`- **Files:** \`${file}\``,
opts.filesRaw !== undefined ? `- **Files:** ${opts.filesRaw}` : `- **Files:** \`${file}\``,
`- **Changes:** Add \`run${n}()\` to \`${file}\`. (new file)`,
];
if (opts.interfaceIn !== undefined) lines.push(`- **Interface in:** ${opts.interfaceIn}`);
@ -34,7 +40,7 @@ export function step(n, opts = {}) {
if (opts.testFirst !== false) {
lines.push(
'- **Test first:**',
` - File: \`${test}\` (new)`,
opts.testFirstRaw !== undefined ? opts.testFirstRaw : ` - File: \`${test}\` (new)`,
` - Verifies: run${n}() returns the value the next step reads`,
' - Pattern: `tests/lib/plan-schema.test.mjs`',
);

12
tests/helpers/timing.mjs
View file

@ -23,3 +23,15 @@ export async function fastestWithin(limitMs, measure, tries = 3) {
const fastest = Math.min(...runs);
return { ok: fastest < limitMs, fastest, runs };
}
// The red-proof control-run test (review 630 r9, brief note 3) reads these.
// Red proof gives its three runs one timeout, and the test needs the result and
// start-point runs to finish under it on a loaded machine: the criteria runner
// saw the start point killed at 2 s under full-suite load (review 703d87e1).
// The timeout is raised on purpose and is not a loosened assertion: the test
// checks which run red proof stopped, not how fast a run is. `slowStartMs`
// simulates a slow cold start, above the old 2 s; `hangMs` outlasts the timeout
// by 30 s with a real timer. This is the raise of the test's own threshold
// (2000 -> 15000 ms, 6000 -> 45000 ms) that the floor guard may report as
// loosened-threshold: plan step 1 of round 3 asks for it.
export const CONTROL_RUN = Object.freeze({ timeoutMs: 15_000, hangMs: 45_000, slowStartMs: 2_500 });

5
tests/hooks/guards-review-regressions.test.mjs
View file

@ -10,6 +10,7 @@ import { writeFileSync, mkdirSync, realpathSync } from 'node:fs';
import { join } from 'node:path';
import { runHook, runHookWithEnv } from '../helpers/hook-helper.mjs';
import { ROOT, makeRepo, gitIn, fakeTool, tmp, pluginCopy, readStubLog, parseJsonOut } from './guard-helpers.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const PRE_BASH = join(ROOT, 'hooks', 'scripts', 'pre-bash-executor.mjs');
const PRE_WRITE = join(ROOT, 'hooks', 'scripts', 'pre-write-executor.mjs');
@ -177,7 +178,7 @@ test('review 5: a guard module that cannot load makes the Bash guard ask, never
});
function lockIn(repo, rel, env = {}) {
const r = spawnSync(process.execPath, [LOCK, 'lock', '--files', rel, '--step', '1'], { cwd: repo, encoding: 'utf8', env: { ...process.env, ...env } });
const r = spawnSync(process.execPath, [LOCK, 'lock', '--files', rel, '--step', '1'], { cwd: repo, encoding: 'utf8', env: childEnv(env) });
assert.equal(r.status, 0, r.stderr);
}
@ -216,7 +217,7 @@ test('review 10: the stop guard finds the lock taken by this session from anothe
// 4, 11, 12. The step check
function runStep(repo) {
const r = spawnSync(process.execPath, [STEP, '--cwd', repo], { encoding: 'utf8', env: { ...process.env, PATH: '/usr/bin:/bin' } });
const r = spawnSync(process.execPath, [STEP, '--cwd', repo], { encoding: 'utf8', env: childEnv({ PATH: '/usr/bin:/bin' }) });
return { code: r.status, json: JSON.parse(r.stdout) };
}
function jsRepo() {

3
tests/hooks/post-compact-flush.test.mjs
View file

@ -12,6 +12,7 @@ import { tmpdir } from 'node:os';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { execFile } from 'node:child_process';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
@ -22,7 +23,7 @@ function runHookIn(cwd, input = {}) {
const child = execFile(
'node',
[HOOK],
{ timeout: 5000, cwd, env: { ...process.env } },
{ timeout: 5000, cwd, env: childEnv() },
(err, stdout, stderr) => {
resolve({
code: child.exitCode ?? 0,

3
tests/hooks/pre-compact-flush.test.mjs
View file

@ -12,6 +12,7 @@ import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'nod
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const HOOK = join(HERE, '..', '..', 'hooks', 'scripts', 'pre-compact-flush.mjs');
@ -43,7 +44,7 @@ function project(checkpoint, { stateStatus = 'in_progress' } = {}) {
}
function run(repo) {
return spawnSync(process.execPath, [HOOK], { input: '{}', encoding: 'utf8', env: { ...process.env, CLAUDE_PROJECT_DIR: repo } });
return spawnSync(process.execPath, [HOOK], { input: '{}', encoding: 'utf8', env: childEnv({ CLAUDE_PROJECT_DIR: repo }) });
}
const progress = (dir) => JSON.parse(readFileSync(join(dir, 'progress.json'), 'utf8'));

3
tests/hooks/test-lock-green.test.mjs
View file

@ -18,6 +18,7 @@ import { join } from 'node:path';
import { runHook } from '../helpers/hook-helper.mjs';
import { ROOT, makeRepo, parseJsonOut } from './guard-helpers.mjs';
import { pathWith } from '../helpers/bin-path.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const CLI = join(ROOT, 'lib', 'util', 'test-lock.mjs');
const STOP = join(ROOT, 'hooks', 'scripts', 'stop-test-lock.mjs');
@ -26,7 +27,7 @@ const SESS = `sess-green-${process.pid}-${Date.now()}`;
const CMD = 'node --test tests/step.test.mjs';
function cli(repo, args, env = {}) {
const r = spawnSync(process.execPath, [CLI, ...args], { cwd: repo, encoding: 'utf8', env: { ...process.env, CLAUDE_CODE_SESSION_ID: SESS, ...env } });
const r = spawnSync(process.execPath, [CLI, ...args], { cwd: repo, encoding: 'utf8', env: childEnv({ CLAUDE_CODE_SESSION_ID: SESS, ...env }) });
return { code: r.status, stdout: r.stdout, stderr: r.stderr };
}

3
tests/hooks/test-lock-guard.test.mjs
View file

@ -15,6 +15,7 @@ import { writeFileSync, mkdirSync, realpathSync } from 'node:fs';
import { join } from 'node:path';
import { runHook } from '../helpers/hook-helper.mjs';
import { ROOT, scriptsFor, makeRepo, parseJsonOut } from './guard-helpers.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const CLI = join(ROOT, 'lib', 'util', 'test-lock.mjs');
const PRE_WRITE = join(ROOT, 'hooks', 'scripts', 'pre-write-executor.mjs');
@ -31,7 +32,7 @@ function lockedRepo({ session = SESS_A, step = 2 } = {}) {
writeFileSync(join(repo, 'tests', 'free.test.js'), 'y\n');
writeFileSync(join(repo, 'nb.ipynb'), '{}\n');
const r = spawnSync(process.execPath, [CLI, 'lock', '--files', 'tests/locked.test.js,nb.ipynb', '--step', String(step)], {
cwd: repo, encoding: 'utf8', env: { ...process.env, CLAUDE_CODE_SESSION_ID: session },
cwd: repo, encoding: 'utf8', env: childEnv({ CLAUDE_CODE_SESSION_ID: session }),
});
assert.equal(r.status, 0, `lock failed: ${r.stderr}`);
return repo;

3
tests/lib/arch-rules.test.mjs
View file

@ -18,6 +18,7 @@ import { mkdtempSync, mkdirSync, writeFileSync, rmSync, chmodSync, existsSync, s
import { join, dirname } from 'node:path';
import { tmpdir } from 'node:os';
import { fileURLToPath } from 'node:url';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const RUNNER = join(HERE, '..', '..', 'lib', 'verification', 'arch-rules.mjs');
@ -67,7 +68,7 @@ function jsRepo({ cycle = false, binary = true } = {}) {
}
function run(args, env = {}) {
const r = spawnSync('node', [RUNNER, ...args], { encoding: 'utf-8', env: { ...process.env, ...env } });
const r = spawnSync('node', [RUNNER, ...args], { encoding: 'utf-8', env: childEnv(env) });
let json = null;
try { json = JSON.parse(r.stdout); } catch { /* asserted by caller */ }
return { code: r.status, json, out: r.stdout, err: r.stderr };

171
tests/lib/artifact-commit-trailer.test.mjs Normal file
View file

@ -0,0 +1,171 @@
// tests/lib/artifact-commit-trailer.test.mjs
// Approval commits carry the caller's `Co-Authored-By:` trailer (plan step 13).
// The CLI takes `--trailer "<line>"`; a missing, multi-line or malformed one
// (an unfilled `{model}`, no `<email>`, another key) or one holding
// `Claude-Session:` is refused with exit 1 before any git call, so "nothing to
// commit" cannot mask it. A usage error (`--store`) stays exit 2 and comes
// first. Every call in commands/*.md passes the trailer, and none carries a
// `Claude-Session:` line.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { readFileSync, readdirSync, realpathSync, rmSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { childEnv } from '../helpers/child-env.mjs';
import { artifactRepo, git } from '../helpers/artifact-repo.mjs';
import { commitArtifacts } from '../../lib/util/artifact-commit.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const CLI = join(ROOT, 'lib', 'util', 'artifact-commit.mjs');
const TRAILER = 'Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>';
function run(args, cwd) {
return spawnSync(process.execPath, [CLI, ...args], { cwd, encoding: 'utf8', env: childEnv() });
}
const head = (d) => git(d, 'rev-parse', 'HEAD').stdout.trim();
test('a passed trailer ends the commit message and git parses it as a trailer', () => {
const { d, brief } = artifactRepo({ prefix: 'artifact-trailer-', project: '2026-10-12-trailer' });
try {
const r = run(['--store', 'local', '--artifact', brief, '--message', 'docs(voyage): approve intent trailer',
'--trailer', TRAILER], d);
assert.equal(r.status, 0, r.stdout + r.stderr);
assert.equal(JSON.parse(r.stdout).committed, true);
const body = git(d, 'log', '-1', '--format=%B').stdout.trimEnd().split('\n');
assert.equal(body[0], 'docs(voyage): approve intent trailer');
assert.equal(body[body.length - 1], TRAILER, 'the trailer is the last line of the message');
const parsed = git(d, 'log', '-1', '--format=%(trailers:key=Co-Authored-By,valueonly)').stdout.trim();
assert.equal(parsed, 'Claude Opus 5.5 <noreply@anthropic.com>');
} finally { rmSync(d, { recursive: true, force: true }); }
});
const REFUSED = [
['a two-line value', `${TRAILER}\nCo-Authored-By: Other <other@example.invalid>`],
['a value holding Claude-Session:', 'Co-Authored-By: Claude-Session: https://example.invalid/x <noreply@anthropic.com>'],
['a value without <email>', 'Co-Authored-By: Claude Opus 5.5'],
['a non-Co-Authored-By key', 'Signed-off-by: Claude Opus 5.5 <noreply@anthropic.com>'],
['the unfilled {model} placeholder', 'Co-Authored-By: Claude {model} <noreply@anthropic.com>'],
['an empty {model} (two spaces before the address)', 'Co-Authored-By: Claude <noreply@anthropic.com>'],
['a name of three spaces', 'Co-Authored-By: <noreply@anthropic.com>'],
['two spaces after the colon', 'Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>'],
['two spaces inside the name', 'Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>'],
];
for (const [name, value] of REFUSED) {
test(`refused with exit 1, nothing committed: ${name}`, () => {
const { d, brief } = artifactRepo({ prefix: 'artifact-trailer-', project: '2026-10-12-trailer' });
try {
const before = head(d);
const r = run(['--store', 'local', '--artifact', brief, '--message', 'x', '--trailer', value], d);
assert.equal(r.status, 1, r.stdout + r.stderr);
const out = JSON.parse(r.stdout);
assert.equal(out.committed, false);
assert.match(out.reason, /trailer/i);
assert.equal(head(d), before, 'HEAD unchanged');
} finally { rmSync(d, { recursive: true, force: true }); }
});
}
// Guard rows: these must stay accepted (review 8420915b tightened the name).
const ACCEPTED = [
['the session trailer', TRAILER],
['a name with a parenthesis', 'Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>'],
];
for (const [name, value] of ACCEPTED) {
test(`accepted and committed as the last line: ${name}`, () => {
const { d, brief } = artifactRepo({ prefix: 'artifact-trailer-', project: '2026-10-12-trailer' });
try {
const r = run(['--store', 'local', '--artifact', brief, '--message', 'x', '--trailer', value], d);
assert.equal(r.status, 0, r.stdout + r.stderr);
assert.equal(JSON.parse(r.stdout).committed, true);
const body = git(d, 'log', '-1', '--format=%B').stdout.trimEnd().split('\n');
assert.equal(body[body.length - 1], value, 'the trailer is the last line of the message');
} finally { rmSync(d, { recursive: true, force: true }); }
});
}
test('a call without a trailer exits 1 with HEAD unchanged', () => {
const { d, brief } = artifactRepo({ prefix: 'artifact-trailer-', project: '2026-10-12-trailer' });
try {
const before = head(d);
const r = run(['--store', 'local', '--artifact', brief, '--message', 'x'], d);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(JSON.parse(r.stdout).reason, /--trailer/);
assert.equal(head(d), before, 'HEAD unchanged');
assert.equal(git(d, 'diff', '--cached', '--name-only').stdout.trim(), '', 'nothing staged either');
} finally { rmSync(d, { recursive: true, force: true }); }
});
test('the trailer check comes before any git call: "nothing to commit" does not mask a missing trailer', () => {
const { d, brief } = artifactRepo({ prefix: 'artifact-trailer-', project: '2026-10-12-trailer', withLog: false });
try {
const r = run(['--store', 'local', '--artifact', brief, '--message', 'x'], d);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(JSON.parse(r.stdout).reason, /--trailer/);
} finally { rmSync(d, { recursive: true, force: true }); }
});
test('commitArtifacts takes the trailer too, and refuses without it', () => {
const { d, brief } = artifactRepo({ prefix: 'artifact-trailer-', project: '2026-10-12-trailer' });
try {
const repoDir = realpathSync(d);
const before = head(d);
const no = commitArtifacts({ repo: repoDir, store: 'local', artifacts: [brief], message: 'x' });
assert.equal(no.exit, 1, JSON.stringify(no));
assert.equal(head(d), before);
const yes = commitArtifacts({ repo: repoDir, store: 'local', artifacts: [brief], message: 'x', trailer: TRAILER });
assert.equal(yes.exit, 0, JSON.stringify(yes));
assert.equal(yes.committed, true);
assert.equal(git(d, 'log', '-1', '--format=%(trailers:key=Co-Authored-By,valueonly)').stdout.trim(),
'Claude Opus 5.5 <noreply@anthropic.com>');
} finally { rmSync(d, { recursive: true, force: true }); }
});
test('usage before trailer: a missing --store is still exit 2, with or without a trailer', () => {
const { d, brief } = artifactRepo({ prefix: 'artifact-trailer-', project: '2026-10-12-trailer' });
try {
assert.equal(run(['--artifact', brief, '--message', 'x'], d).status, 2);
assert.equal(run(['--artifact', brief, '--message', 'x', '--trailer', TRAILER], d).status, 2);
assert.equal(run(['--store', 'cloud', '--artifact', brief, '--message', 'x', '--trailer', 'bad'], d).status, 2);
} finally { rmSync(d, { recursive: true, force: true }); }
});
// ---- the calls in commands/*.md ----
const CMD_DIR = join(ROOT, 'commands');
// A call is `artifact-commit.mjs` followed by its flags, up to the end of the
// line or the closing backtick of an inline call. A prose mention without
// flags is not a call.
const CALL = /artifact-commit\.mjs[ \t]+--[^`\n]*/g;
function calls() {
const out = [];
for (const f of readdirSync(CMD_DIR).filter((n) => n.endsWith('.md')).sort()) {
const text = readFileSync(join(CMD_DIR, f), 'utf8');
for (const m of text.matchAll(CALL)) out.push({ file: f, call: m[0], index: m.index, text });
}
return out;
}
test('every artifact-commit.mjs call in commands/*.md (exactly 3) passes --trailer, and none holds Claude-Session', () => {
const found = calls();
assert.deepEqual(found.map((c) => c.file), ['trekbrief.md', 'trekexecute.md', 'trekplan.md'],
found.map((c) => `${c.file}: ${c.call}`).join('\n'));
for (const c of found) {
assert.match(c.call, /--trailer "Co-Authored-By: Claude \{model\} <noreply@anthropic\.com>"/, `${c.file}: ${c.call}`);
assert.doesNotMatch(c.call, /Claude-Session/, `${c.file}: ${c.call}`);
}
});
test('commands/trekexecute.md says what to do when the approval commit exits 1 or 2', () => {
const c = calls().find((x) => x.file === 'trekexecute.md');
assert.ok(c, 'the inline call in /trekexecute is found');
const after = c.text.slice(c.index, c.index + 800).replace(/\s+/g, ' ');
assert.ok(after.includes('On exit 1 or 2, print the `reason` in one line and continue: the approval is logged but not committed.'),
after);
});

57
tests/lib/artifact-commit.test.mjs
View file

@ -6,38 +6,17 @@
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { artifactRepo, git } from '../helpers/artifact-repo.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const CLI = join(ROOT, 'lib', 'util', 'artifact-commit.mjs');
const LOG = join(ROOT, 'lib', 'util', 'approval-log.mjs');
const H1 = 'c'.repeat(64);
function git(cwd, ...args) {
return spawnSync('git', args, { cwd, encoding: 'utf8' });
}
function repo() {
const d = mkdtempSync(join(tmpdir(), 'artifact-commit-'));
git(d, 'init', '-q');
git(d, 'config', 'user.name', 'Test');
git(d, 'config', 'user.email', 'test@example.invalid');
git(d, 'config', 'commit.gpgsign', 'false');
mkdirSync(join(d, '.hooks-none'));
git(d, 'config', 'core.hooksPath', '.hooks-none');
writeFileSync(join(d, 'README.md'), 'x\n');
git(d, 'add', 'README.md');
git(d, 'commit', '-q', '-m', 'init');
const proj = join(d, '.claude', 'projects', '2026-10-06-toggle');
mkdirSync(proj, { recursive: true });
writeFileSync(join(proj, 'brief.md'), '# brief\n');
spawnSync(process.execPath, [LOG, 'append', '--kind', 'intent', '--subject', join(proj, 'brief.md'), '--sha256', H1], { cwd: d });
return { d, brief: join(proj, 'brief.md') };
}
// Every call passes a valid trailer (the CLI refuses a call without one).
const TRAILER = ['--trailer', 'Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>'];
function run(args, cwd) {
return spawnSync(process.execPath, [CLI, ...args], { cwd, encoding: 'utf8' });
@ -46,11 +25,11 @@ function run(args, cwd) {
const committedFiles = (d) => git(d, 'show', '--name-only', '--format=', 'HEAD').stdout.trim().split('\n').filter(Boolean);
test('store git: the brief and the log land in one commit, and nothing else the operator staged', () => {
const { d, brief } = repo();
const { d, brief } = artifactRepo();
try {
writeFileSync(join(d, 'unrelated.txt'), 'staged by the operator\n');
git(d, 'add', 'unrelated.txt');
const r = run(['--store', 'git', '--artifact', brief, '--message', 'docs(voyage): approve intent toggle'], d);
const r = run(['--store', 'git', '--artifact', brief, '--message', 'docs(voyage): approve intent toggle', ...TRAILER], d);
assert.equal(r.status, 0, r.stdout + r.stderr);
assert.equal(JSON.parse(r.stdout).committed, true);
assert.deepEqual(committedFiles(d).sort(),
@ -61,9 +40,9 @@ test('store git: the brief and the log land in one commit, and nothing else the
});
test('store local: only the log is committed, and the project folder is kept out of git locally', () => {
const { d, brief } = repo();
const { d, brief } = artifactRepo();
try {
const r = run(['--store', 'local', '--artifact', brief, '--message', 'docs(voyage): approve intent toggle'], d);
const r = run(['--store', 'local', '--artifact', brief, '--message', 'docs(voyage): approve intent toggle', ...TRAILER], d);
assert.equal(r.status, 0, r.stdout + r.stderr);
assert.deepEqual(committedFiles(d), ['.claude/voyage/approvals.jsonl']);
const exclude = readFileSync(join(d, '.git', 'info', 'exclude'), 'utf8');
@ -71,27 +50,27 @@ test('store local: only the log is committed, and the project folder is kept out
assert.equal(git(d, 'status', '--porcelain', '--', '.claude/projects').stdout.trim(), '',
'the local-only brief does not show as untracked');
// idempotent: a second run adds no second exclude line
run(['--store', 'local', '--artifact', brief, '--message', 'again'], d);
run(['--store', 'local', '--artifact', brief, '--message', 'again', ...TRAILER], d);
const again = readFileSync(join(d, '.git', 'info', 'exclude'), 'utf8');
assert.equal(again.split('\n').filter((l) => l.includes('2026-10-06-toggle')).length, 1);
} finally { rmSync(d, { recursive: true, force: true }); }
});
test('nothing new to commit is exit 0 with committed:false (a re-run is not an error)', () => {
const { d, brief } = repo();
const { d, brief } = artifactRepo();
try {
assert.equal(run(['--store', 'git', '--artifact', brief, '--message', 'one'], d).status, 0);
const r = run(['--store', 'git', '--artifact', brief, '--message', 'two'], d);
assert.equal(run(['--store', 'git', '--artifact', brief, '--message', 'one', ...TRAILER], d).status, 0);
const r = run(['--store', 'git', '--artifact', brief, '--message', 'two', ...TRAILER], d);
assert.equal(r.status, 0, r.stdout + r.stderr);
assert.equal(JSON.parse(r.stdout).committed, false);
} finally { rmSync(d, { recursive: true, force: true }); }
});
test('a path the repo ignores is not forced in: exit 1 with the reason', () => {
const { d, brief } = repo();
const { d, brief } = artifactRepo();
try {
writeFileSync(join(d, '.gitignore'), '.claude/\n');
const r = run(['--store', 'git', '--artifact', brief, '--message', 'x'], d);
const r = run(['--store', 'git', '--artifact', brief, '--message', 'x', ...TRAILER], d);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(JSON.parse(r.stdout).reason, /ignored/i);
} finally { rmSync(d, { recursive: true, force: true }); }
@ -101,15 +80,15 @@ test('not a git repository: exit 2 (could not run), never a silent pass', () =>
const d = mkdtempSync(join(tmpdir(), 'artifact-commit-nogit-'));
try {
writeFileSync(join(d, 'brief.md'), 'x');
const r = run(['--store', 'git', '--artifact', join(d, 'brief.md'), '--message', 'x'], d);
const r = run(['--store', 'git', '--artifact', join(d, 'brief.md'), '--message', 'x', ...TRAILER], d);
assert.equal(r.status, 2, r.stdout + r.stderr);
} finally { rmSync(d, { recursive: true, force: true }); }
});
test('a missing or unknown --store is a usage error (exit 2)', () => {
const { d, brief } = repo();
const { d, brief } = artifactRepo();
try {
assert.equal(run(['--artifact', brief, '--message', 'x'], d).status, 2);
assert.equal(run(['--store', 'cloud', '--artifact', brief, '--message', 'x'], d).status, 2);
assert.equal(run(['--artifact', brief, '--message', 'x', ...TRAILER], d).status, 2);
assert.equal(run(['--store', 'cloud', '--artifact', brief, '--message', 'x', ...TRAILER], d).status, 2);
} finally { rmSync(d, { recursive: true, force: true }); }
});

62
tests/lib/doc-consistency.test.mjs
View file

@ -139,9 +139,9 @@ test('every command frontmatter name matches its filename', () => {
}
});
test('templates/plan-template.md declares plan_version: 1.8', () => {
test('templates/plan-template.md declares plan_version: 1.9', () => {
const tpl = read('templates/plan-template.md');
assert.match(tpl, /plan_version:\s*['"]?1\.8['"]?/);
assert.match(tpl, /plan_version:\s*['"]?1\.9['"]?/);
});
test('HANDOVER-CONTRACTS.md locates plan_version in the plan header line, not frontmatter (S28)', () => {
@ -162,8 +162,8 @@ test('HANDOVER-CONTRACTS.md locates plan_version in the plan header line, not fr
// "commands/trekexecute.md still parses v1.7 plan schema" was an OR-chain
// (`includes('plan_version') || includes('Step N:') || includes('### Step ')`)
// that is ~always true and asserts almost nothing. The real v1.7 coverage lives
// in the structural pin "templates/plan-template.md declares plan_version: 1.8"
// (above; 1.7 plans stay valid) + the plan-validator / plan-schema behavior tests.
// in the structural pin "templates/plan-template.md declares plan_version: 1.9"
// (above; 1.8 and 1.7 plans stay valid) + the plan-validator / plan-schema behavior tests.
test('settings.json has only known top-level scopes after Spor 0 cleanup', () => {
const cfg = JSON.parse(read('settings.json'));
@ -2079,17 +2079,49 @@ function schemaRow(id) {
}
// The Phase 9 line_ref was measured, then more prose landed above Phase 9 and
// nothing re-measured it. Pinned to the span it names: heading to closing fence.
test('the trekexecute Phase 9 row\'s line_ref spans the Phase 9 heading to the end of its stdin block', () => {
const ref = schemaRow('trekexecute-stats (Phase 9 record)')[4];
const m = ref.match(/^trekexecute\.md:(\d+)-(\d+)$/);
assert.ok(m, `line_ref "${ref}" is not trekexecute.md:A-B`);
const lines = read('commands/trekexecute.md').split('\n');
const heading = lines.findIndex((l) => /^## Phase 9\b/.test(l)) + 1;
const append = lines.findIndex((l) => /^\s*node \$\{CLAUDE_PLUGIN_ROOT\}\/lib\/stats\/stats-append\.mjs trekexecute\b/.test(l)) + 1;
const fence = lines.findIndex((l, i) => i >= append && l.trim() === '```') + 1;
assert.ok(heading > 0 && append > heading && fence > append, 'Phase 9 heading, stats-append line and closing fence');
assert.equal(ref, `trekexecute.md:${heading}-${fence}`);
// nothing re-measured it. The same drift hit the other command stats rows.
// Each row is pinned to the span it names: its phase heading to the closing
// fence of its stats-append stdin block. The trekreview anchor is the full
// "## Phase 8 — Validate output" so the Phase 8.5 / 8.7 headings never match.
const STATS_ROWS = [
{ id: 'trekbrief-stats', cmd: 'trekbrief', heading: /^## Phase 7\b(?!\.)/ },
{ id: 'trekplan-stats', cmd: 'trekplan', heading: /^## Phase 12\b(?!\.)/ },
{ id: 'trekreview-stats', cmd: 'trekreview', heading: /^## Phase 8 — Validate output/ },
{ id: 'trekexecute-stats (Phase 9 record)', cmd: 'trekexecute', heading: /^## Phase 9\b(?!\.)/ },
];
for (const { id, cmd, heading: headingRe } of STATS_ROWS) {
test(`the ${id} row's line_ref spans its phase heading to the end of its stats-append stdin block`, () => {
const ref = schemaRow(id)[4];
const m = ref.match(new RegExp(`^${cmd}\\.md:(\\d+)-(\\d+)$`));
assert.ok(m, `line_ref "${ref}" is not ${cmd}.md:A-B`);
const lines = read(`commands/${cmd}.md`).split('\n');
const headings = lines.map((l, i) => (headingRe.test(l) ? i : -1)).filter((i) => i >= 0);
assert.equal(headings.length, 1, `exactly one heading in commands/${cmd}.md matches ${headingRe}`);
const heading = headings[0] + 1;
const appendRe = new RegExp(`^\\s*node \\$\\{CLAUDE_PLUGIN_ROOT\\}/lib/stats/stats-append\\.mjs ${cmd}\\b`);
const append = lines.findIndex((l, i) => i >= heading && appendRe.test(l)) + 1;
const nextPhase = lines.findIndex((l, i) => i >= heading && /^## Phase /.test(l)) + 1;
assert.ok(append > heading, `a stats-append line follows the heading in commands/${cmd}.md`);
assert.ok(nextPhase === 0 || append < nextPhase, `the stats-append line sits inside the phase (line ${append}, next phase at ${nextPhase})`);
const fence = lines.findIndex((l, i) => i >= append && l.trim() === '```') + 1;
assert.ok(fence > append, 'the stats-append block has a closing fence');
assert.equal(ref, `${cmd}.md:${heading}-${fence}`);
});
}
// A `## Brief Quality` section belongs to the exhausted and force-stopped
// branches only; the passed-gate branch must not tell the model to write one.
test('trekbrief: the passed-gate branch names no `## Brief Quality`, the exhausted branch still does', () => {
const text = read('commands/trekbrief.md');
const passStart = text.indexOf('**If gate passes:**');
const passEnd = text.indexOf('**If gate fails AND iteration count < the class cap:**');
assert.ok(passStart >= 0 && passEnd > passStart, 'passed-gate branch markers');
assert.ok(!text.slice(passStart, passEnd).includes('## Brief Quality'), 'the passed-gate branch names `## Brief Quality`');
const exStart = text.indexOf('**If gate fails AND iteration count == the class cap (loop exhausted):**');
const exEnd = text.indexOf('### Step 4f', exStart);
assert.ok(exStart > passEnd && exEnd > exStart, 'exhausted branch markers');
assert.ok(text.slice(exStart, exEnd).includes('## Brief Quality'), 'the exhausted branch must still name `## Brief Quality`');
});
test('the autonomy-events row\'s line_ref spans emit() in event-emit.mjs', () => {

6
tests/lib/executor-prose-pins.test.mjs
View file

@ -191,10 +191,12 @@ test('review: no sixth round, attempts never reset, a step at 5 is never built a
assert.match(c, /A step at `fix_rounds` 5 is never built again/);
});
test('review: the helper appends its report, and passes tests as one comma-separated list', () => {
test('review: the helper appends its report, and gives red proof no --tests', () => {
const agent = read(AGENT);
assert.match(agent, /APPEND it to `report_path`/);
assert.match(agent, /ONE comma-separated argument/);
const gate = agent.split('\n').find((l) => l.includes('red-proof.mjs --red-only'));
assert.ok(gate, 'the helper has its red-proof --red-only line');
assert.ok(!gate.includes('--tests'), `the helper's red-proof line passes --tests: ${gate}`);
assert.match(agent, /A step with no `Test first:`/);
});

73
tests/lib/files-reader-regressions.test.mjs Normal file
View file

@ -0,0 +1,73 @@
// tests/lib/files-reader-regressions.test.mjs
// 6.4 / regressions A, B and C of the `Files:` reader (fieldPaths in
// lib/parsers/plan-schema.mjs): three inputs 6.3.0 read right and 6.3.1 read
// wrong. Each case goes through stepTestsOnly on a 1.8-shaped step and through
// the validator's PLAN_STEP_TESTS_ONLY on a 1.8 plan. The controls are the
// round 7-10 neighbours of each case and the plain path right after a
// backticked one: they pin 6.3.1's answers and stay as they are.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
import { validatePlanContent } from '../../lib/validators/plan-validator.mjs';
import { stepTestsOnly, untickedPaths } from '../../lib/parsers/plan-schema.mjs';
const step = (files) => `- **Files:** ${files}\n- **Changes:** x\n`;
// A 1.8 plan whose step 2 carries `files` as its `Files:` field.
const planWith = (files) => buildPlan({ steps: [{}, { file: 'src/mod2.mjs', filesRaw: files, endToEnd: 'bash tests/e2e.sh' }] });
const testsOnlyWarned = (files) => validatePlanContent(planWith(files)).warnings
.some((w) => w.code === 'PLAN_STEP_TESTS_ONLY' && /Step 2\b/.test(w.message));
// [case, Files: text, 6.3.0's paths]
const CASES = [
['A: a comma chunk that opens with a plain file ends the description',
'`tests/a.test.mjs` - new test, src/a.mjs - the fix', ['tests/a.test.mjs', 'src/a.mjs']],
['B: a plain slash path inside a description, with no comma, is no file',
'`tests/a.test.mjs` covers the parser in src/parse.mjs', ['tests/a.test.mjs']],
['C: a backticked bare name alone on a new line is a file',
'`tests/a.test.mjs` - new test\n `LICENSE`', ['tests/a.test.mjs', 'LICENSE']],
];
for (const [name, files, paths] of CASES) {
test(`6.4 / regression ${name}`, () => {
const testsOnly = paths.every((p) => p.startsWith('tests/'));
assert.deepEqual(stepTestsOnly(step(files)), { testsOnly, files: paths }, files);
assert.equal(testsOnlyWarned(files), testsOnly,
`PLAN_STEP_TESTS_ONLY ${testsOnly ? 'fires' : 'stays quiet'} on a 1.8 plan: ${files}`);
});
}
// [control, Files: text, 6.3.1's paths]
const CONTROLS = [
['r7 (comma form of B): a plain slash path in its own comma entry is read',
'`tests/x.test.mjs`, the CLI entry src/cli.mjs', ['tests/x.test.mjs', 'src/cli.mjs']],
['r9 (A\'s neighbour): a description runs on past a comma',
'`tests/a.test.mjs` — mocks fs, child_process.spawnSync and path.join', ['tests/a.test.mjs']],
['r10 (C\'s neighbour): a backticked capitalised name inside a description is no file',
'`tests/a.test.mjs` — covers `parse`, `Parser`, `ParseError`', ['tests/a.test.mjs']],
['B\'s neighbour: a plain path right after the backticked one is read',
'`tests/a.test.mjs` src/b.mjs', ['tests/a.test.mjs', 'src/b.mjs']],
['B\'s neighbour: a plain path after backticked paths joined by "+" is read',
'`src/a.mjs` + `src/b.mjs` src/c.mjs', ['src/a.mjs', 'src/b.mjs', 'src/c.mjs']],
['B\'s neighbour: a plain path after backticked paths joined by "&" is read',
'`src/a.mjs` & `src/b.mjs` src/c.mjs', ['src/a.mjs', 'src/b.mjs', 'src/c.mjs']],
['B\'s neighbour: a plain path after backticked paths joined by "and" is read',
'`src/a.mjs` and `src/b.mjs` src/c.mjs', ['src/a.mjs', 'src/b.mjs', 'src/c.mjs']],
['B\'s neighbour: a plain source path after two joined test paths makes the step no tests-only step',
'`tests/a.test.mjs` + `tests/b.test.mjs` src/c.mjs', ['tests/a.test.mjs', 'tests/b.test.mjs', 'src/c.mjs']],
];
for (const [name, files, paths] of CONTROLS) {
test(`6.4 / control ${name}`, () => {
const testsOnly = paths.every((p) => p.startsWith('tests/'));
assert.deepEqual(stepTestsOnly(step(files)), { testsOnly, files: paths }, files);
assert.equal(testsOnlyWarned(files), testsOnly,
`PLAN_STEP_TESTS_ONLY ${testsOnly ? 'fires' : 'stays quiet'} on a 1.8 plan: ${files}`);
});
}
test('6.4 / control: untickedPaths names the plain path after backticked paths joined by "+"', () => {
assert.deepEqual(untickedPaths('`src/a.mjs` + `src/b.mjs` src/c.mjs'), ['src/c.mjs']);
});

5
tests/lib/independent-review-fixes.test.mjs
View file

@ -15,6 +15,7 @@ import { recordVerdict, planHash, acceptOpen, gatePlan } from '../../lib/validat
import { stampBriefReview, checkBriefReview } from '../../lib/validators/brief-review-stamp.mjs';
import { stampIntentApproval } from '../../lib/validators/intent-approval.mjs';
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
@ -30,11 +31,11 @@ function lockRepo(withCmd) {
writeFileSync(join(repo, 'tests', 'a.test.mjs'), "import { test } from 'node:test';\nimport assert from 'node:assert';\ntest('a', () => assert.equal(1, 2));\n");
const args = ['lock', '--files', 'tests/a.test.mjs', '--step', '1'];
if (withCmd) args.push('--cmd', 'node --test tests/a.test.mjs');
const r = spawnSync(process.execPath, [LOCK, ...args], { cwd: repo, encoding: 'utf8', env: { ...process.env, CLAUDE_CODE_SESSION_ID: SESS } });
const r = spawnSync(process.execPath, [LOCK, ...args], { cwd: repo, encoding: 'utf8', env: childEnv({ CLAUDE_CODE_SESSION_ID: SESS }) });
assert.equal(r.status, 0, r.stderr);
return repo;
}
const lockCli = (repo, args) => spawnSync(process.execPath, [LOCK, ...args], { cwd: repo, encoding: 'utf8', env: { ...process.env, CLAUDE_CODE_SESSION_ID: SESS } });
const lockCli = (repo, args) => spawnSync(process.execPath, [LOCK, ...args], { cwd: repo, encoding: 'utf8', env: childEnv({ CLAUDE_CODE_SESSION_ID: SESS }) });
test('green: another allowed command (git status) can never mark a red step green', async () => {
const repo = lockRepo(true);

6
tests/lib/plan-prose-pins.test.mjs
View file

@ -113,10 +113,10 @@ test('Handover 4 documents every PLAN_* code the validator and the gate emit', (
assert.deepEqual(missing, [], `Handover 4 does not document: ${missing.join(', ')}`);
});
test('Handover 4 states 1.8 as current and 1.7 as still valid', () => {
test('Handover 4 states 1.9 as current and 1.8 and 1.7 as still valid', () => {
const contract = read('docs/HANDOVER-CONTRACTS.md');
assert.match(contract, /\| `plan\.md` \| `plan_version` \(plan header line\) \| `1\.8` \(1\.7 still valid\) \|/);
assert.match(contract, /`"1\.8"` \(current\), `"1\.7"` \(valid\)/);
assert.match(contract, /\| `plan\.md` \| `plan_version` \(plan header line\) \| `1\.9` \(1\.8 and 1\.7 still valid\) \|/);
assert.match(contract, /`"1\.9"` \(current\), `"1\.8"` \(valid\), `"1\.7"` \(valid\)/);
});
test('the verification CLIs credit what they borrow and say they install nothing', () => {

2
tests/lib/plan-schema.test.mjs
View file

@ -150,5 +150,5 @@ test('extractPlanVersion — recognizes the canonical plan template (regression
// The real generated artifact: no test exercised the parser against the
// actual template, which is why defect S26 went unnoticed.
const tpl = readFileSync(new URL('../../templates/plan-template.md', import.meta.url), 'utf-8');
assert.equal(extractPlanVersion(tpl), '1.8');
assert.equal(extractPlanVersion(tpl), '1.9');
});

514
tests/lib/plan-version-1-9-docs.test.mjs Normal file
View file

@ -0,0 +1,514 @@
// tests/lib/plan-version-1-9-docs.test.mjs
// plan_version 1.9 is the current target, and the contract docs follow it:
// Handover 4, the plan template and CLAUDE.md name 1.9 and its rule, "the file
// first on every line"; the template, /trekplan and the planning orchestrator
// say every path stands before any annotation; and the rules introduced at 1.8,
// which still bind 1.8 and 1.9 plans, say "1.8 or later" in the sentence that
// carries each of them (never "1.9", which would release a 1.8 plan from them).
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { validatePlanContent } from '../../lib/validators/plan-validator.mjs';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const read = (rel) => readFileSync(join(ROOT, rel), 'utf-8');
const RULE = 'file first on every line';
const ANNOTATION = 'every path stands before any annotation';
const NEW_CODES = ['PLAN_STEP_PATH_NOT_FIRST', 'PLAN_STEP_PATH_NOT_A_PATH', 'PLAN_STEP_PATH_UNTICKED'];
// Sentence boundaries: a full stop or question mark before whitespace, a blank
// line, or the start of a list item. "1.8" and ".mjs" are no boundary.
const BOUNDARY = /[.?!](?=\s)|\n[ \t>]*\n|\n[ \t>]*[-*] /g;
/**
* The sentence of `text` that holds the first match of `anchor`: from the
* boundary before the match to the first boundary after it.
*/
function sentence(text, anchor, label) {
const m = anchor.exec(text);
assert.ok(m, `${label}: anchor ${anchor} not found`);
let start = 0;
let end = text.length;
for (const b of text.matchAll(BOUNDARY)) {
const bEnd = b.index + b[0].length;
if (bEnd <= m.index) start = bEnd;
else if (b.index >= m.index + m[0].length) { end = b.index; break; }
}
return text.slice(start, end);
}
// A line that opens a list item: its indent, then `-`, `*`, `+`, `1.` or `1)`.
const LIST_ITEM_LINE = /^([ \t>]*)(?:[-*+]|\d+[.)]) /;
/**
* The list item of `text` that holds the first match of `anchor`: from the
* start of the line that opens it (`- `) to the next line that opens a list
* item at the same indent (or a smaller one, which closes its list too), a
* blank line, or the end of the text. A wrapped line of the item belongs to it.
*/
function bullet(text, anchor, label) {
const m = anchor.exec(text);
assert.ok(m, `${label}: anchor ${anchor} not found`);
const lines = text.split('\n');
let at = 0;
let i = 0;
while (i < lines.length - 1 && at + lines[i].length < m.index) { at += lines[i].length + 1; i++; }
let first = i;
while (first >= 0 && !LIST_ITEM_LINE.test(lines[first])) {
assert.ok(lines[first].trim() !== '', `${label}: the match of ${anchor} stands in no list item`);
first--;
}
assert.ok(first >= 0, `${label}: the match of ${anchor} stands in no list item`);
const indent = LIST_ITEM_LINE.exec(lines[first])[1].length;
let last = first + 1;
for (; last < lines.length; last++) {
if (lines[last].trim() === '' || /^[ \t>]*$/.test(lines[last])) break;
const item = LIST_ITEM_LINE.exec(lines[last]);
if (item && item[1].length <= indent) break;
}
return lines.slice(first, last).join('\n');
}
function holds18OrLater(text, anchor, label) {
const s = sentence(text, anchor, label);
assert.match(s, /1\.8 or later/, `${label} must say "1.8 or later" in the sentence that carries the rule: ${JSON.stringify(s)}`);
assert.doesNotMatch(s, /1\.9/, `${label} is a 1.8 rule that still binds 1.8 plans; it must not say 1.9: ${JSON.stringify(s)}`);
}
const handover4 = () => {
const c = read('docs/HANDOVER-CONTRACTS.md');
const start = c.indexOf('## Handover 4');
assert.ok(start >= 0, '## Handover 4 missing');
return c.slice(start, c.indexOf('## Handover 5', start));
};
const claudeRow = () => {
const m = /^\| `\/trekplan` \|.*$/m.exec(read('CLAUDE.md'));
assert.ok(m, 'CLAUDE.md has no /trekplan row');
return m[0];
};
test('Handover 4 names plan_version 1.9 and the "file first on every line" rule', () => {
const h4 = handover4();
assert.match(h4, /plan_version 1\.9/);
assert.ok(h4.includes(RULE), `Handover 4 must name the rule "${RULE}"`);
assert.match(h4, /`"1\.9"` \(current\), `"1\.8"` \(valid\), `"1\.7"` \(valid\)/);
});
test('the plan template declares plan_version 1.9 and states the rule', () => {
const t = read('templates/plan-template.md');
assert.ok(t.includes('`plan_version: 1.9` · `ceremony:'), 'the template header line names plan_version 1.9');
assert.ok(t.includes(RULE), `the template must state the rule "${RULE}"`);
});
test('CLAUDE.md /trekplan row names Plan 1.9 and the rule, and labels the Test first / End-to-end rules "From 1.8"', () => {
const row = claudeRow();
assert.match(row, /Plan 1\.9: the file first on every line/);
assert.match(row, /From 1\.8: `Test first:` per step \+ one `End-to-end:` step/);
assert.doesNotMatch(row, /Plan 1\.8:/);
});
test('the template, /trekplan and the planning orchestrator say every path stands before any annotation', () => {
for (const rel of ['templates/plan-template.md', 'commands/trekplan.md', 'agents/planning-orchestrator.md']) {
assert.ok(read(rel).includes(ANNOTATION), `${rel} must say "${ANNOTATION}"`);
}
});
test('the plan template: the Test first and End-to-end rules say "1.8 or later" in their own sentence', () => {
const t = read('templates/plan-template.md');
holds18OrLater(t, /Every step has\s+`Test first:`/, 'template Test first sentence');
holds18OrLater(t, /without an\s+`End-to-end:`\s+step is invalid/, 'template End-to-end sentence');
});
test('plan-critic: the Test first check says "1.8 or later" in its own sentence', () => {
holds18OrLater(read('agents/plan-critic.md'), /does every step's\s+`Test first:`\s+name a test/, 'plan-critic Test first check');
});
test('design template: the opening line and the --strict line say "1.8 or later"', () => {
const d = read('templates/design-template.md');
holds18OrLater(d, /The design part of a plan/, 'design template opening line');
holds18OrLater(d, /`plan-validator --strict` checks/, 'design template --strict line');
});
test('HANDOVER-CONTRACTS: the design-part line and invariant 5 say "1.8 or later"; invariant 7 reads the Test first File line', () => {
// The design-part line stands in Handover 3 (it tells the external slot from
// the plan's own design); invariants 5 and 7 stand in Handover 4.
holds18OrLater(read('docs/HANDOVER-CONTRACTS.md'), /A plan's own design/, 'HANDOVER-CONTRACTS design-part line');
const h4 = handover4();
holds18OrLater(h4, /\(1\.7 plans are not held to these\)/, 'Handover 4 invariant 5');
const bullet = sentence(h4, /\*\*A step of only tests\*\*/, 'Handover 4 invariant 7 tests-only bullet');
assert.ok(bullet.includes('a file on its `Test first:` `File:` line'),
`invariant 7 must say "a file on its \`Test first:\` \`File:\` line": ${JSON.stringify(bullet)}`);
assert.ok(!bullet.includes('its `Test first:` file,'), 'invariant 7 no longer says "its `Test first:` file"');
});
const trekplan19Block = () => {
const p = read('commands/trekplan.md');
const h19 = p.indexOf('**plan_version 1.9 additions**');
assert.ok(h19 >= 0, 'commands/trekplan.md has the "**plan_version 1.9 additions**" block');
const end = p.indexOf('\n>\n> **', h19);
return p.slice(h19, end < 0 ? undefined : end);
};
test('/trekplan: the 1.8 additions heading says "1.8 or later", and a separate 1.9 additions block follows it', () => {
const p = read('commands/trekplan.md');
const h18 = p.indexOf('**plan_version 1.8 or later additions**');
assert.ok(h18 >= 0, 'the heading reads "**plan_version 1.8 or later additions**"');
assert.ok(!p.includes('**plan_version 1.8 additions**'), 'the old 1.8-only heading is gone');
const h19 = p.indexOf('**plan_version 1.9 additions**');
assert.ok(h19 > h18, 'a separate "**plan_version 1.9 additions**" block stands after the 1.8 block');
assert.ok(trekplan19Block().includes(ANNOTATION), `the 1.9 block says "${ANNOTATION}"`);
});
// Step 12: the docs follow the 1.9 fixes. Operator decision 20 (12.10): the
// `Test first:` files are the backticked paths right after `File:`, and the
// red proof checks `--tests` against them (`tests-mismatch`). The 1.9 `File:`
// line takes the numbered markers, a wrapped line joins it, a list item ends
// it. Operator decision 22 (13.10): a backticked placeholder or a mixed-case
// bare name is no path (`./Name` names such a file); the bare word none,
// unticked, stays PLAN_STEP_PATH_NOT_FIRST.
const BOTH_NONES = ['the bare word none', 'a backticked placeholder'];
function holdsBothNones(text, label) {
for (const phrase of BOTH_NONES) {
assert.ok(text.includes(phrase), `${label} must say "${phrase}": ${JSON.stringify(text)}`);
}
}
const invariant7 = () => {
const h4 = handover4();
const m7 = /^7\. \*\*Every plan_version/m.exec(h4);
assert.ok(m7, 'Handover 4 has invariant 7');
const m8 = /^8\. \*\*/m.exec(h4.slice(m7.index));
return h4.slice(m7.index, m8 ? m7.index + m8.index : undefined);
};
const template19Paragraph = () => {
const t = read('templates/plan-template.md');
const start = t.indexOf('From plan_version 1.9 the rule is the file first on every line');
assert.ok(start >= 0, 'the template has its 1.9 paragraph');
const end = t.indexOf('\n### Step 1:', start);
return t.slice(start, end < 0 ? undefined : end);
};
test('Handover 4 invariant 7 names operator decision 20 and the red proof\'s tests-mismatch', () => {
const inv7 = invariant7();
assert.match(inv7, /operator decision 20\b/, `invariant 7 names operator decision 20: ${JSON.stringify(inv7)}`);
assert.ok(inv7.includes('`tests-mismatch`'), `invariant 7 names \`tests-mismatch\`: ${JSON.stringify(inv7)}`);
});
test('Handover 4 invariant 8, the Test first bullet: numbered markers, the wrapped line, a list item ends the entry', () => {
const b = bullet(handover4(), /A line counts only when/, 'invariant 8 Test first bullet');
assert.ok(b.includes('`1.`') && b.includes('`1)`'), `the Test first bullet names the \`1.\` and \`1)\` markers: ${JSON.stringify(b)}`);
assert.match(b, /wrapped/, `the Test first bullet names the wrapped line: ${JSON.stringify(b)}`);
assert.match(b, /any list item\b[^;]*\bends\b/, `the Test first bullet says any list item ends the entry: ${JSON.stringify(b)}`);
});
test('Handover 4 invariant 8, the refusal bullet: operator decision 22 with PLAN_STEP_PATH_NOT_A_PATH, and both nones', () => {
const b = bullet(handover4(), /an entry that does not open with a backticked span/, 'invariant 8 refusal bullet');
const s = sentence(b, /operator decision 22\b/, 'invariant 8 refusal bullet, decision 22');
assert.ok(s.includes('`PLAN_STEP_PATH_NOT_A_PATH`'), `operator decision 22 stands with \`PLAN_STEP_PATH_NOT_A_PATH\`: ${JSON.stringify(s)}`);
holdsBothNones(b, 'the invariant 8 refusal bullet');
});
test('Handover 4 code table: the PLAN_STEP_PATH_NOT_A_PATH row names placeholders and mixed-case bare names', () => {
const m = /^\| `PLAN_STEP_PATH_NOT_A_PATH` \|.*$/m.exec(handover4());
assert.ok(m, 'Handover 4 has a PLAN_STEP_PATH_NOT_A_PATH row');
assert.match(m[0], /placeholder/, `the row names placeholders: ${JSON.stringify(m[0])}`);
assert.match(m[0], /mixed-case/, `the row names mixed-case bare names: ${JSON.stringify(m[0])}`);
});
// Second remediation, Step 8: the docs follow operator decisions 29 to 31
// (14.10). Decision 29: in 1.9 a wrapped line keeps its paths only when the
// entry so far ends with a comma; otherwise it is description, and a `File:`
// line with nothing after it takes its first wrapped line. Decision 31: below
// 1.9 the run of test files goes on past a note in brackets and the words
// and, plus and +. Decision 30: red proof reads the list from the plan, the
// documented call passes no `--tests`, and an unreadable plan is
// `plan-unreadable`.
// A blockquote's text without its `> ` markers, so a phrase wrapped across two
// quoted lines reads as one.
const unquote = (text) => text.replace(/\n>[ \t]?/g, '\n');
test('Handover 4 invariant 8, the Files bullet: operator decision 29, "ends with a comma", "takes its first wrapped line"', () => {
const b = bullet(handover4(), /the entries of a step's `Files:` field/, 'invariant 8 Files bullet');
assert.match(b, /\boperator decision 29\b/, `the Files bullet names operator decision 29: ${JSON.stringify(b)}`);
assert.match(b, /ends\s+with\s+a\s+comma/, `the Files bullet says a wrapped line keeps its paths when the entry ends with a comma: ${JSON.stringify(b)}`);
assert.match(b, /takes\s+its\s+first\s+wrapped\s+line/, `the Files bullet says an empty entry takes its first wrapped line: ${JSON.stringify(b)}`);
});
test('Handover 4 invariant 7 names operator decisions 30 and 31, plan-unreadable and plus', () => {
const inv7 = invariant7();
assert.match(inv7, /\boperator decision 30\b/, `invariant 7 names operator decision 30: ${JSON.stringify(inv7)}`);
assert.match(inv7, /\boperator decision 31\b/, `invariant 7 names operator decision 31: ${JSON.stringify(inv7)}`);
assert.ok(inv7.includes('plan-unreadable'), `invariant 7 names \`plan-unreadable\`: ${JSON.stringify(inv7)}`);
assert.match(inv7, /`plus`/, `invariant 7 names the word \`plus\`: ${JSON.stringify(inv7)}`);
});
test('Handover 4 invariant 7: "This holds at every plan_version" comes before "Below plan_version 1.9"', () => {
const inv7 = invariant7();
const every = inv7.indexOf('This holds at every plan_version');
const below = inv7.indexOf('Below plan_version 1.9');
assert.ok(every >= 0, `invariant 7 says "This holds at every plan_version": ${JSON.stringify(inv7)}`);
assert.ok(below >= 0, `invariant 7 says "Below plan_version 1.9": ${JSON.stringify(inv7)}`);
assert.ok(every < below, 'the every-plan_version sentence stands before the below-1.9 sentence');
});
for (const [label, slice] of [['the /trekplan 1.9 block', trekplan19Block], ['the template 1.9 paragraph', template19Paragraph]]) {
test(`${label} follows operator decision 29: a wrapped line is description unless the entry ends with a comma`, () => {
const text = unquote(slice());
assert.match(text, /entry\s+so\s+far\s+does\s+not\s+end\s+with\s+a\s+comma[^.]*description/,
`${label} says a wrapped line is description once the entry so far does not end with a comma: ${JSON.stringify(text)}`);
assert.match(text, /takes\s+its\s+first\s+wrapped\s+line/,
`${label} says a \`File:\` line with nothing after it takes its first wrapped line: ${JSON.stringify(text)}`);
});
test(`${label} names the placeholders, \`./Name\` and both nones`, () => {
const text = slice();
assert.ok(text.includes('`TBD`') && text.includes('`N/A`'), `${label} names the placeholders (\`TBD\`, \`N/A\`): ${JSON.stringify(text)}`);
assert.ok(text.includes('`./Name`'), `${label} names \`./Name\`: ${JSON.stringify(text)}`);
holdsBothNones(text, label);
});
}
test('the template\'s own example steps validate as 1.9 with none of the new path codes', () => {
const r = validatePlanContent(read('templates/plan-template.md'));
assert.equal(r.parsed.planVersion, '1.9');
const found = [...r.errors, ...r.warnings].map((i) => i.code).filter((c) => NEW_CODES.includes(c));
assert.deepEqual(found, [], JSON.stringify([...r.errors, ...r.warnings].filter((i) => NEW_CODES.includes(i.code)), null, 2));
const soft = validatePlanContent(read('templates/plan-template.md'), { strict: false });
assert.deepEqual([...soft.errors, ...soft.warnings].map((i) => i.code).filter((c) => NEW_CODES.includes(c)), []);
});
// Step 14: the CHANGELOG's top section is the Unreleased 1.9 delivery, with door
// lines the release check accepts. The release check is imported as a module
// namespace, so a missing parseDoors export fails an assertion, not the load.
const releaseCheck = await import('../../scripts/release-check.mjs');
const UNRELEASED = '## Unreleased — plan_version 1.9, the trial-4 findings, session-clean tests';
function firstChangelogSection() {
const lines = read('CHANGELOG.md').split('\n');
const start = lines.findIndex((l) => /^## /.test(l));
assert.ok(start >= 0, 'CHANGELOG.md has no "## " section');
let end = lines.length;
for (let i = start + 1; i < lines.length; i++) {
if (/^## /.test(lines[i])) { end = i; break; }
}
return { heading: lines[start], body: lines.slice(start + 1, end).join('\n') };
}
// The Unreleased section's body, after checking it is the first section.
function unreleasedBody() {
const { heading, body } = firstChangelogSection();
assert.equal(heading, UNRELEASED);
return body;
}
test('CHANGELOG: the first "## " section is the Unreleased plan_version 1.9 section', () => {
assert.equal(firstChangelogSection().heading, UNRELEASED);
});
test('CHANGELOG: parseDoors reads the Unreleased section\'s door lines, none bad, one naming plan_version 1.9', () => {
assert.equal(typeof releaseCheck.parseDoors, 'function', 'scripts/release-check.mjs exports parseDoors');
const body = unreleasedBody();
const d = releaseCheck.parseDoors(body);
assert.equal(d.found, true, 'the Unreleased section has a "### Doors" heading');
assert.ok(d.entries.length > 0, 'the Doors list has door lines');
assert.deepEqual(d.bad, [], 'every top-level door line reads "- one-way: ..." or "- two-way: ..."');
assert.ok(d.entries.some((e) => e.text.includes('plan_version 1.9')),
`one door line names plan_version 1.9: ${JSON.stringify(d.entries.map((e) => e.text))}`);
});
// A `### <name>` body of the Unreleased section, up to the next heading.
const subsection = (body, name) => {
const lines = body.split('\n');
const start = lines.findIndex((l) => l.replace(/\s+$/, '') === `### ${name}`);
assert.ok(start >= 0, `the Unreleased section has a "### ${name}" heading`);
let end = lines.length;
for (let i = start + 1; i < lines.length; i++) {
if (/^###? /.test(lines[i])) { end = i; break; }
}
return lines.slice(start + 1, end).join('\n');
};
const knownIssues = (body) => subsection(body, 'Known issues');
test('CHANGELOG: the Unreleased section\'s Known issues name the session split', () => {
const body = unreleasedBody();
const known = knownIssues(body);
assert.match(known, /session split/, `Known issues name the session split: ${JSON.stringify(known)}`);
});
// Step 14 of the remediation: the CHANGELOG records the review fixes. The door
// line for the Test first files names operator decision 20 (the backticked paths
// right after File:), the workflow put-back names operator decision 23, the
// engine-provenance entry states the tier rank instead of "on a severity tie",
// Known issues name the default path's verdict gate (operator decision 25), and
// the intro's measurement names the second remediation's Step 8 commit. Word
// boundaries keep "operator decision 2026-10-12" from counting as decision 20.
test('CHANGELOG: the door lines name operator decisions 20 and 23, and none says "every test file on the Test first File line"', () => {
const body = unreleasedBody();
const texts = releaseCheck.parseDoors(body).entries.map((e) => e.text);
assert.ok(texts.some((t) => /\boperator decision 20\b/.test(t)),
`a door line names operator decision 20: ${JSON.stringify(texts)}`);
assert.ok(texts.some((t) => /\boperator decision 23\b/.test(t)),
`a door line names operator decision 23: ${JSON.stringify(texts)}`);
assert.ok(!texts.some((t) => /every test file on the Test first File line/.test(t)),
`no door line still says "every test file on the Test first File line": ${JSON.stringify(texts)}`);
});
test('CHANGELOG: the Unreleased section no longer says "on a severity tie"', () => {
const body = unreleasedBody();
assert.doesNotMatch(body, /on\s+a\s+severity\s+tie/);
});
test('CHANGELOG: Known issues name the verdict gate, the engine payloads and operator decision 25', () => {
const body = unreleasedBody();
const known = knownIssues(body);
assert.match(known, /verdict\s+gate/, `Known issues name the verdict gate: ${JSON.stringify(known)}`);
assert.match(known, /engine\s+payloads/, `Known issues name the engine payloads: ${JSON.stringify(known)}`);
assert.match(known, /\boperator\s+decision\s+25\b/, `Known issues name operator decision 25: ${JSON.stringify(known)}`);
});
test('CHANGELOG: the intro names the third remediation\'s Step 2 commit with its measured suite, and no longer acd0e3d, 5b94952 or the Step 13 commit', () => {
const body = unreleasedBody();
const firstSub = body.search(/^### /m);
const intro = firstSub < 0 ? body : body.slice(0, firstSub);
assert.ok(!intro.includes('acd0e3d'), `the intro no longer names acd0e3d: ${JSON.stringify(intro)}`);
assert.ok(!intro.includes('5b94952'), `the intro no longer names 5b94952: ${JSON.stringify(intro)}`);
assert.doesNotMatch(intro, /the\s+remediation's\s+Step\s+13\s+commit/,
`the intro no longer names "the remediation's Step 13 commit": ${JSON.stringify(intro)}`);
assert.match(intro, /the\s+third\s+remediation's\s+Step\s+2\s+commit/,
`the intro names "the third remediation's Step 2 commit": ${JSON.stringify(intro)}`);
assert.doesNotMatch(intro, /the\s+second\s+remediation's\s+Step\s+8\s+commit/,
`the intro no longer names "the second remediation's Step 8 commit": ${JSON.stringify(intro)}`);
// Measured by `npm test` at 829da48, the third remediation's Step 2 commit.
assert.match(intro, /Suite\s+2517\s+\(2515\s+pass,\s+0\s+fail,\s+2\s+skipped\)/,
`the intro names the measured suite "Suite 2517 (2515 pass, 0 fail, 2 skipped)": ${JSON.stringify(intro)}`);
assert.ok(intro.includes('(`829da48`)'), `the intro names the backticked short sha 829da48: ${JSON.stringify(intro)}`);
});
// Second remediation, Step 9: the CHANGELOG records round 2. Door lines name
// operator decisions 29, 30 and 31; the Fixed bullets that name `b4fa1927`,
// `6c31212c` and `cf241526` follow decisions 29, 31 and 30; three new Fixed
// bullets name `08deae68`, `edf2ec56` and `7961a700` (operator decision 27).
// Each bullet is read on its own with bullet(): the CHANGELOG's bullets have no
// blank line between them. Phrases are looked for with runs of whitespace
// read as one space, so a phrase wrapped over two lines still counts.
const oneSpace = (text) => text.replace(/\s+/g, ' ');
test('CHANGELOG: door lines name operator decisions 29, 30 and 31, none bad, and the decision-29 line says "takes its first wrapped line"', () => {
const body = unreleasedBody();
const d = releaseCheck.parseDoors(body);
assert.deepEqual(d.bad, [], 'every top-level door line reads "- one-way: ..." or "- two-way: ..."');
const texts = d.entries.map((e) => e.text);
for (const re of [/\boperator decision 29\b/, /\boperator decision 30\b/, /\boperator decision 31\b/]) {
assert.ok(texts.some((t) => re.test(t)), `a door line matches ${re}: ${JSON.stringify(texts)}`);
}
const d29 = texts.filter((t) => /\boperator decision 29\b/.test(t));
assert.ok(d29.some((t) => /takes\s+its\s+first\s+wrapped\s+line/.test(t)),
`the door line naming operator decision 29 says "takes its first wrapped line": ${JSON.stringify(d29)}`);
});
test('CHANGELOG: the section names 08deae68, edf2ec56 and operator decision 27', () => {
const body = unreleasedBody();
assert.ok(body.includes('08deae68'), 'the Unreleased section names 08deae68');
assert.ok(body.includes('edf2ec56'), 'the Unreleased section names edf2ec56');
assert.match(body, /\boperator decision 27\b/, 'the Unreleased section names operator decision 27');
});
test('CHANGELOG: the b4fa1927 bullet follows operator decision 29', () => {
const body = unreleasedBody();
const b = bullet(body, /b4fa1927/, 'Fixed bullet b4fa1927');
assert.match(b, /\boperator decision 29\b/, `the b4fa1927 bullet names operator decision 29: ${JSON.stringify(b)}`);
assert.ok(b.includes('4b6460e5'), `the b4fa1927 bullet names 4b6460e5: ${JSON.stringify(b)}`);
assert.match(b, /only\s+after\s+a\s+comma/, `the b4fa1927 bullet says "only after a comma": ${JSON.stringify(b)}`);
assert.match(b, /takes\s+its\s+first\s+wrapped\s+line/, `the b4fa1927 bullet says "takes its first wrapped line": ${JSON.stringify(b)}`);
const flat = oneSpace(b);
assert.ok(!flat.includes('a wrapped line joins it'), `the b4fa1927 bullet no longer says "a wrapped line joins it": ${JSON.stringify(b)}`);
assert.ok(!flat.includes('keeps the paths on its wrapped lines'),
`the b4fa1927 bullet no longer says "keeps the paths on its wrapped lines": ${JSON.stringify(b)}`);
});
test('CHANGELOG: the 6c31212c bullet names operator decision 31 below 1.9 and keeps decision 20 at every plan_version', () => {
const body = unreleasedBody();
const b = bullet(body, /6c31212c/, 'Fixed bullet 6c31212c');
assert.match(b, /\boperator decision 31\b/, `the 6c31212c bullet names operator decision 31: ${JSON.stringify(b)}`);
assert.match(b, /\bplus\b/, `the 6c31212c bullet names plus: ${JSON.stringify(b)}`);
assert.match(b, /separated\s+only\s+by\s+commas\s+or\s+spaces,\s+and\s+below\s+plan_version\s+1\.9/,
`the 6c31212c bullet qualifies "separated only by commas or spaces" below 1.9: ${JSON.stringify(b)}`);
assert.match(b, /at\s+every\s+plan_version\s+\(operator\s+decision\s+20\b/,
`the 6c31212c bullet keeps "at every plan_version (operator decision 20": ${JSON.stringify(b)}`);
});
test('CHANGELOG: the edf2ec56, 08deae68 and 7961a700 bullets say what was fixed', () => {
const body = unreleasedBody();
const e = bullet(body, /edf2ec56/, 'Fixed bullet edf2ec56');
assert.match(e, /allow_blocked_by/, `the edf2ec56 bullet names allow_blocked_by: ${JSON.stringify(e)}`);
assert.match(e, /catalogue\s+tier\s+or\s+higher/, `the edf2ec56 bullet says "catalogue tier or higher": ${JSON.stringify(e)}`);
const o = bullet(body, /08deae68/, 'Fixed bullet 08deae68');
assert.match(o, /own\s+severity/, `the 08deae68 bullet says "own severity": ${JSON.stringify(o)}`);
assert.match(o, /pass1_raised_from/, `the 08deae68 bullet names pass1_raised_from: ${JSON.stringify(o)}`);
const k = bullet(body, /7961a700/, 'Fixed bullet 7961a700');
assert.match(k, /\boperator decision 27\b/, `the 7961a700 bullet names operator decision 27: ${JSON.stringify(k)}`);
});
test('CHANGELOG: the cf241526 bullet follows operator decision 30', () => {
const body = unreleasedBody();
const b = bullet(body, /cf241526/, 'Fixed bullet cf241526');
assert.match(b, /\boperator decision 30\b/, `the cf241526 bullet names operator decision 30: ${JSON.stringify(b)}`);
assert.match(b, /pass\s+no\s+`--tests`/, `the cf241526 bullet says the callers pass no \`--tests\`: ${JSON.stringify(b)}`);
assert.match(b, /decision\s+20\s+and,\s+below\s+1\.9,\s+decision\s+31/,
`the cf241526 bullet says "decision 20 and, below 1.9, decision 31": ${JSON.stringify(b)}`);
assert.match(b, /printed\s+list/, `the cf241526 bullet names the printed list: ${JSON.stringify(b)}`);
assert.ok(b.includes('plan-unreadable'), `the cf241526 bullet names plan-unreadable: ${JSON.stringify(b)}`);
assert.ok(b.includes('tests-mismatch'), `the cf241526 bullet names tests-mismatch: ${JSON.stringify(b)}`);
assert.doesNotMatch(b, /as\s+decision\s+20\s+sets\s+them/, `the cf241526 bullet no longer says "as decision 20 sets them": ${JSON.stringify(b)}`);
});
test('CHANGELOG: the old rerun wording is gone from the Unreleased section', () => {
const body = unreleasedBody();
const flat = oneSpace(body);
for (const phrase of ['reruns red proof once with `plan_tests`', 'reruns the same way', 'the list red proof accepted']) {
assert.ok(!flat.includes(phrase), `the Unreleased section no longer says "${phrase}"`);
}
});
// Third remediation, Step 3: the CHANGELOG records round 3. A door line names
// operator decision 41 (below 1.8 the helper's own files go to red proof); the
// Fixed bullets name `703d87e1` and `7b3b59ed`; Known issues name each finding
// operator decision 42 defers, and the accepted deviation of operator decision
// 38. The Known-issues bullets are read from that slice only (knownIssues
// above), so the round-1 Fixed bullet that names `4b6460e5` never counts.
const fixedSlice = (body) => subsection(body, 'Fixed');
test('CHANGELOG: round 3 — the decision-41 door line, two Fixed bullets and the findings deferred by decision 42', () => {
const body = unreleasedBody();
const d = releaseCheck.parseDoors(body);
assert.deepEqual(d.bad, [], 'every top-level door line reads "- one-way: ..." or "- two-way: ..."');
const texts = d.entries.map((e) => e.text);
assert.ok(texts.some((t) => /\boperator decision 41\b/.test(t)),
`a door line names operator decision 41: ${JSON.stringify(texts)}`);
const fixed = fixedSlice(body);
const f703 = bullet(fixed, /703d87e1/, 'Fixed bullet 703d87e1');
assert.ok(f703.includes('tests/helpers/timing.mjs'), `the 703d87e1 bullet names tests/helpers/timing.mjs: ${JSON.stringify(f703)}`);
const f7b3 = bullet(fixed, /7b3b59ed/, 'Fixed bullet 7b3b59ed');
assert.match(f7b3, /\boperator\s+decision\s+41\b/, `the 7b3b59ed bullet names operator decision 41: ${JSON.stringify(f7b3)}`);
const known = knownIssues(body);
for (const id of ['2d624b54', 'c06d5385', '6ff82e75', '4b6460e5', '4e1607cb', '4f22b101']) {
const b = bullet(known, new RegExp(id), `Known-issues bullet ${id}`);
assert.match(b, /\boperator\s+decision\s+42\b/, `the ${id} Known-issues bullet names operator decision 42: ${JSON.stringify(b)}`);
}
const dev = bullet(known, /4dce7c74/, 'Known-issues bullet 4dce7c74');
assert.match(dev, /\boperator\s+decision\s+38\b/, `the 4dce7c74 Known-issues bullet names operator decision 38: ${JSON.stringify(dev)}`);
const c06 = bullet(known, /c06d5385/, 'Known-issues bullet c06d5385');
assert.ok(c06.includes('e031a480'), `the c06d5385 bullet names e031a480: ${JSON.stringify(c06)}`);
assert.ok(c06.includes('original_severity'), `the c06d5385 bullet names original_severity: ${JSON.stringify(c06)}`);
});

225
tests/lib/red-proof.test.mjs
View file

@ -13,6 +13,8 @@ import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { proveRed } from '../../lib/verification/red-proof.mjs';
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
@ -51,8 +53,9 @@ function addTest(dir) {
writeFileSync(join(dir, 'tests', 'sum.test.mjs'), SUM_TEST);
}
/** The CLI with the child environment the suite requires (tests/helpers/child-env.mjs). */
function cli(args) {
const r = spawnSync(process.execPath, [CLI, ...args], { encoding: 'utf8' });
const r = spawnSync(process.execPath, [CLI, ...args], { encoding: 'utf8', env: childEnv() });
return { status: r.status, stdout: r.stdout, stderr: r.stderr };
}
@ -269,3 +272,223 @@ test('red proof: reproduces the 02243c6 red-first measurement (6 of 55) and the
rmSync(tar, { force: true });
}
});
// Review cf241526 (operator decision 20): with --plan and --step, the test list
// is checked against the step's `Test first:` File line, and read from it when
// --tests is not given. An empty plan list (a 1.7 step without `Test first:`, a
// 1.8 File line with no backticked path right after the marker) changes nothing.
const OTHER_TEST = [
"import { test } from 'node:test';",
"test('other', () => {});",
'',
].join('\n');
/** A plan whose Step 1 changes lib/sum.mjs; `opts` overrides Step 1's builder options. */
function planFile(dir, opts = {}, version = '1.8') {
const p = join(dir, 'plan.local.md');
writeFileSync(p, buildPlan({ version, steps: [
{ file: 'lib/sum.mjs', test: 'tests/sum.test.mjs', ...opts },
{ endToEnd: 'bash tests/e2e.sh' },
] }));
return p;
}
test('red proof --plan/--step: a --tests that differs from the step\'s Test first File line is not-run, exit 2 (tests-mismatch)', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
writeFileSync(join(dir, 'tests', 'other.test.mjs'), OTHER_TEST);
const plan = planFile(dir);
for (const given of ['tests/other.test.mjs', 'tests/sum.test.mjs,tests/other.test.mjs']) {
for (const mode of [['--red-only'], []]) {
const c = cli([...mode, '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', given, '--plan', plan, '--step', '1', '--json']);
assert.equal(c.status, 2, `${given} ${mode.join(' ')}: ${c.stdout}${c.stderr}`);
const out = JSON.parse(c.stdout);
assert.equal(out.status, 'not-run', JSON.stringify(out));
assert.match(out.reason, /^tests-mismatch/, JSON.stringify(out));
assert.deepEqual(out.plan_tests, ['tests/sum.test.mjs']);
}
}
// The API says the same.
const r = proveRed({ cwd: dir, base, cmd: TEST_CMD, tests: ['tests/other.test.mjs'], redOnly: true, plan, step: 1 });
assert.equal(r.status, 'not-run', JSON.stringify(r));
assert.match(r.reason, /^tests-mismatch/);
assert.deepEqual(r.plan_tests, ['tests/sum.test.mjs']);
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof --plan/--step without --tests: the step\'s Test first file is the test list', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir); // test written, nothing built yet: red at the start point
const plan = planFile(dir);
const red = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--plan', plan, '--step', '1', '--json']);
assert.equal(red.status, 0, red.stdout + red.stderr);
assert.equal(JSON.parse(red.stdout).status, 'proven');
// Built: the full run proves red, green and the control from the plan's list alone.
writeFileSync(join(dir, 'lib', 'sum.mjs'), 'export function sum(a, b) { return a + b; }\n');
const full = cli(['--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--plan', plan, '--step', '1', '--json']);
assert.equal(full.status, 0, full.stdout + full.stderr);
assert.equal(JSON.parse(full.stdout).status, 'proven');
// A step whose plan list is empty still needs --tests.
const none = planFile(dir, { testFirst: false }, '1.7');
const empty = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--plan', none, '--step', '1', '--json']);
assert.equal(empty.status, 2, empty.stdout + empty.stderr);
assert.match(JSON.parse(empty.stdout).reason, /no test files given/);
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof --plan/--step: a --tests equal to the plan\'s list (normalised) behaves as before', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
const plan = planFile(dir);
for (const given of ['tests/sum.test.mjs', './tests/sum.test.mjs']) {
const before = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', given, '--json']);
const withPlan = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', given, '--plan', plan, '--step', '1', '--json']);
assert.equal(withPlan.status, 0, withPlan.stdout + withPlan.stderr);
assert.equal(withPlan.status, before.status);
const a = JSON.parse(withPlan.stdout);
const b = JSON.parse(before.stdout);
assert.equal(a.status, 'proven');
assert.equal(a.reason, b.reason);
assert.equal('plan_tests' in a, false, JSON.stringify(a));
}
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof --plan/--step: a 1.8 File line naming a source in its description lists only the test (decision 20)', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
// Built: were lib/sum.mjs on the list, it would be copied into the start
// point and the start point would go green (not-red-at-base).
writeFileSync(join(dir, 'lib', 'sum.mjs'), 'export function sum(a, b) { return a + b; }\n');
const plan = planFile(dir, { testFirstRaw: ' - File: `tests/sum.test.mjs` (existing) - covers `lib/sum.mjs`' });
const derived = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--plan', plan, '--step', '1', '--json']);
assert.equal(derived.status, 0, derived.stdout + derived.stderr);
assert.equal(JSON.parse(derived.stdout).status, 'proven');
const given = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/sum.test.mjs', '--plan', plan, '--step', '1', '--json']);
assert.equal(given.status, 0, given.stdout + given.stderr);
assert.doesNotMatch(JSON.parse(given.stdout).reason, /tests-mismatch/);
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof --plan/--step: an empty plan list (1.7 without Test first, 1.8 File line without a backticked path) changes nothing', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
writeFileSync(join(dir, 'tests', 'other.test.mjs'), OTHER_TEST);
// planFile writes one path: read each plan's text before the next one is written.
const plans = [];
plans.push(readFileSync(planFile(dir, { testFirst: false }, '1.7'), 'utf8'));
plans.push(readFileSync(planFile(dir, { testFirstRaw: ' - File: see `tests/sum.test.mjs` for the test' }, '1.8'), 'utf8'));
const args = ['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/other.test.mjs', '--json'];
const before = cli(args);
const b = JSON.parse(before.stdout);
for (const content of plans) {
const p = join(dir, 'plan.local.md');
writeFileSync(p, content);
const c = cli([...args, '--plan', p, '--step', '1']);
assert.equal(c.status, before.status, c.stdout + c.stderr);
const out = JSON.parse(c.stdout);
assert.doesNotMatch(out.reason, /tests-mismatch/, JSON.stringify(out));
assert.equal(out.status, b.status);
assert.equal(out.reason, b.reason);
assert.equal('plan_tests' in out, false);
}
} finally { rmSync(dir, { recursive: true, force: true }); }
});
// Operator decision 30 (14.10; reviews 793fc1aa, 09ae46d8, d01df508, 11ee78c5):
// with --plan and --step, red proof reads the step's test files from the plan,
// prints the list it ran (`tests`, and a ` tests:` line in text mode), and a
// plan or step it cannot read is not-run (`plan-unreadable`), --tests or not.
test('red proof --plan: a plan that cannot be read is not-run, exit 2 (plan-unreadable), with and without --tests', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir); // red at the start point: a missing plan must not read as proven
const cases = [
[join(dir, 'nope.md'), '1', /^plan-unreadable: cannot read the plan: /],
// A readable plan without the step named.
[planFile(dir), '9', /^plan-unreadable: no "### Step 9:" in the plan$/],
];
for (const [plan, step, reason] of cases) {
for (const given of [[], ['--tests', 'tests/sum.test.mjs']]) {
const c = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, ...given, '--plan', plan, '--step', step, '--json']);
assert.equal(c.status, 2, `${given.join(' ')}: ${c.stdout}${c.stderr}`);
const out = JSON.parse(c.stdout);
assert.equal(out.status, 'not-run', JSON.stringify(out));
assert.equal(typeof out.reason, 'string', JSON.stringify(out));
assert.match(out.reason, reason, JSON.stringify(out));
}
}
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof --plan: a green start point with a missing plan and --tests is exit 2 plan-unreadable, not exit 1', () => {
const { dir, base } = repo('a + b');
try {
addTest(dir);
const missing = join(dir, 'nope.md');
for (const mode of [['--red-only'], []]) {
const c = cli([...mode, '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/sum.test.mjs', '--plan', missing, '--step', '1', '--json']);
assert.equal(c.status, 2, `${mode.join(' ')}: ${c.stdout}${c.stderr}`);
const out = JSON.parse(c.stdout);
assert.equal(out.status, 'not-run', JSON.stringify(out));
assert.equal(typeof out.reason, 'string', JSON.stringify(out));
assert.match(out.reason, /^plan-unreadable/, JSON.stringify(out));
}
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof --plan/--step, the documented call (no --tests): the result names the list it ran in `tests`, and text mode prints it', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
const plan = planFile(dir);
const args = ['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--plan', plan, '--step', '1'];
const json = cli([...args, '--json']);
assert.equal(json.status, 0, json.stdout + json.stderr);
const out = JSON.parse(json.stdout);
assert.equal(out.status, 'proven', JSON.stringify(out));
assert.deepEqual(out.tests, ['tests/sum.test.mjs'], JSON.stringify(out));
const text = cli(args);
assert.equal(text.status, 0, text.stdout + text.stderr);
assert.equal(typeof text.stdout, 'string');
assert.match(text.stdout, /^ {2}tests: tests\/sum\.test\.mjs$/m, text.stdout);
// Built: the full run on a red start point and a green result names the same list.
writeFileSync(join(dir, 'lib', 'sum.mjs'), 'export function sum(a, b) { return a + b; }\n');
const full = cli(['--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--plan', plan, '--step', '1', '--json']);
assert.equal(full.status, 0, full.stdout + full.stderr);
const f = JSON.parse(full.stdout);
assert.equal(f.status, 'proven', JSON.stringify(f));
assert.deepEqual(f.tests, ['tests/sum.test.mjs'], JSON.stringify(f));
} finally { rmSync(dir, { recursive: true, force: true }); }
// The full run on a green start point: rejected, with the same list.
const green = repo('a + b');
try {
addTest(green.dir);
const plan = planFile(green.dir);
const c = cli(['--cwd', green.dir, '--base', green.base, '--cmd', TEST_CMD, '--plan', plan, '--step', '1', '--json']);
assert.equal(c.status, 1, c.stdout + c.stderr);
const out = JSON.parse(c.stdout);
assert.equal(out.reason, 'not-red-at-base', JSON.stringify(out));
assert.deepEqual(out.tests, ['tests/sum.test.mjs'], JSON.stringify(out));
} finally { rmSync(green.dir, { recursive: true, force: true }); }
});
test('red proof --plan/--step: a --tests naming another test is exit 2, and text mode prints the plan\'s list on a plan_tests line', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
writeFileSync(join(dir, 'tests', 'other.test.mjs'), OTHER_TEST);
const plan = planFile(dir);
const c = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/other.test.mjs', '--plan', plan, '--step', '1']);
assert.equal(c.status, 2, c.stdout + c.stderr);
assert.equal(typeof c.stdout, 'string');
assert.match(c.stdout, /^ {2}plan_tests: tests\/sum\.test\.mjs$/m, c.stdout);
} finally { rmSync(dir, { recursive: true, force: true }); }
});

3
tests/lib/research-loop-cap.test.mjs
View file

@ -22,6 +22,7 @@ import {
MAX_CONV_TURNS,
MAX_TOTAL_DIMENSIONS,
} from '../../lib/util/research-loop-cap.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const SHIM = join(HERE, '..', '..', 'lib', 'util', 'research-loop-cap.mjs');
@ -40,7 +41,7 @@ function runShim(args, env) {
const out = execFileSync(process.execPath, [SHIM, ...args], {
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'pipe'],
env: { ...process.env, ...env },
env: childEnv(env),
});
return { code: 0, out };
} catch (e) {

2
tests/lib/retterunde-prose-pins.test.mjs
View file

@ -63,7 +63,7 @@ test('1b: plan-critic has it as a finding under Verification gaps, and still 12
test('1c: the executor runs red-proof with --plan/--step and sends a tests_only not-red-at-base up to the plan', () => {
const ex = read('commands/trekexecute.md');
assert.match(ex, /red-proof\.mjs --base \{start_sha\} --cmd "\{Verify command\}" --tests [^\n]*--plan "\{plan_path\}" --step \{N\}[^\n]*--json/);
assert.match(ex, /red-proof\.mjs --base \{start_sha\} --cmd "\{Verify command\}" --plan "\{plan_path\}" --step \{N\}[^\n]*--json/);
assert.match(ex, /`not-red-at-base`, `tests_only: true`[^\n]*straight up to the plan \(Sub-step F, `fix_rounds = 4`, stop class 4\)/);
// The helper's gate, as the executor describes it, carries the same flags.
assert.match(ex, /red-proof\.mjs --red-only --base \{start_sha\} [^\n]*--plan "\{plan_path\}" --step \{N\}/);

29
tests/lib/review-630.test.mjs
View file

@ -20,6 +20,8 @@ import { stepTestsOnly } from '../../lib/parsers/plan-schema.mjs';
import { validateFindings } from '../../lib/review/findings-schema.mjs';
import { lintSkill } from '../../lib/skills/skill-lint.mjs';
import * as routing from '../../lib/skills/routing.mjs';
import { childEnv } from '../helpers/child-env.mjs';
import * as timing from '../helpers/timing.mjs';
const awaitImportSync = () => routing;
@ -364,7 +366,7 @@ test('review 630 r3: the plan validator hands its parsed fields to the tests-onl
assert.equal(typeof schema.stepTestsOnlyFromFields, 'function');
const body = '- **Files:** `tests/a.test.mjs`\n- **Changes:** x\n';
assert.deepEqual(schema.stepTestsOnlyFromFields(schema.parseStepFields(body)), stepTestsOnly(body));
assert.match(read('lib/validators/plan-validator.mjs'), /stepTestsOnlyFromFields\(f\)/);
assert.match(read('lib/validators/plan-validator.mjs'), /stepTestsOnlyFromFields\(f, planVersion\)/);
});
test('review 630 r3: skill frontmatter - a value starting with # is a hazard, an escaped backslash closes a quote', async () => {
@ -980,7 +982,7 @@ test('review 630 r8 / red-proof.mjs:101: a run reports one reading of a stop, no
writeFileSync(join(repo, 'tests', 'a.test.mjs'), 'x\n');
const out = spawnSync(process.execPath, [join(ROOT, 'lib', 'verification', 'red-proof.mjs'), '--cwd', repo, '--base', base,
'--cmd', 'jest tests/a.test.mjs', '--tests', 'tests/a.test.mjs', '--red-only', '--json'],
{ encoding: 'utf8', env: { ...process.env, PATH: bin.path }, timeout: 60_000 });
{ encoding: 'utf8', env: childEnv({ PATH: bin.path }), timeout: 60_000 });
const r = JSON.parse(out.stdout);
assert.equal(r.status, 'not-run', JSON.stringify(r));
assert.match(r.reason, /could not start/, r.reason);
@ -1158,7 +1160,17 @@ test('review 630 r9 / 7b3750e9 plan-schema.mjs:372: a plain path that ends with
assert.deepEqual(stepTestsOnly(step('`tests/a.test.mjs`, note: Makefile and package.json')).files, ['tests/a.test.mjs', 'package.json']);
});
test('review 630 r9 / brief note 3: red proof reports a control run it stopped as not-run, in the control run', async () => {
// Review 703d87e1: under full-suite load the criteria runner saw this test's
// start-point run killed at the old 2 s timeout ("at the start point"), not the
// control run. Its numbers live in tests/helpers/timing.mjs (CONTROL_RUN): the
// timeout leaves headroom over a slow start, the hang outlasts it. Every run
// but the control starts slowly on purpose (slowStartMs), so a timeout without
// headroom fails here on demand, not only under load. BEFORE_703D87E1 is the
// record of the numbers before the fix, taken only when the export is absent.
const BEFORE_703D87E1 = Object.freeze({ timeoutMs: 2000, hangMs: 6000, slowStartMs: 2500 });
test('review 630 r9 / brief note 3 + 703d87e1: red proof reports a control run it stopped as not-run, in the control run, when every other run starts slowly', async () => {
const c = timing.CONTROL_RUN ?? BEFORE_703D87E1;
const { proveRed } = await import('../../lib/verification/red-proof.mjs');
const repo = gitRepo('r630-r9-control-');
try {
@ -1167,18 +1179,21 @@ test('review 630 r9 / brief note 3: red proof reports a control run it stopped a
mkdirSync(join(repo, 'tests'));
mkdirSync(join(repo, 'src'));
// Red at the start point (no src/impl.mjs), green in the working tree, and
// it hangs only in the control: the start-point copy with the change laid on.
// it hangs only in the control: the start-point copy with the change laid
// on. Every other run waits slowStartMs before its one test.
writeFileSync(join(repo, 'tests', 'a.test.mjs'), [
"import { test } from 'node:test';",
"import { existsSync } from 'node:fs';",
"const control = process.cwd().includes('voyage-red-proof-') && existsSync('src/impl.mjs');",
`if (!control) await new Promise((r) => setTimeout(r, ${c.slowStartMs}));`,
"test('impl', async () => {",
" if (!existsSync('src/impl.mjs')) throw new Error('no impl');",
" if (process.cwd().includes('voyage-red-proof-')) await new Promise((r) => setTimeout(r, 6000));",
` if (control) await new Promise((r) => setTimeout(r, ${c.hangMs}));`,
'});', ''].join('\n'));
writeFileSync(join(repo, 'src', 'impl.mjs'), 'export const x = 1;\n');
const r = proveRed({ cwd: repo, base, cmd: 'node --test tests/a.test.mjs', tests: ['tests/a.test.mjs'], timeoutMs: 2000 });
const r = proveRed({ cwd: repo, base, cmd: 'node --test tests/a.test.mjs', tests: ['tests/a.test.mjs'], timeoutMs: c.timeoutMs });
assert.equal(r.status, 'not-run', JSON.stringify(r));
assert.match(r.reason, /^in the control run: the command was killed .*after the 2 s timeout/, r.reason);
assert.match(r.reason, new RegExp(`^in the control run: the command was killed .*after the ${c.timeoutMs / 1000} s timeout`), r.reason);
assert.equal(r.base.exit, 1, 'red at the start point');
assert.equal(r.result.exit, 0, 'green at the result');
assert.ok(r.control, 'the control run is reported');

7
tests/lib/review-prose-pins.test.mjs
View file

@ -179,8 +179,11 @@ test('trekreview — the write-outside-project_dir exception names only the mark
test('armB workflow — its inlined rule-key enum equals the catalogue (no drift)', () => {
const src = read('scripts/trekreview-armB.workflow.mjs');
const block = src.slice(src.indexOf('const RULE_KEYS = ['), src.indexOf('];', src.indexOf('const RULE_KEYS = [')));
const keys = [...block.matchAll(/'([A-Z_]+)'/g)].map((m) => m[1]).sort();
assert.ok(src.includes('const RULE_KEYS = Object.keys(RULE_TIERS);'), 'the enum is the keys of the inlined RULE_TIERS');
const start = src.indexOf('const RULE_TIERS = {');
assert.ok(start !== -1, 'the script carries a RULE_TIERS literal block');
const block = src.slice(start, src.indexOf('};', start));
const keys = [...block.matchAll(/([A-Z_]+):\s*'[A-Z]+'/g)].map((m) => m[1]).sort();
assert.deepEqual(keys, [...RULE_KEYS].sort());
});

228
tests/lib/session-id-isolation.test.mjs Normal file
View file

@ -0,0 +1,228 @@
// tests/lib/session-id-isolation.test.mjs
// The test suite must not run under the live session's id. The lock CLI
// (lib/util/test-lock.mjs) writes a session pointer keyed by
// CLAUDE_CODE_SESSION_ID into <tmpdir>/voyage-test-locks-<uid>/, and the stop
// guard reads it; a test child that inherits the id of the session running
// `npm test` writes a pointer for that live session (S133). Two layers keep
// the id out:
// - `npm test` preloads tests/helpers/no-session-id.mjs, which deletes the
// id from the runner's environment before any test file starts;
// - every test that starts a child builds the child's environment with
// childEnv() (tests/helpers/child-env.mjs), which drops the id, so a
// single file run directly (`node --test <file>`) does not leak it either.
//
// Parts:
// (a) static scan: no file under tests/ spreads the process environment
// into a child, except the helper itself;
// (b) childEnv() drops an inherited id and keeps an id the caller sets;
// (c) the test command holds the preload, and the preload clears the id;
// (d) every test file that drives the lock CLI or the stop guard runs green,
// through the test command's preload and directly, and leaves no
// pointer for a made-up id;
// (e) known-positive control: the lock CLI run with that id does write the
// pointer at the computed path, so (d) cannot pass on a wrong path.
//
// Every child environment here is built without a spread (Object.entries +
// filter), so this file does not depend on the helper and does not trip (a).
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawn, spawnSync } from 'node:child_process';
import { readFileSync, readdirSync, existsSync, mkdtempSync, mkdirSync, writeFileSync, rmSync, realpathSync } from 'node:fs';
import { join, dirname, relative, sep } from 'node:path';
import { tmpdir } from 'node:os';
import { createHash, randomUUID } from 'node:crypto';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { makeRepo } from '../hooks/guard-helpers.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const SELF = 'tests/lib/session-id-isolation.test.mjs';
const HELPER = 'tests/helpers/child-env.mjs';
const PRELOAD = 'tests/helpers/no-session-id.mjs';
const PRELOAD_FLAG = `--import ./${PRELOAD}`;
const LOCK_CLI = join(ROOT, 'lib', 'util', 'test-lock.mjs');
const ID_VAR = 'CLAUDE_CODE_SESSION_ID';
const TIMEOUT = 300_000;
// Built from parts so this file's own text never matches them.
const PENV = 'process' + '\\.env(?![\\w$.])';
const SPREAD_PATTERNS = [
['spread', new RegExp('\\.\\.\\.\\s*' + PENV)],
['env-property', new RegExp('\\benv\\s*:\\s*' + PENV)],
['object-assign', new RegExp('Object\\.assign\\([^)]*' + PENV)],
];
function walk(dir, keep) {
const out = [];
for (const e of readdirSync(dir, { withFileTypes: true })) {
const p = join(dir, e.name);
if (e.isDirectory()) out.push(...walk(p, keep));
else if (keep(e.name)) out.push(relative(ROOT, p).split(sep).join('/'));
}
return out.sort();
}
const madeUpId = () => `isolation-${randomUUID()}`;
// The pointer path, as lib/util/test-lock.mjs:76-78 computes it.
function pointerPath(tmp, id) {
const uid = process.getuid ? process.getuid() : 'u';
const name = `${createHash('sha256').update(String(id)).digest('hex').slice(0, 32)}.json`;
return join(tmp, `voyage-test-locks-${uid}`, name);
}
// The child's environment: ours without NODE_TEST_CONTEXT (so a nested
// `node --test` reports its own exit code) and without the live id, plus a
// made-up id and a private TMPDIR.
function isolatedEnv(id, tmp) {
const env = Object.fromEntries(Object.entries(process.env)
.filter(([k]) => k !== 'NODE_TEST_CONTEXT' && k !== ID_VAR));
env[ID_VAR] = id;
env.TMPDIR = tmp;
return env;
}
const privateTmp = () => realpathSync(mkdtempSync(join(tmpdir(), 'voyage-sid-')));
function testScript() {
return JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf8')).scripts?.test ?? '';
}
// The node flags the test command puts before --test (the preload).
function preloadFlags() {
const words = testScript().split(/\s+/).filter(Boolean);
const start = words.indexOf('node');
const end = words.indexOf('--test');
if (start < 0 || end < 0 || end < start) return [];
return words.slice(start + 1, end);
}
test('(a) no test file spreads the process environment into a child, except the helper', () => {
const files = walk(join(ROOT, 'tests'), (n) => n.endsWith('.mjs'));
assert.ok(files.length >= 140, `the scan saw only ${files.length} .mjs files under tests/; an empty scan proves nothing`);
const hits = [];
for (const rel of files) {
if (rel === HELPER) continue;
const text = readFileSync(join(ROOT, rel), 'utf8');
for (const [kind, re] of SPREAD_PATTERNS) {
if (re.test(text)) hits.push(`${rel} (${kind})`);
}
}
assert.deepEqual(hits, [], `use childEnv() from ${HELPER} instead:\n${hits.join('\n')}`);
});
test('(b) childEnv() drops an inherited id and keeps an id the caller sets', async () => {
let mod;
try {
mod = await import(pathToFileURL(join(ROOT, HELPER)).href);
} catch (e) {
// Only the error's name: its message would read as a load crash to red-proof.
assert.fail(`child-env helper missing: ${HELPER} could not be loaded (${e.name})`);
}
assert.equal(typeof mod.childEnv, 'function', 'child-env helper missing: no childEnv export');
const had = Object.prototype.hasOwnProperty.call(process.env, ID_VAR);
const saved = process.env[ID_VAR];
const inherited = madeUpId();
process.env[ID_VAR] = inherited;
try {
const plain = mod.childEnv();
assert.equal(Object.prototype.hasOwnProperty.call(plain, ID_VAR), false, 'an inherited id must not reach the child');
assert.equal(plain.PATH, process.env.PATH, 'the rest of the environment is kept');
plain.VOYAGE_SID_PROBE = 'x';
assert.equal(process.env.VOYAGE_SID_PROBE, undefined, 'childEnv() returns a copy, not process.env');
const own = mod.childEnv({ [ID_VAR]: 'sess-own', VOYAGE_SID_EXTRA: 'y' });
assert.equal(own[ID_VAR], 'sess-own', 'an id the test sets itself is kept');
assert.equal(own.VOYAGE_SID_EXTRA, 'y', 'overrides are applied');
assert.equal(mod.childEnv({ PATH: '/usr/bin:/bin' }).PATH, '/usr/bin:/bin', 'overrides win over the inherited value');
} finally {
if (had) process.env[ID_VAR] = saved;
else delete process.env[ID_VAR];
}
});
test('(c) the test command preloads the helper, and the preload clears the id', () => {
assert.ok(testScript().includes(PRELOAD_FLAG),
`package.json scripts.test must hold "${PRELOAD_FLAG}"; it is: ${testScript()}`);
const tmp = privateTmp();
try {
const r = spawnSync(process.execPath,
['--import', `./${PRELOAD}`, '-e', `process.stdout.write(JSON.stringify(process.env.${ID_VAR} ?? null))`],
{ cwd: ROOT, encoding: 'utf8', env: isolatedEnv(madeUpId(), tmp) });
assert.equal(r.status, 0, `the preload child did not run: ${r.stderr}`);
assert.equal(r.stdout.trim(), 'null', 'the preload must delete the id');
} finally {
rmSync(tmp, { recursive: true, force: true });
}
});
// Files that name the lock CLI or the stop guard but do not drive either.
// Each must still exist and still match, so the list stays honest.
const NOT_DRIVERS = new Map([
['tests/lib/review-630.test.mjs', 'names the lock CLI only in source-reading assertions (:502, :952-964)'],
['tests/lib/skills-workshop-prose-pins.test.mjs', 'prose pin: checks the skill names the CLI (:85)'],
]);
const DRIVER_RE = /test-lock\.mjs|stop-test-lock/;
// One nested `node --test` run of `rel`, with its own made-up id and private
// TMPDIR. Resolves to the problems it shows.
function runDriver(rel, mode, pre) {
const id = madeUpId();
const tmp = privateTmp();
return new Promise((resolve) => {
const child = spawn(process.execPath, [...pre, '--test', '--test-reporter=tap', rel],
{ cwd: ROOT, env: isolatedEnv(id, tmp), timeout: TIMEOUT });
let out = '';
child.stdout.on('data', (d) => { out += d; });
child.stderr.on('data', (d) => { out += d; });
child.on('close', (status, signal) => {
const problems = [];
const m = out.match(/^# tests (\d+)$/m);
if (status !== 0) problems.push(`${rel} [${mode}]: exit ${status}${signal ? ` (${signal})` : ''}`);
if (!m || Number(m[1]) === 0) problems.push(`${rel} [${mode}]: reported no tests`);
if (existsSync(pointerPath(tmp, id))) problems.push(`${rel} [${mode}]: left a session pointer for the made-up id`);
rmSync(tmp, { recursive: true, force: true });
resolve(problems);
});
});
}
test('(d) the lock-driving test files run green and leave no pointer for a made-up id', { timeout: TIMEOUT }, async () => {
const all = walk(join(ROOT, 'tests'), (n) => n.endsWith('.test.mjs')).filter((f) => f !== SELF);
const matching = all.filter((rel) => DRIVER_RE.test(readFileSync(join(ROOT, rel), 'utf8')));
for (const [rel, why] of NOT_DRIVERS) {
assert.ok(existsSync(join(ROOT, rel)), `excluded file is gone, drop it from the list: ${rel} (${why})`);
assert.ok(matching.includes(rel), `excluded file no longer names the lock CLI, drop it from the list: ${rel}`);
}
const drivers = matching.filter((rel) => !NOT_DRIVERS.has(rel));
assert.ok(drivers.length >= 5, `only ${drivers.length} lock-driving test files found: ${drivers.join(', ')}`);
// The files of one mode run side by side (as `node --test` runs files); the
// two modes run one after the other.
const problems = [];
for (const [mode, pre] of [['test command', preloadFlags()], ['direct', []]]) {
const runs = await Promise.all(drivers.map((rel) => runDriver(rel, mode, pre)));
problems.push(...runs.flat());
}
assert.deepEqual(problems, []);
});
test('(e) known-positive control: the lock CLI writes the pointer at the computed path', () => {
const id = madeUpId();
const tmp = privateTmp();
const repo = makeRepo('main');
try {
mkdirSync(join(repo, 'tests'), { recursive: true });
writeFileSync(join(repo, 'tests', 'a.test.js'), 'a\n');
const r = spawnSync(process.execPath, [LOCK_CLI, 'lock', '--files', 'tests/a.test.js', '--step', '1'],
{ cwd: repo, encoding: 'utf8', env: isolatedEnv(id, tmp) });
assert.equal(r.status, 0, r.stderr);
const p = pointerPath(tmp, id);
assert.ok(existsSync(p), `the lock CLI wrote no pointer at ${p}; the path in (d) would be wrong`);
assert.ok(existsSync(JSON.parse(readFileSync(p, 'utf8')).lock_path), 'the pointer names the lock file');
} finally {
rmSync(tmp, { recursive: true, force: true });
rmSync(repo, { recursive: true, force: true });
}
});

7
tests/lib/session-orders.test.mjs
View file

@ -27,6 +27,7 @@ import { fileURLToPath } from 'node:url';
import {
parseExecutionStrategy, buildWaveOrders, progressFileFor, waveStatus,
} from '../../lib/util/session-orders.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
@ -100,7 +101,7 @@ function fill(script, values) {
const BASH = existsSync('/bin/bash') ? '/bin/bash' : 'bash';
function runBash(script, cwd, env = {}) {
return spawnSync(BASH, ['-c', script], { cwd, encoding: 'utf-8', env: { ...process.env, ...env } });
return spawnSync(BASH, ['-c', script], { cwd, encoding: 'utf-8', env: childEnv(env) });
}
function gitc(cwd, ...args) {
@ -204,7 +205,7 @@ test('CLI on the fixture: wave 1 writes two orders and wave.json, prints the sta
const sessionDir = join(root, '.claude', 'trekplan-sessions', 'demo');
const r = spawnSync(process.execPath, [MODULE, 'write', '--plan', PLAN_REL, '--wave', '1', '--session-dir', sessionDir], {
cwd: root, encoding: 'utf-8', env: { ...process.env, PATH: `${bin}:${process.env.PATH}` },
cwd: root, encoding: 'utf-8', env: childEnv({ PATH: `${bin}:${process.env.PATH}` }),
});
assert.equal(r.status, 0, r.stderr);
assert.equal(existsSync(marker), false, 'nothing may start Claude Code');
@ -555,7 +556,7 @@ test('Check 4 and Step 5 pass a first run (no worktrees yet) under zsh as well a
const step5 = fill(bashBlock('### Step 5 — Cleanup'), { slug: 'demo' });
for (const shell of [BASH, ZSH]) {
for (const [name, block] of [['Check 4', check4], ['Step 5', step5]]) {
const r = spawnSync(shell, ['-c', block], { cwd: root, encoding: 'utf-8', env: { ...process.env, ...env } });
const r = spawnSync(shell, ['-c', block], { cwd: root, encoding: 'utf-8', env: childEnv(env) });
assert.equal(r.status, 0, `${name} under ${shell}: ${r.stderr}`);
assert.doesNotMatch(r.stderr, /no matches found/, `${name} under ${shell}`);
}

5
tests/lib/stats-event-emit.test.mjs
View file

@ -21,6 +21,7 @@ import { tmpdir } from 'node:os';
import { fileURLToPath } from 'node:url';
import { emit, buildRecord, resolveStatsPath, KNOWN_EVENTS } from '../../lib/stats/event-emit.mjs';
import { resolveStatsDir } from '../../lib/stats/stats-append.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const SHIM = join(HERE, '..', '..', 'lib', 'stats', 'event-emit.mjs');
@ -36,7 +37,7 @@ const DATA_REL = join('.claude', 'plugins', 'data', 'voyage-ktg-plugin-marketpla
/** process.env with HOME pinned and CLAUDE_PLUGIN_DATA empty: never the real data dir. */
function isolatedEnv(home) {
const env = { ...process.env, CLAUDE_PLUGIN_DATA: '', HOME: home };
const env = childEnv({ CLAUDE_PLUGIN_DATA: '', HOME: home });
delete env.CLAUDE_COORD_DIR;
return env;
}
@ -169,7 +170,7 @@ test('CLI shim writes via emit when CLAUDE_PLUGIN_DATA is set', () => {
execFileSync(process.execPath, [
SHIM, '--event', 'brief-approved', '--payload', '{"foo":42}',
], {
env: { ...process.env, CLAUDE_PLUGIN_DATA: dir },
env: childEnv({ CLAUDE_PLUGIN_DATA: dir }),
encoding: 'utf-8',
});
const path = join(dir, 'trekexecute-stats.jsonl');

3
tests/lib/stats-landing.test.mjs
View file

@ -23,6 +23,7 @@ import { cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symli
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
@ -48,7 +49,7 @@ function appendLine(sec, kind, rel) {
/** process.env without CLAUDE_COORD_DIR: a test never reads the real coord queue. */
function baseEnv() {
const env = { ...process.env };
const env = childEnv();
delete env.CLAUDE_COORD_DIR;
return env;
}

3
tests/lib/step-check.test.mjs
View file

@ -12,11 +12,12 @@ import { spawnSync } from 'node:child_process';
import { writeFileSync, mkdirSync, appendFileSync } from 'node:fs';
import { join } from 'node:path';
import { ROOT, makeRepo, gitIn, fakeTool, tmp, pluginCopy } from '../hooks/guard-helpers.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const CLI = join(ROOT, 'lib', 'verification', 'step-check.mjs');
function run(cli, args, env = {}) {
const r = spawnSync(process.execPath, [cli, ...args], { encoding: 'utf8', env: { ...process.env, PATH: '/usr/bin:/bin', ...env } });
const r = spawnSync(process.execPath, [cli, ...args], { encoding: 'utf8', env: childEnv({ PATH: '/usr/bin:/bin', ...env }) });
let json = null;
try { json = JSON.parse(r.stdout); } catch { json = null; }
return { code: r.status, stdout: r.stdout, stderr: r.stderr, json };

435
tests/lib/test-first-multi-file.test.mjs Normal file
View file

@ -0,0 +1,435 @@
// tests/lib/test-first-multi-file.test.mjs
// Plan step 3 (files-backticks trial 4): below plan_version 1.9 every test file
// on the `Test first:` `File:` line is read, not only the first backticked
// span. 6.3.1 read the first span alone, so a step whose `Files:` were its two
// `Test first:` files was not seen as tests-only when the second file is not a
// test by its path. One reading: testFirstFiles (plan-schema.mjs), used by
// stepTestsOnly; the executor and the step helper say the same in prose.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import * as m from '../../lib/parsers/plan-schema.mjs';
import { validatePlanContent } from '../../lib/validators/plan-validator.mjs';
import { isTestPath } from '../../lib/util/test-path.mjs';
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
/** A 1.8-shaped step section: `Files:` and the `Test first:` lines given. */
const step = (files, testFirst) => [
'### Step 1: x', '',
`- **Files:** ${files}`,
'- **Changes:** x',
'- **Test first:**',
...testFirst,
'- **Verify:** `node --test tests/a.test.mjs` -> expected: exit 0',
].join('\n');
const TWO = step('`tests/a.test.mjs`, `checks/smoke.mjs`', [
' - File: `tests/a.test.mjs`, `checks/smoke.mjs` (new)',
' - Verifies: both checks run',
]);
const testFirstFiles = (body, version) => {
assert.equal(typeof m.testFirstFiles, 'function', 'plan-schema.mjs exports testFirstFiles');
return m.testFirstFiles(m.parseStepFields(body), version);
};
test('a 1.8 step whose Files are its two Test first files is tests-only', () => {
// checks/smoke.mjs is not a test by its path: only the Test first line makes it one.
assert.equal(m.stepTestsOnly(TWO).testsOnly, true, JSON.stringify(m.stepTestsOnly(TWO)));
});
test('testFirstFiles returns every path on the File: line, below 1.9 and without a version', () => {
assert.deepEqual(testFirstFiles(TWO, '1.8'), ['tests/a.test.mjs', 'checks/smoke.mjs']);
assert.deepEqual(testFirstFiles(TWO, null), ['tests/a.test.mjs', 'checks/smoke.mjs']);
assert.deepEqual(testFirstFiles(TWO), ['tests/a.test.mjs', 'checks/smoke.mjs']);
});
test('testFirstFiles: a call, an option or a URL on the File: line is not read; paths are normalised and deduplicated', () => {
const body = step('`tests/a.test.mjs`', [
' - File: `tests/a.test.mjs`, `parse()`, `--watch`, `https://example.invalid/x.mjs`, `./tests/a.test.mjs`',
' - Verifies: x',
]);
assert.deepEqual(testFirstFiles(body, '1.8'), ['tests/a.test.mjs']);
});
test('testFirstFiles: a Verifies: line, or a span before File: on the line, is never read', () => {
const body = step('`tests/a.test.mjs`', [
' - see `lib/before.mjs` File: `tests/a.test.mjs`',
' - Verifies: `lib/other.mjs` returns `src/x.mjs`',
]);
assert.deepEqual(testFirstFiles(body, '1.8'), ['tests/a.test.mjs']);
const none = step('`tests/a.test.mjs`', [' - Verifies: `tests/a.test.mjs` runs']);
assert.deepEqual(testFirstFiles(none, '1.8'), []);
});
test('a code file beside the test on the File: line still makes the step not tests-only when Files has more', () => {
const body = step('`tests/a.test.mjs`, `src/cli.mjs`', [
' - File: `tests/a.test.mjs`, `parse()` (new)',
' - Verifies: x',
]);
assert.equal(m.stepTestsOnly(body).testsOnly, false);
});
// ---- plan_version 1.9 (plan step 5) ----------------------------------------
// From 1.9 a `Test first:` `File:` line is every line that opens with `File:`
// (after an optional list marker and `**`), backticked or not, and its rest is
// one entry read by entryPaths: the leading backticked paths are the test
// files, the rest is description. The validator puts the same grammar on it.
const PATH_CODES = ['PLAN_STEP_PATH_NOT_FIRST', 'PLAN_STEP_PATH_NOT_A_PATH'];
// A plan of the given version whose first step has `Files:` `files` and the
// `Test first:` `File:` line `fileLine` (both verbatim).
const planWith = (version, files, fileLine) => buildPlan({
version,
steps: [{ filesRaw: files, testFirstRaw: fileLine }, { endToEnd: 'bash tests/e2e.sh' }],
});
test('1.9: every leading backticked path on the File: line is a test file', () => {
const body = step('`tests/a.test.mjs`, `tests/b.test.mjs`', [
' - File: `tests/a.test.mjs`, `tests/b.test.mjs` (new)',
' - Verifies: x',
]);
assert.deepEqual(testFirstFiles(body, '1.9'), ['tests/a.test.mjs', 'tests/b.test.mjs']);
});
test('decision 20: a backticked path in the description of the File: line is not a test file, at 1.9 and below', () => {
const body = step('`tests/a.test.mjs`', [
' - File: `tests/a.test.mjs` (new), covers `tests/b.test.mjs`',
' - Verifies: x',
]);
assert.deepEqual(testFirstFiles(body, '1.9'), ['tests/a.test.mjs']);
// Operator decision 20 (12.10): below 1.9 too, only the paths right after File:.
assert.deepEqual(testFirstFiles(body, '1.8'), ['tests/a.test.mjs']);
});
test('decision 20: a code file named only in the File: line description keeps the step from being tests-only, at 1.9 and 1.8', () => {
const files = '`tests/a.test.mjs`, `checks/smoke.mjs`';
const line = ' - File: `tests/a.test.mjs` (new), covers `checks/smoke.mjs`';
const body = step(files, [line, ' - Verifies: x']);
assert.equal(m.stepTestsOnly(body, '1.9').testsOnly, false, JSON.stringify(m.stepTestsOnly(body, '1.9')));
assert.equal(m.stepTestsOnly(body, '1.8').testsOnly, false, JSON.stringify(m.stepTestsOnly(body, '1.8')));
const v19 = validatePlanContent(planWith('1.9', files, line));
assert.ok(!v19.warnings.some((w) => w.code === 'PLAN_STEP_TESTS_ONLY'), JSON.stringify(v19.warnings));
assert.deepEqual(v19.errors.filter((e) => PATH_CODES.includes(e.code)), [], JSON.stringify(v19.errors));
const v18 = validatePlanContent(planWith('1.8', files, line));
assert.ok(!v18.warnings.some((w) => w.code === 'PLAN_STEP_TESTS_ONLY'), JSON.stringify(v18.warnings));
});
test('1.9: a File: line that opens with `parse()` is refused as no path (error strict, warning soft)', () => {
const text = planWith('1.9', '`tests/a.test.mjs`', ' - File: `parse()`');
const strict = validatePlanContent(text);
const e = strict.errors.find((i) => i.code === 'PLAN_STEP_PATH_NOT_A_PATH');
assert.ok(e, JSON.stringify(strict.errors));
assert.match(e.message, /Step 1/);
assert.match(e.message, /Test first/);
const soft = validatePlanContent(text, { strict: false });
assert.ok(soft.warnings.some((w) => w.code === 'PLAN_STEP_PATH_NOT_A_PATH'), JSON.stringify(soft.warnings));
assert.deepEqual(soft.errors.filter((i) => PATH_CODES.includes(i.code)), []);
});
test('1.9: an unticked File: line gets PLAN_STEP_PATH_NOT_FIRST next to PLAN_STEP_NO_TEST_FIRST', () => {
const r = validatePlanContent(planWith('1.9', '`tests/a.test.mjs`', ' - File: tests/a.test.mjs'));
const codes = r.errors.map((e) => e.code);
assert.ok(codes.includes('PLAN_STEP_PATH_NOT_FIRST'), codes.join(','));
assert.ok(codes.includes('PLAN_STEP_NO_TEST_FIRST'), codes.join(','));
const e = r.errors.find((i) => i.code === 'PLAN_STEP_PATH_NOT_FIRST');
assert.match(e.message, /Step 1/);
assert.match(e.message, /tests\/a\.test\.mjs/);
});
// ---- operator decision 20 (12.10), review 6c31212c --------------------------
// Below 1.9 too, a step's test files are the backticked paths that stand right
// after `File:`, separated only by commas or whitespace. A path in the
// description after them (`covers parse() in src/parser.mjs`) is no test file,
// so a step that builds that source is not tests-only and its source is never
// locked. Each `File:` marker on a line starts its own run, as 6.3.1 read the
// first span after each marker.
// The review's exact input: the description names a call and the source.
const REVIEW_FILES = '`src/parser.mjs`, `tests/parser.test.mjs`';
const REVIEW_LINE = ' - File: `tests/parser.test.mjs` (new) \u2014 covers `parse()` in `src/parser.mjs`';
test('decision 20: the review input (6c31212c) gives only the test below 1.9, is not tests-only, and gets no PLAN_STEP_TESTS_ONLY', () => {
const body = step(REVIEW_FILES, [REVIEW_LINE, ' - Verifies: x']);
for (const v of ['1.8', '1.7', null, undefined]) {
assert.deepEqual(testFirstFiles(body, v), ['tests/parser.test.mjs'], `version ${v}`);
}
assert.equal(m.stepTestsOnly(body, '1.8').testsOnly, false, JSON.stringify(m.stepTestsOnly(body, '1.8')));
assert.equal(m.stepTestsOnly(body).testsOnly, false, JSON.stringify(m.stepTestsOnly(body)));
const v18 = validatePlanContent(planWith('1.8', REVIEW_FILES, REVIEW_LINE));
assert.ok(!v18.warnings.some((w) => w.code === 'PLAN_STEP_TESTS_ONLY'), JSON.stringify(v18.warnings));
});
test('decision 20: two leading tests (comma or space) before a description that names a source give the two tests at 1.8 and 1.9', () => {
for (const lead of ['`tests/a.test.mjs`, `tests/b.test.mjs`', '`tests/a.test.mjs` `tests/b.test.mjs`']) {
const body = step('`tests/a.test.mjs`, `tests/b.test.mjs`, `src/a.mjs`', [
` - File: ${lead} (new) - covers \`run()\` in \`src/a.mjs\``,
' - Verifies: x',
]);
for (const v of ['1.8', '1.9']) {
assert.deepEqual(testFirstFiles(body, v), ['tests/a.test.mjs', 'tests/b.test.mjs'], `${v}: ${lead}`);
}
}
});
test('decision 20: a 1.8 line with two File: markers, each followed by a test, gives both tests (6.3.1 read both)', () => {
const body = step('`tests/a.test.mjs`, `tests/b.test.mjs`', [
' - File: `tests/a.test.mjs` (new) and File: `tests/b.test.mjs` (existing)',
' - Verifies: x',
]);
assert.deepEqual(testFirstFiles(body, '1.8'), ['tests/a.test.mjs', 'tests/b.test.mjs']);
assert.deepEqual(testFirstFiles(body, null), ['tests/a.test.mjs', 'tests/b.test.mjs']);
});
// ---- operator decisions 28 (13.10) and 31 (14.10), review bd2520b0 ---------
// Below 1.9 the run after `File:` also goes on past a note in brackets
// (`(new)`, `*(new)*`, `(existing ...)`) and the whole words `and`, `plus` and
// `+`, to a further path only when it is clearly a test; a path that is not
// ends the run, and so does any other text. At 1.9 nothing changes.
const OLDER = ['1.8', '1.7', null];
const fileLine = (rest) => step('`tests/a.test.mjs`, `tests/b.test.mjs`', [` - File: ${rest}`, ' - Verifies: x']);
test('decisions 28 and 31: below 1.9 a test after a note, and, plus or + is read', () => {
const lines = [
'`tests/a.test.mjs` (new), `tests/b.test.mjs` (new),',
'`tests/a.test.mjs` and `tests/b.test.mjs`',
'`tests/a.test.mjs` (existing) + `tests/b.test.mjs`',
'`tests/a.test.mjs` (existing, the parser) plus `tests/b.test.mjs`',
'`tests/a.test.mjs` *(new)*, `tests/b.test.mjs`',
];
for (const rest of lines) {
for (const v of OLDER) {
assert.deepEqual(testFirstFiles(fileLine(rest), v), ['tests/a.test.mjs', 'tests/b.test.mjs'], `version ${v}: ${rest}`);
}
}
});
test('decisions 28 and 31: below 1.9 a near miss gives only the first test', () => {
// Both read as tests by lib/util/test-path.mjs alone (a specs/ folder); the
// run refuses them all the same: a document or a data file outside tests/.
assert.equal(isTestPath('docs/specs/api.md'), true);
assert.equal(isTestPath('api/specs/openapi.yaml'), true);
const lines = [
'`tests/a.test.mjs` (new) - covers `src/a.mjs`',
'`tests/a.test.mjs` (new), `src/a.mjs`',
'`tests/a.test.mjs` (new) or `tests/b.test.mjs`',
'`tests/a.test.mjs` (new), covers `tests/b.test.mjs`',
'`tests/a.test.mjs` (new) and `src/a.mjs`',
'`tests/a.test.mjs` (new) and `https://example.invalid/tests/b.test.mjs`',
'`tests/a.test.mjs` (new) and `docs/specs/api.md`',
'`tests/a.test.mjs` (new), `api/specs/openapi.yaml`',
'`tests/a.test.mjs` android `tests/b.test.mjs`',
'`tests/a.test.mjs` andand `tests/b.test.mjs`',
'`tests/a.test.mjs` plusand `tests/b.test.mjs`',
];
for (const rest of lines) {
for (const v of OLDER) {
assert.deepEqual(testFirstFiles(fileLine(rest), v), ['tests/a.test.mjs'], `version ${v}: ${rest}`);
}
}
});
test('decisions 28 and 31: below 1.9 the run after a note stops at the first source path', () => {
const rest = '`tests/a.test.mjs` (new), `tests/b.test.mjs`, `src/a.mjs`, `tests/c.test.mjs`';
for (const v of OLDER) {
assert.deepEqual(testFirstFiles(fileLine(rest), v), ['tests/a.test.mjs', 'tests/b.test.mjs'], `version ${v}`);
}
});
test('decisions 28 and 31: at 1.9 the two-test line with notes still gives only the first test', () => {
const body = fileLine('`tests/a.test.mjs` (new), `tests/b.test.mjs` (new),');
assert.deepEqual(testFirstFiles(body, '1.9'), ['tests/a.test.mjs']);
});
// ---- one definition in the prose (plan step 11, review cf241526) -----------
// The executor and the step helper define the step's test files once, as
// decisions 20 and 31 read them; every other place points at that definition.
// Operator decision 30 (14.10): red proof reads that list from the plan, so
// neither the helper nor the executor passes --tests; it prints the list it
// ran (`tests`), and the lock's --files is that printed list. A plan or step
// red proof cannot read is `plan-unreadable`: the helper stops before it
// builds, the executor corrects its own call and restarts the step once with
// a fresh build helper (`build (restart)`), outside the fix rounds. A --tests
// that differs is still refused (`tests-mismatch`, the plan's list in
// `plan_tests`), but the documented call never gives one.
const EXEC = ['commands', 'trekexecute.md'];
const HELPER = ['agents', 'step-implementer.md'];
const readRel = (rel) => readFileSync(join(ROOT, ...rel), 'utf8');
const count = (text, s) => text.split(s).length - 1;
test('prose: neither file reads every backticked span on the File: line any more', () => {
for (const rel of [EXEC, HELPER]) {
const text = readRel(rel);
assert.equal(count(text, 'every backticked span on that line'), 0, `${rel.join('/')} still says "every backticked span on that line"`);
}
});
test('prose: the decision-20 definition stands exactly once in each file and keeps the pinned phrase', () => {
for (const rel of [EXEC, HELPER]) {
const text = readRel(rel);
assert.equal((text.match(/operator decision 20(?!\d)/g) || []).length, 1, `${rel.join('/')} holds the decision-20 definition once`);
assert.ok(text.includes('every file on the `Test first:` `File:` line'), `${rel.join('/')} keeps the pinned phrase`);
}
});
test('prose: the executor definition names decisions 30 and 31, tests-mismatch, plan_tests and plan-unreadable, and keeps its line shape', () => {
const lines = readRel(EXEC).split('\n');
const i = lines.findIndex((l) => l.startsWith("**The step's test files** are"));
assert.ok(i >= 0, 'the definition line is there');
const def = lines[i];
assert.ok(def.includes('operator decision 20'), 'the definition line carries decision 20');
assert.ok(def.includes('operator decision 30'), 'the definition line carries decision 30');
assert.ok(def.includes('`tests-mismatch`'), 'the definition names tests-mismatch');
assert.ok(def.includes('`plan_tests`'), 'the definition names plan_tests');
assert.ok(def.includes('`plan-unreadable`'), 'the definition names plan-unreadable');
assert.ok(def.includes('`--files` is the list red proof printed as `tests`'), 'the definition says the lock follows the printed list');
assert.ok(def.endsWith(' The'), `the definition line still ends with " The": ${def.slice(-40)}`);
assert.ok(lines[i + 1].startsWith('plan template lists them'), `the next line: ${lines[i + 1]}`);
});
test('prose: both definitions name decision 31 and read past (new), and, plus or + below 1.9', () => {
for (const rel of [EXEC, HELPER]) {
const text = readRel(rel).replace(/\n\s*/g, ' ');
const at20 = text.search(/operator decision 20(?!\d)/);
const head = text.lastIndexOf('every file on the `Test first:` `File:` line', at20);
assert.ok(at20 > 0 && head >= 0, `${rel.join('/')}: the definition is there`);
const def = text.slice(head, at20);
assert.ok(def.includes('operator decision 31'), `${rel.join('/')}: the definition names operator decision 31`);
assert.ok(def.includes('`plus`'), `${rel.join('/')}: the definition names plus`);
assert.ok(def.includes('`(new)`'), `${rel.join('/')}: the definition names (new)`);
}
});
test('prose: the executor exit-2 row names plan-unreadable first, the one restart outside the fix rounds, and tests-mismatch with plan_tests', () => {
const row = readRel(EXEC).split('\n').find((l) => l.startsWith('| 2 | `"not-run"`'));
assert.ok(row, 'the exit-2 row is there');
for (const s of ['`plan-unreadable`', '`tests-mismatch`', '`plan_tests`', 'fresh build helper', 'outside the fix rounds',
'absolute path', 'reruns only its own red proof', '`build (restart)`', '`cost_if_wrong`', '`attempts`',
'`iterations_remaining`', '`error`', '`--resume`']) {
assert.ok(row.includes(s), `the exit-2 row names ${s}`);
}
assert.match(row, /decision[- ]log\s+entry/, 'the exit-2 row records a decision-log entry');
});
test('prose: a second plan-unreadable goes up to the plan (stop class 4 row, Back to planning)', () => {
const text = readRel(EXEC);
const row4 = text.split('\n').find((l) => l.startsWith('| 4 | the plan |'));
assert.ok(row4, 'the stop-class-4 row is there');
assert.ok(row4.includes('`plan-unreadable`'), 'the stop-class-4 row names plan-unreadable');
const b = text.indexOf('**Back to planning**');
assert.ok(b > 0, 'the Back to planning paragraph is there');
const end = text.indexOf('\n\n', b);
const para = text.slice(b, end < 0 ? undefined : end);
assert.ok(para.includes('`plan-unreadable`'), 'the Back to planning paragraph names plan-unreadable');
});
test('prose: the exit-1 otherwise row no longer counts tests_only: null', () => {
const row = readRel(EXEC).split('\n').find((l) => l.startsWith('| 1, `not-red-at-base`, otherwise'));
assert.ok(row, 'the exit-1 otherwise row is there');
assert.ok(!row.includes('tests_only: null'), 'the exit-1 otherwise row no longer says tests_only: null');
});
test('prose: no red-proof call line in either file passes --tests (operator decision 30)', () => {
for (const rel of [EXEC, HELPER]) {
const calls = readRel(rel).split('\n').filter((l) => l.includes('red-proof.mjs --red-only') || l.includes('red-proof.mjs --base'));
assert.ok(calls.length > 0, `${rel.join('/')}: a red-proof call line is there`);
for (const l of calls) assert.ok(!l.includes('--tests'), `${rel.join('/')}: the call passes --tests: ${l}`);
}
});
test('prose: below 1.8 a plan that names no test file gets --tests from the files the helper wrote (operator decision 41, review 7b3b59ed)', () => {
const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const phrase = (s) => new RegExp(s.split(/\s+/).map(esc).join('\\s+'));
const d41 = /\boperator\s+decision\s+41(?!\d)/;
const NTFG = '`no test files given`';
const has = (text, s, where) => assert.match(text, phrase(s), `${where} holds "${s}"`);
// The helper: the sentence that holds `no test files given`.
const h = readRel(HELPER);
const at = h.search(phrase(NTFG));
assert.ok(at >= 0, 'the helper names `no test files given`');
const before = h.slice(0, at);
const ends = [...before.matchAll(/\.\s/g)];
const start = ends.length ? ends[ends.length - 1].index + 1 : 0;
const rest = h.slice(at);
const stop = rest.search(/\.(\s|$)/);
const sentence = h.slice(start, stop < 0 ? undefined : at + stop + 1);
has(sentence, 'below 1.8', 'the helper sentence');
has(sentence, '`--tests <the test files you wrote', 'the helper sentence');
assert.match(sentence, d41, 'the helper sentence names operator decision 41');
const pu = h.match(/plan-unreadable[^.]*do not build/);
assert.ok(pu && pu.index < at, 'the helper sentence stands after the plan-unreadable sentence');
// The executor: the definition line and the exit-2 row.
const lines = readRel(EXEC).split('\n');
const def = lines.find((l) => l.startsWith("**The step's test files** are"));
assert.ok(def, 'the executor definition line is there');
for (const s of [NTFG, 'naming the test files it wrote', 'the `tests:` line the helper copied into its report']) {
has(def, s, 'the executor definition line');
}
assert.match(def, d41, 'the executor definition line names operator decision 41');
assert.ok(def.endsWith(' The'), `the definition line still ends with " The": ${def.slice(-40)}`);
const row = lines.find((l) => l.startsWith('| 2 | `"not-run"`'));
assert.ok(row, 'the exit-2 row is there');
const r = row.search(phrase(NTFG));
assert.ok(r >= 0, 'the exit-2 row names `no test files given`');
const after = row.slice(r);
for (const s of ['is not a finding', 'the `tests:` line in the helper\'s report', 'tolerated as `"not-run"`']) {
has(after, s, 'the exit-2 row after `no test files given`');
}
assert.match(after, d41, 'the exit-2 row names operator decision 41');
// Handover 4, invariant 7.
const ho = readRel(['docs', 'HANDOVER-CONTRACTS.md']);
const a7 = ho.indexOf('7. **Every plan_version');
const a8 = ho.indexOf('8. **', a7);
assert.ok(a7 >= 0 && a8 > a7, 'Handover 4 invariants 7 and 8 are there');
const inv7 = ho.slice(a7, a8);
for (const s of [NTFG, 'naming the test files it wrote', 'the `tests:` line in the helper\'s report']) {
has(inv7, s, 'Handover 4 invariant 7');
}
assert.match(inv7, d41, 'Handover 4 invariant 7 names operator decision 41');
});
test('prose: the helper stops on plan-unreadable before the 1.8 branches and locks the printed list', () => {
const text = readRel(HELPER);
const pu = text.match(/plan-unreadable[^.]*do not build/);
assert.ok(pu, 'the helper does not build on plan-unreadable');
const below = text.indexOf('Below 1.8');
assert.ok(below > 0, 'the helper still has its Below 1.8 branch');
assert.ok(pu.index < below, 'the plan-unreadable sentence stands before "Below 1.8"');
assert.ok(text.includes('`--files` is the list red proof printed'), 'the lock follows the printed list');
assert.ok(text.includes('copy that line into the report'), 'the helper copies the tests line into the report');
});
test('prose: the round label build (restart) is named by the executor prompt, the report heading and the helper build mode', () => {
const ex = readRel(EXEC);
assert.ok(ex.split('\n').some((l) => l.includes('(`build`, `build (restart)`, or `fix round R`)')), 'the Sub-step C prompt line names build (restart)');
const h = readRel(HELPER);
assert.ok(h.split('\n').includes('## Step {N} - {build | build (restart) | fix round R}'), 'the helper report heading names build (restart)');
const a = h.indexOf('## Mode `build`');
const z = h.indexOf('## Mode `fix`');
assert.ok(a >= 0 && z > a, 'the helper has both modes');
assert.match(h.slice(a, z), /build \(restart\)[^.]*one\s+copy\s+of\s+each\s+test\s+case/);
});
test('prose: the old rerun wording is gone from both files', () => {
const ex = readRel(EXEC);
for (const s of ['runs it once more with the list it printed', 'reruns once with the list red proof printed', 'the list red proof accepted']) {
assert.ok(!ex.includes(s), `commands/trekexecute.md still says "${s}"`);
}
const h = readRel(HELPER);
assert.doesNotMatch(h, /once\s+more[^.]*plan_tests/);
for (const s of ['after a rerun, `plan_tests`', 'the list red proof accepted']) {
assert.ok(!h.includes(s), `agents/step-implementer.md still says "${s}"`);
}
});

3
tests/lib/test-lock.test.mjs
View file

@ -11,12 +11,13 @@ import { existsSync, readFileSync, writeFileSync, mkdirSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { makeRepo, gitIn, tmp } from '../hooks/guard-helpers.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const CLI = join(HERE, '..', '..', 'lib', 'util', 'test-lock.mjs');
function run(cwd, args, env = {}) {
const r = spawnSync(process.execPath, [CLI, ...args], { cwd, encoding: 'utf8', env: { ...process.env, ...env } });
const r = spawnSync(process.execPath, [CLI, ...args], { cwd, encoding: 'utf8', env: childEnv(env) });
return { code: r.status, stdout: r.stdout, stderr: r.stderr };
}

20
tests/lib/timing-under-load.test.mjs
View file

@ -16,6 +16,7 @@ import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fastestWithin } from '../helpers/timing.mjs';
import * as timing from '../helpers/timing.mjs';
import { HOOK_TIMEOUT_MS, runHookWithEnv } from '../helpers/hook-helper.mjs';
/** A measure that returns the given durations in turn (no real waiting). */
@ -61,3 +62,22 @@ test('hook helper: the kill limit is far above 5 s by default, and a call can se
assert.equal(done.stdout, 'done');
} finally { rmSync(dir, { recursive: true, force: true }); }
});
// The red-proof control-run test (review 630 r9, brief note 3, in
// tests/lib/review-630.test.mjs) reads its timeout, hang and simulated slow
// start from tests/helpers/timing.mjs (CONTROL_RUN, review 703d87e1): the
// criteria runner saw its start point killed at 2 s under full-suite load.
// BEFORE_703D87E1 is the record of the numbers before the fix, taken only when
// the export is absent.
const BEFORE_703D87E1 = Object.freeze({ timeoutMs: 2000, hangMs: 6000, slowStartMs: 2500 });
test('red proof control run (703d87e1): the timeout leaves headroom over a slow start and the hang outlasts it', () => {
const c = timing.CONTROL_RUN ?? BEFORE_703D87E1;
const seen = JSON.stringify(c);
assert.ok(c.slowStartMs > 2000, `the simulated slow start is above the old 2 s timeout: ${seen}`);
assert.ok(c.timeoutMs >= 15000, `the timeout leaves headroom over a loaded machine: ${seen}`);
assert.equal(c.timeoutMs % 1000, 0, `a whole number of seconds (the reason prints Math.round(timeoutMs / 1000)): ${seen}`);
assert.ok(c.timeoutMs >= 6 * c.slowStartMs, `the timeout is at least 6x the simulated slow start: ${seen}`);
assert.ok(c.hangMs >= c.timeoutMs + 30000, `the hang outlasts the timeout by 30 s: ${seen}`);
assert.ok(c.hangMs < 2 ** 31 - 1, `the hang fits a real timer (below 2^31-1 ms): ${seen}`);
});

683
tests/review/brief-conflict-findings.test.mjs Normal file
View file

@ -0,0 +1,683 @@
// tests/review/brief-conflict-findings.test.mjs
// The brief-conflict rule (plan 2026-10-11-files-backticks-trial4, Step 8).
//
// A `/code-review` finding at the MAJOR tier whose recommended action
// contradicts a named brief Non-Goal (or stated choice) is kept, marked with
// `contradicts_brief: "<the Non-Goal, verbatim>"`, and drops to MINOR with
// `brief_lowered_from: 'MAJOR'`. Nothing else is ever lowered by the mark: a
// BLOCKER stays BLOCKER, a `/security-review` finding keeps its severity, a
// reviewer-agent finding (no `engine`) is never marked. The verdict gate
// refuses (exit 2) every shape outside the one legal one.
//
// The new exports are probed via `import * as m` so a missing export is an
// assertion failure, never a load-time crash. Fixtures are read inside tests.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { readFileSync, existsSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import * as m from '../../lib/review/coordinator-contract.mjs';
import { extractFindingsBlock } from '../../lib/review/findings-schema.mjs';
import { toReviewerPayload } from '../../lib/review/engine-findings.mjs';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const GATE = join(ROOT, 'lib', 'review', 'verdict-gate.mjs');
const LEGAL_FIXTURE = join(ROOT, 'tests', 'fixtures', 'review-brief-conflict-legal.md');
const ILLEGAL_FIXTURE = join(ROOT, 'tests', 'fixtures', 'review-brief-conflict-illegal.md');
const NON_GOAL = 'No retry layer around the HTTP client (Non-Goal 2)';
function illegal(f) {
assert.equal(typeof m.illegalBriefLowering, 'function',
'coordinator-contract.mjs must export illegalBriefLowering(f)');
return m.illegalBriefLowering(f);
}
function lowerable(f) {
assert.equal(typeof m.lowerableBriefConflict, 'function',
'coordinator-contract.mjs must export lowerableBriefConflict(f)');
return m.lowerableBriefConflict(f);
}
function codeReviewDefect(extra = {}) {
return {
file: 'lib/http.mjs', line: 42, rule_key: 'CODE_REVIEW_DEFECT', severity: 'MAJOR',
title: 'No retry on transient failure', detail: 'A 503 fails the call.',
recommended_action: 'Wrap the call in a retry with backoff.', engine: 'code-review',
...extra,
};
}
function payload(reviewer, findings) {
return [{ reviewer, findings }];
}
function legalShape(extra = {}) {
return codeReviewDefect({
id: 'd3c22f4b1309d6001a0f3948e377a8e80a4b9c55',
severity: 'MINOR', contradicts_brief: NON_GOAL, brief_lowered_from: 'MAJOR', ...extra,
});
}
function assertAllOutputLegal(result) {
for (const f of result.findings) {
assert.equal(illegal(f), false,
`runContract wrote a finding illegalBriefLowering refuses: ${JSON.stringify(f)}`);
}
}
// ---- runContract (Pass 3) ----------------------------------------------------
test('runContract — a marked /code-review MAJOR drops to MINOR, keeps the Non-Goal, verdict ALLOW, not unverified', () => {
const result = m.runContract(payload('claude-code-review', [codeReviewDefect({ contradicts_brief: NON_GOAL })]));
assert.equal(result.findings.length, 1, 'the brief-conflicting finding is kept, never removed');
const f = result.findings[0];
assert.equal(f.contradicts_brief, NON_GOAL, 'the Non-Goal is named verbatim');
assert.equal(f.severity, 'MINOR');
assert.equal(f.brief_lowered_from, 'MAJOR');
assert.equal(result.verdict, 'ALLOW');
assert.equal(result.unverified.length, 0, 'the mark is not a suppression reason');
assert.deepEqual(result.allow_blocked_by, []);
assertAllOutputLegal(result);
});
test('runContract — the same /code-review MAJOR without the mark stays MAJOR and WARN', () => {
const result = m.runContract(payload('claude-code-review', [codeReviewDefect()]));
assert.equal(result.findings[0].severity, 'MAJOR');
assert.equal('brief_lowered_from' in result.findings[0], false);
assert.equal(result.verdict, 'WARN');
assertAllOutputLegal(result);
});
test('runContract — a marked /code-review BLOCKER-tier finding stays BLOCKER and BLOCK', () => {
const f = codeReviewDefect({ rule_key: 'SECURITY_INJECTION', severity: 'BLOCKER', contradicts_brief: NON_GOAL });
const result = m.runContract(payload('claude-code-review', [f]));
assert.equal(result.findings[0].severity, 'BLOCKER');
assert.equal('brief_lowered_from' in result.findings[0], false);
assert.equal(result.findings[0].contradicts_brief, NON_GOAL, 'the mark is kept on an engine finding');
assert.equal(result.verdict, 'BLOCK');
assertAllOutputLegal(result);
});
test('runContract — a marked /security-review finding keeps BLOCKER', () => {
const f = {
file: 'lib/auth.mjs', line: 9, rule_key: 'SECURITY_REVIEW_VULNERABILITY', severity: 'BLOCKER',
title: 'Token in log', detail: 'd', recommended_action: 'Remove the log line.',
engine: 'security-review', contradicts_brief: NON_GOAL,
};
const result = m.runContract(payload('claude-security-review', [f]));
assert.equal(result.findings[0].severity, 'BLOCKER');
assert.equal('brief_lowered_from' in result.findings[0], false);
assert.equal(result.verdict, 'BLOCK');
assertAllOutputLegal(result);
});
test('runContract — a marked MINOR-tier CODE_REVIEW_IMPROVEMENT stays MINOR with no brief_lowered_from', () => {
const f = codeReviewDefect({ rule_key: 'CODE_REVIEW_IMPROVEMENT', severity: 'MINOR', contradicts_brief: NON_GOAL });
const result = m.runContract(payload('claude-code-review', [f]));
assert.equal(result.findings[0].severity, 'MINOR');
assert.equal('brief_lowered_from' in result.findings[0], false);
assert.equal(result.findings[0].contradicts_brief, NON_GOAL);
assertAllOutputLegal(result);
});
test('runContract — a reviewer finding (no engine) that arrives marked loses the mark and keeps today\'s reset', () => {
const f = {
file: 'lib/b.mjs', line: 7, rule_key: 'MISSING_TEST', severity: 'BLOCKER',
title: 'No test', detail: 'd', recommended_action: 'Add a test.', contradicts_brief: NON_GOAL,
};
const result = m.runContract(payload('code-correctness-reviewer', [f]));
const out = result.findings[0];
assert.equal('contradicts_brief' in out, false, 'only an engine finding is ever marked');
assert.equal(out.severity, 'MAJOR');
assert.equal(out.original_severity, 'BLOCKER');
assert.equal('brief_lowered_from' in out, false);
assertAllOutputLegal(result);
});
test('runContract — an incoming brief_lowered_from is removed from every finding', () => {
const reviewer = {
file: 'lib/b.mjs', line: 7, rule_key: 'MISSING_TEST', severity: 'MAJOR',
title: 'No test', detail: 'd', recommended_action: 'Add a test.', brief_lowered_from: 'MAJOR',
};
const engineBlocker = codeReviewDefect({ line: 50, rule_key: 'SECURITY_INJECTION', severity: 'BLOCKER', brief_lowered_from: 'BLOCKER' });
const engineUnmarked = codeReviewDefect({ line: 60, brief_lowered_from: 'MAJOR' });
const lowerableOne = codeReviewDefect({ line: 70, contradicts_brief: NON_GOAL, brief_lowered_from: 'BLOCKER' });
const result = m.runContract([
{ reviewer: 'code-correctness-reviewer', findings: [reviewer] },
{ reviewer: 'claude-code-review', findings: [engineBlocker, engineUnmarked, lowerableOne] },
]);
const byLine = new Map(result.findings.map((f) => [f.line, f]));
assert.equal('brief_lowered_from' in byLine.get(7), false);
assert.equal('brief_lowered_from' in byLine.get(50), false);
assert.equal(byLine.get(50).severity, 'BLOCKER');
assert.equal('brief_lowered_from' in byLine.get(60), false);
assert.equal(byLine.get(60).severity, 'MAJOR');
assert.equal(byLine.get(70).severity, 'MINOR');
assert.equal(byLine.get(70).brief_lowered_from, 'MAJOR', 'only the lowering itself writes the field');
assertAllOutputLegal(result);
});
test('lowerableBriefConflict — only a marked /code-review finding at MAJOR on a MAJOR-tier key', () => {
assert.equal(lowerable(codeReviewDefect({ contradicts_brief: NON_GOAL })), true);
assert.equal(lowerable(codeReviewDefect()), false, 'no mark');
assert.equal(lowerable(codeReviewDefect({ contradicts_brief: '' })), false, 'empty mark');
assert.equal(lowerable(codeReviewDefect({ contradicts_brief: true })), false, 'non-string mark');
assert.equal(lowerable(codeReviewDefect({ contradicts_brief: NON_GOAL, engine: 'security-review' })), false);
assert.equal(lowerable(codeReviewDefect({ contradicts_brief: NON_GOAL, engine: undefined })), false, 'missing engine');
assert.equal(lowerable(codeReviewDefect({ contradicts_brief: NON_GOAL, rule_key: 'NOPE' })), false, 'unknown key');
assert.equal(lowerable(codeReviewDefect({ contradicts_brief: NON_GOAL, rule_key: 'SECURITY_INJECTION', severity: 'BLOCKER' })), false);
assert.equal(lowerable(codeReviewDefect({ contradicts_brief: NON_GOAL, severity: 'BLOCKER' })), false, 'severity must be MAJOR');
});
// ---- effectiveSeverity (Pass 2 trimming) -------------------------------------
test('judgeFilter — a marked /code-review MAJOR with a long detail is trimmed, its mark intact; unmarked it is kept whole', () => {
const long = 'x'.repeat(900);
const marked = codeReviewDefect({ detail: long, contradicts_brief: NON_GOAL });
const unmarked = codeReviewDefect({ detail: long });
const a = m.judgeFilter([marked]);
assert.equal(a.trimmed.length, 1, 'the marked finding is judged MINOR, so its text may be trimmed');
assert.equal(a.trimmed[0].contradicts_brief, NON_GOAL, 'trimming never touches the mark');
assert.ok(a.trimmed[0].detail.length <= m.JUDGE_DETAIL_MAX);
const b = m.judgeFilter([unmarked]);
assert.equal(b.trimmed.length, 0, 'an unmarked MAJOR keeps every character');
assert.equal(b.kept[0].detail, long);
const result = m.runContract(payload('claude-code-review', [marked]));
assert.equal(result.trimmed_count, 1);
assert.equal(result.findings[0].contradicts_brief, NON_GOAL);
assert.equal(result.findings[0].severity, 'MINOR');
});
// ---- illegalBriefLowering ----------------------------------------------------
test('illegalBriefLowering — the legal shape is false', () => {
assert.equal(illegal(legalShape()), false);
});
test('illegalBriefLowering — a marked BLOCKER-tier engine finding below BLOCKER is true, with and without brief_lowered_from', () => {
const base = { rule_key: 'SECURITY_INJECTION', contradicts_brief: NON_GOAL };
assert.equal(illegal(codeReviewDefect({ ...base, severity: 'MINOR', brief_lowered_from: 'MAJOR' })), true);
assert.equal(illegal(codeReviewDefect({ ...base, severity: 'MAJOR', brief_lowered_from: 'BLOCKER' })), true);
assert.equal(illegal(codeReviewDefect({ ...base, severity: 'MAJOR' })), true);
assert.equal(illegal(codeReviewDefect({ ...base, severity: 'MINOR' })), true);
});
test('illegalBriefLowering — a marked /security-review finding below its tier is true', () => {
const f = {
id: 'a'.repeat(40), file: 'lib/auth.mjs', line: 9, rule_key: 'SECURITY_REVIEW_VULNERABILITY',
engine: 'security-review', contradicts_brief: NON_GOAL,
};
assert.equal(illegal({ ...f, severity: 'MAJOR' }), true);
assert.equal(illegal({ ...f, severity: 'MINOR', brief_lowered_from: 'MAJOR' }), true);
});
test('illegalBriefLowering — a marked reviewer finding (no engine) is true at any severity', () => {
const f = { id: 'b'.repeat(40), file: 'lib/b.mjs', line: 7, rule_key: 'MISSING_TEST', contradicts_brief: NON_GOAL };
assert.equal(illegal({ ...f, severity: 'MINOR' }), true, 'below its tier');
assert.equal(illegal({ ...f, severity: 'MAJOR' }), true, 'at its tier');
assert.equal(illegal({ ...f, severity: 'BLOCKER' }), true, 'above its tier');
});
test('illegalBriefLowering — brief_lowered_from without a mark is true', () => {
assert.equal(illegal(legalShape({ contradicts_brief: undefined })), true,
'brief_lowered_from MAJOR on an otherwise legal finding without a mark');
const reviewer = { id: 'b'.repeat(40), file: 'lib/b.mjs', line: 7, rule_key: 'MISSING_TEST', severity: 'MAJOR', brief_lowered_from: 'MAJOR' };
assert.equal(illegal(reviewer), true, 'brief_lowered_from on an unmarked reviewer finding');
});
test('illegalBriefLowering — a marked engine finding with an unknown or missing rule_key is true', () => {
assert.equal(illegal(codeReviewDefect({ rule_key: 'NOT_A_KEY', contradicts_brief: NON_GOAL })), true);
assert.equal(illegal(codeReviewDefect({ rule_key: undefined, contradicts_brief: NON_GOAL })), true);
});
test('illegalBriefLowering — contradicts_brief: true (not a string) on the otherwise legal shape is true', () => {
assert.equal(illegal(legalShape({ contradicts_brief: true })), true);
});
test('illegalBriefLowering — an unmarked engine finding below its tier is true', () => {
const sec = {
id: 'c'.repeat(40), file: 'lib/auth.mjs', line: 9, rule_key: 'SECURITY_REVIEW_VULNERABILITY',
severity: 'MINOR', engine: 'security-review',
};
assert.equal(illegal(sec), true, 'unmarked /security-review SECURITY_REVIEW_VULNERABILITY at MINOR');
assert.equal(illegal(codeReviewDefect({ severity: 'MINOR' })), true, 'unmarked /code-review CODE_REVIEW_DEFECT at MINOR');
});
test('illegalBriefLowering — false for engine findings at their tier and for unmarked reviewer findings', () => {
assert.equal(illegal(codeReviewDefect({ contradicts_brief: NON_GOAL })), false, 'a marked engine finding at its tier');
assert.equal(illegal(codeReviewDefect()), false, 'an unmarked engine finding at its tier');
const reviewer = { id: 'b'.repeat(40), file: 'lib/b.mjs', line: 7, rule_key: 'MISSING_TEST' };
assert.equal(illegal({ ...reviewer, severity: 'MINOR' }), false,
'an unmarked reviewer MISSING_TEST at MINOR without brief_lowered_from (the quick-mode shape)');
assert.equal(illegal({ ...reviewer, severity: 'BLOCKER' }), false,
'an unmarked reviewer MISSING_TEST asserted BLOCKER (above its tier)');
});
// ---- review → plan handover --------------------------------------------------
test('review → plan handover — the lowered finding is left out by /trekplan\'s severity filter', () => {
const prose = readFileSync(join(ROOT, 'commands', 'trekplan.md'), 'utf-8');
const line = prose.split('\n').find((l) => l.includes('Filter findings to severity'));
assert.ok(line, 'commands/trekplan.md carries the "Filter findings to severity" line');
const set = line.match(/\{([^}]*)\}/);
assert.ok(set, 'the filter line names a {…} severity set');
const severities = new Set(set[1].split(',').map((s) => s.trim()));
assert.deepEqual([...severities].sort(), ['BLOCKER', 'MAJOR']);
const lowered = m.runContract(payload('claude-code-review', [codeReviewDefect({ contradicts_brief: NON_GOAL })]));
assert.equal(lowered.findings.filter((f) => severities.has(f.severity)).length, 0,
'a brief-lowered finding does not become a plan goal');
const unmarked = m.runContract(payload('claude-code-review', [codeReviewDefect()]));
assert.equal(unmarked.findings.filter((f) => severities.has(f.severity)).length, 1,
'known-positive control: the same finding unmarked does');
});
// ---- the verdict gate --------------------------------------------------------
function runGate(path) {
return spawnSync(process.execPath, [GATE, path], { encoding: 'utf-8' });
}
test('verdict gate — exit 0 on the legal brief lowering', () => {
assert.ok(existsSync(LEGAL_FIXTURE), 'tests/fixtures/review-brief-conflict-legal.md exists');
const r = runGate(LEGAL_FIXTURE);
assert.equal(r.status, 0, `legal lowering must pass: ${r.stdout}${r.stderr}`);
});
test('verdict gate — exit 2 on the illegal brief lowering, the reason naming the finding id', () => {
assert.ok(existsSync(ILLEGAL_FIXTURE), 'tests/fixtures/review-brief-conflict-illegal.md exists');
const text = readFileSync(ILLEGAL_FIXTURE, 'utf-8');
const json = JSON.parse(extractFindingsBlock(text));
assert.equal(json.verdict, 'ALLOW', 'the fixture claims ALLOW');
const id = json.findings[0].id;
assert.match(id, /^[0-9a-f]{40}$/);
const r = runGate(ILLEGAL_FIXTURE);
assert.equal(r.status, 2, `a marked reviewer finding must be refused: ${r.stdout}${r.stderr}`);
assert.ok(r.stdout.includes(id), `the reason names the finding id ${id}: ${r.stdout}`);
});
test('verdict gate — imports illegalBriefLowering from coordinator-contract.mjs (the rule lives once)', () => {
const src = readFileSync(GATE, 'utf-8');
assert.match(src,
/^import\s*\{[^}]*\billegalBriefLowering\b[^}]*\}\s*from\s*['"]\.\/coordinator-contract\.mjs['"]/m,
'verdict-gate.mjs must import illegalBriefLowering from ./coordinator-contract.mjs');
assert.doesNotMatch(src, /function\s+illegalBriefLowering\b/,
'verdict-gate.mjs must not carry its own copy of the rule');
});
// ---- the prose states the same rule (Step 10) --------------------------------
//
// The coordinator is a model reading agents/review-coordinator.md; the gate
// above enforces the rule on what it writes. These pins keep the prose saying
// the same rule the code enforces. Every pin reads a bounded cut, never the
// whole file, so a phrase elsewhere in the file cannot satisfy it.
const readDoc = (...rel) => readFileSync(join(ROOT, ...rel), 'utf-8');
/** The text from `startMarker` up to (not including) the first `endMarker` after it. */
function cut(text, startMarker, endMarker, label) {
const start = text.indexOf(startMarker);
assert.ok(start >= 0, `${label}: start marker missing: ${JSON.stringify(startMarker)}`);
const end = text.indexOf(endMarker, start + startMarker.length);
assert.ok(end > start, `${label}: end marker ${JSON.stringify(endMarker)} missing after the start`);
return text.slice(start, end);
}
/** The sentence from `startPhrase` to its full stop (the first '.' after it). */
function sentenceFrom(text, startPhrase, label) {
const start = text.indexOf(startPhrase);
assert.ok(start >= 0, `${label}: sentence start missing: ${JSON.stringify(startPhrase)}`);
const stop = text.indexOf('.', start + startPhrase.length);
assert.ok(stop > start, `${label}: the sentence has no full stop`);
return text.slice(start, stop + 1);
}
/** One hard-rule bullet, from its bold lead to the next bullet. */
function hardRule(coord, lead) {
const at = coord.indexOf('\n## Hard rules');
assert.ok(at >= 0, 'the ## Hard rules section exists');
const rules = coord.slice(at);
return cut(`${rules}\n- **`, `- **${lead}**`, '\n- **', `hard rule "${lead}"`);
}
test('trekreview Phase 6 — passes the brief\'s Non-Goals, Constraints, Preferences and Open Questions decisions to the coordinator', () => {
const cmd = readDoc('commands', 'trekreview.md');
const p6 = cut(cmd, '## Phase 6 — Coordinator dedup + verdict', '\n## ', 'Phase 6');
for (const section of ['`## Non-Goals`', '`## Constraints`', '`## Preferences`', '`## Open Questions / Assumptions`']) {
assert.ok(p6.includes(section), `Phase 6 names the brief section ${section}`);
}
assert.match(p6, /operator decisions in `## Open Questions \/ Assumptions`/, 'only the operator decisions of Open Questions');
assert.match(p6, /verbatim/, 'the choices are passed verbatim');
assert.match(p6, /brief choices/i, 'Phase 6 names the input the coordinator reads');
});
test('trekreview workflow path — engine, contradicts_brief and brief_lowered_from are carried into review.md', () => {
const cmd = readDoc('commands', 'trekreview.md');
const wf = cut(cmd, 'Then continue at **Phase 7**', '\n\n', 'workflow paragraph');
for (const field of ['`engine`', '`contradicts_brief`', '`brief_lowered_from`']) {
assert.ok(wf.includes(field), `the workflow paragraph names ${field}`);
}
assert.match(wf, /review\.md/, 'the fields are written into review.md');
});
test('review-coordinator Input — names "Brief choices" (the four sections) and the mode line names the exception', () => {
const coord = readDoc('agents', 'review-coordinator.md');
const input = cut(coord, '## Input', '\n## Your 4-pass process', 'Input');
assert.match(input, /\*\*Brief choices\*\*/);
const choices = cut(input, '**Brief choices**', '\n- **', 'Brief choices bullet');
for (const section of ['`## Non-Goals`', '`## Constraints`', '`## Preferences`', '`## Open Questions / Assumptions`']) {
assert.ok(choices.includes(section), `Brief choices names ${section}`);
}
assert.match(choices, /verbatim/);
const mode = cut(input, '- **Mode**', '\n- **', 'mode line');
assert.match(mode, /except the brief-conflict rule, which applies in every mode/);
});
test('review-coordinator Pass 3 — only an engine finding is marked, and the one lowering needs severity MAJOR', () => {
const coord = readDoc('agents', 'review-coordinator.md');
const p3 = cut(coord, '### Pass 3', 'In `quick` mode', 'Pass 3');
assert.match(p3, /`contradicts_brief: "<verbatim>"`/, 'the mark carries the Non-Goal or choice verbatim');
assert.match(p3, /engine finding \(`engine: code-review` or `security-review`\)/, 'only an engine finding is marked');
assert.match(p3, /only an engine finding is ever marked/);
assert.match(p3, /only a `\/code-review` finding at the MAJOR tier and at severity MAJOR then drops to MINOR, with `brief_lowered_from: MAJOR`/,
'the one lowering, severity MAJOR required');
assert.match(p3, /a BLOCKER-tier, a MINOR-tier and any `\/security-review` finding only get the mark/);
assert.match(p3, /no engine finding is otherwise written below its catalogue tier/);
assert.match(p3, /`engine` is written only on a finding whose kept Pass 1 survivor itself came from an engine payload/);
assert.match(p3, /a reviewer-agent finding \(no `engine`\) and a finding whose `rule_key` is not in the catalogue are never marked/);
assert.match(p3, /not the rule key `NON_GOAL_VIOLATED`/);
assert.ok(p3.includes('the brief-conflict reason (`contradicts_brief`), a field, never a `suppressed_reason`'),
'the mark is named as a field, never a suppression reason');
assert.match(p3, /verdict gate refuses \(exit 2\) every other marked or lowered shape/);
});
test('review-coordinator — quick mode and high-effort normalization each still apply the brief-conflict rule', () => {
const coord = readDoc('agents', 'review-coordinator.md');
const pass3 = coord.indexOf('### Pass 3');
assert.ok(pass3 >= 0, 'Pass 3 heading');
const quick = cut(coord.slice(pass3), 'In `quick` mode, skip this pass entirely.', '\n\n', 'quick-mode paragraph');
assert.match(quick, /brief-conflict rule[^.]*still applies/, 'the quick-mode paragraph says the rule still applies');
const high = cut(coord, '**High-effort normalization (v5.1.1):**', '\n### ', 'high-effort paragraph');
assert.match(high, /brief-conflict rule[^.]*still applies/, 'the high-effort paragraph says the rule still applies');
});
test('review-coordinator Pass 2 — the trim rule judges a lowerable brief conflict as MINOR, inside its own sentence', () => {
const coord = readDoc('agents', 'review-coordinator.md');
const s = sentenceFrom(coord, 'The trim decision reads the **catalogue** tier', 'trim rule');
assert.match(s, /except a lowerable brief conflict, which is judged as MINOR \(decide the mark before trimming\)/);
assert.match(s, /so a mislabelled MAJOR is never cut\.$/, 'the sentence still ends on its own guarantee');
});
test('review-coordinator hard rules — the brief re-evaluation, the severity floor and quick mode each name the exception', () => {
const coord = readDoc('agents', 'review-coordinator.md');
assert.match(hardRule(coord, 'Bounded operations only.'), /brief-conflict rule/,
'"You do NOT re-evaluate findings against the brief" names its one exception');
assert.match(hardRule(coord, 'Severity floor is the catalogue.'), /except a lowerable brief conflict, which is MINOR/);
assert.match(hardRule(coord, 'Quick mode is documented.'), /except the brief-conflict rule, which applies in every mode/);
});
test('review-coordinator output — the field list and the trailing JSON example carry engine, contradicts_brief, brief_lowered_from', () => {
const coord = readDoc('agents', 'review-coordinator.md');
const fields = cut(coord, 'Each Findings subsection uses the', '### Trailing JSON block', 'field list');
assert.ok(fields.includes('- `- engine: {code-review | security-review, when the finding came from an engine}`'));
assert.ok(fields.includes('- `- contradicts_brief: {verbatim Non-Goal or choice, only when set}`'));
assert.ok(fields.includes('- `- brief_lowered_from: {MAJOR, only when lowered for the brief}`'));
assert.ok(!fields.includes('original_severity'), 'the brief rule does not read original_severity');
const block = coord.match(/### Trailing JSON block\n\nThe LAST fenced block in the file is a `json` block:\n\n```json\n([\s\S]*?)\n```/);
assert.ok(block, 'the trailing JSON example is present');
const objects = block[1].match(/\{[^{}]*\}/g) ?? [];
const lowered = objects.find((o) => /"brief_lowered_from":\s*"MAJOR"/.test(o));
assert.ok(lowered, 'the JSON example shows a finding with brief_lowered_from');
assert.match(lowered, /"engine":\s*"code-review"/);
assert.match(lowered, /"contradicts_brief":\s*"[^"]+"/);
assert.match(lowered, /"severity":\s*"MINOR"/);
assert.match(lowered, /"rule_key":\s*"CODE_REVIEW_DEFECT"/);
});
test('Handover 6 — names the fields, engine findings only, the one lowering and the five shapes the gate refuses', () => {
const contracts = readDoc('docs', 'HANDOVER-CONTRACTS.md');
const h6 = cut(contracts, '## Handover 6 — `review.md` → plan', '\n## Handover 7', 'Handover 6');
for (const field of ['`engine`', '`contradicts_brief`', '`brief_lowered_from`']) {
assert.ok(h6.includes(field), `Handover 6 names ${field}`);
}
assert.match(h6, /only an engine finding is (?:ever )?marked/i, 'engine findings only');
assert.match(h6, /`\/code-review` MAJOR-tier finding[^.]*MINOR[^.]*`brief_lowered_from: MAJOR`/, 'the one legal lowering');
assert.match(h6, /exits 2/);
const shapes = [
/any other `brief_lowered_from`/,
/a mark on a reviewer-agent finding/,
/a marked finding below its tier/,
/a marked finding with an unknown key/,
/an engine finding below its tier, marked or not/,
];
for (const shape of shapes) assert.match(h6, shape, `Handover 6 names the refused shape ${shape}`);
assert.match(h6, /cannot check an `engine` label on the prose path/);
assert.match(h6, /skips a lowered finding/, 'a plan made from the review skips a lowered (MINOR) finding');
});
// ---- Decision 13 (operator 12.10): the source decides, a tie keeps the reviewer --
//
// "Came from an engine" is read from the payload a finding arrived in, never
// from an `engine` field a reviewer wrote on its own finding; and Pass 1 ranks
// a triplet's findings by catalogue tier first (for a catalogue key the tier
// always ties inside one triplet), then a reviewer-agent finding over an engine
// finding, and the survivor carries the highest severity its group asserted,
// so an engine's mark can never lower what a reviewer agent asserted, whatever
// the order (review bc7f4e23).
function reviewerCopy(extra = {}) {
const { engine: _e, ...rest } = codeReviewDefect({ title: 'Reviewer: no retry', ...extra });
return rest;
}
test('decision 13 — a reviewer payload whose finding labels itself engine code-review is not lowered', () => {
const f = codeReviewDefect({ contradicts_brief: NON_GOAL });
const result = m.runContract(payload('code-correctness-reviewer', [f]));
const out = result.findings[0];
assert.equal(out.severity, 'MAJOR', 'a reviewer MAJOR is never lowered by the brief rule');
assert.equal('brief_lowered_from' in out, false);
assert.equal('engine' in out, false, 'engine comes from the payload source, not the field');
assert.equal('contradicts_brief' in out, false, 'only an engine finding is ever marked');
assert.equal(result.verdict, 'WARN');
assertAllOutputLegal(result);
});
test('decision 13 — an engine payload decides the engine: a /security-review payload finding labelled code-review keeps its severity', () => {
const f = codeReviewDefect({ contradicts_brief: NON_GOAL });
const result = m.runContract(payload('claude-security-review', [f]));
const out = result.findings[0];
assert.equal(out.engine, 'security-review');
assert.equal(out.severity, 'MAJOR');
assert.equal('brief_lowered_from' in out, false);
assert.equal(out.contradicts_brief, NON_GOAL, 'the mark is kept on an engine finding');
assertAllOutputLegal(result);
});
// One reviewer finding and one /code-review finding as two payloads, in `order`.
function reviewerAndEnginePayloads(order, reviewerFinding, engineFinding) {
const reviewerPayload = { reviewer: 'code-correctness-reviewer', findings: [reviewerFinding] };
const enginePayload = { reviewer: 'claude-code-review', findings: [engineFinding] };
return order === 'reviewer first' ? [reviewerPayload, enginePayload] : [enginePayload, reviewerPayload];
}
// A reviewer MISSING_TEST and a marked /code-review MAJOR on the same triplet, in `order`.
function reviewerAndMarkedEngine(order, reviewerExtra = {}) {
return reviewerAndEnginePayloads(order,
reviewerCopy({ rule_key: 'MISSING_TEST', ...reviewerExtra }),
codeReviewDefect({ rule_key: 'MISSING_TEST', contradicts_brief: NON_GOAL }));
}
for (const order of ['reviewer first', 'engine first']) {
test(`decision 13 — a tie between a marked /code-review MAJOR and a reviewer MAJOR keeps the reviewer (${order}) → WARN`, () => {
const result = m.runContract(reviewerAndMarkedEngine(order));
assert.equal(result.findings.length, 1);
const out = result.findings[0];
assert.equal(out.severity, 'MAJOR', 'the reviewer agent asserted MAJOR; the engine mark cannot lower it');
assert.equal('brief_lowered_from' in out, false);
assert.equal('engine' in out, false, 'the survivor is the reviewer finding');
assert.equal('contradicts_brief' in out, false);
assert.deepEqual([...out.raised_by].sort(), ['claude-code-review', 'code-correctness-reviewer']);
assert.equal(result.verdict, 'WARN');
assertAllOutputLegal(result);
});
}
for (const order of ['reviewer first', 'engine first']) {
test(`bc7f4e23 — a marked /code-review MAJOR over a reviewer MINOR on the same triplet keeps the reviewer at MAJOR (${order}) → WARN`, () => {
const result = m.runContract(reviewerAndMarkedEngine(order, { severity: 'MINOR' }));
assert.equal(result.findings.length, 1);
const out = result.findings[0];
assert.equal('engine' in out, false, 'the survivor is the reviewer finding');
assert.equal(out.severity, 'MAJOR', 'the group asserted MAJOR; dropping the engine duplicate never lowers it');
assert.equal('brief_lowered_from' in out, false, 'the engine mark never lowers what a reviewer agent asserted');
assert.equal('contradicts_brief' in out, false);
assert.equal(result.verdict, 'WARN');
assertAllOutputLegal(result);
});
test(`bc7f4e23 — dedupByTriplet(ingest(...)) keeps the reviewer finding at the group's MAJOR over a marked /code-review MAJOR (${order})`, () => {
const { findings } = m.ingest(reviewerAndMarkedEngine(order, { severity: 'MINOR' }));
const out = m.dedupByTriplet(findings);
assert.equal(out.length, 1);
assert.equal('engine' in out[0], false, 'the reviewer-agent finding survives the tier tie');
assert.equal(out[0].reviewer, 'code-correctness-reviewer');
assert.equal(out[0].severity, 'MAJOR', 'the survivor carries the highest severity its group asserted');
assert.equal('contradicts_brief' in out[0], false, 'the engine mark is not carried to the survivor');
assert.deepEqual([...out[0].raised_by].sort(), ['claude-code-review', 'code-correctness-reviewer']);
});
}
test('decision 13 — the prose states the tie rule (coordinator Pass 1, Handover 6)', () => {
const coordinator = readDoc('agents', 'review-coordinator.md');
const pass1 = cut(coordinator, '### Pass 1', '### Pass 2', 'coordinator Pass 1');
assert.match(pass1, /prefer\s+a\s+reviewer-agent\s+finding\s+over\s+an\s+engine\s+finding/,
'Pass 1: a tie keeps the reviewer-agent finding');
assert.match(pass1, /never\s+lowers\s+what\s+a\s+reviewer\s+agent\s+asserted/);
const h6 = cut(readDoc('docs', 'HANDOVER-CONTRACTS.md'), '## Handover 6 — `review.md` → plan', '\n## Handover 7', 'Handover 6');
assert.match(h6, /takes? `engine` from the payload a finding arrived in/);
assert.match(h6, /ranked by catalogue tier[^.]*a reviewer-agent finding survives over an engine finding[^.]*highest severity the group asserted/i,
'Handover 6: Pass 1 ranks by catalogue tier and the survivor carries the group severity (bc7f4e23)');
});
test('b406a628 — coordinator Pass 1 ranks in the library order and writes the group severity (decision 24)', () => {
const coordinator = readDoc('agents', 'review-coordinator.md');
const pass1 = cut(coordinator, '### Pass 1', '### Pass 2', 'coordinator Pass 1');
assert.doesNotMatch(pass1, /highest\s+catalogue\s+severity/, 'Pass 1 no longer keeps the highest catalogue severity');
const ranks = [
/higher\s+catalogue\s+tier/,
/prefer\s+a\s+reviewer-agent\s+finding\s+over\s+an\s+engine\s+finding/,
/higher\s+asserted\s+severity/,
/brief-conformance-reviewer/,
/first\s+seen/,
];
let previous = -1;
for (const rank of ranks) {
const at = pass1.search(rank);
assert.ok(at >= 0, `Pass 1 names the rank ${rank}`);
assert.ok(at > previous, `Pass 1 names the rank ${rank} after the one before it`);
previous = at;
}
assert.match(pass1, /not\s+in\s+the\s+catalogue/, 'Pass 1 names the unknown-key fallback');
assert.match(pass1, /highest\s+severity\s+any\s+finding\s+in\s+its\s+group\s+asserted/,
'Pass 1 writes the group severity');
assert.doesNotMatch(coordinator, /not\s+the\s+reviewer's\s+emitted\s+value/,
'no Hard rule still says the tiebreak ignores the emitted value');
assert.match(coordinator, /Pass\s+1\s+ranks\s+by\s+the\s+catalogue\s+tier\s+first/);
});
// ---- The engine of a payload is read from an own property (review 3799ea6c) --
//
// A payload `reviewer` named after an Object.prototype member must not pick up
// an inherited function or object as its engine: such a payload is a reviewer
// payload, its findings carry no `engine`, and the contract's output is plain
// JSON. The same lookup in engine-findings.mjs refuses an engine named after
// one with the existing "engine must be one of" error.
for (const name of ['toString', 'constructor', 'hasOwnProperty', 'valueOf', '__proto__']) {
test(`own property — a payload reviewer named ${name} gives findings no engine, and runContract's output survives a JSON round trip`, () => {
const { engine: _e, ...finding } = codeReviewDefect();
const ingested = m.ingest(payload(name, [finding]));
assert.equal(ingested.findings.length, 1);
assert.equal('engine' in ingested.findings[0], false,
`a payload reviewer named ${name} is not an engine payload`);
const result = m.runContract(payload(name, [finding]));
assert.deepEqual(JSON.parse(JSON.stringify(result)), result,
'runContract output is plain JSON (no inherited function or object as engine)');
assertAllOutputLegal(result);
});
}
test('own property — toReviewerPayload refuses engine toString with the "engine must be one of" error', () => {
const r = toReviewerPayload({ engine: 'toString', findings: [] });
assert.equal(r.ok, false, 'toString is not an engine');
assert.equal(r.errors.length, 1);
assert.match(r.errors[0], /^engine must be one of code-review\|security-review, got "toString"$/);
});
// ---- original_severity names what the kept finding asserted (review 08deae68) --
//
// Pass 1 writes the survivor at its group's highest asserted severity. When
// Pass 3 then resets it to the catalogue tier, `original_severity` must name
// what the kept finding itself asserted, never the severity a duplicate lent
// it. The Pass-1 bookkeeping never leaves the contract.
// A reviewer copy at `reviewerSeverity` and an engine CODE_REVIEW_DEFECT at
// `engineSeverity` on one triplet, in `order`.
function reviewerAndEngine(order, reviewerSeverity, engineSeverity, reviewerExtra = {}) {
return reviewerAndEnginePayloads(order,
reviewerCopy({ severity: reviewerSeverity, ...reviewerExtra }),
codeReviewDefect({ severity: engineSeverity }));
}
function assertNoPass1Field(result) {
for (const f of [...result.findings, ...result.suppressed, ...result.unverified]) {
assert.equal('pass1_raised_from' in f, false, `pass1_raised_from leaked: ${JSON.stringify(f)}`);
}
assert.equal(JSON.stringify(result).includes('pass1_raised_from'), false,
'no part of the result carries pass1_raised_from');
}
for (const order of ['reviewer first', 'engine first']) {
test(`08deae68 — a reviewer MAJOR raised by an engine BLOCKER ends at MAJOR with no original_severity (${order}) → WARN`, () => {
const result = m.runContract(reviewerAndEngine(order, 'MAJOR', 'BLOCKER'));
assert.equal(result.findings.length, 1);
const out = result.findings[0];
assert.equal('engine' in out, false, 'the survivor is the reviewer finding');
assert.equal(out.severity, 'MAJOR', 'the catalogue tier of CODE_REVIEW_DEFECT');
assert.equal('original_severity' in out, false,
'the kept finding asserted MAJOR, its tier; the BLOCKER was the engine duplicate\'s');
assert.equal(result.verdict, 'WARN');
assertAllOutputLegal(result);
assertNoPass1Field(result);
});
test(`08deae68 — a reviewer MINOR raised by an engine MAJOR ends at MAJOR with original_severity MINOR (${order})`, () => {
const result = m.runContract(reviewerAndEngine(order, 'MINOR', 'MAJOR'));
assert.equal(result.findings.length, 1);
const out = result.findings[0];
assert.equal('engine' in out, false, 'the survivor is the reviewer finding');
assert.equal(out.severity, 'MAJOR');
assert.equal(out.original_severity, 'MINOR', 'the kept finding itself asserted MINOR');
assert.equal(result.verdict, 'WARN');
assertAllOutputLegal(result);
assertNoPass1Field(result);
});
test(`08deae68 — a raised survivor set aside by Pass 2 (empty recommended_action) carries no pass1_raised_from (${order})`, () => {
const result = m.runContract(reviewerAndEngine(order, 'MAJOR', 'BLOCKER', { recommended_action: '' }));
assert.equal(result.findings.length, 0);
assert.equal(result.unverified.length, 1);
const u = result.unverified[0];
assert.equal(u.suppressed_reason, 'actionability:empty');
assert.equal(u.reviewer, 'code-correctness-reviewer', 'the raised survivor is the reviewer finding');
assert.equal(u.severity, 'BLOCKER', 'Pass 1 raised it to its group\'s severity');
assert.equal(result.suppressed.length, 1);
assert.equal(result.verdict, 'WARN');
assertNoPass1Field(result);
});
}

3
tests/scripts/release-check.test.mjs
View file

@ -17,6 +17,7 @@ import { mkdtempSync, mkdirSync, readFileSync, writeFileSync, rmSync, existsSync
import { tmpdir } from 'node:os';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { childEnv } from '../helpers/child-env.mjs';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const CHECK = join(ROOT, 'scripts', 'release-check.mjs');
@ -360,7 +361,7 @@ test('release-check — a red suite is red when the check itself runs under node
const control = run(fx);
assert.equal(control.status, 1, `the control: the suite is red: ${control.stdout}`);
const r = spawnSync(process.execPath, [CHECK, '--repo', fx.repo, '--review', fx.review],
{ encoding: 'utf8', env: { ...process.env, NODE_TEST_CONTEXT: 'child-v8' } });
{ encoding: 'utf8', env: childEnv({ NODE_TEST_CONTEXT: 'child-v8' }) });
assert.equal(r.status, 1, `under a parent test runner: ${r.stdout}`);
assert.match(r.stdout, /npm test exited/);
} finally { fx.cleanup(); }

560
tests/scripts/trekreview-armb-workflow.test.mjs
View file

@ -9,6 +9,9 @@ import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { illegalBriefLowering, severityRank as rank } from '../../lib/review/coordinator-contract.mjs';
import { ENGINES } from '../../lib/review/engine-findings.mjs';
import { RULE_CATALOGUE } from '../../lib/review/rule-catalogue.mjs';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const SRC = readFileSync(join(ROOT, 'scripts', 'trekreview-armB.workflow.mjs'), 'utf-8')
@ -21,8 +24,17 @@ const engine = (reviewer, findings = []) => ({ reviewer, findings });
function stubs({ reviewers = [], coordinator = null } = {}) {
let i = 0;
const calls = { coordinatorPrompt: null, coordinatorOpts: null };
return {
agent: async (_prompt, opts) => (opts.label === 'coordinator' ? coordinator : reviewers[i++] ?? null),
calls,
agent: async (prompt, opts) => {
if (opts.label === 'coordinator') {
calls.coordinatorPrompt = prompt;
calls.coordinatorOpts = opts;
return coordinator;
}
return reviewers[i++] ?? null;
},
parallel: async (thunks) => Promise.all(thunks.map((t) => t())),
phase: () => {},
log: () => {},
@ -69,3 +81,549 @@ test('armB — quick mode expects only the /code-review engine', async () => {
assert.equal(r.verdict, 'ALLOW');
assert.deepEqual(r.missing_engines, []);
});
// ---- Review edf2ec56: a coordinator that returns nothing fails closed ----------------
//
// A coordinator that returns no verdict and findings counts as a missing reviewer:
// its partial list is not trusted (the floors read every merged finding, each raised
// to its catalogue tier, never lowered), ALLOW is forbidden, a BLOCK is never lowered,
// and every reviewer, engine and coordinator that did not deliver is named in
// `allow_blocked_by` as `missing-reviewer:<name>`.
const COORD_MISSING = 'missing-reviewer:review-coordinator';
const MEH_MINOR = () => ({ severity: 'MINOR', rule_key: 'MISSING_ERROR_HANDLING', file: 'src/r.mjs', line: 4, title: 't', detail: 'd' });
const CRI_MINOR = () => ({ severity: 'MINOR', rule_key: 'CODE_REVIEW_IMPROVEMENT', file: 'src/e.mjs', line: 8, title: 't', detail: 'd' });
const UC_BLOCKER = () => ({ severity: 'BLOCKER', rule_key: 'UNIMPLEMENTED_CRITERION', file: 'src/r.mjs', line: 2, title: 't', detail: 'd' });
const BOTH_ENGINES = (code = [], security = []) => [engine('claude-code-review', code), engine('claude-security-review', security)];
async function coordCase({ coordinator, reviewerFindings = [MEH_MINOR()], codeFindings = [CRI_MINOR()] }) {
const s = stubs({
reviewers: [{ reviewer: 'brief-conformance', findings: reviewerFindings }, { reviewer: 'code-correctness', findings: [] }],
coordinator,
});
const r = await exec({ ...BASE, enginePayloads: BOTH_ENGINES(codeFindings) }, s);
return { r, s };
}
const failedCoordinators = [
['returns null', () => null],
['returns findings (a partial list) and no verdict', () => ({ findings: [CRI_MINOR()] })],
['returns ALLOW with no findings array', () => ({ verdict: 'ALLOW' })],
];
for (const [label, make] of failedCoordinators) {
test(`edf2ec56 — a coordinator that ${label} → WARN, every merged finding still MINOR, the coordinator named`, async () => {
const { r, s } = await coordCase({ coordinator: make() });
assert.ok(typeof s.calls.coordinatorPrompt === 'string', 'the coordinator was called');
assert.equal(r.verdict, 'WARN');
assert.deepEqual(r.findings.map((f) => f.rule_key).sort(), ['CODE_REVIEW_IMPROVEMENT', 'MISSING_ERROR_HANDLING'],
'the floors read every merged finding, not a partial list');
for (const f of r.findings) assert.equal(f.severity, 'MINOR');
assert.equal(r.missing_reviewers, 1);
assert.deepEqual(r.allow_blocked_by, [COORD_MISSING]);
});
}
test('edf2ec56 — a coordinator returning an empty findings array and no verdict over a reviewer BLOCKER → BLOCK, the BLOCKER in findings', async () => {
const { r } = await coordCase({ coordinator: { findings: [] }, reviewerFindings: [UC_BLOCKER()], codeFindings: [] });
assert.equal(r.verdict, 'BLOCK');
assert.equal(r.findings.filter((f) => f.rule_key === 'UNIMPLEMENTED_CRITERION' && f.severity === 'BLOCKER').length, 1);
assert.deepEqual(r.allow_blocked_by, [COORD_MISSING]);
});
test('edf2ec56 — a coordinator returning BLOCK and no findings array → BLOCK (never lowered)', async () => {
const { r } = await coordCase({ coordinator: { verdict: 'BLOCK' } });
assert.equal(r.verdict, 'BLOCK');
assert.deepEqual(r.allow_blocked_by, [COORD_MISSING]);
});
test('edf2ec56 — a coordinator returning the verdict PASS (outside BLOCK/WARN/ALLOW) → WARN, the coordinator named', async () => {
const { r } = await coordCase({ coordinator: { verdict: 'PASS', findings: [MEH_MINOR(), CRI_MINOR()] } });
assert.equal(r.verdict, 'WARN');
assert.ok(r.allow_blocked_by.includes(COORD_MISSING));
});
test('edf2ec56 — no engine payloads: both engines are named in allow_blocked_by, WARN', async () => {
const s = stubs({
reviewers: [{ reviewer: 'brief-conformance', findings: [MEH_MINOR()] }, { reviewer: 'code-correctness', findings: [] }],
coordinator: { verdict: 'ALLOW', findings: [MEH_MINOR()] },
});
const r = await exec({ ...BASE }, s);
assert.equal(r.verdict, 'WARN');
assert.deepEqual(r.allow_blocked_by, ['missing-reviewer:claude-code-review', 'missing-reviewer:claude-security-review']);
});
test('edf2ec56 — a first reviewer that returns null is named missing-reviewer:brief-conformance-reviewer', async () => {
const s = stubs({
reviewers: [null, { reviewer: 'code-correctness', findings: [MEH_MINOR()] }],
coordinator: { verdict: 'ALLOW', findings: [MEH_MINOR()] },
});
const r = await exec({ ...BASE, enginePayloads: BOTH_ENGINES() }, s);
assert.equal(r.verdict, 'WARN');
assert.deepEqual(r.allow_blocked_by, ['missing-reviewer:brief-conformance-reviewer']);
});
test('edf2ec56 — quick mode: a correctness reviewer that returns null is named missing-reviewer:code-correctness-reviewer, no brief-conformance-reviewer', async () => {
const s = stubs({ reviewers: [null], coordinator: { verdict: 'ALLOW', findings: [CRI_MINOR()] } });
const r = await exec({ ...BASE, quick: true, enginePayloads: [engine('claude-code-review', [CRI_MINOR()])] }, s);
assert.equal(r.verdict, 'WARN');
assert.deepEqual(r.allow_blocked_by, ['missing-reviewer:code-correctness-reviewer']);
assert.ok(!r.allow_blocked_by.includes('missing-reviewer:brief-conformance-reviewer'));
});
test('edf2ec56 — a null coordinator: a MISSING_TEST asserted MINOR is read at MAJOR (its tier), a MISSING_ERROR_HANDLING asserted BLOCKER stays BLOCKER → BLOCK', async () => {
const mt = { severity: 'MINOR', rule_key: 'MISSING_TEST', file: 'src/r.mjs', line: 5, title: 't', detail: 'd' };
const mehBlocker = { severity: 'BLOCKER', rule_key: 'MISSING_ERROR_HANDLING', file: 'src/r.mjs', line: 6, title: 't', detail: 'd' };
const { r } = await coordCase({ coordinator: null, reviewerFindings: [mt, mehBlocker], codeFindings: [] });
const sevOf = (key) => r.findings.find((f) => f.rule_key === key)?.severity;
assert.equal(sevOf('MISSING_TEST'), 'MAJOR', 'raised to its catalogue tier');
assert.equal(sevOf('MISSING_ERROR_HANDLING'), 'BLOCKER', 'a raise, never a reset');
assert.equal(r.verdict, 'BLOCK');
});
test('edf2ec56 control — a coordinator returning ALLOW with the MINOR findings and both engines → ALLOW, nothing named', async () => {
const { r } = await coordCase({ coordinator: { verdict: 'ALLOW', findings: [MEH_MINOR(), CRI_MINOR()] } });
assert.equal(r.verdict, 'ALLOW');
assert.equal(r.missing_reviewers, 0);
assert.deepEqual(r.allow_blocked_by, []);
});
test('edf2ec56 control — no findings at all → ALLOW, the coordinator skipped and not named', async () => {
const { r, s } = await coordCase({ coordinator: null, reviewerFindings: [], codeFindings: [] });
assert.equal(s.calls.coordinatorPrompt, null, 'the coordinator was skipped');
assert.equal(r.verdict, 'ALLOW');
assert.deepEqual(r.allow_blocked_by, []);
assert.ok(!r.allow_blocked_by.includes(COORD_MISSING));
});
test('edf2ec56 control — a BLOCKER with a null coordinator → BLOCK', async () => {
const { r } = await coordCase({ coordinator: null, reviewerFindings: [UC_BLOCKER()], codeFindings: [] });
assert.equal(r.verdict, 'BLOCK');
assert.deepEqual(r.allow_blocked_by, [COORD_MISSING]);
});
test('edf2ec56 — the workflow section of commands/trekreview.md and Handover 6 name the coordinator; the section names allow_blocked_by', () => {
const cmd = readFileSync(join(ROOT, 'commands', 'trekreview.md'), 'utf-8');
const handover = readFileSync(join(ROOT, 'docs', 'HANDOVER-CONTRACTS.md'), 'utf-8');
const wf = section(cmd, '## Phase 5–6 via the Workflow substrate', '## Phase 7');
assert.match(wf, /missing-reviewer:review-coordinator/);
assert.match(wf, /allow_blocked_by/);
const h6 = section(handover, '## Handover 6 — `review.md` → plan', '\n## Handover 7');
assert.match(h6, /missing-reviewer:review-coordinator/);
});
// ---- Step 9: the brief-conflict rule on the Workflow path ---------------------------
//
// Each case stubs the reviewers' and engines' payloads (so `merged` holds the
// input), the coordinator's returned findings and its verdict, then asserts the
// workflow's verdict. `merged` is read from the prompt the coordinator stub
// receives. For every case the test states the value of `illegalBriefLowering`
// on the subject finding as returned, with `engine` set from its counterpart's
// source (removed when the counterpart is a reviewer finding or absent), and
// checks the floor raised a finding exactly where that value is true — plus
// case 7, where a false `engine` label alone raises.
const MARK = 'Non-Goal: no new runtime dependency';
const FILE = 'src/a.mjs';
const LINE = 10;
const F = (rule_key, severity, extra = {}) => ({ severity, rule_key, file: FILE, line: LINE, title: 't', detail: 'd', ...extra });
const PROMPT_JSON_MARK = '=== PRE-DEDUPED FINDINGS (JSON) ===';
const triplet = (f) => `${f.file}\u0000${f.line}\u0000${f.rule_key}`;
async function floorCase({ reviewerFindings = [], codeFindings = [], securityFindings = [], quick = false, returned, verdict }) {
const before = structuredClone(returned);
const coordinator = { verdict, findings: structuredClone(returned) };
const reviewers = quick
? [{ reviewer: 'code-correctness', findings: structuredClone(reviewerFindings) }]
: [{ reviewer: 'brief-conformance', findings: structuredClone(reviewerFindings) }, { reviewer: 'code-correctness', findings: [] }];
const enginePayloads = quick
? [engine('claude-code-review', structuredClone(codeFindings))]
: [engine('claude-code-review', structuredClone(codeFindings)), engine('claude-security-review', structuredClone(securityFindings))];
const s = stubs({ reviewers, coordinator });
const r = await exec({ ...BASE, quick, enginePayloads }, s);
const prompt = s.calls.coordinatorPrompt;
assert.ok(typeof prompt === 'string', 'the coordinator ran');
const merged = JSON.parse(prompt.slice(prompt.indexOf(PROMPT_JSON_MARK) + PROMPT_JSON_MARK.length));
// Only indexes present in `before` are compared: the floor may append engine
// findings it puts back (decision 23), so `returned` can be longer.
const raised = r.findings.some((f, i) => i < before.length && rank(f.severity) < rank(before[i].severity));
return { r, merged, before, raised, s };
}
// The subject (index 0 of `returned`) with `engine` from its counterpart's source.
function subjectIllegal(before, merged, idx = 0) {
const subject = { ...before[idx] };
const counterpart = merged.find((m) => triplet(m) === triplet(subject));
if (counterpart && counterpart.engine) subject.engine = counterpart.engine;
else delete subject.engine;
return illegalBriefLowering(subject);
}
function checkRaise({ before, merged, raised }, statedIllegal, extraRaise = false) {
assert.equal(subjectIllegal(before, merged), statedIllegal, 'illegalBriefLowering on the subject');
assert.equal(raised, statedIllegal || extraRaise, 'the floor raised a finding exactly where the shape is illegal');
}
test('armB brief rule (1) — an unlabelled SECURITY_REVIEW_VULNERABILITY returned at MINOR over a /security-review BLOCKER → BLOCK', async () => {
const c = await floorCase({
securityFindings: [F('SECURITY_REVIEW_VULNERABILITY', 'BLOCKER')],
returned: [F('SECURITY_REVIEW_VULNERABILITY', 'MINOR')],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
assert.equal(c.r.findings[0].engine, 'security-review');
checkRaise(c, true);
});
test('armB brief rule (2) — an unlabelled CODE_REVIEW_DEFECT returned at MINOR over a /code-review MAJOR with ALLOW → WARN', async () => {
const c = await floorCase({
codeFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
returned: [F('CODE_REVIEW_DEFECT', 'MINOR')],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'WARN');
checkRaise(c, true);
});
for (const withEngine of [true, false]) {
test(`armB brief rule (3) — the legal lowering keeps ALLOW (returned ${withEngine ? 'with' : 'without'} engine) and carries engine code-review`, async () => {
const lowered = F('CODE_REVIEW_DEFECT', 'MINOR', { contradicts_brief: MARK, brief_lowered_from: 'MAJOR' });
if (withEngine) lowered.engine = 'code-review';
const c = await floorCase({
codeFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
returned: [lowered],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'ALLOW');
assert.equal(c.r.findings[0].engine, 'code-review');
assert.equal(c.r.findings[0].severity, 'MINOR');
checkRaise(c, false);
});
}
test('armB brief rule (4) — the legal shape over a reviewer CODE_REVIEW_DEFECT asserted BLOCKER (relabelled engine) → BLOCK, no engine', async () => {
const c = await floorCase({
reviewerFindings: [F('CODE_REVIEW_DEFECT', 'BLOCKER')],
returned: [F('CODE_REVIEW_DEFECT', 'MINOR', { engine: 'code-review', contradicts_brief: MARK, brief_lowered_from: 'MAJOR' })],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
assert.equal(c.r.findings[0].engine, undefined);
checkRaise(c, true);
});
test('armB brief rule (5) — a reviewer finding that carries engine code-review is not an engine finding: the source decides → WARN', async () => {
const c = await floorCase({
reviewerFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR', { engine: 'code-review' })],
returned: [F('CODE_REVIEW_DEFECT', 'MINOR', { engine: 'code-review', contradicts_brief: MARK, brief_lowered_from: 'MAJOR' })],
verdict: 'ALLOW',
});
assert.equal(c.merged[0].engine, undefined, 'a reviewer engine field never reaches merged');
assert.equal(c.r.verdict, 'WARN');
checkRaise(c, true);
});
test('armB brief rule (6) — a marked reviewer MISSING_TEST asserted BLOCKER, returned at MAJOR with WARN → BLOCK', async () => {
const c = await floorCase({
reviewerFindings: [F('MISSING_TEST', 'BLOCKER')],
returned: [F('MISSING_TEST', 'MAJOR', { contradicts_brief: MARK })],
verdict: 'WARN',
});
assert.equal(c.r.verdict, 'BLOCK');
checkRaise(c, true);
});
test('armB brief rule (7) — a reviewer MISSING_TEST asserted BLOCKER, returned at MAJOR with only engine security-review → BLOCK, no engine', async () => {
const c = await floorCase({
reviewerFindings: [F('MISSING_TEST', 'BLOCKER')],
returned: [F('MISSING_TEST', 'MAJOR', { engine: 'security-review' })],
verdict: 'WARN',
});
assert.equal(c.r.verdict, 'BLOCK');
assert.equal(c.r.findings[0].engine, undefined);
checkRaise(c, false, true);
});
test('armB brief rule (8) — a stray brief_lowered_from on a reviewer finding asserted BLOCKER, returned at MAJOR with WARN → BLOCK', async () => {
const c = await floorCase({
reviewerFindings: [F('MISSING_TEST', 'BLOCKER')],
returned: [F('MISSING_TEST', 'MAJOR', { brief_lowered_from: 'MAJOR' })],
verdict: 'WARN',
});
assert.equal(c.r.verdict, 'BLOCK');
checkRaise(c, true);
});
test('armB brief rule (9) — a marked finding with no counterpart and an unknown rule_key, returned at MINOR with ALLOW → BLOCK', async () => {
const other = { severity: 'MINOR', rule_key: 'MISSING_ERROR_HANDLING', file: 'src/b.mjs', line: 3, title: 't', detail: 'd' };
const c = await floorCase({
reviewerFindings: [other],
returned: [{ severity: 'MINOR', rule_key: 'NOT_IN_THE_CATALOGUE', file: 'src/ghost.mjs', line: 7, title: 't', detail: 'd', contradicts_brief: MARK }, other],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
checkRaise(c, true);
});
test('armB brief rule (10) — a marked BLOCKER-tier /code-review finding returned at MINOR → BLOCK', async () => {
const c = await floorCase({
codeFindings: [F('SECURITY_INJECTION', 'BLOCKER')],
returned: [F('SECURITY_INJECTION', 'MINOR', { engine: 'code-review', contradicts_brief: MARK })],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
checkRaise(c, true);
});
test('armB brief rule (11) — bc7f4e23: dedup ranks by tier and keeps the reviewer over a /code-review MAJOR on the same triplet, at the group\'s MAJOR → WARN', async () => {
const c = await floorCase({
reviewerFindings: [F('CODE_REVIEW_DEFECT', 'MINOR')],
codeFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
returned: [F('CODE_REVIEW_DEFECT', 'MINOR')],
verdict: 'ALLOW',
});
assert.equal(c.merged.length, 1);
assert.equal(c.merged[0].severity, 'MAJOR', 'the survivor carries the highest severity its group asserted');
assert.equal('engine' in c.merged[0], false, 'the reviewer finding survives the tier tie');
assert.equal(c.r.deduped_count, 1);
assert.equal(c.r.raw_finding_count, 2);
assert.equal(c.r.verdict, 'WARN');
checkRaise(c, false, true);
});
test('armB brief rule (12) — an unlabelled reviewer MISSING_TEST asserted BLOCKER, returned at MAJOR (catalogue reset) with WARN stays WARN', async () => {
const c = await floorCase({
reviewerFindings: [F('MISSING_TEST', 'BLOCKER')],
returned: [F('MISSING_TEST', 'MAJOR')],
verdict: 'WARN',
});
assert.equal(c.r.verdict, 'WARN');
checkRaise(c, false);
});
test('armB brief rule (13) — a marked MINOR-tier engine finding at its tier (CODE_REVIEW_IMPROVEMENT) keeps ALLOW', async () => {
const c = await floorCase({
codeFindings: [F('CODE_REVIEW_IMPROVEMENT', 'MINOR')],
returned: [F('CODE_REVIEW_IMPROVEMENT', 'MINOR', { engine: 'code-review', contradicts_brief: MARK })],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'ALLOW');
assert.equal(c.r.findings[0].engine, 'code-review');
checkRaise(c, false);
});
test('armB brief rule (14) — an unlabelled reviewer MISSING_TEST at MINOR (quick mode) keeps the coordinator verdict', async () => {
const c = await floorCase({
quick: true,
reviewerFindings: [F('MISSING_TEST', 'MINOR')],
returned: [F('MISSING_TEST', 'MINOR')],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'ALLOW');
checkRaise(c, false);
});
test('armB brief rule — RULE_TIERS in the script equals the catalogue (same keys, same tiers)', () => {
const start = SRC.indexOf('const RULE_TIERS = {');
assert.ok(start !== -1, 'the script carries a RULE_TIERS literal block');
const block = SRC.slice(start, SRC.indexOf('};', start));
const tiers = Object.fromEntries([...block.matchAll(/([A-Z_]+):\s*'([A-Z]+)'/g)].map((m) => [m[1], m[2]]));
const catalogue = Object.fromEntries(RULE_CATALOGUE.map((r) => [r.rule_key, r.severity]));
assert.deepEqual(tiers, catalogue);
});
test('armB brief rule — ENGINE_OF_PAYLOAD in the script maps each ENGINES reviewer name to its engine', () => {
const start = SRC.indexOf('const ENGINE_OF_PAYLOAD = {');
assert.ok(start !== -1, 'the script carries an ENGINE_OF_PAYLOAD literal block');
const block = SRC.slice(start, SRC.indexOf('};', start));
const map = Object.fromEntries([...block.matchAll(/'([a-z-]+)':\s*'([a-z-]+)'/g)].map((m) => [m[1], m[2]]));
const expected = Object.fromEntries(Object.entries(ENGINES).map(([engine, spec]) => [spec.reviewer, engine]));
assert.deepEqual(map, expected);
});
test('armB brief rule — VERDICT_SCHEMA declares engine, contradicts_brief and brief_lowered_from; the prompt states the rule', async () => {
const c = await floorCase({
codeFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
returned: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
verdict: 'WARN',
});
const props = c.s.calls.coordinatorOpts.schema.properties.findings.items.properties;
assert.deepEqual(props.engine, { type: 'string', enum: ['code-review', 'security-review'] });
assert.deepEqual(props.contradicts_brief, { type: 'string' });
assert.deepEqual(props.brief_lowered_from, { type: 'string', enum: ['MAJOR'] });
const prompt = c.s.calls.coordinatorPrompt;
assert.ok(prompt.includes('contradicts_brief'));
assert.ok(prompt.includes('brief_lowered_from'));
assert.ok(prompt.includes('## Non-Goals'));
assert.ok(prompt.includes('## Constraints'));
assert.ok(prompt.includes('## Preferences'));
assert.ok(prompt.includes('## Open Questions / Assumptions'));
assert.ok(/only an engine finding/i.test(prompt), 'engine findings only');
assert.ok(/kept and marked, never removed/i.test(prompt));
});
// Decision 13 (operator 12.10), bc7f4e23: dedup ranks by catalogue tier, which
// always ties inside a catalogue triplet, so a reviewer-agent finding survives
// over an engine finding and carries the group's highest asserted severity; the
// engine's mark can never lower what a reviewer agent asserted (the same order
// as coordinator-contract Pass 1).
test('armB brief rule (15) — a tie between a reviewer MAJOR and a /code-review MAJOR keeps the reviewer: the legal shape returned with ALLOW → WARN, no engine', async () => {
const c = await floorCase({
reviewerFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
codeFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
returned: [F('CODE_REVIEW_DEFECT', 'MINOR', { contradicts_brief: MARK, brief_lowered_from: 'MAJOR', engine: 'code-review' })],
verdict: 'ALLOW',
});
assert.equal(c.merged.length, 1);
assert.equal(c.merged[0].severity, 'MAJOR');
assert.equal('engine' in c.merged[0], false, 'the tie keeps the reviewer finding');
assert.equal(c.r.verdict, 'WARN');
assert.equal(c.r.findings[0].severity, 'MAJOR');
assert.equal('engine' in c.r.findings[0], false);
checkRaise(c, true);
});
// ---- Decision 23 (operator 13.10, review ab1b5d9f): no engine finding leaves ----
// the workflow review unreturned. Every engine finding of the input (before
// dedup) with no returned finding at its exact (file, line, rule_key) is put
// back at its own severity; a returned finding is never below the lower of its
// reviewer counterpart's severity and its catalogue tier.
const at = (findings, file, line, rule_key) =>
findings.filter((f) => f && f.file === file && f.line === line && f.rule_key === rule_key);
test('decision 23 — a /security-review BLOCKER returned as a CODE_REVIEW_IMPROVEMENT MINOR on the same line → BLOCK, the BLOCKER is put back', async () => {
const c = await floorCase({
securityFindings: [F('SECURITY_REVIEW_VULNERABILITY', 'BLOCKER')],
returned: [F('CODE_REVIEW_IMPROVEMENT', 'MINOR')],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
const back = at(c.r.findings, FILE, LINE, 'SECURITY_REVIEW_VULNERABILITY');
assert.equal(back.length, 1, 'the BLOCKER is in the findings');
assert.equal(back[0].severity, 'BLOCKER');
assert.equal(back[0].engine, 'security-review');
assert.equal('brief_lowered_from' in back[0], false);
});
test('decision 23 — the same BLOCKER returned one line lower at MINOR → BLOCK, the BLOCKER is back at its own line', async () => {
const c = await floorCase({
securityFindings: [F('SECURITY_REVIEW_VULNERABILITY', 'BLOCKER')],
returned: [F('SECURITY_REVIEW_VULNERABILITY', 'MINOR', { line: LINE + 1 })],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
const back = at(c.r.findings, FILE, LINE, 'SECURITY_REVIEW_VULNERABILITY');
assert.equal(back.length, 1, 'the BLOCKER at its own line is in the findings');
assert.equal(back[0].severity, 'BLOCKER');
assert.equal(back[0].engine, 'security-review');
});
test('decision 23 — the same BLOCKER dropped (returned empty with ALLOW) → BLOCK', async () => {
const c = await floorCase({
securityFindings: [F('SECURITY_REVIEW_VULNERABILITY', 'BLOCKER')],
returned: [],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
const back = at(c.r.findings, FILE, LINE, 'SECURITY_REVIEW_VULNERABILITY');
assert.equal(back.length, 1);
assert.equal(back[0].severity, 'BLOCKER');
assert.equal(back[0].engine, 'security-review');
});
test('decision 23 — a /security-review BLOCKER shadowed in dedup by a reviewer MINOR on the same triplet, returned empty → BLOCK', async () => {
const c = await floorCase({
reviewerFindings: [F('SECURITY_REVIEW_VULNERABILITY', 'MINOR')],
securityFindings: [F('SECURITY_REVIEW_VULNERABILITY', 'BLOCKER')],
returned: [],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
const back = at(c.r.findings, FILE, LINE, 'SECURITY_REVIEW_VULNERABILITY');
assert.equal(back.length, 1);
assert.equal(back[0].severity, 'BLOCKER');
assert.equal(back[0].engine, 'security-review');
});
test('decision 23 — a /code-review MAJOR dropped → WARN', async () => {
const c = await floorCase({
codeFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
returned: [],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'WARN');
const back = at(c.r.findings, FILE, LINE, 'CODE_REVIEW_DEFECT');
assert.equal(back.length, 1);
assert.equal(back[0].severity, 'MAJOR');
assert.equal(back[0].engine, 'code-review');
});
for (const order of ['MAJOR first', 'BLOCKER first']) {
test(`decision 23 — two /code-review findings on one triplet (${order}), returned empty → BLOCK, one copy, at BLOCKER`, async () => {
const pair = [F('CODE_REVIEW_DEFECT', 'MAJOR', { title: 'major' }), F('CODE_REVIEW_DEFECT', 'BLOCKER', { title: 'blocker' })];
const c = await floorCase({
codeFindings: order === 'MAJOR first' ? pair : [pair[1], pair[0]],
returned: [],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'BLOCK');
const back = at(c.r.findings, FILE, LINE, 'CODE_REVIEW_DEFECT');
assert.equal(back.length, 1, 'exactly one finding at the triplet');
assert.equal(back[0].severity, 'BLOCKER', 'the higher-severity copy is put back');
assert.equal(back[0].title, 'blocker');
assert.equal(back[0].engine, 'code-review');
});
}
test('decision 23 floor (b) — a reviewer finding with a rule_key outside the catalogue asserted MAJOR, returned at MINOR → raised to MAJOR, WARN', async () => {
const c = await floorCase({
reviewerFindings: [F('NOT_IN_THE_CATALOGUE', 'MAJOR')],
returned: [F('NOT_IN_THE_CATALOGUE', 'MINOR')],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'WARN');
assert.equal(c.r.findings.length, 1);
assert.equal(c.r.findings[0].severity, 'MAJOR', "the bound for an unknown key is the counterpart's asserted severity");
});
test('decision 23 — the one legal lowering (marked /code-review MAJOR returned as MINOR with the mark) still gives ALLOW', async () => {
const c = await floorCase({
codeFindings: [F('CODE_REVIEW_DEFECT', 'MAJOR')],
returned: [F('CODE_REVIEW_DEFECT', 'MINOR', { contradicts_brief: MARK, brief_lowered_from: 'MAJOR' })],
verdict: 'ALLOW',
});
assert.equal(c.r.verdict, 'ALLOW');
assert.equal(c.r.findings.length, 1, 'nothing is put back over a returned triplet');
assert.equal(c.r.findings[0].severity, 'MINOR');
assert.equal(c.r.findings[0].engine, 'code-review');
});
// ---- the docs state the workflow floor ------------------------------------------
// Slice from the first `start` to the next `end` after it; fail when either is missing.
function section(text, start, end) {
const i = text.indexOf(start);
assert.ok(i !== -1, `section start not found: ${start}`);
const j = text.indexOf(end, i + start.length);
assert.ok(j !== -1, `section end not found after ${start}: ${JSON.stringify(end)}`);
return text.slice(i, j);
}
const CMD_TEXT = readFileSync(join(ROOT, 'commands', 'trekreview.md'), 'utf-8');
const HANDOVER_TEXT = readFileSync(join(ROOT, 'docs', 'HANDOVER-CONTRACTS.md'), 'utf-8');
const PUT_BACK = /put\s+back\s+at\s+its\s+own\s+severity/;
const FLOOR_B = /lower\s+of\s+its\s+reviewer\s+counterpart's\s+severity\s+and\s+its\s+catalogue\s+tier/;
test('decision 23 — commands/trekreview.md workflow section states the put-back and floor (b); the port is no longer the SAME pipeline', () => {
const wf = section(CMD_TEXT, '## Phase 5–6 via the Workflow substrate', '## Phase 7');
assert.match(wf, PUT_BACK);
assert.match(wf, FLOOR_B);
assert.doesNotMatch(CMD_TEXT, /the\s+SAME\s+Phase/);
});
test('decision 23 — Handover 6 states the dedup rank on both paths, the put-back and floor (b)', () => {
const h6 = section(HANDOVER_TEXT, '## Handover 6 — `review.md` → plan', '\n## Handover 7');
assert.match(h6, /dedup\s+ranks\s+as\s+Pass\s+1\s+does\s+\(both\s+paths\)/);
assert.match(h6, PUT_BACK);
assert.match(h6, FLOOR_B);
});

56
tests/validators/brief-review-stamp-order.test.mjs Normal file
View file

@ -0,0 +1,56 @@
// tests/validators/brief-review-stamp-order.test.mjs
// /trekbrief stamps only text the reviewer saw. A `small` brief gets one
// review; its gaps are written as assumptions. When that writing came before
// the stamp, the stamp covered text no reviewer read and /trekplan skipped its
// own review. The gaps are now written after the stamp, on purpose, so the
// stamp reads stale and /trekplan reviews the brief once more.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { stampBriefReview, checkBriefReview } from '../../lib/validators/brief-review-stamp.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const brief = readFileSync(join(ROOT, 'commands', 'trekbrief.md'), 'utf8');
const STAMP = 'brief-review-stamp.mjs --stamp';
const GAPS = "Class `small` only: now write the reviewer's gaps";
const PASS_START = '**If gate passes:**';
const PASS_END = '**If gate fails AND iteration count < the class cap:**';
function unique(text, needle) {
const at = text.indexOf(needle);
assert.ok(at >= 0, `found: ${needle}`);
assert.equal(text.lastIndexOf(needle), at, `unique in commands/trekbrief.md: ${needle}`);
return at;
}
test('trekbrief: the stamp comes before the small-class gaps are written, both in the passed-gate branch', () => {
const start = unique(brief, PASS_START);
const end = unique(brief, PASS_END);
const stamp = unique(brief, STAMP);
const gaps = unique(brief, GAPS);
assert.ok(start < end, 'passed-gate branch precedes the failed-gate branch');
assert.ok(start < stamp && stamp < end, 'the stamp sits in the passed-gate branch');
assert.ok(start < gaps && gaps < end, 'the gaps instruction sits in the passed-gate branch');
assert.ok(stamp < gaps, 'order: stamp, then the small-class gaps');
const after = brief.slice(gaps, end);
assert.match(after, /They come after the stamp on purpose/);
assert.match(after, /## Open Questions \/ Assumptions/);
});
test('guard: stamping a brief and then appending an assumption reads as stale', () => {
const text = [
'---', 'type: trekbrief', 'brief_version: "2.2"', 'task: t', 'slug: s', 'ceremony: small', '---', '',
'## TL;DR', '', 'A toy.', '', '## Intent', '', 'Why it matters.', '', '## Goal', '', 'What.', '',
'## Open Questions / Assumptions', '', '- None.', '',
].join('\n');
const s = stampBriefReview(text);
assert.equal(s.stamped, true);
assert.equal(checkBriefReview(s.text).state, 'current');
const withGap = s.text + '- Assumed, not asked: the setting defaults to off (reviewer gap).\n';
assert.equal(checkBriefReview(withGap).state, 'stale');
});

3
tests/validators/intent-approval.test.mjs
View file

@ -20,6 +20,7 @@ import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync, existsSync
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
@ -78,7 +79,7 @@ function commandLine(rel, verb) {
function run(line, subs, env = {}) {
let cmd = line.split('${CLAUDE_PLUGIN_ROOT}').join(ROOT);
for (const [k, v] of Object.entries(subs)) cmd = cmd.split(k).join(v);
return spawnSync('bash', ['-c', cmd], { encoding: 'utf8', env: { ...process.env, ...env } });
return spawnSync('bash', ['-c', cmd], { encoding: 'utf8', env: childEnv(env) });
}
function gate(briefPath) {

112
tests/validators/plan-validator-parse-once.test.mjs Normal file
View file

@ -0,0 +1,112 @@
// tests/validators/plan-validator-parse-once.test.mjs
// The plan validator parses each step's fields once (review finding 93382464):
// one private `withFields(sections)` maps each section to its parsed fields,
// and every check reads `s.fields` instead of calling `parseStepFields` again.
// The output must not change: the expected codes below were recorded from
// `validatePlanContent` at the start point of the change (1621352), each with
// the step number its message names, in order.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
import { validatePlanContent } from '../../lib/validators/plan-validator.mjs';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const SOURCE = readFileSync(join(ROOT, 'lib', 'validators', 'plan-validator.mjs'), 'utf8');
// The body of a top-level function: from its declaration to the next
// top-level `function` or `export`. '' when the function is not there.
function bodyOf(name) {
const start = SOURCE.search(new RegExp(`^function ${name}\\(`, 'm'));
if (start < 0) return '';
const rest = SOURCE.slice(start + 1);
const next = rest.search(/^(?:async )?function |^export /m);
return next < 0 ? SOURCE.slice(start) : SOURCE.slice(start, start + 1 + next);
}
// Each issue as CODE@step (the step its message names), so order and place are pinned.
const tag = (x) => {
const m = /\b[Ss]tep (\d+)/.exec(x.message);
return m ? `${x.code}@${m[1]}` : x.code;
};
const codes = (r) => ({ errors: r.errors.map(tag), warnings: r.warnings.map(tag) });
const E2E = { endToEnd: 'bash tests/e2e.sh' };
const TESTS_ONLY = { file: 'tests/only.test.mjs', test: 'tests/only.test.mjs', manifestPaths: ['tests/only.test.mjs'] };
const STEP3 = '### Step 3: Fix it\n\n- **Files:** `src/fix.mjs`, `tests/fix.test.mjs`\n- **Changes:** fix it\n' +
'- **On failure:** escalate\n';
const STEP4 = '### Step 4: Check it\n\n- **Files:** `src/check.mjs`\n- **Changes:** check it\n' +
'- **Verify:** `curl http://localhost/x` → expected: 200\n- **Regression:** `npm test | cat` → expected: exit 0\n' +
'- **On failure:** maybe later\n';
const appended = (plan, ...steps) => plan.replace('\n## Test Strategy', `\n${steps.join('\n')}\n## Test Strategy`);
test('parse once: the validator source holds one parseStepFields( call, inside function withFields', () => {
const calls = SOURCE.match(/parseStepFields\(/g) ?? [];
assert.equal(calls.length, 1, `parseStepFields( occurs ${calls.length} times in lib/validators/plan-validator.mjs`);
const body = bodyOf('withFields');
assert.ok(body !== '', 'lib/validators/plan-validator.mjs declares function withFields');
assert.match(body, /parseStepFields\(/, 'the one parseStepFields( call stands inside function withFields');
});
for (const name of ['checkSteps', 'checkRunnable', 'checkRecovery', 'checkPathGrammar']) {
test(`parse once: ${name} reads .fields in its body`, () => {
const body = bodyOf(name);
assert.ok(body !== '', `function ${name} is declared`);
assert.match(body, /\.fields\b/, `${name} reads the parsed fields (.fields) instead of parsing again`);
});
}
test('parse once: a 1.7 and a 1.8 plan with a tests-only step give the codes recorded at the start point', () => {
for (const version of ['1.7', '1.8']) {
const plan = buildPlan({ version, steps: [TESTS_ONLY, E2E] });
assert.deepEqual(codes(validatePlanContent(plan)), { errors: [], warnings: ['PLAN_STEP_TESTS_ONLY@1'] }, version);
}
});
test('parse once: a 1.8 plan with an unticked path gives the codes recorded at the start point', () => {
const plan = buildPlan({ version: '1.8', steps: [{ filesRaw: '`src/mod1.mjs`, src/plain.mjs - the parser' }, E2E] });
const want = { errors: [], warnings: ['PLAN_STEP_PATH_UNTICKED@1'] };
assert.deepEqual(codes(validatePlanContent(plan)), want);
assert.deepEqual(codes(validatePlanContent(plan, { strict: false })), want);
});
test('parse once: a 1.9 plan with refused entries gives the codes recorded at the start point', () => {
const plan = buildPlan({
version: '1.9',
steps: [{ filesRaw: 'the parser `src/mod1.mjs`', testFirstRaw: ' - File: `parse()` then `tests/mod1.test.mjs`' }, E2E],
});
const refused = ['PLAN_STEP_PATH_NOT_FIRST@1', 'PLAN_STEP_PATH_NOT_A_PATH@1', 'PLAN_STEP_NO_TEST_FIRST@1'];
assert.deepEqual(codes(validatePlanContent(plan)), { errors: refused, warnings: [] });
assert.deepEqual(codes(validatePlanContent(plan, { strict: false })), { errors: [], warnings: refused });
});
test('parse once: --steps-from gives the codes recorded at the start point', () => {
assert.deepEqual(codes(validatePlanContent(appended(buildPlan({ version: '1.7' }), STEP3), { strict: true, stepsFrom: 3 })), {
errors: ['MANIFEST_MISSING@3', 'PLAN_STEP_NO_TEST_FIRST@3', 'PLAN_STEP_VERIFY_NOT_RUNNABLE@3'],
warnings: [],
});
assert.deepEqual(codes(validatePlanContent(appended(buildPlan({ version: '1.8' }), STEP3), { strict: true, stepsFrom: 3 })), {
errors: ['MANIFEST_MISSING@3', 'PLAN_STEP_VERIFY_NOT_RUNNABLE@3', 'PLAN_STEP_NO_TEST_FIRST@3'],
warnings: [],
});
const two = appended(buildPlan({ version: '1.7' }), STEP3, STEP4);
assert.deepEqual(codes(validatePlanContent(two, { strict: true, stepsFrom: 3 })), {
errors: [
'MANIFEST_MISSING@3', 'MANIFEST_MISSING@4', 'PLAN_STEP_REGRESSION_NOT_RUNNABLE@4',
'PLAN_STEP_NO_TEST_FIRST@3', 'PLAN_STEP_NO_TEST_FIRST@4',
'PLAN_STEP_VERIFY_NOT_RUNNABLE@4', 'PLAN_STEP_VERIFY_NOT_RUNNABLE@3',
],
warnings: ['PLAN_STEP_ON_FAILURE_UNKNOWN@4'],
});
assert.deepEqual(codes(validatePlanContent(two, { strict: false, stepsFrom: 3 })), {
errors: ['MANIFEST_MISSING@3', 'MANIFEST_MISSING@4'],
warnings: [
'PLAN_STEP_REGRESSION_NOT_RUNNABLE@4', 'PLAN_STEP_ON_FAILURE_UNKNOWN@4',
'PLAN_STEP_NO_TEST_FIRST@3', 'PLAN_STEP_NO_TEST_FIRST@4',
'PLAN_STEP_VERIFY_NOT_RUNNABLE@4', 'PLAN_STEP_VERIFY_NOT_RUNNABLE@3',
],
});
});

427
tests/validators/plan-version-1-9.test.mjs Normal file
View file

@ -0,0 +1,427 @@
// tests/validators/plan-version-1-9.test.mjs
// plan_version 1.9: each `Files:` entry opens with its backticked path(s), and
// everything after the first other text is description that is never read.
// An entry that opens with anything else is refused (an error under --strict,
// a warning under --soft). Decision 16 (operator 11.10): only a 1.9 plan is
// refused; a plan below 1.9 is read as before and gets one warning,
// PLAN_STEP_PATH_UNTICKED, for a path its reader found outside backticks.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import * as schema from '../../lib/parsers/plan-schema.mjs';
import { validatePlanContent } from '../../lib/validators/plan-validator.mjs';
import { gatePlan, recordVerdict, planHash } from '../../lib/validators/plan-gate.mjs';
import { buildPlan } from '../helpers/plan-v18-builder.mjs';
import { plannedPaths } from '../../lib/verification/plan-scope-sweep.mjs';
import { childEnv } from '../helpers/child-env.mjs';
import { spawnSync } from 'node:child_process';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, readFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const errCodes = (r) => r.errors.map((e) => e.code);
const warnCodes = (r) => r.warnings.map((w) => w.code);
const PATH_CODES = ['PLAN_STEP_PATH_NOT_FIRST', 'PLAN_STEP_PATH_NOT_A_PATH'];
const pathIssues = (list) => list.filter((i) => PATH_CODES.includes(i.code));
// A 1.9 plan whose first step's `Files:` value is `files` (verbatim).
const plan19 = (files) => buildPlan({ version: '1.9', steps: [{ filesRaw: files }, { endToEnd: 'bash tests/e2e.sh' }] });
const BOTH = [{ agent: 'plan-critic', present: true }, { agent: 'scope-guardian', present: true }];
const passedCritic = (text) => recordVerdict(text, { findings: [], sources: BOTH, dedup_stats: { total_in: 0, total_out: 0 } },
{ reviewed: planHash(text) });
function entryPaths(text) {
assert.equal(typeof schema.entryPaths, 'function', 'plan-schema exports entryPaths');
return schema.entryPaths(text);
}
test('1.9 is the current target: PLAN_VERSION_CURRENT === \'1.9\'', () => {
assert.equal(schema.PLAN_VERSION_CURRENT, '1.9');
});
test('1.9: a plan_version 1.9 plan is supported - no PLAN_VERSION_MISMATCH, and the builder plan validates strict', () => {
const r = validatePlanContent(buildPlan({ version: '1.9' }));
assert.equal(r.warnings.find((w) => w.code === 'PLAN_VERSION_MISMATCH'), undefined, JSON.stringify(r.warnings));
assert.deepEqual(r.errors, [], JSON.stringify(r.errors, null, 2));
assert.equal(r.parsed.planVersion, '1.9');
});
test('1.9: a Files entry without a leading backticked path is an error under --strict and a warning under --soft', () => {
const text = plan19('src/mod1.mjs - the parser');
const strict = validatePlanContent(text);
assert.equal(strict.valid, false);
const e = strict.errors.find((i) => i.code === 'PLAN_STEP_PATH_NOT_FIRST');
assert.ok(e, errCodes(strict).join(','));
assert.match(e.message, /Step 1/);
assert.match(e.message, /src\/mod1\.mjs - the parser/);
const soft = validatePlanContent(text, { strict: false });
assert.ok(warnCodes(soft).includes('PLAN_STEP_PATH_NOT_FIRST'), warnCodes(soft).join(','));
assert.deepEqual(pathIssues(soft.errors), []);
});
test('1.9: "`tests/a.test.mjs` - covers `src/parse.mjs`" names only the test file', () => {
const r = entryPaths('`tests/a.test.mjs` - covers `src/parse.mjs`');
assert.deepEqual(r.files, ['tests/a.test.mjs']);
assert.deepEqual(r.issues, []);
const v = validatePlanContent(plan19('`tests/a.test.mjs` - covers `src/parse.mjs`'));
assert.deepEqual(pathIssues(v.errors), [], JSON.stringify(v.errors));
});
test('1.9: several leading paths separated by commas or whitespace are all files', () => {
assert.deepEqual(entryPaths('`src/a.mjs`, `./src/b.mjs` `tests/a.test.mjs` *(new)*, `src/c.mjs`').files,
['src/a.mjs', 'src/b.mjs', 'tests/a.test.mjs']);
});
test('1.9: a wrapped description line is not an entry', () => {
const text = '`src/a.mjs` - the parser, which\n reads src/b.mjs and the CLI options';
const r = entryPaths(text);
assert.deepEqual(r.files, ['src/a.mjs']);
assert.deepEqual(r.issues, []);
const v = validatePlanContent(plan19(text));
assert.deepEqual(pathIssues(v.errors), [], JSON.stringify(v.errors));
});
test('1.9: an empty first line with a nested list is not an entry; each list item is one', () => {
const text = '\n - `src/a.mjs` - the parser\n - `tests/a.test.mjs` (new)';
const r = entryPaths(text);
assert.deepEqual(r.files, ['src/a.mjs', 'tests/a.test.mjs']);
assert.deepEqual(r.issues, []);
const v = validatePlanContent(plan19(text));
assert.deepEqual(pathIssues(v.errors), [], JSON.stringify(v.errors));
// A list item that does not open with a path is refused.
const bad = entryPaths('\n - `src/a.mjs`\n - the CLI in src/cli.mjs');
assert.deepEqual(bad.files, ['src/a.mjs']);
assert.deepEqual(bad.issues.map((i) => i.code), ['PLAN_STEP_PATH_NOT_FIRST']);
assert.match(bad.issues[0].entry, /the CLI in src\/cli\.mjs/);
});
test('1.9: a leading `parse()` is refused as no path', () => {
const r = entryPaths('`parse()` - the parser entry');
assert.deepEqual(r.files, []);
assert.deepEqual(r.issues.map((i) => i.code), ['PLAN_STEP_PATH_NOT_A_PATH']);
const v = validatePlanContent(plan19('`parse()` - the parser entry'));
assert.ok(errCodes(v).includes('PLAN_STEP_PATH_NOT_A_PATH'), errCodes(v).join(','));
});
test('1.9: the shape test is the reader\'s leading-position test', () => {
assert.equal(typeof schema.isPathShape, 'function', 'plan-schema exports isPathShape');
for (const s of ['LICENSE', 'Makefile', 'src/a.mjs', 'package.json', 'docs/']) assert.equal(schema.isPathShape(s), true, s);
for (const s of ['parse()', '-x', 'https://example.com/a.md', 'format']) assert.equal(schema.isPathShape(s), false, s);
});
test('1.9: `none` and an absent `Files:` are refused', () => {
const none = entryPaths('none');
assert.deepEqual(none.files, []);
assert.deepEqual(none.issues.map((i) => i.code), ['PLAN_STEP_PATH_NOT_FIRST']);
const empty = entryPaths('');
assert.deepEqual(empty.files, []);
assert.deepEqual(empty.issues.map((i) => i.code), ['PLAN_STEP_PATH_NOT_FIRST']);
assert.ok(errCodes(validatePlanContent(plan19('none'))).includes('PLAN_STEP_PATH_NOT_FIRST'));
const absent = buildPlan({ version: '1.9' }).replace(/^- \*\*Files:\*\* `src\/mod1\.mjs`\n/m, '');
assert.ok(!/Files:\*\* `src\/mod1\.mjs`/.test(absent), 'the fixture drops Step 1\'s Files line');
const r = validatePlanContent(absent);
const e = r.errors.find((i) => i.code === 'PLAN_STEP_PATH_NOT_FIRST');
assert.ok(e, errCodes(r).join(','));
assert.match(e.message, /Step 1/);
});
test('1.9: the tests-only check reads Files by the grammar', () => {
const body = '- **Files:** `tests/a.test.mjs` - covers `src/parse.mjs`\n- **Changes:** x\n';
assert.deepEqual(schema.stepTestsOnly(body, '1.9'), { testsOnly: true, files: ['tests/a.test.mjs'] });
});
test('below 1.9: an unticked path gets PLAN_STEP_PATH_UNTICKED (a warning under --strict too), the 6.3.1 paths, and the gate passes', () => {
const files = '`tests/mod1.test.mjs`, src/mod1.mjs';
const text = buildPlan({ steps: [{ filesRaw: files }, { endToEnd: 'bash tests/e2e.sh' }] });
assert.ok(text.includes(`- **Files:** ${files}\n`), 'the builder writes the raw Files value');
const r = validatePlanContent(text);
assert.deepEqual(r.errors, [], JSON.stringify(r.errors, null, 2));
const w = r.warnings.filter((i) => i.code === 'PLAN_STEP_PATH_UNTICKED');
assert.equal(w.length, 1, warnCodes(r).join(','));
assert.match(w[0].message, /Step 1/);
assert.match(w[0].message, /src\/mod1\.mjs/);
assert.equal(r.warnings.filter((i) => PATH_CODES.includes(i.code)).length, 0);
const soft = validatePlanContent(text, { strict: false });
assert.ok(warnCodes(soft).includes('PLAN_STEP_PATH_UNTICKED'));
// Read as before: the same paths 6.3.1 reads.
assert.equal(typeof schema.untickedPaths, 'function', 'plan-schema exports untickedPaths');
assert.deepEqual(schema.untickedPaths(files), ['src/mod1.mjs']);
assert.deepEqual(schema.fieldPaths(files), ['tests/mod1.test.mjs', 'src/mod1.mjs']);
assert.deepEqual(schema.stepTestsOnly(`- **Files:** ${files}\n`).files, ['tests/mod1.test.mjs', 'src/mod1.mjs']);
// The gate passes it: a warning never stops a plan.
const rec = passedCritic(text);
assert.equal(rec.code, 0, rec.message);
const g = gatePlan(rec.text);
assert.equal(g.code, 0, JSON.stringify(g.reasons));
});
test('below 1.9: a plan whose Files are all backticked gets no new warning', () => {
const r = validatePlanContent(buildPlan());
assert.equal(r.warnings.find((w) => w.code === 'PLAN_STEP_PATH_UNTICKED'), undefined, JSON.stringify(r.warnings));
// The same unticked step in a 1.9 plan is refused rather than warned about.
const v = validatePlanContent(plan19('src/mod1.mjs'));
assert.ok(errCodes(v).includes('PLAN_STEP_PATH_NOT_FIRST'), errCodes(v).join(','));
assert.equal(v.warnings.find((w) => w.code === 'PLAN_STEP_PATH_UNTICKED'), undefined);
});
// ---- Step 6: K6 and the red proof read a 1.9 plan by the grammar ----
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const RED_PROOF = join(ROOT, 'lib', 'verification', 'red-proof.mjs');
const COVERS = '`tests/a.test.mjs` - covers `src/parse.mjs`';
// Step 1 names only its test, and the description mentions the source it covers.
// The manifest names the test alone, so it adds nothing K6 would not plan anyway.
const coversPlan = (version) => buildPlan({ version, steps: [
{ filesRaw: COVERS, test: 'tests/a.test.mjs', manifestPaths: ['tests/a.test.mjs'] },
{ endToEnd: 'bash tests/e2e.sh' },
] });
test('1.9: K6 plans the files a 1.9 Files: entry names, never a path its description mentions', () => {
const text = coversPlan('1.9');
assert.ok(text.includes(`- **Files:** ${COVERS}\n`), 'the builder writes the raw Files value');
assert.deepEqual(plannedPaths(text), ['src/mod2.mjs', 'tests/a.test.mjs', 'tests/mod2.test.mjs']);
});
test('1.9: K6 plans the files a 1.9 Test first File: line names, never its description', () => {
const text = buildPlan({ version: '1.9', steps: [
{ filesRaw: '`tests/a.test.mjs`', testFirstRaw: ` - File: ${COVERS}`, test: 'tests/a.test.mjs', manifestPaths: ['tests/a.test.mjs'] },
{ endToEnd: 'bash tests/e2e.sh' },
] });
assert.ok(text.includes(` - File: ${COVERS}\n`), 'the builder writes the raw File line');
assert.deepEqual(plannedPaths(text), ['src/mod2.mjs', 'tests/a.test.mjs', 'tests/mod2.test.mjs']);
});
test('below 1.9: K6 plans what it planned before - the same input plans both paths', () => {
assert.deepEqual(plannedPaths(coversPlan('1.8')), ['src/mod2.mjs', 'src/parse.mjs', 'tests/a.test.mjs', 'tests/mod2.test.mjs']);
const tf = buildPlan({ version: '1.8', steps: [
{ filesRaw: '`tests/a.test.mjs`', testFirstRaw: ` - File: ${COVERS}`, test: 'tests/a.test.mjs', manifestPaths: ['tests/a.test.mjs'] },
{ endToEnd: 'bash tests/e2e.sh' },
] });
assert.deepEqual(plannedPaths(tf), ['src/mod2.mjs', 'src/parse.mjs', 'tests/a.test.mjs', 'tests/mod2.test.mjs']);
});
test('decision 27 — K6 keeps its own norm and LIST_ITEM is a literal', () => {
const sweep = readFileSync(join(ROOT, 'lib', 'verification', 'plan-scope-sweep.mjs'), 'utf8');
assert.doesNotMatch(sweep, /\bnormPlanPath\b/, 'K6 does not import the parser\'s normaliser');
assert.ok(sweep.includes("const norm = p => p.replace(/\\\\/g, '/').replace(/^\\.\\//, '');"), 'K6 keeps its own norm line');
const parser = readFileSync(join(ROOT, 'lib', 'parsers', 'plan-schema.mjs'), 'utf8');
assert.doesNotMatch(parser, /\bLIST_MARKER\b/, 'LIST_ITEM is not built from LIST_MARKER');
assert.ok(parser.includes('const LIST_ITEM = /^\\s*(?:[-*+]|\\d+[.)])\\s+/;'), 'LIST_ITEM is the literal');
});
test('1.9: the red proof\'s tests_only reads Files by the grammar (CLI, temp git repo); 1.8 reads it as before', () => {
const repo = mkdtempSync(join(tmpdir(), 'plan19-redproof-'));
const git = (...args) => spawnSync('git', args, { cwd: repo, encoding: 'utf8', env: childEnv() });
try {
for (const args of [['init', '-q'], ['config', 'user.email', 't@example.invalid'], ['config', 'user.name', 't'], ['config', 'commit.gpgsign', 'false']]) {
assert.equal(git(...args).status, 0);
}
writeFileSync(join(repo, 'README.md'), 'x\n');
assert.equal(git('add', '-A').status, 0);
assert.equal(git('commit', '-q', '-m', 'base').status, 0);
const base = git('rev-parse', 'HEAD').stdout.trim();
// The helper wrote the step's test alone; it passes at the start point.
mkdirSync(join(repo, 'tests'));
writeFileSync(join(repo, 'tests', 'a.test.mjs'), "import { test } from 'node:test';\ntest('a', () => {});\n");
mkdirSync(join(repo, 'plans'));
for (const [version, want] of [['1.9', true], ['1.8', false]]) {
const plan = join(repo, 'plans', `plan-${version}.md`);
writeFileSync(plan, coversPlan(version));
const r = spawnSync('node', [RED_PROOF, '--red-only', '--base', base, '--cmd', 'node --test tests/a.test.mjs',
'--tests', 'tests/a.test.mjs', '--plan', plan, '--step', '1', '--json'], { cwd: repo, encoding: 'utf8', env: childEnv() });
assert.equal(r.status, 1, `${version}: ${r.stdout}${r.stderr}`);
const out = JSON.parse(r.stdout);
assert.equal(out.reason, 'not-red-at-base', JSON.stringify(out));
assert.equal(out.tests_only, want, `plan_version ${version}: ${JSON.stringify(out.tests_only_note ?? null)}`);
}
} finally { rmSync(repo, { recursive: true, force: true }); }
});
// The validator reads the step's test the way the executor does (S139): from
// 1.9 `Test first:` names its test only through a line that opens with
// `File:` and its backticked path (testFirstFiles), so a `File:` mid-line is
// no test. Below 1.9 the step check is read as before (decision 16).
const planTestFirst19 = (version, line) =>
buildPlan({ version, steps: [{ testFirstRaw: line }, { endToEnd: 'bash tests/e2e.sh' }] });
const step1Fields = (text) => schema.parseStepFields(text.split('### Step 1:')[1].split('### Step 2:')[0]);
test('1.9: a Test first whose File: stands mid-line names no test - PLAN_STEP_NO_TEST_FIRST under --strict, a warning under --soft', () => {
for (const line of [
' - see File: `tests/mod1.test.mjs` for the test',
' - Pattern: File: `tests/mod1.test.mjs`',
]) {
const text = planTestFirst19('1.9', line);
assert.ok(text.includes(`${line}\n`), 'the builder writes the raw File line');
assert.deepEqual(schema.testFirstFiles(step1Fields(text), '1.9'), [],
'the executor reads no test file here');
const r = validatePlanContent(text, { strict: true });
assert.equal(r.valid, false, `${line}: ${errCodes(r).join(',')}`);
const no = r.errors.filter((e) => e.code === 'PLAN_STEP_NO_TEST_FIRST');
assert.equal(no.length, 1, `${line}: ${errCodes(r).join(',')}`);
assert.match(no[0].message, /Step 1/);
const soft = validatePlanContent(text, { strict: false });
assert.ok(warnCodes(soft).includes('PLAN_STEP_NO_TEST_FIRST'), `${line}: ${warnCodes(soft).join(',')}`);
}
});
test('1.9: a File: line that opens the line names its test - no PLAN_STEP_NO_TEST_FIRST', () => {
for (const line of [
' - File: `tests/mod1.test.mjs` (new)',
' - **File:** `tests/mod1.test.mjs` *(new)*',
' - File: `tests/mod1.test.mjs`, `tests/mod1b.test.mjs` (new)',
]) {
const r = validatePlanContent(planTestFirst19('1.9', line), { strict: true });
assert.equal(r.errors.find((e) => e.code === 'PLAN_STEP_NO_TEST_FIRST'), undefined, `${line}: ${errCodes(r).join(',')}`);
}
});
test('below 1.9: the step check reads a mid-line File: as before', () => {
const r = validatePlanContent(planTestFirst19('1.8', ' - see File: `tests/mod1.test.mjs` for the test'), { strict: true });
assert.equal(r.errors.find((e) => e.code === 'PLAN_STEP_NO_TEST_FIRST'), undefined, errCodes(r).join(','));
});
// ---- Step 3 (trial 4): a wrapped 1.9 entry keeps its paths, and a numbered
// `File:` line names its test (reviews b4fa1927, 4b6460e5) ----
const WRAPPED = '`tests/a.test.mjs`,\n `src/a.mjs`';
test('1.9: a Files list wrapped after a comma names both files and no issue', () => {
const r = entryPaths(WRAPPED);
assert.deepEqual(r.files, ['tests/a.test.mjs', 'src/a.mjs']);
assert.deepEqual(r.issues, []);
const v = validatePlanContent(plan19(WRAPPED));
assert.deepEqual(pathIssues(v.errors), [], JSON.stringify(v.errors));
// An issue keeps the entry's first line number.
const bad = entryPaths('- `src/a.mjs`\n- the CLI,\n `src/b.mjs`');
assert.deepEqual(bad.files, ['src/a.mjs']);
assert.deepEqual(bad.issues.map((i) => [i.code, i.line]), [['PLAN_STEP_PATH_NOT_FIRST', 2]]);
});
test('1.9: a step whose wrapped Files list names its source is not tests-only', () => {
const body = `- **Files:** ${WRAPPED}\n- **Changes:** x\n- **Test first:**\n - File: \`tests/a.test.mjs\` (new)\n - Verifies: x\n`;
assert.deepEqual(schema.stepTestsOnly(body, '1.9'), { testsOnly: false, files: ['tests/a.test.mjs', 'src/a.mjs'] });
});
test('1.9: K6 plans the second file of a wrapped Files list', () => {
const text = buildPlan({ version: '1.9', steps: [
{ filesRaw: WRAPPED, test: 'tests/a.test.mjs', manifestPaths: ['tests/a.test.mjs'] },
{ endToEnd: 'bash tests/e2e.sh' },
] });
assert.ok(text.includes(`- **Files:** ${WRAPPED}\n`), 'the builder writes the raw Files value');
assert.ok(plannedPaths(text).includes('src/a.mjs'), JSON.stringify(plannedPaths(text)));
});
test('1.9: a numbered File line (`1.` and `1)`) names its test - no PLAN_STEP_NO_TEST_FIRST', () => {
for (const line of [' 1. File: `tests/mod1.test.mjs` (new)', ' 1) File: `tests/mod1.test.mjs` (new)']) {
const text = planTestFirst19('1.9', line);
assert.ok(text.includes(`${line}\n`), 'the builder writes the raw File line');
assert.deepEqual(schema.testFirstFiles(step1Fields(text), '1.9'), ['tests/mod1.test.mjs'], line);
const r = validatePlanContent(text, { strict: true });
assert.equal(r.errors.find((e) => e.code === 'PLAN_STEP_NO_TEST_FIRST'), undefined, `${line}: ${errCodes(r).join(',')}`);
}
});
test('1.9: a wrapped File line keeps its second test; a wrapped description stays description', () => {
const wrapped = planTestFirst19('1.9', ' - File: `tests/mod1.test.mjs`,\n `tests/mod1b.test.mjs` (new)');
assert.deepEqual(schema.testFirstFiles(step1Fields(wrapped), '1.9'), ['tests/mod1.test.mjs', 'tests/mod1b.test.mjs']);
const described = planTestFirst19('1.9', ' - File: `tests/mod1.test.mjs` - covers\n `src/parse.mjs` and the CLI');
assert.deepEqual(schema.testFirstFiles(step1Fields(described), '1.9'), ['tests/mod1.test.mjs']);
});
test('1.9: a Verifies item that wraps onto a backticked source path gives only the File line\'s test', () => {
const text = planTestFirst19('1.9',
' - File: `tests/mod1.test.mjs` (new)\n - Verifies: the parser in\n `src/mod1.mjs` reads the CLI options');
assert.deepEqual(schema.testFirstFiles(step1Fields(text), '1.9'), ['tests/mod1.test.mjs']);
});
test('1.9: two unmarked File lines in a row give both tests', () => {
const text = planTestFirst19('1.9', ' File: `tests/mod1.test.mjs`\n File: `tests/mod1b.test.mjs`');
assert.deepEqual(schema.testFirstFiles(step1Fields(text), '1.9'), ['tests/mod1.test.mjs', 'tests/mod1b.test.mjs']);
});
// ---- Step 2 (second remediation): a 1.9 wrapped line keeps its paths only
// after a comma, or in an entry with nothing on its own line (operator
// decision 29, 14.10; reviews d21ff81e, eaeb398e) ----
test('1.9: a line wrapped onto a backticked call after an entry with no comma is description - no issue', () => {
const r = entryPaths('`tests/a.test.mjs`\n `parse()` reads the input');
assert.deepEqual(r.files, ['tests/a.test.mjs']);
assert.deepEqual(r.issues, []);
});
test('1.9: a line wrapped onto a backticked option after an entry with no comma is description - no issue, and the validator agrees', () => {
const pair = '`src/a.mjs`\n `--strict` makes it an error';
const r = entryPaths(pair);
assert.deepEqual(r.files, ['src/a.mjs']);
assert.deepEqual(r.issues, []);
const v = validatePlanContent(plan19(pair), { strict: true });
assert.deepEqual(pathIssues(v.errors), [], JSON.stringify(v.errors));
});
test('1.9: once a wrapped line is description, a comma at its end does not open the entry again', () => {
const r = entryPaths('`src/a.mjs`\n `src/b.mjs`,\n `src/c.mjs`');
assert.deepEqual(r.files, ['src/a.mjs']);
assert.deepEqual(r.issues, []);
});
test('1.9: a File line wrapped onto a backticked source path gives only the test, and the step is not tests-only', () => {
const file = ' - File: `tests/mod1.test.mjs`\n `src/mod1.mjs` is the parser under test';
const text = planTestFirst19('1.9', file);
assert.ok(text.includes(`${file}\n`), 'the builder writes the raw File line');
assert.deepEqual(schema.testFirstFiles(step1Fields(text), '1.9'), ['tests/mod1.test.mjs']);
const body = `- **Files:** \`tests/mod1.test.mjs\`, \`src/mod1.mjs\`\n- **Changes:** x\n- **Test first:**\n${file}\n - Verifies: x\n`;
assert.deepEqual(schema.stepTestsOnly(body, '1.9'), { testsOnly: false, files: ['tests/mod1.test.mjs', 'src/mod1.mjs'] });
});
test('1.9: a File line wrapped onto a backticked call gives no PLAN_STEP_PATH_NOT_A_PATH under --strict', () => {
const text = planTestFirst19('1.9', ' - File: `tests/mod1.test.mjs`\n `parse()` reads the input');
const r = validatePlanContent(text, { strict: true });
assert.equal(r.errors.find((e) => e.code === 'PLAN_STEP_PATH_NOT_A_PATH'), undefined, JSON.stringify(r.errors));
assert.deepEqual(schema.testFirstFiles(step1Fields(text), '1.9'), ['tests/mod1.test.mjs']);
});
test('1.9 controls: an empty File line takes its first wrapped line, a comma keeps the list, a comma after no path still refuses', () => {
const empty = planTestFirst19('1.9', ' - File:\n `tests/mod1.test.mjs` (new)');
assert.deepEqual(schema.testFirstFiles(step1Fields(empty), '1.9'), ['tests/mod1.test.mjs']);
assert.deepEqual(entryPaths(WRAPPED).files, ['tests/a.test.mjs', 'src/a.mjs']);
const bad = entryPaths('- `src/a.mjs`\n- the CLI,\n `src/b.mjs`');
assert.deepEqual(bad.issues.map((i) => [i.code, i.line]), [['PLAN_STEP_PATH_NOT_FIRST', 2]]);
});
// ---- Step 4 (trial 4): a 1.9 plan refuses a placeholder and a mixed-case bare
// name as a file (operator decision 22, 13.10; review 33f37f53) ----
test('1.9: isPathShape is false for a placeholder and a mixed-case bare name, true for a capitals name and ./Name', () => {
for (const s of ['None', 'none', 'TBD', 'TODO', 'TBA', 'N/A', 'n/a', 'NA', '...', 'Parser']) {
assert.equal(schema.isPathShape(s), false, s);
}
for (const s of ['LICENSE', 'README', 'Makefile', './Parser', 'TBC', 'NULL']) assert.equal(schema.isPathShape(s), true, s);
});
test('1.9: a Files entry that is a backticked None or TBD is PLAN_STEP_PATH_NOT_A_PATH under --strict, a warning under --soft', () => {
for (const files of ['`None`', '`TBD`']) {
const text = plan19(files);
const strict = validatePlanContent(text);
assert.equal(strict.valid, false, `${files}: ${errCodes(strict).join(',')}`);
const e = strict.errors.find((i) => i.code === 'PLAN_STEP_PATH_NOT_A_PATH');
assert.ok(e, `${files}: ${errCodes(strict).join(',')}`);
assert.match(e.message, /Step 1/);
assert.match(e.hint, /placeholder/i);
assert.match(e.hint, /TBD/);
assert.match(e.hint, /\.\/Name/);
const soft = validatePlanContent(text, { strict: false });
assert.ok(warnCodes(soft).includes('PLAN_STEP_PATH_NOT_A_PATH'), `${files}: ${warnCodes(soft).join(',')}`);
assert.deepEqual(pathIssues(soft.errors), []);
}
});
test('1.9: a Test first File line naming TBD gives no test, and K6 does not plan it', () => {
const text = planTestFirst19('1.9', ' - File: `TBD`');
assert.ok(text.includes(' - File: `TBD`\n'), 'the builder writes the raw File line');
assert.deepEqual(schema.testFirstFiles(step1Fields(text), '1.9'), []);
assert.ok(!plannedPaths(text).includes('TBD'), JSON.stringify(plannedPaths(text)));
});
test('below 1.9: fieldPaths of a backticked None still returns it (6.3.1\'s answer)', () => {
assert.deepEqual(schema.fieldPaths('`None`'), ['None']);
});