feat(execute): one fresh helper per step, a controller on the diff, proof by command

/trekexecute no longer builds steps itself. Each plan step is built by one
fresh helper (new agent step-implementer: opus, effort high, no Agent tool)
and judged by one controller (a general helper on opus) that reads the
diff from a recorded start_sha and gives two verdicts, spec and quality.
Fix rounds go up, maximum 5: three fixes by a fresh helper, then back to
the plan, then to the operator. No cheaper-model ladder.

Proof is a command, never the model's word:
- lib/verification/red-proof.mjs: the step's test must fail at the start
  point and pass at the result (exit 0/1/2; 2 is never green). Reproduces
  the 02243c6 red-first measurement (6 of 55) through the shim fixture.
- lib/verification/test-integrity.mjs: the floor guard on the diff
  (removed asserts, skipped/deleted tests, muted checks, stubs, loosened
  thresholds). Over the last 150 commits: 149 clean, 1 real finding.
- lib/verification/done-line.mjs: DONE step N only when the test command
  exits 0.

Test first is required from plan_version 1.8. The test lock (K8) is used
through its CLI and reported as not-locked while it is missing. A run
stops only on a closed list of four classes; every other call goes into a
decision log with its cost if wrong. Without a passing run-the-thing
criterion the result is not verified end to end, never green.

Handover 5 gains optional per-step start_sha, report_path, red_proof,
test_lock, fix_rounds (0-5) and run-level decisions and end_to_end; old
progress files still validate and resume.

No isolation on the helper: measured, a worktree-isolated subagent
branches from the default branch (02f320f), not the executor's HEAD
(2c89a6b), so it would miss the earlier steps.

Borrowings credited in commands/trekexecute.md ## Credits (Superpowers,
ai-native-sdlc, Kiro, GitHub Spec Kit, agent-skills, Cherny).

Tests: 1411 -> 1457 (1455 pass, 0 fail, 2 skipped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-06 19:58:18 +02:00
commit abc3083c1e
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
28 changed files with 2052 additions and 103 deletions

View file

@ -29,7 +29,7 @@ Voyage — a contract-driven Claude Code pipeline: brief, research, plan, execut
| `/trekbrief` | Brief — interactive interview produces a task brief with explicit research plan; ends with the operator approving `## Intent` + `## Goal` (`--approve <dir>` does only that, then writes the trekbrief record); optionally orchestrates the pipeline | session | | `/trekbrief` | Brief — interactive interview produces a task brief with explicit research plan; ends with the operator approving `## Intent` + `## Goal` (`--approve <dir>` does only that, then writes the trekbrief record); optionally orchestrates the pipeline | session |
| `/trekresearch` | Research — deep local + external research, produces structured research brief. Opt-in `--engine {swarm\|deep-research}` delegates the external phase to Claude Code's built-in `/deep-research` workflow (swarm default) | session | | `/trekresearch` | Research — deep local + external research, produces structured research brief. Opt-in `--engine {swarm\|deep-research}` delegates the external phase to Claude Code's built-in `/deep-research` workflow (swarm default) | session |
| `/trekplan` | Plan — brief-reviewer, explore, plan, review. Requires `--brief` or `--project`; halts unless the brief's intent is approved (hash-bound, stale on edit). Auto-discovers `architecture/overview.md` if present | session | | `/trekplan` | Plan — brief-reviewer, explore, plan, review. Requires `--brief` or `--project`; halts unless the brief's intent is approved (hash-bound, stale on edit). Auto-discovers `architecture/overview.md` if present | session |
| `/trekexecute` | Execute — disciplined plan/session-spec executor with failure recovery | session | | `/trekexecute` | Execute — disciplined plan/session-spec executor: one fresh `step-implementer` helper per plan step, one controller that reads the diff (spec + quality verdicts), proof of red / floor guard / done line as commands, fix rounds that go up (max 5: helper → plan → operator), closed stop list + decision log in the final report | session |
| `/trekreview` | Review — independent post-hoc review of delivered code against the brief. Produces `review.md` with severity-tagged findings (Handover 6) | session | | `/trekreview` | Review — independent post-hoc review of delivered code against the brief. Produces `review.md` with severity-tagged findings (Handover 6) | session |
| `/trekcontinue` | Continue — resumes the next session of a multi-session voyage project. Reads `.session-state.local.json` (Handover 7) and immediately begins executing | session | | `/trekcontinue` | Continue — resumes the next session of a multi-session voyage project. Reads `.session-state.local.json` (Handover 7) and immediately begins executing | session |
| `/trekendsession` | End-session — mark the current session complete and write session-state pointing at the next session. Helper for informal multi-session flows | session | | `/trekendsession` | End-session — mark the current session complete and write session-state pointing at the next session. Helper for informal multi-session flows | session |
@ -74,15 +74,16 @@ Full flag reference for each command (modes, `--gates`, `--profile`, breaking ch
| plan-critic | opus | Adversarial plan review (10 dimensions) | | plan-critic | opus | Adversarial plan review (10 dimensions) |
| scope-guardian | opus | Scope alignment (creep + gaps) | | scope-guardian | opus | Scope alignment (creep + gaps) |
| session-decomposer | opus | Splits plans into headless sessions with dependency graph | | session-decomposer | opus | Splits plans into headless sessions with dependency graph |
| step-implementer | opus | `/trekexecute` step helper — one fresh helper per plan step: test first, proof of red, test lock, then exactly the step's Changes; ran / saw / did-NOT-check report (no Agent tool) |
| synthesis-agent | opus | Distills Phase-5/7 exploration outputs into a findings digest (NW3 PoC — **dormant**, not wired; delegating Phase 7 alone yields Δ main-context ≈ 0, see `docs/T1-synthesis-poc-results.md`) | | synthesis-agent | opus | Distills Phase-5/7 exploration outputs into a findings digest (NW3 PoC — **dormant**, not wired; delegating Phase 7 alone yields Δ main-context ≈ 0, see `docs/T1-synthesis-poc-results.md`) |
| docs-researcher | opus | Official documentation, RFCs, vendor docs (Tavily, MS Learn) | | docs-researcher | opus | Official documentation, RFCs, vendor docs (Tavily, MS Learn) |
| community-researcher | opus | Community experience: issues, blogs, discussions | | community-researcher | opus | Community experience: issues, blogs, discussions |
| security-researcher | opus | CVEs, audit history, supply chain risks | | security-researcher | opus | CVEs, audit history, supply chain risks |
| contrarian-researcher | opus | Counter-evidence, overlooked alternatives | | contrarian-researcher | opus | Counter-evidence, overlooked alternatives |
> **Inventory (S33 reconcile, updated v5.10.1).** 23 agent files = **20 spawnable** (one, `synthesis-agent`, ships **dormant** — Δ≈0, wired to nothing) **+ 3 orchestrator reference docs** (`planning-/research-/review-orchestrator` document the inline `/trek*` workflow, not spawnable capabilities). All 23 stay `model: opus` (operator pin `40d8742`); the glue/mechanical/retrieval/dormant roles were reconsidered for a sonnet downgrade and **kept opus** — decision record: `docs/voyage-vs-cc-balance-analysis.md` §10. > **Inventory (S33 reconcile, updated wave 1).** 24 agent files = **21 spawnable** (one, `synthesis-agent`, ships **dormant** — Δ≈0, wired to nothing) **+ 3 orchestrator reference docs** (`planning-/research-/review-orchestrator` document the inline `/trek*` workflow, not spawnable capabilities). All 24 stay `model: opus` (operator pin `40d8742`); the glue/mechanical/retrieval/dormant roles were reconsidered for a sonnet downgrade and **kept opus** — decision record: `docs/voyage-vs-cc-balance-analysis.md` §10.
> **Model & effort.** On the Anthropic API, `opus` = Opus 5.5 and `sonnet` = Sonnet 5.5 (both default reasoning effort `medium`); `fable` = Fable 5.1 (the most capable model in Claude Code; default `high`, so xhigh must be chosen explicitly). An agent without `effort:` inherits the session level. Eight agents pin native per-spawn `effort: high` and the rest follow the session; none pins below `high` — a different axis from brief `phase_signals.effort` (orchestration shape: which agents/passes run). Per-agent table + axes → `docs/profiles.md` §Model & effort axes. > **Model & effort.** On the Anthropic API, `opus` = Opus 5.5 and `sonnet` = Sonnet 5.5 (both default reasoning effort `medium`); `fable` = Fable 5.1 (the most capable model in Claude Code; default `high`, so xhigh must be chosen explicitly). An agent without `effort:` inherits the session level. Nine agents pin native per-spawn `effort: high` and the rest follow the session; none pins below `high` — a different axis from brief `phase_signals.effort` (orchestration shape: which agents/passes run). Per-agent table + axes → `docs/profiles.md` §Model & effort axes.
## Reference docs (read on demand) ## Reference docs (read on demand)

View file

@ -34,7 +34,7 @@ Changes to this plugin must preserve:
| `commands/trekresearch.md` | The `/trekresearch` slash command — research orchestration | | `commands/trekresearch.md` | The `/trekresearch` slash command — research orchestration |
| `commands/trekplan.md` | The `/trekplan` slash command — planning orchestration | | `commands/trekplan.md` | The `/trekplan` slash command — planning orchestration |
| `commands/trekexecute.md` | The `/trekexecute` slash command — execution orchestration | | `commands/trekexecute.md` | The `/trekexecute` slash command — execution orchestration |
| `agents/*.md` | 19 specialized agents for research, exploration, review, and orchestration | | `agents/*.md` | 24 agent files (21 spawnable + 3 orchestrator reference docs) for research, exploration, review, execution, and orchestration |
| `templates/plan-template.md` | Structured plan output format | | `templates/plan-template.md` | Structured plan output format |
| `templates/research-brief-template.md` | Research brief format with triangulation and confidence | | `templates/research-brief-template.md` | Research brief format with triangulation and confidence |
| `templates/spec-template.md` | Spec file format | | `templates/spec-template.md` | Spec file format |

View file

@ -296,7 +296,25 @@ Exploration uses 6–8 agents in parallel (architecture-mapper, dependency-trace
Reads a plan from `/trekplan` and implements it with strict discipline. No guessing, no improvising — follows the plan exactly. Reads a plan from `/trekplan` and implements it with strict discipline. No guessing, no improvising — follows the plan exactly.
Per step: apply Changes exactly as written → run Verify (exit code is truth) → manifest audit (expected paths, forbidden paths, commit pattern) → follow the plan's failure clause if anything fails (revert / retry / skip / escalate) → Checkpoint commit. After all steps: independent Phase 7.5 manifest audit from git log + filesystem (ignoring agent bookkeeping); drift triggers Phase 7.6 bounded recovery. Per step: record the start point → **one fresh helper** (`step-implementer`) writes the step's test first, proves it red, locks it, then applies Changes exactly as written → the executor proves it by command: **red at the start point, green at the result** (`red-proof.mjs`), Verify (exit code is truth), manifest audit, and the **floor guard** on the diff (`test-integrity.mjs`: removed asserts, skipped or deleted tests, muted checks, stubs, loosened thresholds) → **one controller** reads the diff — never the helper's summary — and gives two verdicts, *spec* and *quality* → a script writes the step's **done line** only when the test command exits 0 (`done-line.mjs`) → Checkpoint commit. After all steps: independent Phase 7.5 manifest audit from git log + filesystem (ignoring agent bookkeeping); drift triggers Phase 7.6 bounded recovery. A plan with no passing run-the-thing criterion ends **not verified end to end**, never green.
### The step loop
| Part | What it does | Why |
|------|--------------|-----|
| Fresh helper per step | `voyage:step-implementer` (opus, effort high, no Agent tool) builds one step and writes a report: **ran · saw · did NOT check** | Each step starts from the repository and the plan, not from a long context |
| Start point | `start_sha` recorded before the step; every check reads `git diff {start_sha}` | The review package is the diff, not a story |
| Proof of red | the step's test must fail at the start point and pass at the result — run by `red-proof.mjs`, not claimed | A test that passes without the change proves nothing |
| Test lock | the helper locks its test files after the red proof (`test-lock.mjs`, when installed; otherwise "not-locked" is reported) | The code moves to meet the test, never the reverse |
| Floor guard | `test-integrity.mjs` reads the diff for weakened tests; exit 2 ("could not run") is never green | Cheating the tests is caught by a command |
| Controller | one general helper, two verdicts (spec, quality), re-checks only what a fix round changed | An independent judge of the diff |
| Done line | `done-line.mjs` appends `DONE step N` only when the test command exits 0 | "Done" is an exit code |
The helper works in your working tree, not in a worktree of its own: measured 2026-10-06, a worktree-isolated subagent branches from the default branch rather than your `HEAD`, so it would not see the earlier steps (only the operator setting `worktree.baseRef: "head"` changes that).
### Unattended: what stops a run
A run stops only on a **closed list**: something that cannot be undone · something security-sensitive · something outside the working copy (merge to main, push, publish) · a plan so broken that everything after it is guesswork. Everything else the executor decides itself and logs in the final report's **Decisions taken** table, each with what it costs if it is wrong.
### Modes ### Modes
@ -337,11 +355,11 @@ Use `--fg` to run every step sequentially in the current session instead, even w
### Failure recovery ### Failure recovery
- **3-attempt retry cap** — retries twice, then stops (never loops forever) - **Fix rounds go up, maximum 5** — rounds 1–3: a fresh helper fixes exactly the controller's findings; after three failed fixes the question is the architecture: round 4 sends the step back to planning (`/trekplan` revises it, then `--resume`), round 5 to you. Never a fourth guess, never a cheaper model
- **On failure: revert** — undo changes, stop - **On failure: revert** — fix rounds, then undo the step's changes when it goes up
- **On failure: retry** — try alternative approach, then revert if still failing - **On failure: retry** — the alternative approach goes to the first fix round; revert when it goes up
- **On failure: skip** — non-critical step, continue - **On failure: skip** — non-critical step, continue
- **On failure: escalate** — stop everything, needs human judgment - **On failure: escalate** — stop at the first failed round, needs human judgment
### Security hardening ### Security hardening
@ -623,7 +641,7 @@ Solid arrows are the forward pipeline; dashed are conditional/internal edges (`-
### Agents per phase ### Agents per phase
Every command orchestrates its agents inline from the main context (all sub-agents are `model: opus`-pinned). `/trekexecute` is the exception — it spawns **no** sub-agents. Every command orchestrates its agents inline from the main context (all sub-agents are `model: opus`-pinned). `/trekexecute` spawns no swarm: one fresh `step-implementer` per plan step, in order, and one controller per round.
```mermaid ```mermaid
flowchart TB flowchart TB
@ -640,8 +658,8 @@ flowchart TB
PLC["Phase 9 — parallel · plan-critic · scope-guardian"] PLC["Phase 9 — parallel · plan-critic · scope-guardian"]
PLG --> PLS --> PLC PLG --> PLS --> PLC
end end
subgraph EX["/trekexecute · no sub-agents"] subgraph EX["/trekexecute · one helper per step"]
EXI["inline step-loop + session orders per wave (worktrees)<br/>+ Phase 7.5/7.6 manifest audit (deterministic)"] EXI["per step: step-implementer → controller (spec + quality)<br/>+ red proof · floor guard · done line (commands)<br/>+ session orders per wave (worktrees) + Phase 7.5/7.6 audit"]
end end
subgraph RV["/trekreview"] subgraph RV["/trekreview"]
RVR["Phase 5 — parallel · code-correctness-reviewer · brief-conformance-reviewer*"] RVR["Phase 5 — parallel · code-correctness-reviewer · brief-conformance-reviewer*"]
@ -803,7 +821,7 @@ Default JSONL stats stream (`${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl`) is unchan
The default `premium` profile runs **Opus on every phase** of the pipeline's agent work — the exploration and review swarms (5–10 sub-agents per command; spawn sites inject the composed brief > profile > frontmatter resolution, with `agents/*.md` `model: opus` pins as the fallback) and the executor (one per plan session). The command orchestrator itself is not profile-controlled: as of v5.9, command frontmatter omits `model:`, so the orchestrator follows the session model. For cheaper runs, opt into `--profile balanced` (Sonnet on brief/research/execute/continue, Opus on plan + review) or `--profile economy` (Sonnet everywhere); for maximum quality, `--profile fable` (Fable 5.1 on every phase). Per-command cost is published in `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` if you want exact numbers. The default `premium` profile runs **Opus on every phase** of the pipeline's agent work — the exploration and review swarms (5–10 sub-agents per command; spawn sites inject the composed brief > profile > frontmatter resolution, with `agents/*.md` `model: opus` pins as the fallback) and the executor (one per plan session). The command orchestrator itself is not profile-controlled: as of v5.9, command frontmatter omits `model:`, so the orchestrator follows the session model. For cheaper runs, opt into `--profile balanced` (Sonnet on brief/research/execute/continue, Opus on plan + review) or `--profile economy` (Sonnet everywhere); for maximum quality, `--profile fable` (Fable 5.1 on every phase). Per-command cost is published in `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` if you want exact numbers.
On the Anthropic API the `opus` alias resolves to **Opus 5.5** and `sonnet` to **Sonnet 5.5** (both default to reasoning effort `medium`), and `fable` to **Fable 5.1** (the most capable model in Claude Code; default reasoning effort `high` — xhigh must be chosen explicitly, see [`docs/profiles.md`](docs/profiles.md)). Other providers resolve `opus` and `sonnet` to other versions. Note two distinct effort axes that share the word "effort": brief `phase_signals.effort` (low/standard/high) tunes *orchestration shape* — how many agents and passes run — while native `effort:` (eight agents pin `high`; the rest follow the session) tunes the *per-spawn reasoning budget*. See [`docs/profiles.md`](docs/profiles.md) § Model & effort axes. On the Anthropic API the `opus` alias resolves to **Opus 5.5** and `sonnet` to **Sonnet 5.5** (both default to reasoning effort `medium`), and `fable` to **Fable 5.1** (the most capable model in Claude Code; default reasoning effort `high` — xhigh must be chosen explicitly, see [`docs/profiles.md`](docs/profiles.md)). Other providers resolve `opus` and `sonnet` to other versions. Note two distinct effort axes that share the word "effort": brief `phase_signals.effort` (low/standard/high) tunes *orchestration shape* — how many agents and passes run — while native `effort:` (nine agents pin `high`; the rest follow the session) tunes the *per-spawn reasoning budget*. See [`docs/profiles.md`](docs/profiles.md) § Model & effort axes.
For per-profile cost estimates, see [`docs/profiles.md`](docs/profiles.md). For per-profile cost estimates, see [`docs/profiles.md`](docs/profiles.md).
@ -819,8 +837,8 @@ Top-level layout:
``` ```
trekplan/ trekplan/
├── agents/ 23 specialized agents (all `model: opus`-pinned; per-phase model set by `--profile`) ├── agents/ 24 specialized agents (all `model: opus`-pinned; per-phase model set by `--profile`)
│ └ 20 spawnable (1 dormant: synthesis-agent, Δ≈0) + 3 orchestrator reference docs (not spawned) │ └ 21 spawnable (1 dormant: synthesis-agent, Δ≈0) + 3 orchestrator reference docs (not spawned)
├── commands/ 6 slash commands (trekbrief, trekresearch, trekplan, trekexecute, trekreview, trekcontinue) + trekendsession helper ├── commands/ 6 slash commands (trekbrief, trekresearch, trekplan, trekexecute, trekreview, trekcontinue) + trekendsession helper
├── templates/ Frontmatter templates for brief, research, plan, session ├── templates/ Frontmatter templates for brief, research, plan, session
├── hooks/ 8 hooks (pre-bash, pre-write, pre-agent-cap, session-title, post-bash-stats, pre-compact-flush, post-compact-flush, otel-export) ├── hooks/ 8 hooks (pre-bash, pre-write, pre-agent-cap, session-title, post-bash-stats, pre-compact-flush, post-compact-flush, otel-export)

109
agents/step-implementer.md Normal file
View file

@ -0,0 +1,109 @@
---
name: step-implementer
description: |
Use this agent when /trekexecute hands one plan step to a fresh helper: it
writes the step's test first, proves it red, locks it, implements exactly the
step's Changes, and writes a ran / saw / did-NOT-check report to a file.
model: opus
effort: high
color: green
tools: ["Read", "Write", "Edit", "Bash", "Glob", "Grep"]
---
You are the helper for ONE step of a Voyage plan. You start fresh: you know
nothing about earlier steps except what is in the repository and the files the
executor names. You do one step, report it in a file, and stop. You never start
another helper (you have no Agent tool, by design), never commit, never push,
and never touch a file the step does not list.
## What the executor gives you
- `plan` — path to the plan; `step` — the step number; the step's text verbatim
(Files, Changes, Reuses, Test first, Verify, and when present Interface in,
Interface out, Review focus).
- `start_sha` — the commit the step starts from. Everything you change is read
later as `git diff {start_sha}`, so the diff is your real report.
- `report_path` — where your report goes.
- `mode` — `build` (the step's first helper) or `fix` (a fix round: the
controller's findings to fix are included; fix only those).
- `voyage_root` — the plugin root, for the CLIs below.
## Mode `build`
1. **Test first.** Read the step's `Test first:` and write that test (and only
that test) first. Follow the `Pattern:` file's style when one is named.
Then register every new file you created so the executor's diff sees it:
`git add --intent-to-add <path>` (this stages nothing).
2. **Prove it red before you build.** Run, from the repository root:
`node {voyage_root}/lib/verification/red-proof.mjs --red-only --base {start_sha} --cmd "<the step's Verify command>" --tests <test files>`
Exit 0 = the test fails at the start point: continue. Exit 1 = the test
already passes at the start point, so it proves nothing: rewrite the test
until it fails for the reason the step exists, and run the check again.
Exit 2 = it could not run (the Verify command is not one test-runner
invocation, or a file is missing): do not build; say so in the report and stop.
3. **Lock the tests** (when the CLI exists):
`node {voyage_root}/lib/util/test-lock.mjs lock --files <test files> --step {step}`.
If the file does not exist, write "not-locked (test-lock.mjs missing)" in the
report. Never silent. Once locked you cannot edit the tests, and that is the
point: the code changes to meet the test, never the reverse.
4. **Implement exactly the step's `Changes:`** in the files its `Files:` lists.
Read `Reuses:` first. No extra scope, no refactors on the side, no "while I
was here". If the step cannot be done as written, stop and say why in the
report: the executor sends it up to the plan; you do not improvise.
5. **Run the step's `Verify:` command** yourself, fresh. Its exit code is the
truth, not your reading of the output.
## Mode `fix`
You get the controller's findings (file, line, problem). Fix those and nothing
else. The tests stay locked: if a finding says the test itself is wrong, do not
work around it - say so in the report; that goes up to the plan. Run the
step's `Verify:` again.
## Never
- Never edit a test to make it pass: no deleted assert, no skip, no looser
threshold, no `|| true`. The executor's floor guard reads your diff for
exactly that, and finds it.
- Never write a stub ("not implemented", `TODO: implement`) as the step's body.
- Never write a `DONE` line. The executor's script writes it, and only when
the test command exits 0.
- Never commit, push, merge or tag. Never write outside the repository.
## The report (write it to `report_path`, then answer with one line)
```markdown
## Step {N} - {build | fix round R}
### Ran
- `{command}` -> exit {code}
(every command you ran, in order, with its exit code)
### Saw
- {what the output actually showed: the failing test name at the start point,
the passing count after, a warning you did not fix}
### Did NOT check
- {what you did not verify, plainly: other platforms, the full suite, an edge
case the plan names but no test covers, anything you assumed}
### Files touched
- {path} ({new | modified})
### Test lock
- {locked | not-locked (test-lock.mjs missing) | not-locked (reason)}
```
"Did NOT check" is never empty: if you truly checked everything the step
names, say what lies outside the step that you did not look at.
Your final answer to the executor is one line: `report: {report_path}`. The
executor does not read your summary to decide anything; it reads the diff, and a
controller judges it.
## Credits
The loop this helper lives in is adapted from obra/superpowers@8ca22db (MIT,
(c) 2025 Jesse Vincent), subagent-driven-development: a fresh helper per task,
hand-over in files. The ran / saw / did-NOT-check report contract is borrowed
from bashebr/ai-native-sdlc@11fb8d7 (MIT).

View file

@ -2,8 +2,8 @@
name: trekexecute name: trekexecute
description: Disciplined plan executor — single-session, or multi-session through session orders the user starts, with failure recovery description: Disciplined plan executor — single-session, or multi-session through session orders the user starts, with failure recovery
argument-hint: "[--project <dir>] [--fg | --resume | --dry-run | --validate | --step N | --session N] [plan.md]" argument-hint: "[--project <dir>] [--fg | --resume | --dry-run | --validate | --step N | --session N] [plan.md]"
allowed-tools: Read, Write, Edit, Bash, Glob, Grep, AskUserQuestion allowed-tools: Read, Write, Edit, Bash, Glob, Grep, AskUserQuestion, Agent
disallowed-tools: Agent, TeamCreate disallowed-tools: TeamCreate
--- ---
# Ultraexecute Local # Ultraexecute Local
@ -12,9 +12,17 @@ Disciplined executor for trekplan plans. Reads a plan file, detects if it has
an Execution Strategy (multi-session), and either executes directly or an Execution Strategy (multi-session), and either executes directly or
prepares one session order per session, wave by wave — all to realize one plan. prepares one session order per session, wave by wave — all to realize one plan.
Each plan step is built by **one fresh helper** (`voyage:step-implementer`)
and judged by **one controller** that reads the step's diff. The executor
itself writes no product code: it records the start point, launches the
helper, runs the mechanical checks (proof of red, the floor guard, the
manifest, the done line), and decides from exit codes and the controller's
verdicts (Phase 6).
Voyage never starts a Claude Code session. Work that needs another session Voyage never starts a Claude Code session. Work that needs another session
becomes an order plus a starter line the user gives a session they open becomes an order plus a starter line the user gives a session they open
themselves (Phase 2.6). themselves (Phase 2.6). Helpers inside this session are subagents, and helpers
never start helpers.
## Phase 1 — Parse mode and validate input ## Phase 1 — Parse mode and validate input
@ -120,15 +128,34 @@ Extract every `### Step N: {description}` heading (in order). For each step, ext
- **Files** — file paths to create or modify - **Files** — file paths to create or modify
- **Changes** — what to modify - **Changes** — what to modify
- **Reuses** — existing code to leverage (informational) - **Reuses** — existing code to leverage (informational)
- **Test first** — test to run before implementation (optional) - **Test first** — the test the step writes before it builds: its `File:`
- **Verify** — command to run after implementation (existing | new), what it `Verifies:`, an optional `Pattern:` (required from
plan_version ≥ 1.8, see below)
- **Verify** — command to run after implementation. It is also the step's
**test command**: proof of red (Sub-step B) and the done line (Sub-step G)
run it, so it must be one test-runner invocation
- **On failure** — recovery action (revert/retry/skip/escalate) - **On failure** — recovery action (revert/retry/skip/escalate)
- **Checkpoint** — git commit command after success - **Checkpoint** — git commit command after success
- **Manifest** — YAML block following Checkpoint (v1.7+) - **Manifest** — YAML block following Checkpoint (v1.7+)
- **Interface in** / **Interface out** / **Review focus** — optional (K4):
what the step consumes and produces, and what its controller must look at
hardest. Read when present and passed to the helper and the controller;
never required, and the executor tolerates their absence in every plan
version.
If a step is missing `On failure`, default to `escalate` and record a parse warning. If a step is missing `On failure`, default to `escalate` and record a parse warning.
If a step is missing `Verify`, record a parse warning. If a step is missing `Verify`, record a parse warning.
**`Test first:` is required from plan_version ≥ 1.8.** A ≥ 1.8 step without a
`Test first:` field (or whose `File:` is missing) is not executable: STOP with
```
Error: plan_version={v} but step {N} has no Test first: field.
Re-run /trekplan - from plan_version 1.8 every step proves red before it builds.
```
For plans below 1.8 the field stays optional: a step without it runs, its
`red_proof` is recorded as `"not-run"`, and the final report counts it among
the steps without proof of red (never hidden, never called proven).
### Parse the Manifest block (v1.7+) ### Parse the Manifest block (v1.7+)
Read the plan header for a `plan_version` marker. Treat ≥ `1.7` as strict Read the plan header for a `plan_version` marker. Treat ≥ `1.7` as strict
@ -1023,16 +1050,35 @@ progress file.
"current_step": 0, "current_step": 0,
"status": "{in_progress | completed | failed | stopped | partial}", "status": "{in_progress | completed | failed | stopped | partial}",
"steps": { "steps": {
"1": { "status": "pending", "attempts": 0, "error": null, "completed_at": null, "commit": null } "1": { "status": "pending", "attempts": 0, "fix_rounds": 0, "start_sha": null, "report_path": null, "red_proof": null, "test_lock": null, "error": null, "completed_at": null, "commit": null }
}, },
"iterations_remaining": 25, "iterations_remaining": 25,
"entry_condition_checked": false, "entry_condition_checked": false,
"exit_condition_checked": false, "exit_condition_checked": false,
"plan_verification": null, "plan_verification": null,
"end_to_end": null,
"decisions": [],
"summary": null "summary": null
} }
``` ```
**The step-loop fields (Handover 5, additive-optional).** Written by Phase 6;
a progress file without them still validates and resumes:
| Field | Written | Meaning |
|---|---|---|
| `steps.{N}.start_sha` | Sub-step 0 | the commit the step started from; every check reads `git diff {start_sha}` |
| `steps.{N}.report_path` | Sub-step 0 | the step's report file (helper rounds + the done line) |
| `steps.{N}.red_proof` | Sub-step B | `"proven" \| "rejected" \| "not-run"` — the exit of `red-proof.mjs` |
| `steps.{N}.test_lock` | Sub-steps B, G | `"locked" \| "released" \| "not-locked"` (K8) |
| `steps.{N}.fix_rounds` | Sub-step F | 0–5: 0 = passed its first control, 1–3 = fix rounds, 4 = sent up to the plan, 5 = sent up to the operator |
| `steps.{N}.attempts` | Sub-steps C, F | helpers launched for the step: 1 + the fix rounds that launched one |
| `end_to_end` | Phase 7 | `"verified" \| "not-verified"` (P9 (6)) |
| `decisions` | any phase | `[{step, decision, cost_if_wrong}]` — every call the executor made on its own (Phase 5.5) |
`progress-validator.mjs` rejects `fix_rounds` outside 0–5 (there is no sixth
round) and a `decisions` entry without its `cost_if_wrong`.
`plan_version` is the value Phase 2 parsed from the plan header (required by `plan_version` is the value Phase 2 parsed from the plan header (required by
`progress-validator.mjs`). The status words here are the validator's: `progress-validator.mjs`). The status words here are the validator's:
`PROGRESS_VALID_STATUSES` for the run, `PROGRESS_STEP_STATUSES` for a step. `PROGRESS_VALID_STATUSES` for the run, `PROGRESS_STEP_STATUSES` for a step.
@ -1047,8 +1093,8 @@ condition instead) and on early stop. Unknown keys are tolerated by
recovery/retry budget against the global hard cap (unit = **iterations**, per the recovery/retry budget against the global hard cap (unit = **iterations**, per the
brief — token/cost budgeting is out of scope). Seeded to brief — token/cost budgeting is out of scope). Seeded to
`TREKEXECUTE_MAX_RECOVERY_ITERATIONS` (default 25) at start. **Single writer:** the `TREKEXECUTE_MAX_RECOVERY_ITERATIONS` (default 25) at start. **Single writer:** the
executor decrements it by 1 on **every recovery/retry iteration** — each step executor decrements it by 1 on **every recovery/retry iteration** — each fix
attempt beyond the first, and each Phase 7.6 recovery dispatch. **Backfill-on-absent:** round of a step (Sub-step F), and each Phase 7.6 recovery dispatch. **Backfill-on-absent:**
when resuming a legacy `progress.json` that lacks the field, seed it to the cap when resuming a legacy `progress.json` that lacks the field, seed it to the cap
rather than hard-failing (forward/backward-compatible; shape-validated by rather than hard-failing (forward/backward-compatible; shape-validated by
`progress-validator.mjs` — non-negative integer, never required-top). The field is `progress-validator.mjs` — non-negative integer, never required-top). The field is
@ -1160,6 +1206,47 @@ If `has_execution_strategy = true`, show a preview of multi-session orchestratio
Stop after the dry-run report. Do not execute anything. Stop after the dry-run report. Do not execute anything.
## Phase 5.5 — Autonomy: what stops a run, and the decision log
A run is long and mostly unattended. It does not stop to ask: what it decides
on its own it writes down, and it stops only for the four reasons below.
### Closed stop list
The list is **closed**: a run stops on these four classes and on nothing else.
Every stop names its class in the final report (Phase 8 `### Stops`).
1. **Something that cannot be undone.** A step or command whose effect cannot
be reverted from git: deleting data outside the repo's tracked files, a
migration against a live store, anything the plan marks irreversible.
2. **Something security-sensitive.** A command the executor denylist blocks
(Phase 2.4, Sub-step D1), a write to a security-sensitive path (Hard
rule 16), credentials or secrets in a diff.
3. **Something outside the working copy.** Merge to the main branch, push,
tag, publish, a write outside the repository (Hard rule 15). The main-merge
gate (Phase 8) is this class: it always pauses.
4. **A plan so broken that everything after it is guesswork.** The plan fails
to parse, a ≥ 1.8 step has no `Test first:`, a step's own `On failure:
escalate` fired (the plan's author declared that past this point is
guesswork), or a step went up the fix ladder to the plan (Sub-step F).
Not on the list, and therefore never a stop: a failing test (that is a fix
round), an ambiguity the plan leaves open (decide, and log it), a style
choice, a missing optional field.
### Decision log
Every decision the executor (or a helper, through its report) takes that the
plan did not dictate goes into progress `decisions` as one entry:
`{"step": N, "decision": "...", "cost_if_wrong": "..."}`. `cost_if_wrong` is
concrete: what has to be redone, by whom, how big — "one rename commit",
"step 4's API shape; steps 5–7 build on it". The final report prints the log
(Phase 8 `### Decisions taken`), so the operator reviews the calls instead of
being asked them one by one.
Borrowed: Superpowers subagent-driven-development (MIT) — the stop classes and
the log of choices made while unattended.
## Phase 6 — Step execution loop ## Phase 6 — Step execution loop
The core execution phase. Runs for modes: `execute`, `resume`, `step`. The core execution phase. Runs for modes: `execute`, `resume`, `step`.
@ -1179,6 +1266,33 @@ Update progress: `steps.{N}.status = "in_progress"`, `current_step = N`, `update
--- Step {N}/{total}: {description} --- --- Step {N}/{total}: {description} ---
``` ```
One step at a time. Inside one session the steps run in order in one working
tree; parallel steps are the multi-session wave path (Phase 2.6), never two
helpers in one tree.
#### Sub-step 0 — Record the start point
Before anything touches the tree, record where the step begins:
```bash
git rev-parse HEAD
git status --porcelain
```
- `steps.{N}.start_sha` = the `HEAD` sha. Every later check of this step reads
`git diff {start_sha}`: the review package is the diff from this recorded
start point, not anything the helper says. On `--resume` of a step that
already has a `start_sha`, keep it.
- `steps.{N}.report_path` = `{progress-dir}/steps/step-{N}.report.md` (create
the directory). Every helper round of the step appends to it, and the done
line lands in it (Sub-step G). Hand-over between the executor, the helper
and the controller is in files: the plan, this report, the diff.
- A non-empty `git status --porcelain` (beyond Voyage's own progress and report
files) means work the plan does not know about would mix into this step's
diff. Not a stop (Phase 5.5): log it as a decision
(`cost_if_wrong`: "the controller judges unrelated changes as part of step N")
and name the paths in the report.
#### Sub-step A — Scope fence check (session specs only) #### Sub-step A — Scope fence check (session specs only)
Before touching any file, verify that every file in the step's `Files:` field is Before touching any file, verify that every file in the step's `Files:` field is
@ -1192,24 +1306,97 @@ Escalating — this step cannot be executed within this session's scope.
Treat this as an automatic `escalate`. Jump to the stop-and-report logic. Treat this as an automatic `escalate`. Jump to the stop-and-report logic.
#### Sub-step B — Test first (if present) #### Sub-step B — Test first and proof of red
If the step has a `Test first:` field: The step's test is written first, and a command proves it red — never the
1. If test file is marked `(new)`: note it will be created during implementation. model's word that "it failed first". This sub-step blocks: a step whose test
2. If test file exists: run it. Expect failure (RED state). is not proven red at its start point and green at its result does not pass.
3. If test unexpectedly passes: warn but continue — step may already be done. (A plan below 1.8 whose step has no `Test first:` skips it: `red_proof` =
`"not-run"`, counted in the final report, never called proven.)
Do not block on test-first failures — they are expected. **The test command** is the step's `Verify:` command. It must be ONE
test-runner invocation without shell operators (the allowlist of
`lib/verification/criteria-runner.mjs`); the tools below refuse anything else
with exit 2.
#### Sub-step C — Implement changes 1. **The helper proves red before it builds** (Sub-step C): it writes the test
from `Test first:`, runs the gate
`red-proof.mjs --red-only --base {start_sha} --cmd "{Verify}" --tests {files}`
(the test must FAIL at the start point), locks the test files, and only then
implements.
2. **The executor proves it both ways after the helper returns:**
```bash
node ${CLAUDE_PLUGIN_ROOT}/lib/verification/red-proof.mjs --base {start_sha} --cmd "{Verify command}" --tests {Test first File paths, comma-separated} --json
```
The test must FAIL against the start point (`start_sha` unpacked with
`git archive`, the step's test files copied in) and must PASS against the
result (the working tree). The start point holds tracked files only: add
`--link node_modules` (or the repo's untracked dependency directory) when
the test needs it.
Read the step's `Files:` and `Changes:` fields. Implement exactly as described. | Exit | `steps.{N}.red_proof` | Executor |
|------|-----------------------|----------|
| 0 | `"proven"` | continue |
| 1, `not-red-at-base` | `"rejected"` | the test passes without the change, so it proves nothing: a finding for Sub-step F |
| 1, `not-green-at-result` | `"rejected"` | the change does not make the test pass: a finding for Sub-step F |
| 2 | `"not-run"` | never a pass and never proven. When the reason is the command (the `Verify:` is not one test-runner invocation) on a plan ≥ 1.8, the plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4). Any other reason (a missing test file, an unresolvable start point) is a finding for Sub-step F |
**Rules:** The exit code is the verdict, the output text is not. `base.tap` (when the
- Follow `Changes:` exactly — do not improvise, add scope, or optimize runner prints TAP) says what the red was: `fail 1 of 1` on a node test file is
- Use Edit for modifications, Write for new files usually a load error, not a named failing test — say so in the step report.
- If `Reuses:` references existing code, read that code first for context
- Only touch files listed in `Files:` — nothing else **The test lock (K8).** After its red gate the helper runs
`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs lock --files {test files} --step {N}`;
from then on the guard refuses Write/Edit/NotebookEdit on those files, so the
code moves to meet the test and never the reverse. The executor reads
`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs status` and records
`steps.{N}.test_lock = "locked"` when it exits 0 and names the step's files.
If `lib/util/test-lock.mjs` is not in this plugin, record `"not-locked"` and
write "not locked: test-lock.mjs missing" in the step report — never silent.
The floor guard (Sub-step D3) still reads every test change in the diff. The
lock is released in Sub-step G, or when the run stops.
#### Sub-step C — Implement through one fresh helper
The executor does not write the step's code. **Launch** one fresh helper for
the step with the Agent tool:
Launch `subagent_type: "voyage:step-implementer"` with a short `description`
("step {N}: {title}") and a prompt that carries, as files and values, not as a
summary: the plan path, the step number, the step's text verbatim (every
field, including `Interface in:`, `Interface out:` and `Review focus:` when
present), `start_sha`, `report_path`, `mode: build`, and
`voyage_root: ${CLAUDE_PLUGIN_ROOT}`.
- **Never pass the Agent tool's `name` parameter.** A named spawn becomes an
`in_process_teammate`: it does the work, but its result never reaches this
orchestrator (measured: named 0/5 returned, unnamed 3/3 —
`docs/agent-return-channel-defect.md`). The same holds for the controller
(Sub-step E).
- **Never pass `isolation`.** Measured 2026-10-06 (Claude Code 2.1.291): a
worktree-isolated helper branched from the repository's default branch
(`02f320f`), not from the executor's `HEAD` (`2c89a6b`), so it did not see
the earlier steps. Only the operator's `worktree.baseRef: "head"` setting
changes that, and a plugin cannot set it. The helper works in this tree,
from the recorded `start_sha`.
- **Wait for the helper to finish** before anything else; when it runs in the
background, its completion notification is the signal. Never start the next
sub-step or step while a helper runs.
- `steps.{N}.attempts` = the number of helpers launched for the step (1 here).
The helper's one-line answer is not evidence. After it returns:
1. The report must exist at `report_path` with its `### Ran`, `### Saw` and
`### Did NOT check` sections (a missing report is a finding for Sub-step F).
2. Register the step's new files so the diff sees them: for each path in
`Files:` that `git status --porcelain` shows as `??`, run
`git add --intent-to-add {path}` (stages nothing).
3. A changed path outside the step's `Files:` is a scope finding (Hard rule 2)
for the controller. The executor does not revert it on its own.
**Rules the helper is held to** (its agent file carries them; the controller
checks them in the diff): follow `Changes:` exactly — no improvising, no added
scope; touch only the files in `Files:`; read `Reuses:` first; never weaken a
test; never write a stub; never commit.
#### Sub-step D1 — Command verification #### Sub-step D1 — Command verification
@ -1259,7 +1446,9 @@ status = `blocked` and apply `On failure: escalate`. Skip D2 manifest
verification (nothing to verify for a read-only sandbox test). Commit none — verification (nothing to verify for a read-only sandbox test). Commit none —
jump straight to Phase 7 with a structured "sandbox-blocked" reason. jump straight to Phase 7 with a structured "sandbox-blocked" reason.
If **PASS**: proceed to Sub-step D2 (manifest verification). If **PASS**: proceed to Sub-step D2 (manifest verification). If **FAIL**: the
failure output is a finding for Sub-step F; still run D2 and D3, so the
controller and the next fix round see every failure at once.
#### Sub-step D2 — Manifest verification #### Sub-step D2 — Manifest verification
@ -1273,10 +1462,10 @@ holds, regardless of Verify exit code.
file exists in the repo. Count how many exist. file exists in the repo. Count how many exist.
2. **min_file_count satisfied:** count from (1) must be ≥ `min_file_count`. 2. **min_file_count satisfied:** count from (1) must be ≥ `min_file_count`.
3. **Forbidden paths untouched:** for each `forbidden_paths` entry, run 3. **Forbidden paths untouched:** for each `forbidden_paths` entry, run
`git diff --name-only HEAD~{attempts} HEAD -- {path}` (since this step `git diff --name-only {start_sha} -- {path}` (since this step
began). Any modified forbidden path = fail. began). Any modified forbidden path = fail.
4. **Bash syntax check:** for each entry in `bash_syntax_check` AND any 4. **Bash syntax check:** for each entry in `bash_syntax_check` AND any
`.sh` file that appears in `git diff --name-only HEAD~1 HEAD` (safety `.sh` file that appears in `git diff --name-only {start_sha}` (safety
net for unlisted scripts), run `bash -n {path}`. Non-zero exit = fail. net for unlisted scripts), run `bash -n {path}`. Non-zero exit = fail.
5. **must_contain patterns:** for each `{path, pattern}` pair, run 5. **must_contain patterns:** for each `{path, pattern}` pair, run
`grep -E "{pattern}" {path}`. No match = fail. `grep -E "{pattern}" {path}`. No match = fail.
@ -1288,45 +1477,144 @@ Manifest verification FAIL for step {N}:
- {which check failed with detail} - {which check failed with detail}
``` ```
Apply the step's `On failure:` clause (same as Sub-step D1 failure). Increment A manifest failure is a finding for Sub-step F, exactly like a command
attempts. Manifest failures count against the retry cap equally with command failure: it counts against the same fix rounds.
failures.
If all manifest checks pass: proceed to Sub-step F (checkpoint). Either way, proceed to Sub-step D3 (floor guard).
Record per-step manifest audit result in progress file: Record per-step manifest audit result in progress file:
`steps.{N}.manifest_audit = "pass" | "fail"`, `steps.{N}.manifest_audit = "pass" | "fail"`,
`steps.{N}.manifest_drift = [{check: reason, ...}]` on fail. `steps.{N}.manifest_drift = [{check: reason, ...}]` on fail.
#### Sub-step E — On failure handling #### Sub-step D3 — Floor guard
If **FAIL**, read the `On failure:` clause. Apply the retry cap: **maximum 2 retries** Read the step's diff for weakened tests — by a command:
(3 total attempts). Track attempts in `steps.{N}.attempts`.
**`On failure: revert`** ```bash
- If attempts < 3: analyze the failure, re-implement with adjustments, re-verify. node ${CLAUDE_PLUGIN_ROOT}/lib/verification/test-integrity.mjs --base {start_sha} --json
``` ```
Attempt {A}/3 failed. Retrying...
It reports a removed assert, a deleted or skipped test, a muted check
(`|| true`, continue-on-error, an inline lint/type suppression), a stub body
("not implemented") and a threshold loosened in the permissive direction.
| Exit | Meaning | Executor |
|------|---------|----------|
| 0 | clean | continue |
| 1 | findings | each finding goes to the controller (Sub-step E), which rules it `justified` only by quoting the plan line that asks for the change; every other finding is a `violation` |
| 2 | could not run | exit 2 is never green: "floor guard not run: {reason}" is a finding for Sub-step F, and the step cannot pass this round |
A finding is never dropped silently: the step report and the final report list
every one with its ruling.
#### Sub-step E — Control
**One controller, two verdicts.** After the mechanical checks (B, D1, D2, D3)
— whatever they said — launch one controller with the Agent tool:
`subagent_type: "general-purpose"`, `model: "opus"`, no `name` (Sub-step C),
no `isolation`. It is a general helper with the prompt below, not a second
Voyage agent. It reads the diff, not the helper's summary: it is never given
the step report, and it judges what changed, not what the helper says changed.
The prompt (fill in every `{…}`; pass the mechanical results verbatim as JSON):
```text
You are the controller for step {N} of the plan {plan path}. You judge; you
change nothing: no Write, no Edit, no git command that writes.
Read the step's diff yourself: git diff {start_sha} --stat, then
git diff {start_sha}. The diff is the evidence. You are not given the
helper's report and must not look for it.
The step, verbatim:
{every field of the step}
Review focus from the plan (look hardest here): {Review focus, or "none"}
Mechanical results: proof of red {red-proof JSON}; Verify exit {code};
manifest {pass | fail + drift}; floor guard {test-integrity JSON}.
Give two verdicts:
- "spec": does the diff do what the step's Changes, Files and Interface say -
no more, no less? Name any changed path outside Files.
- "quality": does the code hold - correctness, error handling, edge cases the
step names, no stub, no weakened test, the repo's own patterns?
Rule every floor-guard finding: "justified: <the plan line, quoted>" or
"violation". List what you did not judge.
Answer with exactly one JSON block:
{"spec": "pass|fail", "quality": "pass|fail",
"findings": [{"verdict": "spec|quality", "file": "...", "line": 0, "problem": "...", "fix": "..."}],
"floor_guard": [{"kind": "...", "file": "...", "ruling": "justified: ... | violation"}],
"not_judged": ["..."]}
``` ```
- If attempts == 3: revert this step's changes:
Wait for the controller to finish. Parse its JSON block; an answer without a
parseable block is a failed control (a finding for Sub-step F), never a pass.
Append both verdicts, the findings and `not_judged` to the step report.
**The round passes** only when ALL hold: `red_proof` is `"proven"` (or, below
plan_version 1.8, `"not-run"` because the step has no `Test first:`), Verify
PASS, manifest pass, every floor-guard finding ruled `justified` (and the guard
ran), `"spec": "pass"` and `"quality": "pass"`. Then go to Sub-step G.
Otherwise go to Sub-step F with every finding.
**Re-control after a fix** checks only what was fixed. A fresh controller gets
the previous round's findings and the round's own change — before the fix
round the executor saves `git diff {start_sha}` to
`{report dir}/step-{N}-round-{R}.diff`, and afterwards hands over
`diff -u` of that file against the new `git diff {start_sha}` — and answers,
per previous finding, fixed or not, plus whether the round changed anything
the findings did not name (that is a new finding). It does not re-judge the
rest of the step.
#### Sub-step F — Fix rounds
A failed round never loops sideways. **The fix rounds are capped at maximum 5 fix rounds per step, and they escalate upward** — to a fresh helper, then to
the plan, then to the operator. The executor does not improvise a new design,
and there is no ladder of cheaper models: every helper and every controller
runs `opus` (Voyage runs no helper below Opus 5.5 High).
| `fix_rounds` | Goes to | What happens |
|---|---|---|
| 1, 2, 3 | a fresh helper | Launch `voyage:step-implementer` again (no `name`, no `isolation`), `mode: fix`, with the findings (file, line, problem, fix) and the mechanical failures. It fixes only those; the tests stay locked. Then B (both ways), D1, D2, D3 run again and a fresh controller re-controls only what was fixed (Sub-step E). `attempts` +1; `iterations_remaining` −1 |
| 4 | the plan | **After three failed fix rounds the question is the architecture, not a fourth fix.** The step goes back to planning (below). Stop class 4 |
| 5 | the operator | A step already sent to the plan (`fix_rounds` 4) that fails its control again after re-planning goes to the operator. The executor never builds this step again: on a later `--resume` it only runs the checks (B, D1, D2, D3, E) on whatever the operator changed. Stop class 4 |
**Back to planning** (`fix_rounds = 4`): save `git diff {start_sha}` to
`{report dir}/step-{N}-upward.diff` and write `{report dir}/step-{N}-upward.md`:
the step verbatim, every round's findings and verdicts, and the question for
the planner — which assumption in the step (its Changes, its Interface, its
test) makes it fail. Then stop: step `failed`, run `stopped`, and tell the
operator to re-plan the step (`/trekplan` revises it) and run
`/trekexecute --resume`, which launches a fresh `build` helper for the revised
step and keeps `fix_rounds` at 4. A step whose proof of red cannot run on a
≥ 1.8 plan because its `Verify:` is not one test-runner invocation goes here
directly (Sub-step B).
**The plan's `On failure:` clause** decides what happens around the ladder:
**`On failure: revert`** and **`On failure: retry`**
- Fix rounds 1–3 run as above; for `retry`, the clause's alternative approach
is handed to the round-1 fix helper.
- When the step goes up (round 4 or 5), first save the upward files, then
revert this step's changes (nothing of the step is committed yet):
```bash ```bash
git checkout -- {files from Files: field} git checkout -- {files from Files: field}
``` ```
Record failure. **Do NOT proceed to next step.** Jump to Phase 7. Record failure. **Do NOT proceed to next step.** Jump to Phase 7.
**`On failure: retry`**
- If attempts < 3: use the alternative approach described in the On failure clause.
- If attempts == 3: revert and stop. Jump to Phase 7.
**`On failure: skip`** **`On failure: skip`**
- Mark step as skipped regardless of attempt count. Continue to next step. - Mark step as skipped at its first failed round, without fix rounds. Continue
to next step.
``` ```
Step {N}: SKIPPED (non-critical per plan) Step {N}: SKIPPED (non-critical per plan)
``` ```
Update `steps.{N}.status = "skipped"`. Update `steps.{N}.status = "skipped"`.
**`On failure: escalate`** **`On failure: escalate`**
- Stop immediately regardless of attempt count. - Stop at the first failed round, without fix rounds (stop class 4: the plan's
author declared that past this point is guesswork).
``` ```
Step {N}: ESCALATED — requires human judgment Step {N}: ESCALATED — requires human judgment
``` ```
@ -1338,9 +1626,33 @@ If **FAIL**, read the `On failure:` clause. Apply the retry cap: **maximum 2 ret
``` ```
Jump to Phase 7. Jump to Phase 7.
#### Sub-step F — Checkpoint The same wip commit of completed work runs whenever a step goes up (round 4
or 5). Release the step's test lock before stopping
(`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs unlock`, `test_lock` =
`"released"`), so the operator and the planner can edit the test.
Run the `Checkpoint:` git commit command exactly as written in the plan. #### Sub-step G — Done line, unlock, checkpoint
1. **The done line.** A step is done only when a script says so, and the script
writes its line only when the test command exits 0:
```bash
node ${CLAUDE_PLUGIN_ROOT}/lib/verification/done-line.mjs --step {N} --cmd "{Verify command}" --report {report_path}
```
Exit 0 = `DONE step {N}: …` appended to the step report. Exit 1 = the test
command failed after all: back to Sub-step F. Exit 2 = it could not run:
never green — on a ≥ 1.8 plan that is the plan's `Verify:` (up to the plan,
as in Sub-step B); below 1.8, a step whose `Verify:` is not a test-runner
command records "no done line (Verify is not a test-runner command)" in its
report and rests on Sub-step D1's exit code. Confirm with
`node ${CLAUDE_PLUGIN_ROOT}/lib/verification/done-line.mjs --check --step {N} --report {report_path}`
(exit 0 = present). Neither the helper nor the executor ever types a DONE
line.
2. **Release the test lock**:
`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs unlock` → `test_lock` =
`"released"` (a step that was `"not-locked"` stays so).
3. **Checkpoint.** Stage the step's `Files:` — and only those, never
`git add -A` — then run the `Checkpoint:` git commit command exactly as
written in the plan.
If the commit fails (nothing to commit, etc.): warn but do NOT fail the step. If the commit fails (nothing to commit, etc.): warn but do NOT fail the step.
The step's verification already passed — the commit is bookkeeping. The step's verification already passed — the commit is bookkeeping.
@ -1474,6 +1786,23 @@ now enforced on the single-session path and not only on the multi-session one.
Continue to Phase 7.5 either way: the manifest audit is independent, and a Continue to Phase 7.5 either way: the manifest audit is independent, and a
verification failure must not hide a manifest drift. verification failure must not hide a manifest drift.
### End to end — "run the thing" (P9 (6))
Tests passing is not the product working. From the runner's JSON, set
`end_to_end`:
- `"verified"` — at least one criterion that **passed** says in its text that
it runs the thing end to end (its text contains `end to end`, `end-to-end`
or `run the thing`, any case);
- `"not-verified"` — otherwise, including a session spec and a plan whose
verification did not pass.
`"not-verified"` is never green: the report says **not verified end to end**
in plain words, and the completion gate (Phase 7.5, clause 4) turns
`completed` into `partial`. The fix is in the plan — a criterion that runs the
built thing — not in the run. (This reads the existing `## Verification`
criteria; it adds no plan field.)
## Phase 7.5 — Manifest audit (independent) ## Phase 7.5 — Manifest audit (independent)
**Runs for all modes except dry-run.** This is the last-line-of-defense **Runs for all modes except dry-run.** This is the last-line-of-defense
@ -1505,6 +1834,13 @@ produces an independent truth based on objective state, not self-narrative.
6. **Forbidden-path sweep:** for each `scope_forbidden` entry, check 6. **Forbidden-path sweep:** for each `scope_forbidden` entry, check
`git diff --name-only {start_sha} HEAD -- {path}` is empty. `git diff --name-only {start_sha} HEAD -- {path}` is empty.
7. **Done-line sweep:** for every step with `status=completed` that carries a
`report_path`, run
`node ${CLAUDE_PLUGIN_ROOT}/lib/verification/done-line.mjs --check --step {N} --report {report_path}`.
Exit 1 on a plan ≥ 1.8 = drift (`{check: "done-line", expected: "DONE step N", actual: "absent"}`);
below 1.8 it is drift only when the step's report does not say why it has
none (Sub-step G).
Compare the audit result against the executor's `progress.status`: Compare the audit result against the executor's `progress.status`:
- **Audit pass + progress=completed:** status stays `completed`. - **Audit pass + progress=completed:** status stays `completed`.
@ -1548,6 +1884,9 @@ Record in progress file:
> result is OVERRIDDEN `completed → partial`, exactly as the audit-override > result is OVERRIDDEN `completed → partial`, exactly as the audit-override
> already does (`:1221`). So the counter's liveness is auditable at the gate, > already does (`:1221`). So the counter's liveness is auditable at the gate,
> not merely asserted in prose. > not merely asserted in prose.
> 4. **End to end** — for a trekplan, Phase 7's `end_to_end == "verified"`.
> `"not-verified"` OVERRIDES `completed → partial` with the reason
> "not verified end to end": tests that pass are not the thing running.
> >
> Termination authority therefore rests on the deterministic audit + an > Termination authority therefore rests on the deterministic audit + an
> objective stop-signal, never on the model's narrative. Native `/goal` (when > objective stop-signal, never on the model's narrative. Native `/goal` (when
@ -1611,16 +1950,19 @@ Do NOT dispatch a third recovery. Report to the user.
Every recovery/retry loop in this executor has a documented numeric bound. Every recovery/retry loop in this executor has a documented numeric bound.
These are **three distinct axes** — not one nested counter: These are **three distinct axes** — not one nested counter:
- **Per-step retry cap** — `attempts ≤ 3` (initial + **maximum 2 retries**; - **Per-step fix-round cap** — `fix_rounds ≤ 5`: maximum 5 fix rounds per step, escalating upward
Hard Rule 5 / Sub-step E). Bounds the retries of a *single* step. (rounds 1–3 fix with a fresh helper, 4 goes to the plan, 5 to the operator;
Hard Rule 5 / Sub-step F). `attempts` counts the helpers a step launched.
Bounds the rounds of a *single* step.
- **Per-session recovery cap** — `recovery_depth < 2` (Phase 7.6 hard cap). - **Per-session recovery cap** — `recovery_depth < 2` (Phase 7.6 hard cap).
Bounds how deep recovery *dispatch* nests. Bounds how deep recovery *dispatch* nests.
- **Global recovery/retry budget** — a new env-overridable constant - **Global recovery/retry budget** — a new env-overridable constant
`TREKEXECUTE_MAX_RECOVERY_ITERATIONS` (default **25**) — the aggregate ceiling `TREKEXECUTE_MAX_RECOVERY_ITERATIONS` (default **25**) — the aggregate ceiling
on the **total count of recovery + retry iterations across the whole on the **total count of recovery + retry iterations across the whole
execution**. This is the case the two caps above do NOT bound: many steps each execution**. This is the case the two caps above do NOT bound: many steps each
retrying up to 3× can sum unbounded. It primarily backstops the **retry using up to 3 fix rounds can sum unbounded. It primarily backstops the **fix-round
aggregate** (recovery dispatch is already hard-capped at 2). Surfaced and aggregate** (recovery dispatch is already hard-capped at 2). Reaching 0 stops
the run (stop class 4: a plan that needs that many fixes is guesswork). Surfaced and
enforced via the `iterations_remaining` counter (Phase 3 schema; decremented enforced via the `iterations_remaining` counter (Phase 3 schema; decremented
on every recovery/retry iteration; reconciled at the Phase 7.5 completion on every recovery/retry iteration; reconciled at the Phase 7.5 completion
gate). gate).
@ -1787,15 +2129,37 @@ Phase 2.3 (validate exit) and Phase 5 (dry-run) intentionally do not write
**Type:** {plan | session-spec} **Type:** {plan | session-spec}
**Mode:** {execute | resume | step N} **Mode:** {execute | resume | step N}
**Result:** {COMPLETED | FAILED at step N | STOPPED (escalation) | PARTIAL (N/total passed)} **Result:** {COMPLETED | FAILED at step N | STOPPED (escalation) | PARTIAL (N/total passed)}
**End to end:** {verified | not verified end to end}
### Step Results ### Step Results
| Step | Description | Result | Attempts | Commit | Manifest | | Step | Description | Result | Red proof | Fix rounds | Test lock | Done line | Attempts | Commit | Manifest |
|------|-------------|--------|----------|--------|----------| |------|-------------|--------|-----------|------------|-----------|-----------|----------|--------|----------|
| 0 | Sandbox pre-flight | PASS | 1 | — | n/a | | 0 | Sandbox pre-flight | PASS | n/a | 0 | n/a | n/a | 1 | — | n/a |
| 1 | {desc} | PASS | 1 | abc1234 | pass | | 1 | {desc} | PASS | proven | 0 | released | yes | 1 | abc1234 | pass |
| 2 | {desc} | FAIL | 3 | — | — | | 2 | {desc} | FAIL | rejected | 4 (sent to the plan) | released | no | 4 | — | — |
| 3 | {desc} | — | 0 | — | — | | 3 | {desc} | — | — | 0 | — | — | 0 | — | — |
Steps without proof of red: {N} ({step numbers}; below plan_version 1.8 only)
### Controller verdicts
Per step and round: spec {pass | fail}, quality {pass | fail}, findings (file:line — problem),
floor-guard rulings, and what the controller did not judge. Report paths: {report_path per step}.
### Stops
{none — the run ended without a stop | the stop that ended the run, with its class from the
closed stop list (Phase 5.5): 1 cannot be undone · 2 security-sensitive · 3 outside the working
copy · 4 plan so broken that everything after it is guesswork, and the file or command that shows it}
### Decisions taken
| Step | Decision | Cost if wrong |
|------|----------|---------------|
| {N} | {what the executor or a helper decided that the plan did not dictate} | {what must be redone, by whom, how big} |
{none, when the plan dictated every call}
### Plan Verification (Phase 7, trekplans only) ### Plan Verification (Phase 7, trekplans only)
@ -1819,8 +2183,12 @@ Phase 2.3 (validate exit) and Phase 5 (dry-run) intentionally do not write
- Not reached: {N} - Not reached: {N}
- Blocked (sandbox): {N} - Blocked (sandbox): {N}
{if all passed + exit condition passed + plan verification passed}: {if all passed + exit condition passed + plan verification passed + end to end verified}:
All steps completed. Exit condition: PASS. Plan verification: PASS. All steps completed. Exit condition: PASS. Plan verification: PASS. End to end: verified.
{if all passed but end_to_end = not-verified}:
All steps passed their checks, but the plan has no passing run-the-thing criterion:
not verified end to end. Result: PARTIAL.
{if failed/stopped}: {if failed/stopped}:
### Failure Details ### Failure Details
@ -1842,7 +2210,8 @@ To continue (multi-session waves): re-run the same /trekexecute command (not --r
- `completed` — all steps passed AND Phase 7.5 manifest audit passed AND - `completed` — all steps passed AND Phase 7.5 manifest audit passed AND
(for trekplans) Phase 7 plan verification passed (for trekplans) Phase 7 plan verification passed
- `partial` — steps passed per executor but Phase 7.5 found drift, OR - `partial` — steps passed per executor but Phase 7.5 found drift, OR
Phase 7.6 recovery incomplete Phase 7.6 recovery incomplete, OR the run is not verified end to end
(Phase 7, gate clause 4)
- `blocked` — Step 0 sandbox pre-flight exited 77; no real work attempted - `blocked` — Step 0 sandbox pre-flight exited 77; no real work attempted
- `failed` — a step failed and On failure was revert/retry (retry cap hit) - `failed` — a step failed and On failure was revert/retry (retry cap hit)
- `stopped` — On failure: escalate triggered - `stopped` — On failure: escalate triggered
@ -1871,7 +2240,11 @@ To continue (multi-session waves): re-run the same /trekexecute command (not --r
"recovery_depth": 0, "recovery_depth": 0,
"iterations_remaining": 25, "iterations_remaining": 25,
"legacy_plan": false, "legacy_plan": false,
"progress_file": "{path}" "progress_file": "{path}",
"end_to_end": "{verified | not-verified | n/a}",
"steps_without_red_proof": 0,
"stop_class": "{none | 1 | 2 | 3 | 4}",
"decisions": 0
} }
} }
``` ```
@ -1968,9 +2341,10 @@ strategy (gates auto-escalation, parallel-wave choice — see High-effort
behavior below). The resolver does NOT control the orchestrator's own behavior below). The resolver does NOT control the orchestrator's own
model — that is fixed at invocation time (command frontmatter omits model — that is fixed at invocation time (command frontmatter omits
`model:`, so it follows the session model) and cannot be switched mid-turn. `model:`, so it follows the session model) and cannot be switched mid-turn.
`/trekexecute` spawns no sub-agent swarm (Hard Rule 10), so `/trekexecute` spawns no swarm: one step helper at a time and one controller
`phase_signal_result.model` has no spawn site here; it is returned for (Hard Rule 10), and both run `opus` — the step agent's pin and the
cross-command uniformity and stats. controller's `model: "opus"` — so `phase_signal_result.model` still has no
spawn site here; it is returned for cross-command uniformity and stats.
For `/trekexecute` specifically: `effort == 'low'` activates `--gates open` For `/trekexecute` specifically: `effort == 'low'` activates `--gates open`
+ sequential-only execution (no worktree-isolated parallel waves — runs + sequential-only execution (no worktree-isolated parallel waves — runs
@ -2022,7 +2396,9 @@ code-path.
1. **No AskUserQuestion for execution decisions.** All execution decisions come 1. **No AskUserQuestion for execution decisions.** All execution decisions come
from the plan's On failure clauses. If the plan says escalate, stop and from the plan's On failure clauses. If the plan says escalate, stop and
report — never ask. report — never ask. What the plan leaves open, the executor decides and
writes into the decision log with its `cost_if_wrong`; it stops only on the
closed stop list (Phase 5.5).
2. **No scope creep.** Only touch files listed in the step's `Files:` field. 2. **No scope creep.** Only touch files listed in the step's `Files:` field.
If a file outside the list seems to need changing, record it as a finding If a file outside the list seems to need changing, record it as a finding
@ -2034,7 +2410,10 @@ code-path.
4. **Fresh verification.** Re-run the Verify command from scratch every time. 4. **Fresh verification.** Re-run the Verify command from scratch every time.
Never trust cached or prior results. Never trust cached or prior results.
5. **Retry cap = 3 attempts.** Initial + 2 retries, then stop. Never loop forever. 5. **Fix rounds go up — maximum 5 fix rounds per step, escalating upward.**
Rounds 1–3 are fixes by a fresh helper, round 4 sends the step back to
planning, round 5 to the operator (Sub-step F). Never a fourth guess at the
same level, never a cheaper model, never a loop.
6. **Never corrupt completed work.** Only revert files from the failing step. 6. **Never corrupt completed work.** Only revert files from the failing step.
Never touch files from earlier passed steps. Never touch files from earlier passed steps.
@ -2047,11 +2426,15 @@ code-path.
9. **Progress file is ground truth.** Resume uses the progress file, not git log. 9. **Progress file is ground truth.** Resume uses the progress file, not git log.
10. **No sub-agents.** The executor reads and implements directly. 10. **One fresh helper per plan step, one controller, nothing more.** The
No Agent tool, no TeamCreate, no delegation. Enforced (not just executor writes no product code: each step is built by a fresh
documented) via `disallowed-tools: Agent, TeamCreate` in the frontmatter, `voyage:step-implementer` (Sub-step C) and judged by one controller, a
which removes both tools from the pool — `allowed-tools` alone does not, general-purpose helper that reads the diff (Sub-step E).
since omission leaves a tool callable (CC 2.1.152). Helpers never start helpers: the step agent has no Agent tool, and the
controller is told it changes nothing. Never the Agent tool's `name` (a named spawn's
result never returns), never `isolation` (measured: wrong base). No
TeamCreate — kept out via `disallowed-tools: TeamCreate`, since omission
from `allowed-tools` alone leaves a tool callable (CC 2.1.152).
11. **Worktree isolation is mandatory for multi-session execution.** Every 11. **Worktree isolation is mandatory for multi-session execution.** Every
session order names its own git worktree. Never run two or more sessions session order names its own git worktree. Never run two or more sessions
@ -2123,14 +2506,49 @@ code-path.
failure (worktree removed, branch deleted, work lost) into a recoverable failure (worktree removed, branch deleted, work lost) into a recoverable
one (push succeeded, branch preserved on remote). Source: research/02 R3. one (push succeeded, branch preserved on remote). Source: research/02 R3.
20. **Three distinct iteration axes.** Per-step `attempts` (≤3), per-session 20. **Three distinct iteration axes.** Per-step `fix_rounds` (≤5, upward),
`recovery_depth` (<2), and the global `TREKEXECUTE_MAX_RECOVERY_ITERATIONS` per-session `recovery_depth` (<2), and the global
budget (default 25 — aggregate recovery+retry backstop) are independent `TREKEXECUTE_MAX_RECOVERY_ITERATIONS` budget (default 25 — aggregate
bounds, not one nested counter. Every recovery/retry loop is bounded by at fix-round + recovery backstop) are independent bounds, not one nested
least one of them. See "Iteration caps + global recovery/retry budget". counter. Every recovery/retry loop is bounded by at least one of them. See
"Iteration caps + global recovery/retry budget".
21. **Never start Claude Code.** No `claude` CLI invocation, no Agent SDK, no 21. **Never start Claude Code.** No `claude` CLI invocation, no Agent SDK, no
background process that runs a session — not for waves, not for recovery, background process that runs a session — not for waves, not for recovery,
not for a test. Work that needs another session becomes a session order not for a test. Work that needs another session becomes a session order
plus a starter line the user gives a session they open themselves plus a starter line the user gives a session they open themselves
(Phase 2.6, Phase 7.6). (Phase 2.6, Phase 7.6).
22. **Proof, not word.** Red before green, an unweakened test and "done" are
each a command's exit code — `red-proof.mjs`, `test-integrity.mjs`,
`done-line.mjs` — never the helper's or the executor's statement. Exit 2
from any of them ("could not run") is never green.
## Credits
Every borrowing in this executor is named here, never silent. All sources are
MIT-licensed open source except Kiro (a closed product: only its published
documentation was read) and Cherny (public talks and posts). Read 2026-10-06;
pins and quotes in the local practice analysis.
- **Superpowers** — adapted from obra/superpowers@8ca22db (MIT, © 2025 Jesse
Vincent): `subagent-driven-development` — a fresh helper per task, hand-over
in files, a review package from a recorded start point, a reviewer between
tasks, the stop classes and the decision log of an unattended run (P8, P4);
`executing-plans/scripts/task-done` — the completion line appended only when
the test command exits 0, here `done-line.mjs` (P9 (4));
`systematic-debugging` — after three failed fixes, question the
architecture (P9 (7)). Not borrowed: its ladder of cheaper models.
- **ai-native-sdlc** — bashebr/ai-native-sdlc@11fb8d7 (MIT): the helper's
report contract, ran / saw / did NOT check (P8).
- **Kiro** (documentation only): the task runner takes the diff, not the
agent's account of it, as the reviewer's input (P8). Its background daemon is
not borrowed; only the step loop.
- **GitHub Spec Kit** (MIT), github/spec-kit@2e941fe: test-first as a rule,
mechanized here as `red-proof.mjs` (P9 (1)).
- **agent-skills** — addyosmani/agent-skills@9d0c60d (MIT): the floor guard of
constraint-driven development, as `test-integrity.mjs` (P9 (2)).
- **Cherny** (public primary sources): "run the thing" — a plan is verified
end to end only by a criterion that runs what was built (P9 (6)).
- **Claude Code** itself: subagents for the helper and the controller; the
`PreToolUse` guard behind the test lock (K8).

View file

@ -270,7 +270,7 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
## Handover 5 — `progress.json` (resume contract) ## Handover 5 — `progress.json` (resume contract)
**Producer:** `/trekexecute` per-step (after Verify + Manifest audit + Checkpoint). **Producer:** `/trekexecute` per-step (start point recorded before the step; then proof of red + Verify + Manifest audit + floor guard + control + done line + Checkpoint).
**Consumer:** `/trekexecute --resume` (re-entry) + `pre-compact-flush` hook (drift detection before context compaction). **Consumer:** `/trekexecute --resume` (re-entry) + `pre-compact-flush` hook (drift detection before context compaction).
@ -296,6 +296,8 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
| `session_start_sha` | string | optional | git sha at execute start | | `session_start_sha` | string | optional | git sha at execute start |
| `session_end_sha` | string | optional | git sha at execute end | | `session_end_sha` | string | optional | git sha at execute end |
| `steps` | object | yes | Map of step number → step record | | `steps` | object | yes | Map of step number → step record |
| `end_to_end` | string | optional | `verified \| not-verified` — a passing run-the-thing criterion in the plan's `## Verification`; `not-verified` turns `completed` into `partial` (wave 1, P9 (6)) |
| `decisions` | array | optional | `[{step, decision, cost_if_wrong}]` — every call the executor made that the plan did not dictate; both strings non-empty (wave 1, P4) |
**Per-step record:** **Per-step record:**
@ -308,6 +310,15 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
| `commit` | string \| null | git sha after Checkpoint | | `commit` | string \| null | git sha after Checkpoint |
| `manifest_audit` | string | `pass \| fail \| pass-with-note \| n/a` | | `manifest_audit` | string | `pass \| fail \| pass-with-note \| n/a` |
| `note` | string | optional human-readable annotation | | `note` | string | optional human-readable annotation |
| `start_sha` | string \| null | optional (wave 1) — git sha the step started from; every check of the step reads `git diff {start_sha}` |
| `report_path` | string \| null | optional (wave 1) — the step's report file: the helper's ran / saw / did-NOT-check rounds and the script-written `DONE step N` line |
| `red_proof` | string \| null | optional (wave 1) — `proven \| rejected \| not-run`, the exit of `lib/verification/red-proof.mjs` |
| `test_lock` | string \| null | optional (wave 1) — `locked \| released \| not-locked` (K8, `lib/util/test-lock.mjs`) |
| `fix_rounds` | number | optional (wave 1) — 0..5: 0 = passed its first control, 1–3 = fix rounds, 4 = sent up to the plan, 5 = sent up to the operator. 6 is invalid |
The wave-1 fields are **additive-optional on read**: a `progress.json` written
before them validates and resumes unchanged (no `schema_version` bump). `null`
means "not set yet". When present, `progress-validator.mjs` shape-checks them.
**Validation strategy:** `progress-validator.mjs` runs at: **Validation strategy:** `progress-validator.mjs` runs at:
1. `/trekexecute --validate` (alongside plan-validator) 1. `/trekexecute --validate` (alongside plan-validator)
@ -325,6 +336,9 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
- `PROGRESS_STEP_RANGE` → `current_step` outside `[0, total_steps]`; resume halts - `PROGRESS_STEP_RANGE` → `current_step` outside `[0, total_steps]`; resume halts
- `PROGRESS_ALREADY_DONE` → `status === completed`; nothing to resume - `PROGRESS_ALREADY_DONE` → `status === completed`; nothing to resume
- `PROGRESS_STEP_COUNT_MISMATCH` → warning; not a blocker - `PROGRESS_STEP_COUNT_MISMATCH` → warning; not a blocker
- `PROGRESS_FIX_ROUNDS_INVALID` → `fix_rounds` not an integer 0..5; resume halts
- `PROGRESS_START_SHA_INVALID` / `PROGRESS_REPORT_PATH_INVALID` / `PROGRESS_RED_PROOF_INVALID` / `PROGRESS_TEST_LOCK_INVALID` → a wave-1 step field of the wrong shape; resume halts
- `PROGRESS_DECISION_INVALID` → `decisions` is not an array of `{decision, cost_if_wrong}`; resume halts
--- ---

View file

@ -73,10 +73,11 @@ native `effort:`.
**Native `effort:` on agents.** Voyage sets the reasoning axis statically on **Native `effort:` on agents.** Voyage sets the reasoning axis statically on
selected agents, overriding the session level: selected agents, overriding the session level:
- **Eight agents pin `high`:** the retrieval agents `task-finder`, - **Nine agents pin `high`:** the retrieval agents `task-finder`,
`git-historian`, `dependency-tracer`, `architecture-mapper`, and the `git-historian`, `dependency-tracer`, `architecture-mapper`, the
adversarial-reasoning agents `plan-critic`, `risk-assessor`, adversarial-reasoning agents `plan-critic`, `risk-assessor`,
`contrarian-researcher`, `review-coordinator`. No agent pins below `high` `contrarian-researcher`, `review-coordinator`, and `/trekexecute`'s step
helper `step-implementer`. No agent pins below `high`
(operator rule, 2026-10-05: standard is Opus 5.5 at effort High). (operator rule, 2026-10-05: standard is Opus 5.5 at effort High).
- **All other agents:** unset → inherits the session's effort level (a session - **All other agents:** unset → inherits the session's effort level (a session
that nothing sets runs at its model's default — `medium` on Opus 5.5). that nothing sets runs at its model's default — `medium` on Opus 5.5).

View file

@ -6,6 +6,9 @@
// PROGRESS_REQUIRED_TOP, so legacy progress.json without it still validates. The field's // PROGRESS_REQUIRED_TOP, so legacy progress.json without it still validates. The field's
// liveness (never-decremented) is enforced deterministically by the Phase 7.5 completion-gate // liveness (never-decremented) is enforced deterministically by the Phase 7.5 completion-gate
// cross-check in trekexecute, not by this per-validation shape check. // cross-check in trekexecute, not by this per-validation shape check.
// The same holds for the wave-1 step-loop fields (per step: start_sha, report_path,
// red_proof, test_lock, fix_rounds 0..5; per run: decisions): shape-checked when
// present, never required.
import { readFileSync, existsSync } from 'node:fs'; import { readFileSync, existsSync } from 'node:fs';
import { issue, fail } from '../util/result.mjs'; import { issue, fail } from '../util/result.mjs';
@ -17,6 +20,45 @@ export const PROGRESS_VALID_STATUSES = ['pending', 'in_progress', 'completed', '
// know (lib/util/session-orders.mjs ENDED_BAD). // know (lib/util/session-orders.mjs ENDED_BAD).
export const PROGRESS_STEP_STATUSES = ['completed', 'in_progress', 'failed', 'pending', 'deferred', 'skipped', 'blocked']; export const PROGRESS_STEP_STATUSES = ['completed', 'in_progress', 'failed', 'pending', 'deferred', 'skipped', 'blocked'];
// Wave 1 (P8/P9) per-step fields, all optional on read so an older progress.json
// still resumes; `null` means "not set yet". fix_rounds is the step's place on
// the upward ladder: 0 = passed its first control, 1-3 = fix rounds, 4 = sent up
// to the plan, 5 = sent up to the operator. There is no sixth round.
export const PROGRESS_MAX_FIX_ROUNDS = 5;
export const PROGRESS_RED_PROOF_VALUES = ['proven', 'rejected', 'not-run'];
export const PROGRESS_TEST_LOCK_VALUES = ['locked', 'released', 'not-locked'];
function checkStepFields(k, s, errors) {
const set = (v) => v !== undefined && v !== null;
if (s.fix_rounds !== undefined &&
!(Number.isInteger(s.fix_rounds) && s.fix_rounds >= 0 && s.fix_rounds <= PROGRESS_MAX_FIX_ROUNDS)) {
errors.push(issue('PROGRESS_FIX_ROUNDS_INVALID',
`steps["${k}"].fix_rounds=${JSON.stringify(s.fix_rounds)} must be an integer 0..${PROGRESS_MAX_FIX_ROUNDS}`));
}
if (set(s.start_sha) && !(typeof s.start_sha === 'string' && /^[0-9a-f]{7,64}$/.test(s.start_sha))) {
errors.push(issue('PROGRESS_START_SHA_INVALID', `steps["${k}"].start_sha must be a git sha, got ${JSON.stringify(s.start_sha)}`));
}
if (set(s.report_path) && !(typeof s.report_path === 'string' && s.report_path.trim() !== '')) {
errors.push(issue('PROGRESS_REPORT_PATH_INVALID', `steps["${k}"].report_path must be a non-empty path`));
}
if (set(s.red_proof) && !PROGRESS_RED_PROOF_VALUES.includes(s.red_proof)) {
errors.push(issue('PROGRESS_RED_PROOF_INVALID', `steps["${k}"].red_proof "${s.red_proof}" not in [${PROGRESS_RED_PROOF_VALUES.join(', ')}]`));
}
if (set(s.test_lock) && !PROGRESS_TEST_LOCK_VALUES.includes(s.test_lock)) {
errors.push(issue('PROGRESS_TEST_LOCK_INVALID', `steps["${k}"].test_lock "${s.test_lock}" not in [${PROGRESS_TEST_LOCK_VALUES.join(', ')}]`));
}
}
// The run-level decision log (P4): every decision the executor took on its own,
// with what it costs if it is wrong. Optional; when present, every entry carries both.
function checkDecisions(decisions, errors) {
if (decisions === undefined) return;
const ok = Array.isArray(decisions) && decisions.every((d) => d && typeof d === 'object' &&
typeof d.decision === 'string' && d.decision.trim() !== '' &&
typeof d.cost_if_wrong === 'string' && d.cost_if_wrong.trim() !== '');
if (!ok) errors.push(issue('PROGRESS_DECISION_INVALID', 'decisions must be an array of {decision, cost_if_wrong} with both non-empty'));
}
export function validateProgressContent(jsonText, opts = {}) { export function validateProgressContent(jsonText, opts = {}) {
let parsed; let parsed;
try { parsed = JSON.parse(jsonText); } try { parsed = JSON.parse(jsonText); }
@ -82,9 +124,12 @@ export function validateProgressObject(parsed, opts = {}) {
if (s.status !== undefined && !PROGRESS_STEP_STATUSES.includes(s.status)) { if (s.status !== undefined && !PROGRESS_STEP_STATUSES.includes(s.status)) {
warnings.push(issue('PROGRESS_STEP_BAD_STATUS', `steps["${k}"].status "${s.status}" unrecognized`)); warnings.push(issue('PROGRESS_STEP_BAD_STATUS', `steps["${k}"].status "${s.status}" unrecognized`));
} }
checkStepFields(k, s, errors);
} }
} }
checkDecisions(parsed.decisions, errors);
return { valid: errors.length === 0, errors, warnings, parsed }; return { valid: errors.length === 0, errors, warnings, parsed };
} }

View file

@ -0,0 +1,96 @@
// lib/verification/done-line.mjs
// The done line (P9 (4)). A step is "done" only when this script has written
// its line, and it writes the line only when the test command exits 0:
//
// DONE step <N>: `<command>` exit 0 at <HEAD sha> <ISO time>
//
// appended to the step's report file. Exit 0 = written, 1 = the command failed
// (nothing written), 2 = could not run (nothing written; never green): no
// --step, a command outside the test-runner allowlist, a command that cannot
// start, a working tree that is not a git repo.
//
// `--check --step <N> --report <file>`: exit 0 when the report carries a
// well-formed line for step N, 1 when it does not. A hand-typed "DONE" is not
// the script's line.
//
// Adapted from obra/superpowers@8ca22db (MIT, (c) 2025 Jesse Vincent),
// skills/executing-plans/scripts/task-done: the completion line is appended
// only when the test command exits 0.
import { appendFileSync, existsSync, readFileSync, realpathSync } from 'node:fs';
import { spawnSync } from 'node:child_process';
import { resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { allowedCommand, screenCommand } from './criteria-runner.mjs';
import { commandEnv, DEFAULT_TIMEOUT_MS, repoRoot } from './red-proof.mjs';
export function doneLineRe(step) {
return new RegExp(`^DONE step ${step}: \`.+\` exit 0 at [0-9a-f]{40} \\d{4}-\\d{2}-\\d{2}T\\S+$`, 'm');
}
export function hasDoneLine(reportPath, step) {
if (!existsSync(reportPath)) return false;
return doneLineRe(step).test(readFileSync(reportPath, 'utf8'));
}
/** @returns {{status: 'written'|'failed'|'not-run', reason: string, line?: string, exit?: number|null}} */
export function writeDoneLine({ cwd, step, cmd, report, timeoutMs = DEFAULT_TIMEOUT_MS }) {
if (!/^\d+$/.test(String(step ?? ''))) return { status: 'not-run', reason: '--step must be a step number' };
if (!report) return { status: 'not-run', reason: '--report is required' };
// The plan's Verify command is relative to the repo root, as in red-proof.mjs.
const dir = repoRoot(resolve(cwd ?? process.cwd()));
if (!dir) return { status: 'not-run', reason: `${cwd ?? process.cwd()} is not inside a git repository` };
const allowed = allowedCommand(cmd, { cwd: dir });
if (!allowed.allowed) return { status: 'not-run', reason: `command not run: ${allowed.reason}` };
const screen = screenCommand(cmd);
if (!screen.allowed) return { status: 'not-run', reason: `command blocked: ${screen.rule}` };
const head = spawnSync('git', ['rev-parse', '--verify', '--quiet', 'HEAD'], { cwd: dir, encoding: 'utf8' });
if (head.status !== 0) return { status: 'not-run', reason: `no git HEAD in ${dir}` };
const r = spawnSync('/bin/sh', ['-c', cmd], { cwd: dir, env: commandEnv(), encoding: 'utf8', timeout: timeoutMs, maxBuffer: 64 * 1024 * 1024 });
if (r.error || r.status === null || r.status === 126 || r.status === 127) {
return { status: 'not-run', reason: `the command did not run to an exit code (${r.error?.message ?? r.signal ?? `exit ${r.status}`})`, exit: r.status };
}
if (r.status !== 0) return { status: 'failed', reason: `the test command exited ${r.status}`, exit: r.status };
const line = `DONE step ${step}: \`${cmd}\` exit 0 at ${head.stdout.trim()} ${new Date().toISOString()}`;
appendFileSync(report, `${line}\n`);
return { status: 'written', reason: 'the test command exited 0', line, exit: 0 };
}
const USAGE =
'usage: done-line.mjs --step <N> --cmd "<test command>" --report <file> [--cwd <dir>] [--json]\n' +
' done-line.mjs --check --step <N> --report <file>';
export function main(argv) {
const o = {};
let check = false;
let json = false;
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === '--step' && argv[i + 1]) o.step = argv[++i];
else if (a === '--cmd' && argv[i + 1]) o.cmd = argv[++i];
else if (a === '--report' && argv[i + 1]) o.report = resolve(argv[++i]);
else if (a === '--cwd' && argv[i + 1]) o.cwd = resolve(argv[++i]);
else if (a === '--timeout' && argv[i + 1]) o.timeoutMs = Number(argv[++i]);
else if (a === '--check') check = true;
else if (a === '--json') json = true;
else { process.stderr.write(`done-line: unknown argument ${a}\n${USAGE}\n`); return 2; }
}
if (check) {
if (!/^\d+$/.test(String(o.step ?? '')) || !o.report) { process.stderr.write(`done-line: --check needs --step and --report\n${USAGE}\n`); return 2; }
const ok = hasDoneLine(o.report, o.step);
process.stdout.write(json ? `${JSON.stringify({ step: Number(o.step), done: ok })}\n` : `step ${o.step}: ${ok ? 'DONE line present' : 'no DONE line - not done'}\n`);
return ok ? 0 : 1;
}
let r;
try { r = writeDoneLine(o); }
catch (e) { r = { status: 'not-run', reason: `done-line crashed: ${e.message}` }; }
process.stdout.write(json ? `${JSON.stringify(r)}\n` : `done line: ${r.status.toUpperCase()} - ${r.line ?? r.reason}\n`);
return r.status === 'written' ? 0 : r.status === 'failed' ? 1 : 2;
}
function isMain() {
try { return realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url)); }
catch { return false; }
}
if (process.argv[1] && isMain()) process.exitCode = main(process.argv.slice(2));

View file

@ -0,0 +1,198 @@
// lib/verification/red-proof.mjs
// Proof of red (P9 (1)). A step's test is run by a command twice:
//
// 1. against the START POINT (the step's recorded start_sha, with the step's
// test files copied in): it must FAIL - otherwise the test proves nothing
// about the change;
// 2. against the RESULT (the working tree as the step left it): it must PASS.
//
// `--red-only` runs step 1 alone: the gate the step helper passes BEFORE it
// builds (and locks the test). The executor's own check afterwards is the full
// two-sided run.
//
// Exit 0 = proven, 1 = rejected (`not-red-at-base` or `not-green-at-result`),
// 2 = could not run (`not-run`: no test files, an unknown start point, a command
// outside the test-runner allowlist, a command that cannot start). 2 is never a
// pass and never "proven".
//
// The start point is materialized with `git archive <start_sha>` into a temp
// directory, never by checking anything out in the working tree: the step's
// work is not touched, and no worktree is registered in the repo. Tracked files
// only - pass `--link node_modules` (comma list) to symlink an untracked
// dependency directory from the working tree into the start-point copy.
//
// The command must be ONE test-runner invocation without shell operators
// (criteria-runner.mjs `allowedCommand`), and it is screened through the
// plugin's own PreToolUse denylist (`screenCommand`): a command spawned from
// node never passes through the Bash tool, so the hook cannot fire by itself.
//
// When the output is TAP (`--test-reporter=tap`, pytest-tap, ...), the counts
// `# tests N` / `# fail N` are reported for both runs. They show WHAT the red
// was: "fail 1 of 1" on a node test file is usually a load error (an import the
// start point lacks), not named failing tests - see
// tests/fixtures/red-first/02243c6-always-allow-shim.mjs, re-measured through
// this tool in tests/lib/red-proof.test.mjs.
//
// Borrowed: GitHub Spec Kit (MIT, github/spec-kit@2e941fe) - its test-first
// rule is an instruction there; here a command proves it.
import { cpSync, existsSync, mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync } from 'node:fs';
import { spawnSync } from 'node:child_process';
import { tmpdir } from 'node:os';
import { dirname, isAbsolute, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { allowedCommand, screenCommand } from './criteria-runner.mjs';
export const DEFAULT_TIMEOUT_MS = 900_000;
const TAIL = 2000;
function tap(output) {
const tests = String(output).match(/^# tests (\d+)$/m);
const fail = String(output).match(/^# fail (\d+)$/m);
return tests && fail ? { tests: Number(tests[1]), fail: Number(fail[1]) } : null;
}
// The command's exit code must be its own. Under `node --test` a child inherits
// NODE_TEST_CONTEXT, and a nested `node --test` then reports to that parent and
// exits 0 whatever its tests did (measured 2026-10-06: a red start point read as
// green). So the variable never reaches the command.
export function commandEnv(env = process.env) {
const { NODE_TEST_CONTEXT, ...rest } = env;
return rest;
}
function run(cmd, cwd, timeoutMs) {
const r = spawnSync('/bin/sh', ['-c', cmd], { cwd, env: commandEnv(), encoding: 'utf8', timeout: timeoutMs, maxBuffer: 64 * 1024 * 1024 });
const output = `${r.stdout ?? ''}${r.stderr ?? ''}`;
return { exit: r.status, signal: r.signal ?? null, tap: tap(output), tail: output.slice(-TAIL), spawnError: r.error ? r.error.message : null };
}
const notRun = (reason, extra = {}) => ({ status: 'not-run', reason, ...extra });
// A shell that could not find or execute the command (126/127), a signal (the
// timeout), or a spawn error is not a test verdict at all.
function unrunnable(r) {
if (r.spawnError) return `the command could not start: ${r.spawnError}`;
if (r.exit === null) return `the command was killed (${r.signal ?? 'timeout'})`;
if (r.exit === 126 || r.exit === 127) return `the shell could not run the command (exit ${r.exit})`;
return null;
}
/**
* @param {{cwd: string, base: string, cmd: string, tests: string[], link?: string[], timeoutMs?: number}} o
* @returns {{status: 'proven'|'rejected'|'not-run', reason: string, base?: object, result?: object}}
*/
export function repoRoot(dir) {
const top = spawnSync('git', ['rev-parse', '--show-toplevel'], { cwd: dir, encoding: 'utf8' });
return top.status === 0 ? top.stdout.trim() : null;
}
export function proveRed(o) {
const given = resolve(o.cwd ?? process.cwd());
const timeoutMs = o.timeoutMs ?? DEFAULT_TIMEOUT_MS;
const tests = (o.tests ?? []).filter(Boolean);
if (!existsSync(given)) return notRun(`--cwd ${given} does not exist`);
// Plan paths and commands are relative to the repo root, and `git archive`
// run from a subdirectory packs only that subdirectory (measured
// 2026-10-06). So everything below runs at the root of --cwd's repo.
const cwd = repoRoot(given);
if (!cwd) return notRun(`--cwd ${given} is not inside a git repository`);
if (tests.length === 0) return notRun('no test files given: a step without its test cannot prove red');
for (const t of tests) {
if (isAbsolute(t) || t.split('/').includes('..')) return notRun(`test path must be repo-relative: ${t}`);
if (!existsSync(join(cwd, t))) return notRun(`test file not found in the working tree: ${t}`);
}
const allowed = allowedCommand(o.cmd, { cwd });
if (!allowed.allowed) return notRun(`command not run: ${allowed.reason}`);
const screen = screenCommand(o.cmd);
if (!screen.allowed) return notRun(`command blocked: ${screen.rule}`);
const sha = spawnSync('git', ['rev-parse', '--verify', '--quiet', `${o.base}^{commit}`], { cwd, encoding: 'utf8' });
if (sha.status !== 0) return notRun(`cannot resolve the start point ${o.base} in ${cwd}`);
const baseSha = sha.stdout.trim();
// The result first: it is the working tree as it stands. `redOnly` is the
// gate before building (the step helper's own check): the result is not run.
let result;
if (!o.redOnly) {
result = run(o.cmd, cwd, timeoutMs);
const resultBroken = unrunnable(result);
if (resultBroken) return notRun(`at the result: ${resultBroken}`, { result });
}
const tmp = mkdtempSync(join(tmpdir(), 'voyage-red-proof-'));
try {
const tree = join(tmp, 'tree');
mkdirSync(tree);
const tar = join(tmp, 'start.tar');
const a = spawnSync('git', ['archive', '--format=tar', '-o', tar, baseSha], { cwd, encoding: 'utf8' });
if (a.status !== 0) return notRun(`git archive ${baseSha} failed: ${String(a.stderr).trim()}`, { result });
const x = spawnSync('tar', ['-xf', tar, '-C', tree], { encoding: 'utf8' });
if (x.status !== 0) return notRun(`tar could not unpack the start point: ${String(x.stderr).trim()}`, { result });
for (const t of tests) {
mkdirSync(dirname(join(tree, t)), { recursive: true });
cpSync(join(cwd, t), join(tree, t));
}
for (const l of o.link ?? []) {
if (!l || isAbsolute(l) || l.split('/').includes('..')) return notRun(`--link must be repo-relative: ${l}`, { result });
if (existsSync(join(cwd, l)) && !existsSync(join(tree, l))) {
mkdirSync(dirname(join(tree, l)), { recursive: true });
symlinkSync(join(cwd, l), join(tree, l));
}
}
const base = run(o.cmd, tree, timeoutMs);
const baseBroken = unrunnable(base);
if (baseBroken) return notRun(`at the start point: ${baseBroken}`, { base, result });
const out = { base_sha: baseSha, base, result };
if (base.exit === 0) return { status: 'rejected', reason: 'not-red-at-base', ...out };
if (o.redOnly) return { status: 'proven', reason: 'red at the start point (--red-only: the result was not run)', base_sha: baseSha, base };
if (result.exit !== 0) return { status: 'rejected', reason: 'not-green-at-result', ...out };
return { status: 'proven', reason: 'red at the start point, green at the result', ...out };
} finally {
rmSync(tmp, { recursive: true, force: true });
}
}
// --- CLI ------------------------------------------------------------------------
const USAGE =
'usage: red-proof.mjs --base <start_sha> --cmd "<test command>" --tests <a,b> [--red-only] [--cwd <dir>] [--link <dir,...>] [--timeout <ms>] [--json]';
const list = (v) => String(v).split(',').map((s) => s.trim()).filter(Boolean);
export function main(argv) {
const o = { cwd: process.cwd(), tests: [], link: [] };
let json = false;
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === '--base' && argv[i + 1]) o.base = argv[++i];
else if (a === '--cmd' && argv[i + 1]) o.cmd = argv[++i];
else if (a === '--tests' && argv[i + 1]) o.tests = list(argv[++i]);
else if (a === '--cwd' && argv[i + 1]) o.cwd = resolve(argv[++i]);
else if (a === '--link' && argv[i + 1]) o.link = list(argv[++i]);
else if (a === '--red-only') o.redOnly = true;
else if (a === '--timeout' && argv[i + 1]) o.timeoutMs = Number(argv[++i]);
else if (a === '--json') json = true;
else { process.stderr.write(`red-proof: unknown argument ${a}\n${USAGE}\n`); return 2; }
}
if (!o.base || !o.cmd) { process.stderr.write(`red-proof: --base and --cmd are required\n${USAGE}\n`); return 2; }
if (o.timeoutMs !== undefined && !Number.isFinite(o.timeoutMs)) { process.stderr.write(`red-proof: --timeout must be a number\n${USAGE}\n`); return 2; }
let r;
try { r = proveRed(o); }
catch (e) { r = notRun(`red-proof crashed: ${e.message}`); }
if (json) process.stdout.write(JSON.stringify(r, null, 2) + '\n');
else {
const counts = (x) => (x?.tap ? ` (tap: ${x.tap.fail} of ${x.tap.tests} failed)` : '');
process.stdout.write(`red proof: ${r.status.toUpperCase()} - ${r.reason}\n`);
if (r.base) process.stdout.write(` start point ${r.base_sha ?? o.base}: exit ${r.base.exit}${counts(r.base)}\n`);
if (r.result) process.stdout.write(` result: exit ${r.result.exit}${counts(r.result)}\n`);
}
return r.status === 'proven' ? 0 : r.status === 'rejected' ? 1 : 2;
}
function isMain() {
try { return realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url)); }
catch { return false; }
}
if (process.argv[1] && isMain()) process.exitCode = main(process.argv.slice(2));

View file

@ -0,0 +1,283 @@
// lib/verification/test-integrity.mjs
// The floor guard (P9 (2)): read a step's diff and report every change that
// makes the tests weaker instead of the code better.
//
// removed-assert a test file loses more assertions than it gains
// deleted-test a test file loses more test declarations than it gains
// skipped-test a test is skipped, marked todo/xfail, or focused (.only)
// muted-check a check is silenced: `|| true`, continue-on-error,
// --passWithNoTests, an inline lint/type suppression ...
// stub non-test code gains a "not implemented" body
// loosened-threshold a numeric bound in a test or a check config moves in the
// permissive direction (`ms < 200` -> `ms < 2000`)
//
// Exit 0 = clean, 1 = findings, 2 = could not run (no input, unreadable input,
// an unresolvable --base). 2 is never green: an absent measurement is reported
// as absent, not as a pass.
//
// A finding is not a verdict. /trekexecute hands every finding to its
// controller, which either quotes the plan line that asked for the change or
// fails the step; nothing is dropped silently.
//
// Borrowed: the floor guard of addyosmani/agent-skills@9d0c60d (MIT),
// constraint-driven development - mechanized here as a diff reader so a
// command, not the model's word, says whether a test was weakened.
import { readFileSync, realpathSync } from 'node:fs';
import { spawnSync } from 'node:child_process';
import { resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
// --- what counts as a test / a check config ----------------------------------
const TEST_PATH_RE = [
/(^|\/)(tests?|__tests__|spec|specs)\//,
/\.(test|spec)\.[a-z0-9]+$/i,
/_(test|spec)\.[a-z0-9]+$/i,
/(^|\/)test_[^/]*\.py$/,
];
const CHECK_CONFIG_RE =
/(^|\/)(\.github|\.forgejo|\.gitea|\.woodpecker)\/|(^|\/)(\.gitlab-ci\.yml|Makefile|package\.json|pyproject\.toml|setup\.cfg|tox\.ini|\.pre-commit-config\.yaml|tsconfig[^/]*\.json|\.eslintrc[^/]*|eslint\.config\.[a-z]+|(jest|vitest|playwright)\.config\.[a-z]+|\.c8rc[^/]*|\.nycrc[^/]*|codecov\.ya?ml)$/;
export function isTestPath(p) {
return TEST_PATH_RE.some((re) => re.test(p));
}
const isCheckPath = (p) => isTestPath(p) || CHECK_CONFIG_RE.test(p);
// --- the patterns -------------------------------------------------------------
const ASSERT_RE =
/\bassert\w*\b|\bexpect\s*\(|\.should\b|\bt\.(equal|is|ok|true|false|deepEqual|throws|notThrows|match)\b|\bself\.assert\w+|\brequire\.\w+\(|\bt\.(Error|Fatal)f?\(/g;
const TEST_DECL_RE =
/^\s*(test|it|describe|suite)(\.\w+)?\s*\(|^\s*(async\s+)?def\s+test_\w*|^\s*func\s+Test\w*|^\s*#\[test\]|@Test\b/;
const SKIP_RE =
/\b(test|it|describe|suite)\.(skip|todo|only)\s*\(|\bx(it|describe|test)\s*\(|\bf(it|describe)\s*\(|\{\s*(skip|todo)\s*:\s*(true|['"`])|@pytest\.mark\.(skip|skipif|xfail)\b|\bpytest\.skip\s*\(|@unittest\.skip|\bt\.Skip(Now|f)?\s*\(|#\[ignore\]|@(Disabled|Ignore)\b/;
// Inline suppressions silence a check wherever they appear.
const SUPPRESS_RE =
/@ts-ignore|@ts-nocheck|eslint-disable|#\s*type:\s*ignore|#\s*noqa\b|\/\/\s*nolint\b|#\s*pragma:\s*no\s*cover|istanbul\s+ignore|c8\s+ignore/;
// Exit-code mutes count only where a check runs: tests, scripts under tests/, CI
// and check configs. In ordinary code `|| true` is a normal shell idiom.
const MUTE_RE =
/\|\|\s*true\b|\|\|\s*:\s*($|[;)])|\|\|\s*exit\s+0\b|continue-on-error\s*:\s*true|--passWithNoTests|--pass-with-no-tests|\bset\s+\+e\b|--no-verify\b|allow_failure\s*:\s*true/;
const STUB_RE =
/throw\s+new\s+Error\s*\(\s*['"`]\s*(not\s+implemented|unimplemented|todo)\b|\braise\s+NotImplementedError\b|\b(unimplemented|todo)!\s*\(|panic\s*\(\s*"(not\s+implemented|todo)|(\/\/|#)\s*stub\b|TODO:?\s*implement/i;
// Number after a "smaller is stricter" bound: bigger = looser.
const UPPER_BOUND_RE = /(<=?|toBeLessThan(OrEqual)?|lessThan|\bmax\w*|timeout\w*|tolerance|epsilon|delta|within|retries)\W*$/i;
// Number after a "bigger is stricter" bound: smaller = looser.
const LOWER_BOUND_RE = /(>=?|toBeGreaterThan(OrEqual)?|greaterThan|\bmin\w*|threshold|coverage|branches|lines|functions|statements)\W*$/i;
const NUM_RE = /\d+(?:\.\d+)?/g;
// --- diff parsing ---------------------------------------------------------------
/**
* Parse a unified diff into files with hunks of {type: ' '|'-'|'+', text, line}.
* Returns null when non-empty input carries no diff header at all.
*/
export function parseDiff(text) {
const src = String(text ?? '');
if (src.trim() === '') return [];
if (!/^(diff --git |--- |\+\+\+ )/m.test(src)) return null;
const files = [];
let file = null;
let hunk = null;
let oldLine = 0;
let newLine = 0;
// Lines still owed to the current hunk, from its @@ header. While a hunk is
// owed lines, every line is content - even one that reads "--- x" or
// "+++ y" (a removed "-- x" / an added "++ y"). Headers only between hunks.
let oldLeft = 0;
let newLeft = 0;
const lines = src.split('\n');
for (const l of lines) {
if (hunk && (oldLeft > 0 || newLeft > 0)) {
if (l.startsWith('\\')) continue; // "\ No newline at end of file"
if (l.startsWith('-')) { hunk.push({ type: '-', text: l.slice(1), line: oldLine++ }); oldLeft--; continue; }
if (l.startsWith('+')) { hunk.push({ type: '+', text: l.slice(1), line: newLine++ }); newLeft--; continue; }
if (l.startsWith(' ') || l === '') {
hunk.push({ type: ' ', text: l.slice(1), line: newLine });
oldLine++; newLine++; oldLeft--; newLeft--;
continue;
}
// Anything else ends a malformed hunk early; fall through to headers.
oldLeft = 0; newLeft = 0;
}
if (l.startsWith('diff --git ')) {
const m = l.match(/^diff --git a\/(.+) b\/(.+)$/);
file = { path: m ? m[2] : l.slice(11), deleted: false, hunks: [] };
files.push(file);
hunk = null;
continue;
}
if (l.startsWith('--- ')) {
if (!file || file.hunks.length > 0) { file = { path: '', deleted: false, hunks: [] }; files.push(file); }
const oldPath = l.slice(4).replace(/^a\//, '');
if (!file.path && oldPath !== '/dev/null') file.path = oldPath;
hunk = null;
continue;
}
if (l.startsWith('+++ ') && file) {
const newPath = l.slice(4).replace(/^b\//, '');
if (newPath === '/dev/null') file.deleted = true;
else file.path = newPath;
continue;
}
const h = l.match(/^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@/);
if (h && file) {
oldLine = Number(h[1]);
newLine = Number(h[3]);
oldLeft = h[2] === undefined ? 1 : Number(h[2]);
newLeft = h[4] === undefined ? 1 : Number(h[4]);
hunk = [];
file.hunks.push(hunk);
}
}
return files;
}
// --- detectors ------------------------------------------------------------------
const count = (lines, re) => lines.reduce((n, l) => n + (l.text.match(new RegExp(re.source, 'g')) || []).length, 0);
const norm = (s) => s.trim().replace(/\s+/g, ' ');
function addedNotRemoved(removed, added, re) {
const before = new Set(removed.filter((l) => re.test(l.text)).map((l) => norm(l.text)));
return added.filter((l) => re.test(l.text) && !before.has(norm(l.text)));
}
/** A paired -/+ line whose only change is numbers; returns the looser numbers. */
function loosened(oldText, newText) {
const shape = (s) => norm(s).replace(NUM_RE, '#');
if (shape(oldText) !== shape(newText)) return null;
const a = [...norm(oldText).matchAll(NUM_RE)];
const b = [...norm(newText).matchAll(NUM_RE)];
for (let i = 0; i < a.length; i++) {
const before = Number(a[i][0]);
const after = Number(b[i][0]);
if (before === after) continue;
const lead = norm(newText).slice(Math.max(0, b[i].index - 30), b[i].index);
if (UPPER_BOUND_RE.test(lead) && after > before) return `${before} -> ${after}`;
if (LOWER_BOUND_RE.test(lead) && after < before) return `${before} -> ${after}`;
}
return null;
}
function finding(kind, file, l, detail) {
return { kind, file, line: l ? l.line : null, text: l ? l.text.trim() : '', ...(detail ? { detail } : {}) };
}
export function analyzeDiff(text) {
const files = parseDiff(text);
if (files === null) return { status: 'unreadable', findings: [], reason: 'the input is not a unified diff' };
const findings = [];
for (const f of files) {
const all = f.hunks.flat();
const removed = all.filter((l) => l.type === '-');
const added = all.filter((l) => l.type === '+');
const test = isTestPath(f.path);
if (test) {
const lostAsserts = count(removed, ASSERT_RE) - count(added, ASSERT_RE);
if (lostAsserts > 0) {
findings.push(finding('removed-assert', f.path, removed.find((l) => new RegExp(ASSERT_RE.source).test(l.text)), `${lostAsserts} fewer`));
}
const lostTests = removed.filter((l) => TEST_DECL_RE.test(l.text)).length - added.filter((l) => TEST_DECL_RE.test(l.text)).length;
if (lostTests > 0 || (f.deleted && removed.some((l) => TEST_DECL_RE.test(l.text)))) {
findings.push(finding('deleted-test', f.path, removed.find((l) => TEST_DECL_RE.test(l.text)), f.deleted ? 'test file deleted' : `${lostTests} fewer`));
}
for (const l of addedNotRemoved(removed, added, SKIP_RE)) findings.push(finding('skipped-test', f.path, l));
}
for (const l of addedNotRemoved(removed, added, SUPPRESS_RE)) findings.push(finding('muted-check', f.path, l));
if (isCheckPath(f.path)) {
for (const l of addedNotRemoved(removed, added, MUTE_RE)) findings.push(finding('muted-check', f.path, l));
for (const h of f.hunks) {
const minus = h.filter((l) => l.type === '-');
const plus = h.filter((l) => l.type === '+');
for (const m of minus) {
for (const p of plus) {
const d = loosened(m.text, p.text);
if (d) { findings.push(finding('loosened-threshold', f.path, p, d)); break; }
}
}
}
}
if (!test) for (const l of addedNotRemoved(removed, added, STUB_RE)) findings.push(finding('stub', f.path, l));
}
return { status: findings.length ? 'findings' : 'clean', findings };
}
// --- CLI ------------------------------------------------------------------------
const USAGE = 'usage: test-integrity.mjs (--diff <file|-> | --base <ref> [--head <ref>]) [--cwd <dir>] [--json]';
function diffFromGit(cwd, base, head) {
const ok = spawnSync('git', ['rev-parse', '--verify', '--quiet', `${base}^{commit}`], { cwd, encoding: 'utf8' });
if (ok.status !== 0) return { error: `cannot resolve --base ${base} in ${cwd}` };
if (head) {
const okHead = spawnSync('git', ['rev-parse', '--verify', '--quiet', `${head}^{commit}`], { cwd, encoding: 'utf8' });
if (okHead.status !== 0) return { error: `cannot resolve --head ${head} in ${cwd}` };
}
const args = ['diff', '--no-color', '--no-ext-diff', '--find-renames', base, ...(head ? [head] : [])];
const r = spawnSync('git', args, { cwd, encoding: 'utf8', maxBuffer: 256 * 1024 * 1024 });
if (r.status !== 0) return { error: `git diff failed: ${String(r.stderr).trim()}` };
return { text: r.stdout };
}
export function main(argv) {
let diffPath = null;
let base = null;
let head = null;
let cwd = process.cwd();
let json = false;
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === '--diff' && argv[i + 1] !== undefined) diffPath = argv[++i];
else if (a === '--base' && argv[i + 1]) base = argv[++i];
else if (a === '--head' && argv[i + 1]) head = argv[++i];
else if (a === '--cwd' && argv[i + 1]) cwd = resolve(argv[++i]);
else if (a === '--json') json = true;
else { process.stderr.write(`test-integrity: unknown argument ${a}\n${USAGE}\n`); return 2; }
}
if ((diffPath === null) === (base === null)) {
process.stderr.write(`test-integrity: give exactly one of --diff or --base\n${USAGE}\n`);
return 2;
}
let text;
if (diffPath !== null) {
try { text = readFileSync(diffPath === '-' ? 0 : diffPath, 'utf8'); }
catch (e) { return out({ status: 'unreadable', findings: [], reason: `cannot read ${diffPath}: ${e.message}` }, json); }
} else {
const g = diffFromGit(cwd, base, head);
if (g.error) return out({ status: 'unreadable', findings: [], reason: g.error }, json);
text = g.text;
}
return out(analyzeDiff(text), json);
}
function out(r, json) {
if (json) process.stdout.write(JSON.stringify(r, null, 2) + '\n');
else if (r.status === 'unreadable') process.stdout.write(`floor guard: NOT RUN - ${r.reason} (exit 2, never green)\n`);
else if (r.status === 'clean') process.stdout.write('floor guard: clean\n');
else {
process.stdout.write(`floor guard: ${r.findings.length} finding(s)\n`);
for (const f of r.findings) process.stdout.write(` ${f.kind} ${f.file}:${f.line ?? '-'} ${f.text}${f.detail ? ` (${f.detail})` : ''}\n`);
}
return r.status === 'clean' ? 0 : r.status === 'findings' ? 1 : 2;
}
function isMain() {
try { return realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url)); }
catch { return false; }
}
if (process.argv[1] && isMain()) process.exitCode = main(process.argv.slice(2));

View file

@ -168,3 +168,61 @@ test('SC5 control: the extractors report the old vocabulary', () => {
const bad = offenders(writtenStatuses(old)).join('\n'); const bad = offenders(writtenStatuses(old)).join('\n');
for (const word of ['"in-progress"', '"passed"', '"running"']) assert.ok(bad.includes(word), `control must flag ${word}:\n${bad}`); for (const word of ['"in-progress"', '"passed"', '"running"']) assert.ok(bad.includes(word), `control must flag ${word}:\n${bad}`);
}); });
// --- Handover 5, wave 1 (P8/P9): the per-step fields of the step loop ---------
// fix_rounds counts the rounds a step went through after its first control:
// 0 = passed first time, 1-3 = fix rounds, 4 = sent up to the plan, 5 = sent up
// to the operator. There is no sixth round, so 6 is invalid, not a warning.
// All new fields are optional on read: a progress.json written before them
// still resumes (SC4 above validates the old shape).
const { validateProgressObject } = progressValidator;
function withStep(extra) {
return {
schema_version: '1', plan: 'plan.md', plan_version: '1.8', mode: 'execute', status: 'in_progress',
total_steps: 1, current_step: 1,
steps: { 1: { status: 'in_progress', attempts: 1, ...extra } },
};
}
const codes = (r) => r.errors.map((e) => e.code);
test('H5: fix_rounds 0-5 is accepted, 6 is rejected', () => {
for (const n of [0, 1, 2, 3, 4, 5]) {
assert.deepEqual(codes(validateProgressObject(withStep({ fix_rounds: n }))), [], `fix_rounds ${n}`);
}
for (const bad of [6, -1, 2.5, '3']) {
assert.deepEqual(codes(validateProgressObject(withStep({ fix_rounds: bad }))), ['PROGRESS_FIX_ROUNDS_INVALID'], `fix_rounds ${JSON.stringify(bad)}`);
}
});
test('H5: start_sha, report_path, red_proof and test_lock are shape-checked when present', () => {
const ok = { start_sha: '2c89a6b580f689070638f3ff83aea82d355f312f', report_path: '.claude/projects/x/steps/step-1.report.md', red_proof: 'proven', test_lock: 'not-locked' };
assert.deepEqual(codes(validateProgressObject(withStep(ok))), []);
for (const v of ['proven', 'rejected', 'not-run']) assert.deepEqual(codes(validateProgressObject(withStep({ red_proof: v }))), [], v);
for (const v of ['locked', 'released', 'not-locked']) assert.deepEqual(codes(validateProgressObject(withStep({ test_lock: v }))), [], v);
assert.deepEqual(codes(validateProgressObject(withStep({ start_sha: 'HEAD' }))), ['PROGRESS_START_SHA_INVALID']);
assert.deepEqual(codes(validateProgressObject(withStep({ report_path: '' }))), ['PROGRESS_REPORT_PATH_INVALID']);
assert.deepEqual(codes(validateProgressObject(withStep({ red_proof: 'green' }))), ['PROGRESS_RED_PROOF_INVALID']);
assert.deepEqual(codes(validateProgressObject(withStep({ test_lock: 'yes' }))), ['PROGRESS_TEST_LOCK_INVALID']);
// null = not set yet (the schema fence's fresh step).
assert.deepEqual(codes(validateProgressObject(withStep({ start_sha: null, report_path: null, red_proof: null, test_lock: null }))), []);
});
test('H5: the run-level decision log is an array of {decision, cost_if_wrong}', () => {
const run = (decisions) => ({ ...withStep({}), decisions });
assert.deepEqual(codes(validateProgressObject(run([{ step: 2, decision: 'kept the old name', cost_if_wrong: 'one rename commit' }]))), []);
assert.deepEqual(codes(validateProgressObject(run([{ step: 2, decision: 'kept the old name' }]))), ['PROGRESS_DECISION_INVALID']);
assert.deepEqual(codes(validateProgressObject(run('none'))), ['PROGRESS_DECISION_INVALID']);
});
test('H5: the progress schema fence carries the new per-step fields and the decision log', () => {
const schema = schemaObject(read());
const step = schema.steps['1'];
for (const k of ['start_sha', 'report_path', 'red_proof', 'test_lock', 'fix_rounds']) {
assert.ok(k in step, `the schema's step record must carry ${k}`);
}
assert.equal(step.fix_rounds, 0);
assert.ok(Array.isArray(schema.decisions), 'the schema must carry the run-level decisions log');
});

View file

@ -109,11 +109,16 @@ test('trekexecute — global recovery/retry budget TREKEXECUTE_MAX_RECOVERY_ITER
test('trekexecute — every recovery/retry loop bounded + 3-axis cap hierarchy (SC b)', () => { test('trekexecute — every recovery/retry loop bounded + 3-axis cap hierarchy (SC b)', () => {
const text = read(); const text = read();
assert.match(text, /maximum 2 retries|Retry cap = 3 attempts/, 'per-step retry cap must be explicit'); // Wave 1 (P8): the per-step cap is at most 5 fix rounds, and they go UP (to the
// plan, then to the operator) - never sideways into a fourth guess, never down
// a ladder of cheaper models. The old "3 attempts" cap is gone.
assert.match(text, /maximum 5 fix rounds[^\n]*upward/i, 'per-step cap must be "maximum 5 fix rounds", escalating upward');
assert.doesNotMatch(text, /maximum 2 retries|Retry cap = 3 attempts/, 'the old 3-attempt retry cap must be gone');
assert.match(text, /recovery_depth < 2/, 'recovery-dispatch cap must be explicit'); assert.match(text, /recovery_depth < 2/, 'recovery-dispatch cap must be explicit');
const capIdx = text.indexOf('Iteration caps'); const capIdx = text.indexOf('Iteration caps');
assert.ok(capIdx >= 0, 'cap-hierarchy section ("Iteration caps") must exist'); assert.ok(capIdx >= 0, 'cap-hierarchy section ("Iteration caps") must exist');
const section = text.slice(capIdx, capIdx + 1500); const section = text.slice(capIdx, capIdx + 1500);
assert.match(section, /fix_rounds/, 'hierarchy must name the per-step fix_rounds axis');
assert.match(section, /attempts/, 'hierarchy must name per-step attempts axis'); assert.match(section, /attempts/, 'hierarchy must name per-step attempts axis');
assert.match(section, /recovery_depth/, 'hierarchy must name per-session recovery_depth axis'); assert.match(section, /recovery_depth/, 'hierarchy must name per-session recovery_depth axis');
assert.match(section, /TREKEXECUTE_MAX_RECOVERY_ITERATIONS/, 'hierarchy must name the global budget axis'); assert.match(section, /TREKEXECUTE_MAX_RECOVERY_ITERATIONS/, 'hierarchy must name the global budget axis');

View file

@ -22,7 +22,7 @@
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:961-976 | profile, phase_models, profile_source | none | | trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:961-976 | profile, phase_models, profile_source | none |
| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | commands/trekresearch.md (orchestrator-emit Stats tracking) | trekresearch.md:634-676 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none | | trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | commands/trekresearch.md (orchestrator-emit Stats tracking) | trekresearch.md:634-676 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | commands/trekplan.md (orchestrator-emit Phase 12) | trekplan.md:805-826 | profile, phase_models, parallel_agents, profile_source | none | | trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | commands/trekplan.md (orchestrator-emit Phase 12) | trekplan.md:805-826 | profile, phase_models, parallel_agents, profile_source | none |
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:1890-1912 | profile, phase_models, profile_source | none | | trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2263-2285 | profile, phase_models, profile_source | none |
| trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:78-99 | payload.profile, payload.phase_models, payload.profile_source | none | | trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:78-99 | payload.profile, payload.phase_models, payload.profile_source | none |
| trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) | | trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | commands/trekreview.md (orchestrator-emit Phase 8) | trekreview.md:255 | profile, phase_models, profile_source | none | | trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | commands/trekreview.md (orchestrator-emit Phase 8) | trekreview.md:255 | profile, phase_models, profile_source | none |

View file

@ -0,0 +1,22 @@
diff --git a/lib/sum.mjs b/lib/sum.mjs
index 1111111..2222222 100644
--- a/lib/sum.mjs
+++ b/lib/sum.mjs
@@ -1,3 +1,4 @@
export function sum(a, b) {
- return a - b;
+ // fixed: addition, not subtraction
+ return a + b;
}
diff --git a/tests/sum.test.mjs b/tests/sum.test.mjs
index 3333333..4444444 100644
--- a/tests/sum.test.mjs
+++ b/tests/sum.test.mjs
@@ -3,3 +3,7 @@ import { sum } from '../lib/sum.mjs';
test('sum adds', () => {
assert.equal(sum(1, 2), 3);
});
+
+test('sum adds negatives', () => {
+ assert.equal(sum(-1, -2), -3);
+});

View file

@ -0,0 +1,11 @@
diff --git a/tests/sum.test.mjs b/tests/sum.test.mjs
index 3333333..4444444 100644
--- a/tests/sum.test.mjs
+++ b/tests/sum.test.mjs
@@ -3,6 +3,3 @@ import { sum } from '../lib/sum.mjs';
test('sum adds', () => {
assert.equal(sum(1, 2), 3);
});
-test('sum adds negatives', () => {
- sum(-1, -2);
-});

View file

@ -0,0 +1,9 @@
diff --git a/tests/latency.test.mjs b/tests/latency.test.mjs
index 7777777..8888888 100644
--- a/tests/latency.test.mjs
+++ b/tests/latency.test.mjs
@@ -4,3 +4,3 @@ test('tail latency', () => {
const ms = measure();
- assert.ok(ms < 200, `tail latency ${ms}ms`);
+ assert.ok(ms < 2000, `tail latency ${ms}ms`);
});

View file

@ -0,0 +1,8 @@
diff --git a/tests/run.sh b/tests/run.sh
index 5555555..6666666 100644
--- a/tests/run.sh
+++ b/tests/run.sh
@@ -1,2 +1,2 @@
#!/bin/sh
-node --test tests/
+node --test tests/ || true

View file

@ -0,0 +1,9 @@
diff --git a/tests/sum.test.mjs b/tests/sum.test.mjs
index 3333333..4444444 100644
--- a/tests/sum.test.mjs
+++ b/tests/sum.test.mjs
@@ -3,5 +3,4 @@ import { sum } from '../lib/sum.mjs';
test('sum adds', () => {
assert.equal(sum(1, 2), 3);
- assert.equal(sum(2, 2), 4);
});

View file

@ -0,0 +1,9 @@
diff --git a/tests/sum.test.mjs b/tests/sum.test.mjs
index 3333333..4444444 100644
--- a/tests/sum.test.mjs
+++ b/tests/sum.test.mjs
@@ -3,3 +3,3 @@ import { sum } from '../lib/sum.mjs';
-test('sum adds', () => {
+test.skip('sum adds', () => {
assert.equal(sum(1, 2), 3);
});

View file

@ -0,0 +1,9 @@
diff --git a/lib/sum.mjs b/lib/sum.mjs
index 1111111..2222222 100644
--- a/lib/sum.mjs
+++ b/lib/sum.mjs
@@ -1,3 +1,3 @@
export function sum(a, b) {
- return a - b;
+ throw new Error('not implemented');
}

View file

@ -0,0 +1,9 @@
diff --git a/tests/latency.test.mjs b/tests/latency.test.mjs
index 7777777..8888888 100644
--- a/tests/latency.test.mjs
+++ b/tests/latency.test.mjs
@@ -4,3 +4,3 @@ test('tail latency', () => {
const ms = measure();
- assert.ok(ms < 2000, `tail latency ${ms}ms`);
+ assert.ok(ms < 200, `tail latency ${ms}ms`);
});

View file

@ -0,0 +1,2 @@
this is not a diff
just prose that a broken pipe left behind

View file

@ -1667,9 +1667,11 @@ for (const cmd of ['trekresearch', 'trekplan', 'trekreview', 'trekexecute']) {
// The derived set is asserted exactly, so the pin cannot go vacuous if a // The derived set is asserted exactly, so the pin cannot go vacuous if a
// command stops spawning or a new spawning command is added. // command stops spawning or a new spawning command is added.
const SPAWNING_COMMANDS = ['trekbrief', 'trekplan', 'trekresearch', 'trekreview']; // Wave 1 (P8): /trekexecute joined the set - one fresh step helper per plan step
// plus one controller - so the no-name rule now covers five commands.
const SPAWNING_COMMANDS = ['trekbrief', 'trekexecute', 'trekplan', 'trekresearch', 'trekreview'];
test('S85: the set of agent-spawning commands is exactly the four that carry the no-name rule', () => { test('S85: the set of agent-spawning commands is exactly the five that carry the no-name rule', () => {
const spawns = listMd('commands') const spawns = listMd('commands')
.filter((f) => /^Launch\b/m.test(read(`commands/${f}`))) .filter((f) => /^Launch\b/m.test(read(`commands/${f}`)))
.map((f) => f.replace(/\.md$/, '')) .map((f) => f.replace(/\.md$/, ''))

View file

@ -0,0 +1,88 @@
// tests/lib/done-line.test.mjs
// The done line (P9 (4)): a step's "done" is written by a script, and only when
// the test command exits 0. Exit 0 = written, 1 = the command failed (nothing
// written), 2 = could not run (nothing written, never green).
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const CLI = join(ROOT, 'lib', 'verification', 'done-line.mjs');
function sh(cwd, cmd, args) {
const r = spawnSync(cmd, args, { cwd, encoding: 'utf8' });
assert.equal(r.status, 0, `${cmd} ${args.join(' ')}: ${r.stderr}`);
return r.stdout.trim();
}
function repo() {
const dir = mkdtempSync(join(tmpdir(), 'done-line-'));
sh(dir, 'git', ['init', '-q']);
sh(dir, 'git', ['config', 'user.email', 't@example.invalid']);
sh(dir, 'git', ['config', 'user.name', 'test']);
mkdirSync(join(dir, 'tests'));
writeFileSync(join(dir, 'tests', 'ok.test.mjs'), "import { test } from 'node:test';\ntest('ok', () => {});\n");
writeFileSync(join(dir, 'tests', 'bad.test.mjs'), "import { test } from 'node:test';\ntest('bad', () => { throw new Error('red'); });\n");
sh(dir, 'git', ['add', 'tests']);
sh(dir, 'git', ['commit', '-q', '-m', 'tests']);
return dir;
}
const run = (args) => spawnSync(process.execPath, [CLI, ...args], { encoding: 'utf8' });
test('done line: the test command exits 0 -> one DONE line appended, exit 0', () => {
const dir = repo();
const report = join(dir, 'step-3.report.md');
try {
writeFileSync(report, '# Step 3 report\n');
const r = run(['--cwd', dir, '--step', '3', '--cmd', 'node --test tests/ok.test.mjs', '--report', report]);
assert.equal(r.status, 0, r.stdout + r.stderr);
const head = sh(dir, 'git', ['rev-parse', 'HEAD']);
const lines = readFileSync(report, 'utf8').split('\n').filter((l) => l.startsWith('DONE '));
assert.equal(lines.length, 1);
assert.match(lines[0], new RegExp(`^DONE step 3: \`node --test tests/ok\\.test\\.mjs\` exit 0 at ${head} \\d{4}-\\d{2}-\\d{2}T`));
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('done line: the test command fails -> nothing written, exit 1', () => {
const dir = repo();
const report = join(dir, 'step-3.report.md');
try {
writeFileSync(report, '# Step 3 report\n');
const r = run(['--cwd', dir, '--step', '3', '--cmd', 'node --test tests/bad.test.mjs', '--report', report]);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.equal(readFileSync(report, 'utf8'), '# Step 3 report\n');
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('done line: a command outside the test-runner allowlist is not run -> exit 2, nothing written', () => {
const dir = repo();
const report = join(dir, 'step-3.report.md');
try {
for (const cmd of ['echo done', 'node --test tests/bad.test.mjs || true', 'true']) {
const r = run(['--cwd', dir, '--step', '3', '--cmd', cmd, '--report', report]);
assert.equal(r.status, 2, `${cmd}: ${r.stdout}${r.stderr}`);
}
assert.equal(existsSync(report), false);
assert.equal(run(['--cwd', dir, '--cmd', 'node --test tests/ok.test.mjs', '--report', report]).status, 2, 'no --step');
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('done line: --check finds the line for that step only', () => {
const dir = repo();
const report = join(dir, 'step-3.report.md');
try {
assert.equal(run(['--check', '--step', '3', '--report', report]).status, 1, 'no report file = not done');
writeFileSync(report, '# Step 3 report\nDONE step 3 (typed by hand)\n');
assert.equal(run(['--check', '--step', '3', '--report', report]).status, 1, 'a hand-typed line is not the script line');
run(['--cwd', dir, '--step', '3', '--cmd', 'node --test tests/ok.test.mjs', '--report', report]);
assert.equal(run(['--check', '--step', '3', '--report', report]).status, 0);
assert.equal(run(['--check', '--step', '4', '--report', report]).status, 1);
} finally { rmSync(dir, { recursive: true, force: true }); }
});

View file

@ -0,0 +1,199 @@
// tests/lib/executor-prose-pins.test.mjs
// Wave 1, process 1 (the executor): prose pins for P8 (fresh helper per step,
// one controller with two verdicts, fix rounds that go up), P9 on the executor
// side (proof of red, the floor guard, the done line, end to end, back to the
// architecture after three failed fixes) and P4 (the closed stop list and the
// decision log). The mechanisms themselves are tested as code in
// tests/lib/{red-proof,test-integrity,done-line}.test.mjs; these pins hold the
// command prose to them. Fix the SOURCE a pin guards, not the pin.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { existsSync, readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const read = (rel) => readFileSync(join(ROOT, rel), 'utf8');
const CMD = 'commands/trekexecute.md';
const AGENT = 'agents/step-implementer.md';
function frontmatter(text) {
const m = text.match(/^---\n([\s\S]*?)\n---/);
assert.ok(m, 'frontmatter missing');
return m[1];
}
const fmValue = (fm, key) => (fm.match(new RegExp(`^${key}:\\s*(.+)$`, 'm')) || [])[1];
/** The text between `## {start}` (or `#### {start}`) and the next heading of the same or higher level. */
function section(text, heading) {
const at = text.indexOf(heading);
assert.ok(at >= 0, `${CMD} must have "${heading}"`);
const level = heading.match(/^#+/)[0].length;
const rest = text.slice(at + heading.length);
const next = rest.search(new RegExp(`\\n#{1,${level}} `));
return heading + (next >= 0 ? rest.slice(0, next) : rest);
}
// --- P8: the Agent tool is back, for one purpose -------------------------------
test('P8: /trekexecute may spawn - Agent is allowed and no longer disallowed', () => {
const fm = frontmatter(read(CMD));
assert.match(fmValue(fm, 'allowed-tools'), /\bAgent\b/, 'allowed-tools must list Agent');
const disallowed = fmValue(fm, 'disallowed-tools') || '';
assert.doesNotMatch(disallowed, /\bAgent\b/, 'disallowed-tools must no longer remove Agent');
assert.match(disallowed, /\bTeamCreate\b/, 'TeamCreate stays removed - helpers, never a team');
});
test('P8: Hard rule 10 is rewritten - one fresh helper per plan step, and helpers never start helpers', () => {
const text = read(CMD);
assert.doesNotMatch(text, /\*\*No sub-agents\.\*\*/, 'the old "No sub-agents" rule must be gone');
const rule = text.slice(text.indexOf('10. **'), text.indexOf('11. **'));
assert.match(rule, /fresh helper per plan step/i);
assert.match(rule, /voyage:step-implementer/);
assert.match(rule, /helpers never start helpers/i);
assert.match(rule, /controller/i);
});
test('P8: the step agent exists, is opus at effort >= high, and cannot spawn', () => {
assert.ok(existsSync(join(ROOT, AGENT)), `${AGENT} must exist`);
const fm = frontmatter(read(AGENT));
assert.equal(fmValue(fm, 'name'), 'step-implementer');
assert.equal(fmValue(fm, 'model'), 'opus');
assert.ok(['high', 'xhigh', 'max'].includes(fmValue(fm, 'effort')), 'effort must be >= high');
assert.doesNotMatch(fmValue(fm, 'tools') || '', /\bAgent\b/, 'a helper must not hold the Agent tool');
// Measured 2026-10-06 (CC 2.1.291): a worktree-isolated helper branched from
// the default branch (02f320f), not from the executor's HEAD (2c89a6b), and so
// did not see the earlier steps. A plugin cannot set worktree.baseRef. So no
// isolation: the helper works in the executor's tree, from a recorded start_sha.
assert.doesNotMatch(fm, /^isolation:/m, 'no isolation: worktree on the step agent (measured: wrong base)');
});
test('P8: the helper report contract is ran / saw / did NOT check', () => {
const body = read(AGENT);
for (const h of ['### Ran', '### Saw', '### Did NOT check']) assert.ok(body.includes(h), `${AGENT} report must have "${h}"`);
assert.ok(read(CMD).includes('Did NOT check'), 'the command must name the report contract too');
});
test('P8: hand-over is in files, from a recorded start point', () => {
const text = read(CMD);
assert.match(text, /start_sha/);
assert.match(text, /report_path/);
assert.match(text, /git diff \{start_sha\}/, 'the review package is the diff from the recorded start point');
});
test('P8: one controller, two verdicts, reads the diff not the summary, re-checks only what was fixed', () => {
const ctl = section(read(CMD), '#### Sub-step E — Control');
assert.match(ctl, /one controller/i);
assert.match(ctl, /two verdicts/i);
assert.match(ctl, /"spec"/);
assert.match(ctl, /"quality"/);
assert.match(ctl, /reads the diff, not the helper's (summary|report)/i);
assert.match(ctl, /only what (was|the round) fixed/i);
assert.match(ctl, /general-purpose/, 'the controller is a general helper with a prompt from the command, not agent no. 2');
});
test('P8: fix rounds go up - maximum 5, plan then operator, never a cheaper-model ladder', () => {
const fix = section(read(CMD), '#### Sub-step F — Fix rounds');
assert.match(fix, /maximum 5 fix rounds[^\n]*upward/i);
assert.match(fix, /back to (the )?planning/i);
assert.match(fix, /operator/i);
assert.match(fix, /does not improvise/i);
assert.match(fix, /no ladder of cheaper models/i);
// P9 (7): after three failed fixes the question is the architecture, not a fourth fix.
assert.match(fix, /three failed fix(es| rounds)[^\n]*architecture/i);
});
test('P8: the no-name rule covers the step helper and the controller', () => {
const text = read(CMD);
assert.ok(text.includes('in_process_teammate'));
assert.match(text, /never pass .*`name`/i);
assert.match(text, /^Launch\b/m, 'the spawn site starts with "Launch" (the S85 detector reads it)');
});
// --- P9 (executor side) ---------------------------------------------------------
test('P9 (1): proof of red is a command and it blocks - the old "do not block" sentence is gone', () => {
const text = read(CMD);
assert.doesNotMatch(text, /Do not block on test-first failures/);
assert.match(text, /lib\/verification\/red-proof\.mjs/);
const red = section(text, '#### Sub-step B — Test first and proof of red');
assert.match(red, /must FAIL/);
assert.match(red, /must PASS/);
assert.match(red, /not-run/);
assert.match(red, /never (a pass|green|proven)/i);
});
test('K4: Test first is required from plan_version 1.8; Interface in/out and Review focus are read, not required', () => {
const text = read(CMD);
assert.match(text, /plan_version ≥ 1\.8[^\n]*`Test first:`|`Test first:`[^\n]*plan_version ≥ 1\.8/);
for (const f of ['Interface in:', 'Interface out:', 'Review focus:']) assert.ok(text.includes(f), `${CMD} must read ${f}`);
assert.match(text, /tolerat(e|es) (their|its) absence|never required/i);
});
test('K8: the test lock is used through its CLI, and a missing CLI is said out loud', () => {
const text = read(CMD);
assert.match(text, /lib\/util\/test-lock\.mjs lock --files/);
assert.match(text, /test-lock\.mjs unlock/);
assert.match(text, /"not-locked"/);
assert.match(text, /never silent/i);
});
test('P9 (2): the floor guard runs on the step diff and exit 2 is never green', () => {
const text = read(CMD);
assert.match(text, /lib\/verification\/test-integrity\.mjs --base \{start_sha\}/);
assert.match(text, /exit 2[^\n]*never (green|a pass)/i);
});
test('P9 (4): the done line is written by a script, only when the test command exits 0', () => {
const text = read(CMD);
assert.match(text, /lib\/verification\/done-line\.mjs/);
assert.match(text, /done-line\.mjs --check/);
assert.match(text, /only (when|if) the test command exits 0/i);
});
test('P9 (6): without a run-the-thing check the result is "not verified end to end", never green', () => {
const text = read(CMD);
assert.match(text, /not verified end to end/);
assert.match(text, /end_to_end/);
});
test('P9: the three new CLIs the prose names exist', () => {
for (const f of ['red-proof.mjs', 'test-integrity.mjs', 'done-line.mjs']) {
assert.ok(existsSync(join(ROOT, 'lib', 'verification', f)), `lib/verification/${f}`);
}
});
// --- P4: the closed stop list and the decision log ------------------------------
test('P4: the run stops only on the closed list of four, and the final report shows it', () => {
const text = read(CMD);
const list = section(text, '### Closed stop list');
for (const cls of [/cannot be undone/i, /security-sensitive/i, /outside the working copy/i, /so broken that (every|any)thing (after it )?is guesswork/i]) {
assert.match(list, cls);
}
assert.match(list, /closed/i);
// Phase 8's report template is a fence that holds its own "## " heading, so
// the phase is read up to Phase 9, not up to the next heading.
const report = text.slice(text.indexOf('## Phase 8 — Final report'), text.indexOf('## Phase 9'));
assert.match(report, /### Stops/);
assert.match(report, /### Decisions taken/);
});
test('P4: every decision the executor took is logged with what it costs if it is wrong', () => {
const text = read(CMD);
assert.match(text, /cost_if_wrong/);
assert.match(text, /"decisions"/);
});
// --- Credits: every borrowing is named, never silent ----------------------------
test('every borrowing in the executor is credited', () => {
const text = read(CMD);
const credits = section(text, '## Credits');
for (const src of [/Superpowers/, /subagent-driven-development/, /ai-native-sdlc/, /Kiro/, /Spec Kit/, /agent-skills/, /Cherny/]) {
assert.match(credits, src);
}
assert.match(credits, /MIT/);
});

View file

@ -0,0 +1,200 @@
// tests/lib/red-proof.test.mjs
// Proof of red (P9 (1)): the step's test is run by a command against the start
// point, where it must FAIL, and against the result, where it must PASS. The
// model's word that "the test was red first" is never the proof.
// Exit 0 = proven, 1 = rejected (green at the start point, or red at the result),
// 2 = could not run (never green).
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { proveRed } from '../../lib/verification/red-proof.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const CLI = join(ROOT, 'lib', 'verification', 'red-proof.mjs');
const TEST_CMD = 'node --test tests/sum.test.mjs';
const SUM_TEST = [
"import { test } from 'node:test';",
"import { strict as assert } from 'node:assert';",
"import { sum } from '../lib/sum.mjs';",
"test('sum adds', () => { assert.equal(sum(1, 2), 3); });",
'',
].join('\n');
function sh(cwd, cmd, args) {
const r = spawnSync(cmd, args, { cwd, encoding: 'utf8' });
assert.equal(r.status, 0, `${cmd} ${args.join(' ')}: ${r.stderr}`);
return r.stdout.trim();
}
/** A repo whose base commit has `body` as sum(); returns {dir, base}. */
function repo(body) {
const dir = mkdtempSync(join(tmpdir(), 'red-proof-'));
sh(dir, 'git', ['init', '-q']);
sh(dir, 'git', ['config', 'user.email', 't@example.invalid']);
sh(dir, 'git', ['config', 'user.name', 'test']);
mkdirSync(join(dir, 'lib'));
writeFileSync(join(dir, 'lib', 'sum.mjs'), `export function sum(a, b) { return ${body}; }\n`);
sh(dir, 'git', ['add', 'lib/sum.mjs']);
sh(dir, 'git', ['commit', '-q', '-m', 'base']);
return { dir, base: sh(dir, 'git', ['rev-parse', 'HEAD']) };
}
function addTest(dir) {
mkdirSync(join(dir, 'tests'), { recursive: true });
writeFileSync(join(dir, 'tests', 'sum.test.mjs'), SUM_TEST);
}
function cli(args) {
const r = spawnSync(process.execPath, [CLI, ...args], { encoding: 'utf8' });
return { status: r.status, stdout: r.stdout, stderr: r.stderr };
}
test('red proof: red at the start point, green at the result -> proven, exit 0', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
writeFileSync(join(dir, 'lib', 'sum.mjs'), 'export function sum(a, b) { return a + b; }\n');
const r = proveRed({ cwd: dir, base, cmd: TEST_CMD, tests: ['tests/sum.test.mjs'] });
assert.equal(r.status, 'proven', JSON.stringify(r));
assert.notEqual(r.base.exit, 0);
assert.equal(r.result.exit, 0);
const c = cli(['--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/sum.test.mjs', '--json']);
assert.equal(c.status, 0, c.stdout + c.stderr);
assert.equal(JSON.parse(c.stdout).status, 'proven');
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof: a test that already PASSES at the start point is rejected, exit 1', () => {
const { dir, base } = repo('a + b');
try {
addTest(dir);
const r = proveRed({ cwd: dir, base, cmd: TEST_CMD, tests: ['tests/sum.test.mjs'] });
assert.equal(r.status, 'rejected');
assert.equal(r.reason, 'not-red-at-base');
const c = cli(['--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/sum.test.mjs']);
assert.equal(c.status, 1, c.stdout + c.stderr);
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof: a test that still FAILS at the result is rejected, exit 1', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
const r = proveRed({ cwd: dir, base, cmd: TEST_CMD, tests: ['tests/sum.test.mjs'] });
assert.equal(r.status, 'rejected');
assert.equal(r.reason, 'not-green-at-result');
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof --red-only: the gate before building - red at the start point only, the result is not run', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir); // test written, nothing built yet: the working tree is still red
const r = proveRed({ cwd: dir, base, cmd: TEST_CMD, tests: ['tests/sum.test.mjs'], redOnly: true });
assert.equal(r.status, 'proven', JSON.stringify(r));
assert.equal(r.result, undefined, '--red-only never runs the result');
const c = cli(['--red-only', '--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/sum.test.mjs']);
assert.equal(c.status, 0, c.stdout + c.stderr);
} finally { rmSync(dir, { recursive: true, force: true }); }
const green = repo('a + b');
try {
addTest(green.dir);
const r = proveRed({ cwd: green.dir, base: green.base, cmd: TEST_CMD, tests: ['tests/sum.test.mjs'], redOnly: true });
assert.equal(r.reason, 'not-red-at-base');
} finally { rmSync(green.dir, { recursive: true, force: true }); }
});
test('red proof: what cannot be measured is "not-run", exit 2 - never proven', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
writeFileSync(join(dir, 'lib', 'sum.mjs'), 'export function sum(a, b) { return a + b; }\n');
const args = (over) => ['--cwd', dir, '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/sum.test.mjs', ...over];
// A command outside the test-runner allowlist never reaches a shell.
assert.equal(proveRed({ cwd: dir, base, cmd: 'node tests/sum.test.mjs', tests: ['tests/sum.test.mjs'] }).status, 'not-run');
assert.equal(proveRed({ cwd: dir, base, cmd: `${TEST_CMD} || true`, tests: ['tests/sum.test.mjs'] }).status, 'not-run');
// An unknown start point, a missing test file, no test file at all.
assert.equal(proveRed({ cwd: dir, base: 'no-such-ref', cmd: TEST_CMD, tests: ['tests/sum.test.mjs'] }).status, 'not-run');
assert.equal(proveRed({ cwd: dir, base, cmd: TEST_CMD, tests: ['tests/missing.test.mjs'] }).status, 'not-run');
assert.equal(proveRed({ cwd: dir, base, cmd: TEST_CMD, tests: [] }).status, 'not-run');
assert.equal(cli(args([])).status, 0, 'control: the same arguments do prove');
assert.equal(cli(['--cwd', dir, '--base', base, '--cmd', TEST_CMD]).status, 2, 'no --tests');
assert.equal(cli(['--cwd', join(dir, 'nope'), '--base', base, '--cmd', TEST_CMD, '--tests', 'tests/sum.test.mjs']).status, 2);
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof: --cwd in a subdirectory still runs at the repo root (plan paths are root-relative)', () => {
// Measured 2026-10-06: `git archive` run from a subdirectory packs only that
// subdirectory, so a start point built from a subdir cwd would miss lib/.
const { dir, base } = repo('a - b');
try {
addTest(dir);
writeFileSync(join(dir, 'lib', 'sum.mjs'), 'export function sum(a, b) { return a + b; }\n');
mkdirSync(join(dir, 'docs'));
const r = proveRed({ cwd: join(dir, 'docs'), base, cmd: TEST_CMD, tests: ['tests/sum.test.mjs'] });
assert.equal(r.status, 'proven', JSON.stringify({ status: r.status, reason: r.reason }));
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('red proof: the start-point run leaves the working tree untouched', () => {
const { dir, base } = repo('a - b');
try {
addTest(dir);
writeFileSync(join(dir, 'lib', 'sum.mjs'), 'export function sum(a, b) { return a + b; }\n');
const before = sh(dir, 'git', ['status', '--porcelain']);
proveRed({ cwd: dir, base, cmd: TEST_CMD, tests: ['tests/sum.test.mjs'] });
assert.equal(sh(dir, 'git', ['status', '--porcelain']), before);
assert.equal(readFileSync(join(dir, 'lib', 'sum.mjs'), 'utf8'), 'export function sum(a, b) { return a + b; }\n');
} finally { rmSync(dir, { recursive: true, force: true }); }
});
// Reuse of tests/fixtures/red-first/02243c6-always-allow-shim.mjs: the measured
// red-first claim of 02243c6 ("6 of 55"), re-run through red-proof instead of by
// hand. Needs this repo's git history; a `git archive` tree has none, and there
// the test says so and skips rather than pretending to pass.
const HAS_HISTORY = spawnSync('git', ['cat-file', '-e', '02243c6^{commit}'], { cwd: ROOT }).status === 0;
test('red proof: reproduces the 02243c6 red-first measurement (6 of 55) and the bare load error (1 of 1)', { skip: !HAS_HISTORY && 'no git history (git archive tree)' }, () => {
const dir = mkdtempSync(join(tmpdir(), 'red-proof-02243c6-'));
const RUNNER = 'lib/verification/criteria-runner.mjs';
const TEST_FILE = 'tests/lib/criteria-runner.test.mjs';
const tar = join(dir, '..', `${dir.split('/').pop()}.tar`);
try {
sh(ROOT, 'git', ['archive', '--format=tar', '-o', tar, '02243c6']);
sh(dir, 'tar', ['-xf', tar]);
const parent = spawnSync('git', ['show', `02243c6^:${RUNNER}`], { cwd: ROOT, encoding: 'utf8' }).stdout;
const real = readFileSync(join(dir, RUNNER), 'utf8');
const shim = readFileSync(join(ROOT, 'tests', 'fixtures', 'red-first', '02243c6-always-allow-shim.mjs'), 'utf8');
sh(dir, 'git', ['init', '-q']);
sh(dir, 'git', ['config', 'user.email', 't@example.invalid']);
sh(dir, 'git', ['config', 'user.name', 'test']);
sh(dir, 'git', ['add', '.']);
// Start point 1: the parent's runner as it was - the test file cannot even load.
writeFileSync(join(dir, RUNNER), parent);
sh(dir, 'git', ['commit', '-q', '-am', 'parent runner']);
const bare = sh(dir, 'git', ['rev-parse', 'HEAD']);
// Start point 2: the parent's runner plus the always-allow shim.
writeFileSync(join(dir, RUNNER), parent + shim);
sh(dir, 'git', ['commit', '-q', '-am', 'parent runner + shim']);
const shimmed = sh(dir, 'git', ['rev-parse', 'HEAD']);
writeFileSync(join(dir, RUNNER), real);
const cmd = `node --test --test-reporter=tap ${TEST_FILE}`;
const withShim = proveRed({ cwd: dir, base: shimmed, cmd, tests: [TEST_FILE] });
assert.equal(withShim.status, 'proven', JSON.stringify(withShim));
assert.deepEqual(withShim.base.tap, { tests: 55, fail: 6 });
assert.deepEqual(withShim.result.tap, { tests: 55, fail: 0 });
const bareRun = proveRed({ cwd: dir, base: bare, cmd, tests: [TEST_FILE] });
assert.equal(bareRun.status, 'proven');
assert.deepEqual(bareRun.base.tap, { tests: 1, fail: 1 }, 'the parent tree fails as ONE load error, not as named tests');
} finally {
rmSync(dir, { recursive: true, force: true });
rmSync(tar, { force: true });
}
});

View file

@ -0,0 +1,126 @@
// tests/lib/test-integrity.test.mjs
// The floor guard (P9 (2)): a diff that weakens the tests is a finding, never a
// pass. Exit 0 = clean, 1 = findings, 2 = could not run (never green).
// Borrowed: agent-skills "floor guard" (MIT) - see lib/verification/test-integrity.mjs.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { analyzeDiff, isTestPath } from '../../lib/verification/test-integrity.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const CLI = join(ROOT, 'lib', 'verification', 'test-integrity.mjs');
const FIX = join(ROOT, 'tests', 'fixtures', 'test-integrity');
const fixture = (name) => readFileSync(join(FIX, name), 'utf8');
function cli(args, opts = {}) {
const r = spawnSync(process.execPath, [CLI, ...args], { encoding: 'utf8', ...opts });
return { status: r.status, stdout: r.stdout, stderr: r.stderr };
}
const kinds = (r) => r.findings.map((f) => f.kind);
test('floor guard: a clean diff (fix + a new test) is clean -> exit 0', () => {
const r = analyzeDiff(fixture('clean.diff'));
assert.equal(r.status, 'clean', JSON.stringify(r.findings));
assert.deepEqual(r.findings, []);
assert.equal(cli(['--diff', join(FIX, 'clean.diff')]).status, 0);
});
test('floor guard: a diff that removes an assert -> finding removed-assert, exit 1', () => {
const r = analyzeDiff(fixture('removed-assert.diff'));
assert.equal(r.status, 'findings');
assert.deepEqual(kinds(r), ['removed-assert']);
assert.equal(r.findings[0].file, 'tests/sum.test.mjs');
assert.equal(cli(['--diff', join(FIX, 'removed-assert.diff')]).status, 1);
});
test('floor guard: a diff that skips a test -> finding skipped-test, exit 1', () => {
const r = analyzeDiff(fixture('skipped-test.diff'));
assert.deepEqual(kinds(r), ['skipped-test']);
assert.equal(cli(['--diff', join(FIX, 'skipped-test.diff')]).status, 1);
});
test('floor guard: a deleted test, a muted check, a stub and a loosened threshold are each found', () => {
assert.deepEqual(kinds(analyzeDiff(fixture('deleted-test.diff'))), ['deleted-test']);
assert.deepEqual(kinds(analyzeDiff(fixture('muted-check.diff'))), ['muted-check']);
assert.deepEqual(kinds(analyzeDiff(fixture('stub.diff'))), ['stub']);
assert.deepEqual(kinds(analyzeDiff(fixture('loosened-threshold.diff'))), ['loosened-threshold']);
});
test('floor guard: a TIGHTENED threshold is not a finding (direction is read, not just change)', () => {
assert.deepEqual(analyzeDiff(fixture('tightened-threshold.diff')).findings, []);
});
test('floor guard: an unreadable diff -> status unreadable, exit 2 (never green)', () => {
assert.equal(analyzeDiff(fixture('unreadable.diff')).status, 'unreadable');
const r = cli(['--diff', join(FIX, 'unreadable.diff')]);
assert.equal(r.status, 2, r.stdout + r.stderr);
assert.equal(cli(['--diff', join(FIX, 'does-not-exist.diff')]).status, 2);
assert.equal(cli([]).status, 2, 'no input is "could not run", not clean');
});
test('floor guard: hunk line counts decide what is content - a removed "-- x" line is not a file header', () => {
const diff = [
'diff --git a/tests/a.test.mjs b/tests/a.test.mjs',
'--- a/tests/a.test.mjs',
'+++ b/tests/a.test.mjs',
'@@ -1,4 +1,3 @@',
" test('a', () => {",
'--- an old comment',
'+++ counter',
'- assert.equal(1, 1);',
' });',
'\\ No newline at end of file',
].join('\n');
const r = analyzeDiff(diff);
assert.deepEqual(r.findings.map((f) => [f.kind, f.file]), [['removed-assert', 'tests/a.test.mjs']]);
});
test('floor guard: an empty diff is clean (nothing changed weakens nothing)', () => {
assert.equal(analyzeDiff('').status, 'clean');
});
test('floor guard: --json prints the findings with file and line', () => {
const r = cli(['--diff', join(FIX, 'removed-assert.diff'), '--json']);
const out = JSON.parse(r.stdout);
assert.equal(out.status, 'findings');
assert.equal(out.findings[0].kind, 'removed-assert');
assert.equal(out.findings[0].file, 'tests/sum.test.mjs');
assert.equal(typeof out.findings[0].line, 'number');
});
test('floor guard: --base reads the diff from git in --cwd', () => {
const dir = mkdtempSync(join(tmpdir(), 'floor-guard-'));
const git = (...a) => {
const r = spawnSync('git', a, { cwd: dir, encoding: 'utf8' });
assert.equal(r.status, 0, `git ${a.join(' ')}: ${r.stderr}`);
return r.stdout.trim();
};
try {
git('init', '-q');
git('config', 'user.email', 't@example.invalid');
git('config', 'user.name', 'test');
writeFileSync(join(dir, 'a.test.mjs'), "test('a', () => {\n assert.equal(1, 1);\n assert.equal(2, 2);\n});\n");
git('add', 'a.test.mjs');
git('commit', '-q', '-m', 'base');
const base = git('rev-parse', 'HEAD');
writeFileSync(join(dir, 'a.test.mjs'), "test('a', () => {\n assert.equal(1, 1);\n});\n");
const r = cli(['--base', base, '--cwd', dir, '--json']);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.deepEqual(JSON.parse(r.stdout).findings.map((f) => f.kind), ['removed-assert']);
assert.equal(cli(['--base', 'no-such-ref', '--cwd', dir]).status, 2, 'an unresolvable base is "could not run"');
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('floor guard: test-path recognition covers the common layouts', () => {
for (const p of ['tests/a.mjs', 'test/a.py', 'src/__tests__/a.ts', 'a.test.js', 'pkg/a_test.go', 'app/test_a.py', 'spec/a_spec.rb', 'a.spec.ts']) {
assert.ok(isTestPath(p), p);
}
for (const p of ['lib/a.mjs', 'src/contest.ts', 'README.md']) assert.ok(!isTestPath(p), p);
});