fix(execute): the helper and the executor give red proof no list
Operator decisions 30 and 31 (14.10); review findings 11ee78c5 and d01df508. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
73e53f3134
commit
a70f082f6e
SSH key fingerprint:
SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
6 changed files with 224 additions and 43 deletions
|
|
@ -1451,7 +1451,7 @@ is not proven red at its start point and green at its result does not pass.
|
|||
skill is the advice; the gate below, `red-proof.mjs`, is the guarantee, and it
|
||||
stays a command whatever the skill says.
|
||||
|
||||
**The step's test files** are every file on the `Test first:` `File:` line: the backticked paths that stand right after `File:`, separated only by commas or spaces; a path in the description after them is not a test file (every plan_version; operator decision 20, 2026-10-12). `red-proof.mjs --plan --step` reads the same list from the plan and refuses a `--tests` that differs (exit 2, `tests-mismatch`); on that answer the executor runs it once more with the list it printed (`plan_tests`), and a second exit 2 is read as the exit-2 row below says. The lock's `--files` is the list red proof accepted. The
|
||||
**The step's test files** are every file on the `Test first:` `File:` line: the backticked paths that stand right after `File:`, separated only by commas or spaces, and below plan_version 1.9 also each further path after a note in brackets such as `(new)` or after `and`, `plus` or `+`, as long as it is clearly a test (the first path that is not ends the list; operator decision 31); a path in the description after them is not a test file (every plan_version; operator decision 20, 2026-10-12). `red-proof.mjs --plan --step` reads that list from the plan, so neither the helper nor the executor passes `--tests` (operator decision 30, 2026-10-14); it prints the list it ran (`tests`), refuses a `--tests` that differs (exit 2, `tests-mismatch`, the plan's list in `plan_tests`) and a plan or step it cannot read (exit 2, `plan-unreadable`). The lock's `--files` is the list red proof printed as `tests`. The
|
||||
plan template lists them apart from `Files:`, so everywhere below **the step's
|
||||
scope = its `Files:` + its test files**: both are built, both are committed at
|
||||
the checkpoint, and neither is a scope finding.
|
||||
|
|
@ -1465,16 +1465,16 @@ already rejects such a step (`PLAN_STEP_VERIFY_NOT_RUNNABLE`).
|
|||
|
||||
1. **The helper proves red before it builds** (Sub-step C): it writes the test
|
||||
from `Test first:`, runs the gate
|
||||
`red-proof.mjs --red-only --base {start_sha} --cmd "{Verify}" --tests {test files, ONE comma-separated argument} --plan "{plan_path}" --step {N}`
|
||||
`red-proof.mjs --red-only --base {start_sha} --cmd "{Verify}" --plan "{plan_path}" --step {N}`
|
||||
(the test must FAIL at the start point), locks the test files, and only then
|
||||
implements. A `tests_only: true` answer stops the helper before it builds
|
||||
implements. A `tests_only: true` or `plan-unreadable` answer stops the helper before it builds
|
||||
(it does not rewrite the test); the executor's own run below then sends the
|
||||
step up to the plan. The plan's `Files:` alone decides `tests_only`. `--tests` names the step's test files, as Sub-step B defines them.
|
||||
step up to the plan, or (`plan-unreadable`) acts as the exit-2 row says. The plan's `Files:` alone decides `tests_only`. Red proof reads the step's test files, as Sub-step B defines them, from the plan.
|
||||
2. **The executor proves it both ways after the helper returns:**
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/verification/red-proof.mjs --base {start_sha} --cmd "{Verify command}" --tests {test files, ONE comma-separated argument} --plan "{plan_path}" --step {N} --json
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/verification/red-proof.mjs --base {start_sha} --cmd "{Verify command}" --plan "{plan_path}" --step {N} --json
|
||||
```
|
||||
`--tests` names the step's test files, as Sub-step B defines them. The test must FAIL against the start point and PASS against the result
|
||||
Red proof reads the step's test files, as Sub-step B defines them, from the plan. The test must FAIL against the start point and PASS against the result
|
||||
(the working tree), and a **control run** must PASS: the start-point copy
|
||||
with the step's whole change laid on top. The start point is a temp
|
||||
`git clone --shared` checked out at `start_sha` (a real git checkout), with
|
||||
|
|
@ -1487,9 +1487,9 @@ already rejects such a step (`PLAN_STEP_VERIFY_NOT_RUNNABLE`).
|
|||
|------|-----------------------|----------|
|
||||
| 0 | `"proven"` | continue |
|
||||
| 1, `not-red-at-base`, `tests_only: true` | `"rejected"` | every file in the step's `Files:` is a test (the plan validator's `PLAN_STEP_TESTS_ONLY`, the same reading). The plan's `Files:` alone decides: the step's diff is not read, and a path changed outside `Files:` is the controller's scope finding (Sub-step C). The behaviour its tests cover already exists at the start point (an earlier step built it, or it was there before the plan), so the step can never be red and no rewrite of the test can fix that. The plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4), with no fix round — the planner moves the test into the step that builds the behaviour, or makes a test meant to pass from the start the `Regression:` of the step that changes the code |
|
||||
| 1, `not-red-at-base`, otherwise | `"rejected"` | the test passes without the change, so the TEST is wrong: a finding for Sub-step F, and that fix round is the one exception to the lock — the executor releases the step's test lock for it, the fix helper rewrites the test, passes the `--red-only` gate and locks again. `tests_only: null` (the plan or the step could not be read) counts here too |
|
||||
| 1, `not-red-at-base`, otherwise | `"rejected"` | the test passes without the change, so the TEST is wrong: a finding for Sub-step F, and that fix round is the one exception to the lock — the executor releases the step's test lock for it, the fix helper rewrites the test, passes the `--red-only` gate and locks again |
|
||||
| 1, `not-green-at-result` | `"rejected"` | the change does not make the test pass: a finding for Sub-step F |
|
||||
| 2 | `"not-run"` | never a pass and never proven. **`tests-mismatch` first:** a reason that starts `tests-mismatch` (the `--tests` given differs from the plan's `Test first:` `File:` line) is neither the plan's failure nor a finding at first: the executor reruns once with the list red proof printed (`plan_tests`) and reads that answer; only a second exit 2 goes on below. **plan_version ≥ 1.8:** when the reason is the command (the `Verify:` is not one test-runner invocation), the plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4); any other reason (a missing test file, an unresolvable start point, a control run that stays red) is a finding for Sub-step F. **Below 1.8:** a step with no `Test first:`, or whose `Verify:` is not a test-runner command, is tolerated as `"not-run"` — counted in the final report among the steps without proof of red, never called proven; any other reason is a finding |
|
||||
| 2 | `"not-run"` | never a pass and never proven. **`plan-unreadable` first:** a reason that starts `plan-unreadable` means the executor's own `--plan` or `--step` is wrong (a path, a step number), which no helper can fix: the executor corrects `{plan_path}` to the absolute path of the plan file it loaded in Phase 1 and `{N}` to the number in the step's own `### Step N:` heading. When the helper's report shows it built (proven and locked), the executor reruns only its own red proof with the corrected call. When the helper stopped on the same answer before it built, the executor records the restart (progress `red_proof: "not-run"`, the reason in the step's `error`, and a decision-log entry with `cost_if_wrong` "the step's first helper run"), and starts the step again with a fresh build helper (Sub-step C, round label `build (restart)`; `attempts` +1 and `iterations_remaining` −1 as for any launch; the fresh helper starts from the tree as the first helper left it, nothing is reverted, and it keeps one copy of each test case it writes), once, outside the fix rounds (`fix_rounds` does not change); a step whose `error` already holds a `plan-unreadable` restart, on a `--resume` too, does not restart again; a second `plan-unreadable` goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4), and the upward file's question is red proof's reason (the plan path or step heading it could not read), not a step assumption. A `tests-mismatch` (a `--tests` that differs from the plan's list, in `plan_tests`) cannot come from the documented call, which passes no `--tests`; when it does, it is a finding for Sub-step F. **plan_version ≥ 1.8:** when the reason is the command (the `Verify:` is not one test-runner invocation), the plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4); any other reason (a missing test file, an unresolvable start point, a control run that stays red) is a finding for Sub-step F. **Below 1.8:** a step with no `Test first:`, or whose `Verify:` is not a test-runner command, is tolerated as `"not-run"` — counted in the final report among the steps without proof of red, never called proven; any other reason is a finding |
|
||||
|
||||
The exit code is the verdict, the output text is not. `base.tap` (when the
|
||||
runner prints TAP) says what the red was: `fail 1 of 1` on a node test file is
|
||||
|
|
@ -1497,7 +1497,7 @@ usually a load error, not a named failing test — say so in the step report.
|
|||
|
||||
**The test lock (K8).** After its red gate the helper runs
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs lock --files {test files} --step {N} --cmd "{Verify command}"`.
|
||||
`--files` names the step's test files, as Sub-step B defines them (the list red proof accepted). From then on the guard refuses Write/Edit/NotebookEdit on those files, so the
|
||||
`--files` names the step's test files, as Sub-step B defines them (the list red proof printed as `tests`). From then on the guard refuses Write/Edit/NotebookEdit on those files, so the
|
||||
code moves to meet the test and never the reverse. The executor reads
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs status` and records
|
||||
`steps.{N}.test_lock = "locked"` when it exits 0 and names the step's files.
|
||||
|
|
@ -1516,7 +1516,7 @@ Launch `subagent_type: "voyage:step-implementer"` with a short `description`
|
|||
summary: the plan path, `plan_version`, the step number, the step's text
|
||||
verbatim (every field, including `Interface in:`, `Interface out:` and
|
||||
`Review focus:` when present), `start_sha`, `report_path`, `mode: build`, the
|
||||
round label (`build`, or `fix round R`), `voyage_root: ${CLAUDE_PLUGIN_ROOT}`,
|
||||
round label (`build`, `build (restart)`, or `fix round R`), `voyage_root: ${CLAUDE_PLUGIN_ROOT}`,
|
||||
and the skill to work by: `skill: test-first` for a build round,
|
||||
`skill: debugging` for a fix round (S2; Sub-steps B and F).
|
||||
|
||||
|
|
@ -1836,7 +1836,7 @@ session's effort, Sub-step E).
|
|||
| `fix_rounds` | Goes to | What happens |
|
||||
|---|---|---|
|
||||
| 1, 2, 3 | a fresh helper | First `node ${CLAUDE_PLUGIN_ROOT}/lib/util/test-lock.mjs pending` (the step is no longer seen green; D1 marks it again). Launch `voyage:step-implementer` again (no `name`, no `isolation`), `mode: fix`, with the findings (file, line, problem, fix) and the mechanical failures. It fixes only those; the tests stay locked (except in a `not-red-at-base` round, Sub-step B). Then B (both ways), D1, D2, D3, D4 run again and a fresh controller re-controls only what was fixed (Sub-step E). `attempts` +1; `iterations_remaining` −1 |
|
||||
| 4 | the plan | **After three failed fix rounds the question is the architecture, not a fourth fix.** The step goes back to planning (below). Sub-step B sends two reds here directly, with no fix round: a `"not-run"` caused by the `Verify:` command, and a `not-red-at-base` with `tests_only: true` (a step of only tests). Stop class 4 |
|
||||
| 4 | the plan | **After three failed fix rounds the question is the architecture, not a fourth fix.** The step goes back to planning (below). Sub-step B sends three answers here directly, with no fix round: a `"not-run"` caused by the `Verify:` command, a second `plan-unreadable` (the executor's own call still wrong after one correction), and a `not-red-at-base` with `tests_only: true` (a step of only tests). Stop class 4 |
|
||||
| 5 | the operator | A step already sent to the plan (`fix_rounds` 4) that fails its control again after re-planning goes to the operator. The executor never builds this step again: on a later `--resume` it only runs the checks (B, D1, D2, D3, D4, E) on whatever the operator changed. If they fail again the step stays at 5 — there is no sixth round, `fix_rounds` never exceeds 5 — and the run stops again. Stop class 4 |
|
||||
|
||||
**Back to planning** (`fix_rounds = 4`): save `git diff {start_sha}` to
|
||||
|
|
@ -1848,7 +1848,7 @@ operator to re-plan the step (`/trekplan` revises it) and run
|
|||
`/trekexecute --resume`, which launches a fresh `build` helper for the revised
|
||||
step (`attempts` +1, `iterations_remaining` −1) and keeps `fix_rounds` at 4. A step whose proof of red cannot run on a
|
||||
≥ 1.8 plan because its `Verify:` is not one test-runner invocation goes here
|
||||
directly (Sub-step B).
|
||||
directly (Sub-step B), and so does a second `plan-unreadable` after one corrected restart, with red proof's reason as the upward file's question.
|
||||
|
||||
**The plan's `On failure:` clause** decides what happens around the ladder:
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue