fix: round 6 of the release review of 6.3.0
Round 6 (v6.2.0..c3641ff, WARN: 11 MAJOR, 3 MINOR; 0 BLOCKER). The round-5
fixes hold (code-correctness-reviewer); three of the new MAJOR were side
effects of running checks with no shell. Each fixed behind a test that
failed first on c3641ff.
- git checks stay in the repo too: no path that climbs out with .., starts
at / or reads <rev>:../x, and no --no-index (both reviewers).
- A trailing `# comment` on a shell-fence line is dropped: with no shell,
its words made an honest check NOT RUN, among them the first check of the
shipped example plan.
- A glob reaches `node --test` only on Node 21 and later, which expands it
itself; on an older Node it is refused with that reason (engines >=18).
- With no shell, exit 126/127 is the test program's own: red proof reads it
as a red (the control run still tells an environment fault apart), the
done line and the test lock as a failure.
- A runner that could not start (not installed) is NOT RUN in the criteria
runner and not-run (exit 2) in regression-check, as red-proof and
done-line already said; never a red suite.
- Tests-only: a backticked `Makefile` alone in its entry is a path, and red
proof confirms the plan's tests-only reading against the step's own diff
(the altitude finding: the diff is a fact, the Files: prose is not).
- release-check stops a suite that never ends (15 minutes) and reads it as
not measured (exit 2).
- The two lines this release added to pre-bash-executor.mjs are ASCII.
- MINOR: git and the other runners share one plain-word pattern.
Not in this round: MINOR f00a2eb8 (shared norm/ticked-path and isMain
helpers) and MINOR on plan-review-dedup's repeated tokenizing.
Suite 2194 (2192/0/2).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
c3641ffa42
commit
45aa9e107a
17 changed files with 262 additions and 62 deletions
|
|
@ -1486,7 +1486,7 @@ already rejects such a step (`PLAN_STEP_VERIFY_NOT_RUNNABLE`).
|
|||
| Exit | `steps.{N}.red_proof` | Executor |
|
||||
|------|-----------------------|----------|
|
||||
| 0 | `"proven"` | continue |
|
||||
| 1, `not-red-at-base`, `tests_only: true` | `"rejected"` | every file in the step's `Files:` is a test (the plan validator's `PLAN_STEP_TESTS_ONLY`, the same reading): the behaviour its tests cover already exists at the start point (an earlier step built it, or it was there before the plan), so the step can never be red and no rewrite of the test can fix that. The plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4), with no fix round — the planner moves the test into the step that builds the behaviour, or makes a test meant to pass from the start the `Regression:` of the step that changes the code |
|
||||
| 1, `not-red-at-base`, `tests_only: true` | `"rejected"` | every file in the step's `Files:` is a test (the plan validator's `PLAN_STEP_TESTS_ONLY`, the same reading), and so is every file the step changed since its start point: the behaviour its tests cover already exists at the start point (an earlier step built it, or it was there before the plan), so the step can never be red and no rewrite of the test can fix that. The plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4), with no fix round — the planner moves the test into the step that builds the behaviour, or makes a test meant to pass from the start the `Regression:` of the step that changes the code |
|
||||
| 1, `not-red-at-base`, otherwise | `"rejected"` | the test passes without the change, so the TEST is wrong: a finding for Sub-step F, and that fix round is the one exception to the lock — the executor releases the step's test lock for it, the fix helper rewrites the test, passes the `--red-only` gate and locks again. `tests_only: null` (the plan or the step could not be read) counts here too |
|
||||
| 1, `not-green-at-result` | `"rejected"` | the change does not make the test pass: a finding for Sub-step F |
|
||||
| 2 | `"not-run"` | never a pass and never proven. **plan_version ≥ 1.8:** when the reason is the command (the `Verify:` is not one test-runner invocation), the plan is what failed: the step goes straight up to the plan (Sub-step F, `fix_rounds = 4`, stop class 4); any other reason (a missing test file, an unresolvable start point, a control run that stays red) is a finding for Sub-step F. **Below 1.8:** a step with no `Test first:`, or whose `Verify:` is not a test-runner command, is tolerated as `"not-run"` — counted in the final report among the steps without proof of red, never called proven; any other reason is a finding |
|
||||
|
|
@ -1742,7 +1742,7 @@ CLI already speaks progress.json's words (`"blocked"` → `"unrunnable"`,
|
|||
|------|------------------------|----------|
|
||||
| 0 | `"passed"`, or `"absent"` when the step has no `Regression:` (nothing new happens) | continue |
|
||||
| 1 | `"failed"` (the suite went red) or `"unrunnable"` (not one allowlisted command; `"blocked"` by the denylist counts here too) | the step is not done: a finding for Sub-step F, exactly like a red Verify — it goes into the fix rounds, never to the checkpoint |
|
||||
| 2 | `"not-run"` | the CLI could not run (plan unreadable, no such step): a finding for Sub-step F, never green |
|
||||
| 2 | `"not-run"` | the CLI could not run (plan unreadable, no such step), or the runner could not start (not installed): a finding for Sub-step F, never green |
|
||||
|
||||
#### Sub-step E — Control
|
||||
|
||||
|
|
@ -1986,8 +1986,9 @@ If any fail: record which failed. Include in final report.
|
|||
|
||||
Run the criteria runner over the plan file parsed in Phase 1
|
||||
(`{plan_path}`). It parses the `## Verification` section, screens every command
|
||||
through the plugin's own PreToolUse denylist before it reaches a shell, and
|
||||
runs what survives — in the FOREGROUND, never backgrounded. (A command spawned
|
||||
through the plugin's own PreToolUse denylist before it runs (with no shell:
|
||||
its words reach the runner as written), and runs what survives — in the
|
||||
FOREGROUND, never backgrounded. (A command spawned
|
||||
from node does not pass through the Bash tool, so that hook cannot fire by
|
||||
itself; the runner invokes it over its documented stdin protocol instead of
|
||||
carrying a second copy of the denylist.)
|
||||
|
|
|
|||
|
|
@ -793,9 +793,10 @@ Write the plan following the template structure. The plan MUST include:
|
|||
grep`) — with no pipe, redirect or `&&`. It runs with no shell: its words
|
||||
reach the runner as written, so a word with a quote, backslash, `$`,
|
||||
brace, or a leading `~` or `@` is refused, a glob only reaches
|
||||
`node --test` (which expands its own patterns), no path climbs out with
|
||||
`..` or starts at `/`, each runner takes a short list of read-only options,
|
||||
and nothing may follow `make test`. Anything else is reported NOT
|
||||
`node --test` on Node 21 and later (which expands its own patterns), no
|
||||
path climbs out with `..` or starts at `/` (git included), each runner
|
||||
takes a short list of read-only options, and nothing may follow
|
||||
`make test`. A trailing `# comment` on a line of a shell fence is dropped. Anything else is reported NOT
|
||||
RUN, never executed and never failed. A check that needs more than that is
|
||||
wrapped in a script under `tests/` and declared as `bash tests/<script>.sh`.
|
||||
9. **Estimated Scope** — File counts and complexity rating.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue