Round 8 (0f400d3..29b4ac9, BLOCK: 1 BLOCKER, 12 MAJOR, 1 MINOR). Each fix
began with a test that failed on 29b4ac9 (13 tests, all red there); suite
2221 (2219/0/2).
- BLOCKER: a backticked check whose trailing comment holds a quote, $,
backtick or backslash is refused, not dropped. The comment was never
screened, while the executor hands the Verify span as written to red
proof, the test lock and the done line inside --cmd "...", where a $(...)
runs and a quote ends the argument. trekplan's text and the reviewer's
NOT RUN table say so.
- A span that is only a comment is not-a-command, not a placeholder. The
dropComment text says the allowlist still refuses a word with a # in it.
- The criteria runner, comparison-gate and end-state-gate use isMain: they
did nothing and exited 0 under a symlinked path. The test now refuses any
process.argv[1] comparison outside lib/util/is-main.mjs.
- Files: in a field with backticked paths, a plain word counts only before
the entry's description (a dash, or a colon after the path); a name with
no slash only first in its entry or after and/or/+/&. "mocks
fs.readFileSync", "runs on Node.js 22.x" or "no change to package.json"
no longer make a tests-only step look like it builds something.
Lowercase makefile, rakefile and justfile are build files. Measured on the
520 Files: fields in 48 plans of the marketplace and the test repo:
3 fields read differently (two notes no longer read as files, one
duplicate name dropped), tests-only changed in 0.
- The test lock compares --cmd without its trailing comment, and the test
lock and the done line read a stop with stoppedBy, the one reading.
red-proof drops its second reading, spawnError.
- release-check reads a relative --review in --repo or, when only that one
exists, from the current directory as 6.3.0 did; two different files are
exit 2, never guessed. The runbook says so.
- One trailing-json reader: a closing fence may be longer than three
backticks, in the block reader and the verdict gate alike, and the review
validator and the synthesis digest use it, so three backticks inside a
JSON string no longer hide a BLOCKER from the remediation-step check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 7 (516dffa..45aa9e1, WARN: 12 MAJOR, 0 BLOCKER) and the fence defect
found while writing its review.md. Each fix began with a test that failed on
0f400d3 (v6.3.0); suite 2208 (2206/0/2), 14 tests more.
- Tests-only (operator 2026-10-11): the plan's Files: alone decides, and the
text says so (red-proof, trekexecute Sub-step B, step-implementer,
HANDOVER-CONTRACTS). red-proof no longer reads the step's diff, so a change
already in the tree when the step began no longer counts against it, and a
plan placed outside .claude/ reads the same. The round-6 test that had the
diff overrule the plan is rewritten to the new rule.
- The Files: reader keeps a plain file beside a backticked path
(", Dockerfile", "the CLI entry src/cli.mjs", "and Makefile"). Measured on
the 520 Files: fields in 182 plans: no field reads differently.
- Red proof reports a command its timeout killed as killed after the timeout,
not "could not start"; the release check tells a timeout from output past
its buffer and from another signal. One reading of a stop
(criteria-runner stoppedBy) for red proof, the criteria runner and the
release check.
- A trailing `# comment` is dropped in a backticked check too, and in a
command passed as written, so the words screened are the words run.
- The release check reads a relative --review in --repo.
- The 15-minute timeout is defined once (criteria-runner.mjs).
- The findings check reads each cited file once per validation.
- One isMain helper (lib/util/is-main.mjs) for every CLI: the 31 copies of
the realpath guard and the 20 modules still on the file:// guard, which
exited 0 having done nothing under a symlink or a path with a space.
- The Bash guard writes its no-one-to-ask refusal once, on a direct condition.
- The findings check ends a ```json block only at a fence on its own line, so
three backticks inside a JSON string no longer end it (also in 6.2.0).
- Brief-conformance notes: a test for test-lock green on exit 127 (it
passed at once; it fails when 127 is read as could-not-run); the prose now
names the git revision `@~1` as the one leading-@ exception.
Not changed: the example plan's steps 1-2 keep their tests-only warning
(operator decision 2026-10-09; the r5 test records it). Brief note 4 (a
Regression: whose runner is not installed goes to the fix rounds) is a
routing question for the operator.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 6 (v6.2.0..c3641ff, WARN: 11 MAJOR, 3 MINOR; 0 BLOCKER). The round-5
fixes hold (code-correctness-reviewer); three of the new MAJOR were side
effects of running checks with no shell. Each fixed behind a test that
failed first on c3641ff.
- git checks stay in the repo too: no path that climbs out with .., starts
at / or reads <rev>:../x, and no --no-index (both reviewers).
- A trailing `# comment` on a shell-fence line is dropped: with no shell,
its words made an honest check NOT RUN, among them the first check of the
shipped example plan.
- A glob reaches `node --test` only on Node 21 and later, which expands it
itself; on an older Node it is refused with that reason (engines >=18).
- With no shell, exit 126/127 is the test program's own: red proof reads it
as a red (the control run still tells an environment fault apart), the
done line and the test lock as a failure.
- A runner that could not start (not installed) is NOT RUN in the criteria
runner and not-run (exit 2) in regression-check, as red-proof and
done-line already said; never a red suite.
- Tests-only: a backticked `Makefile` alone in its entry is a path, and red
proof confirms the plan's tests-only reading against the step's own diff
(the altitude finding: the diff is a fact, the Files: prose is not).
- release-check stops a suite that never ends (15 minutes) and reads it as
not measured (exit 2).
- The two lines this release added to pre-bash-executor.mjs are ASCII.
- MINOR: git and the other runners share one plain-word pattern.
Not in this round: MINOR f00a2eb8 (shared norm/ticked-path and isMain
helpers) and MINOR on plan-review-dedup's repeated tokenizing.
Suite 2194 (2192/0/2).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 5 (v6.2.0..d37e538, WARN: 11 MAJOR, 2 MINOR). The patching did not
converge (the same two places every round), so the operator chose a
structural fix on 2026-10-11. Each change behind a test that failed first.
- A check runs with no shell. criteria-runner's runCommand starts the runner
with the very words the screen checked (spawnSync(words[0], words.slice(1)),
without NODE_TEST_CONTEXT); Phase 7 criteria, Regression: (its exec copy is
gone, MINOR 0e98d5c7), proof of red, the done line and the test lock all
run through it. No quote, glob, tilde or variable can rewrite a checked
word any more. The screen stays and says why: a word that means something
else in a shell (quote, backslash, $, brace, leading ~) is refused; a glob
reaches only node --test, which expands its own patterns (sh read
`tests/.?/` as `tests/../`); no path climbs out with .. or starts at /;
a word starting with @ is refused (pytest reads @file as more arguments).
- The K6 plan-file sweep has v6.2.0's own Files: reader again (516dffa):
backticked paths, and plain words only in a field without them. The
shared reader from round 3 planned directories a description named and
dropped paths v6.2.0 planned. fieldPaths stays for the tests-only check,
with a path test on spans (`parse()`, `p.parse()` are no paths), plain
descriptions ignored when the field has backticked paths, dotfiles,
backslash paths and glued spans read.
- Skill frontmatter reads a value as YAML does: trailing space is nothing,
and a comment ends a plain value, a flow list or a block indicator
(`model: opus # pinned`). A comment that cuts a plain description, or is
the whole value, is still a hazard.
- release-check reads the diff with --no-renames: a test moved into an
allowed place after the review shows as removed.
- The example plan's step 2 heading no longer says "must FAIL initially";
steps 1-2 keep the tests-only warning (operator 2026-10-09, 73262b6).
Open: MINOR c1fef1ad (one shared isMain helper) is not in this round.
Measured: 0 of 463 Files: fields with backticked paths (520 fields in 182
plans across the marketplace repos) carry another path-like word.
Suite 2185 (2183/0/2).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1. A step of only tests can never be proven red. One reading,
stepTestsOnly (lib/parsers/plan-schema.mjs, test paths from the new
lib/util/test-path.mjs): the plan validator warns PLAN_STEP_TESTS_ONLY,
red-proof --plan/--step adds tests_only to a not-red-at-base, and
/trekexecute and the step helper send such a step straight up to the
plan instead of into fix rounds that cannot succeed. /trekplan, the plan
template and plan-critic (dimension 8, still 12) say the test belongs in
the step that builds the behaviour.
2. Timing tests under load: the hook helper's kill limit is 30 s and per
call overridable; SC #14 passes on the fastest of up to three runs
against the same 800 ms limit (tests/helpers/timing.mjs).
3. The example plan's On failure lines open with a K11 word.
4. /trekreview saves the reviewer's whole answer for the findings check and
names FINDING_PROSE_LINE_OUT_OF_RANGE (error: re-ask; warning: report).
5. The criteria runner's commands run without NODE_TEST_CONTEXT
(commandEnv, moved to lib/util/command-env.mjs).
6. The resolver returns effort only when the brief's phase signal sets one;
the four commands said it always did.
Also: the K2 hook tests used fixed session ids in a per-user temp dir, so
two suites running at once took each other's approval records; the ids are
now unique per run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Independent read-only review of c56f852..2d29966 (2026-10-08), each defect
reproduced by the reviewer; each new test fails on that code.
- test-lock green ran ANY allowlisted command: `green --cmd "git status"`
marked a red step green and the stop guard let it stop. The lock now
stores the step's test command (`lock --cmd`); `green` runs only that
one (a different --cmd, or a lock without one, exits 2). trekexecute D1
calls `green` with no command, which also removes the quoting hazard.
- plan-gate --accept-open worked at round 1 and passed the gate; it now
refuses while rounds are left.
- findings-schema --repo-root swallowed a following flag, or took a root
that does not exist, and silently skipped the line check: exit 2.
- brief-review-stamp hashed only the body, so a changed `framing` or
`research_topics` stayed "current"; it now covers the reviewed
frontmatter too (only the post-review stamps are left out).
- red-proof red_kind read an assertion message naming SyntaxError as a
load error; the signatures are now specific.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
voyage-lekerepo end-to-end trial (2026-10-06), report points 8, 10, 12, 15, 16:
- approval-log: «Approve (Recommended)» / «Godkjenn (anbefalt)» is the
approval it labels; the suffix never turns a hollow answer into a yes.
- trekexecute Phase 8: the state-file block imports atomic-write through
${CLAUDE_PLUGIN_ROOT}; a relative import only ran with cwd = plugin root
(the test runs the block from another directory).
- trekreview, trekresearch, trekcontinue: stats go through stats-append like
the other commands, and a failed write is reported, never skipped
silently (closes the open STATE finding for all three). The schema
fixture's stale line refs for these and trekplan are re-measured.
- annotate prints an absolute path, so file://{$ANNOT_HTML} is a live link.
- red-proof: `red_kind` = load-error | test-failure on a proven red.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An independent review of abc3083 found 15 defects, the two worst verified
in scratch repos. Each fix started with a test that failed.
- Proof of red could be "proven" by the environment: a start point
unpacked with git archive had no node_modules and no .git, so any test
failed there. The start point is now a git clone --shared checked out at
start_sha, gitignored top-level entries are linked in, and a control run
(start point + the change) must pass, or the result is not-run.
- The Test first file was never committed (the template lists it apart
from Files:) and leaked into the next step's diff. The step's scope is
now Files: + test files, staged at the checkpoint.
- New files outside Files: were invisible to every check. Every untracked
file is registered with --intent-to-add; a revert unregisters them.
- The floor guard paired renames (foo.test.js -> .bak read as clean) and
missed skipIf and node's t.skip/t.todo; prose files no longer count as
stubs.
- Phase 7.5 audited from the last step's start_sha; it now uses the run's
session_start_sha.
- The done line is written after the checkpoint and checked with --commit.
- fix_rounds never passes 5; attempts are never reset; re-control keeps
the spec/quality verdicts and earlier floor-guard rulings; the closed
stop list maps every existing stop; failed vs stopped made consistent;
the controller's effort is said, not assumed.
Tests: 1470 (1468 pass, 0 fail, 2 skipped).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/trekexecute no longer builds steps itself. Each plan step is built by one
fresh helper (new agent step-implementer: opus, effort high, no Agent tool)
and judged by one controller (a general helper on opus) that reads the
diff from a recorded start_sha and gives two verdicts, spec and quality.
Fix rounds go up, maximum 5: three fixes by a fresh helper, then back to
the plan, then to the operator. No cheaper-model ladder.
Proof is a command, never the model's word:
- lib/verification/red-proof.mjs: the step's test must fail at the start
point and pass at the result (exit 0/1/2; 2 is never green). Reproduces
the 02243c6 red-first measurement (6 of 55) through the shim fixture.
- lib/verification/test-integrity.mjs: the floor guard on the diff
(removed asserts, skipped/deleted tests, muted checks, stubs, loosened
thresholds). Over the last 150 commits: 149 clean, 1 real finding.
- lib/verification/done-line.mjs: DONE step N only when the test command
exits 0.
Test first is required from plan_version 1.8. The test lock (K8) is used
through its CLI and reported as not-locked while it is missing. A run
stops only on a closed list of four classes; every other call goes into a
decision log with its cost if wrong. Without a passing run-the-thing
criterion the result is not verified end to end, never green.
Handover 5 gains optional per-step start_sha, report_path, red_proof,
test_lock, fix_rounds (0-5) and run-level decisions and end_to_end; old
progress files still validate and resume.
No isolation on the helper: measured, a worktree-isolated subagent
branches from the default branch (02f320f), not the executor's HEAD
(2c89a6b), so it would miss the earlier steps.
Borrowings credited in commands/trekexecute.md ## Credits (Superpowers,
ai-native-sdlc, Kiro, GitHub Spec Kit, agent-skills, Cherny).
Tests: 1411 -> 1457 (1455 pass, 0 fail, 2 skipped).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
