fix: the second lekerepo trial's faults that made a result wrong or misleading

voyage-lekerepo trial 2 (2026-10-08), report section A; the operator decided
they are fixed before the next release:
- A1: a /code-review finding whose category is simplification, efficiency,
  style or readability enters as CODE_REVIEW_IMPROVEMENT (MINOR), not MAJOR.
- A2: recommended_action is the engine's own fix when transcribed, otherwise
  a test-first step; never the failure scenario dressed as a remedy.
- A3: a /code-review finding outside the exact range it was given is declined
  and listed, but no longer holds ALLOW back; /security-review's still does.
- A4: the manifest must_contain recipe uses grep -E -e ... -- so a pattern
  like --sep is not read as an option.
- A5: indented sub-points belong to their criterion (counted), and a
  criterion with sub-points and no command warns BRIEF_SC_SUBPOINTS.
- A6: a brief requiring bash 3.2 whose criterion runs plain bash warns
  BRIEF_SC_BASH_NOT_PINNED; /trekbrief proposes /bin/bash.
- A7: the plan template's and /trekplan's checkpoints stage the step's files.

Tests fail on 5f9faee first (tests/lib/trial2-fixes.test.mjs, 11 of 14).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-08 15:12:07 +02:00
commit 7c11c2aa9f
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
18 changed files with 300 additions and 39 deletions

View file

@ -332,7 +332,9 @@ sentence with your recommended answer first.
testable conditions — commands to run, observations, or metrics."
A command the review can run is one test-runner call (`npm test`,
`node --test <file>`, `bash tests/<script>.sh`); propose a script under
`tests/` for anything else (the product's own binary, a pipe, `jq`).
`tests/` for anything else (the product's own binary, a pipe, `jq`). When
the brief requires bash 3.2, propose `/bin/bash tests/<script>.sh`: plain
`bash` is whichever bash comes first on PATH, which may be newer.
- _Sharpen_ (criterion is vague): "'{quoted criterion}' is subjective.
Which command, observation, or metric would prove this is met?"
- _Quantify_ (performance/quality claim): "You mentioned it should be
@ -830,7 +832,11 @@ criterion whose command `/trekreview` will never run (the product's own
binary, `jq`, `grep`, a pipe). Name each one to the user in plain words and
offer to rewrite it as a script under `tests/` that the criterion runs
(`bash tests/<script>.sh` exits 0) before the brief is approved; if the user
keeps it, it stays a criterion a human judges, and the report says so.
keeps it, it stays a criterion a human judges, and the report says so. The
same goes for `BRIEF_SC_SUBPOINTS` (a criterion with sub-points and no command
of its own: offer one script that checks them all, or one criterion per
sub-point) and `BRIEF_SC_BASH_NOT_PINNED` (the brief requires bash 3.2 but a
criterion runs plain `bash`: offer `/bin/bash`).
**Build the operator-annotation HTML, then print the report.** After the
brief is validated, run `scripts/annotate.mjs` to produce a self-contained

View file

@ -1521,7 +1521,8 @@ holds, regardless of Verify exit code.
`.sh` file that appears in `git diff --name-only {start_sha}` (safety
net for unlisted scripts), run `bash -n {path}`. Non-zero exit = fail.
5. **must_contain patterns:** for each `{path, pattern}` pair, run
`grep -E "{pattern}" {path}`. No match = fail.
`grep -E -e "{pattern}" -- {path}` (`-e` and `--`: a pattern such as `--sep`
is never read as an option). No match = fail.
**On manifest failure:**

View file

@ -629,7 +629,7 @@ Do NOT write this synthesis to disk. It is internal working context only.
> - Pattern: `src/middleware/cors.test.ts` (follow this style)
> - **Verify:** `npm test -- jwt.test.ts` → expected: `3 passing`
> - **On failure:** revert — `git checkout -- src/middleware/jwt.ts src/middleware/jwt.test.ts`
> - **Checkpoint:** `git commit -m "feat(auth): add JWT verification middleware"`
> - **Checkpoint:** `git add src/middleware/jwt.ts src/middleware/jwt.test.ts && git commit -m "feat(auth): add JWT verification middleware"`
> - **Manifest:**
> ```yaml
> manifest:
@ -704,7 +704,7 @@ Write the plan following the template structure. The plan MUST include:
each pinned to a test; and any Global Constraint the step could break,
quoted verbatim.
- **On failure:** — recovery action (revert/retry/skip/escalate)
- **Checkpoint:** — git commit command after success
- **Checkpoint:** — `git add` of the step's Files and test files, then the git commit, after success
Steps are vertical slices: each cuts a narrow but complete path through the
layers it touches and is verifiable on its own. Never "all the tests, then

View file

@ -307,8 +307,11 @@ code, never posts anywhere, and never uploads the repo to a cloud review.
later in this conversation. Do not start Phase 6 before both engines have
delivered or have been recorded as not run. For each engine that delivered,
transcribe every finding — file, line (0 when none), the one-sentence summary,
the failure scenario, the category if given, and a narrower catalogue
`rule_key` only when one plainly fits — into
the failure scenario, the category if given (it decides the tier:
`simplification`, `efficiency`, `style` and `readability` enter as MINOR), the
fix the engine proposes (`fix`) only when its text states one — never write one
the engine did not give — and a narrower catalogue `rule_key` only when one
plainly fits — into
`{project_dir}/.review-engine-{code-review|security-review}.json` as
`{"engine": "...", "findings": [...]}`. Transcribe; never re-judge, merge or
drop an engine finding on the way. Then convert it:
@ -325,9 +328,12 @@ Exit 0 prints a reviewer payload (`claude-code-review` /
`claude-security-review`) for Phase 6 plus `out_of_scope`: engine findings on
files outside the review's SHA range (`/security-review` diffs against
origin's default branch, which can be wider). Pass `out_of_scope` to the
coordinator as well: each one is `unverified` (`engine:out-of-scope`), is
listed under `## Declined to Judge` and in Coverage, and forbids `ALLOW` —
never silently dropped. Absolute paths under the repo root are made relative
coordinator as well: each one is listed under `## Declined to Judge`
(`engine:out-of-scope`) and in Coverage — never silently dropped. One from
`/security-review` is `unverified` and forbids `ALLOW`: its base is wider than
this review and nobody here judged it. One from `/code-review` does not hold
`ALLOW` back: that engine was given exactly this range, so a citation outside
it is a remark about code this review did not change. Absolute paths under the repo root are made relative
before the scope match (`--repo-root`). Exit 1 = the transcription is
malformed, or the scope list is empty: fix it from the engine's own text (or
the scope from Phase 3) and re-run; never feed a payload that did not pass.