fix: the second lekerepo trial's faults that made a result wrong or misleading
voyage-lekerepo trial 2 (2026-10-08), report section A; the operator decided
they are fixed before the next release:
- A1: a /code-review finding whose category is simplification, efficiency,
style or readability enters as CODE_REVIEW_IMPROVEMENT (MINOR), not MAJOR.
- A2: recommended_action is the engine's own fix when transcribed, otherwise
a test-first step; never the failure scenario dressed as a remedy.
- A3: a /code-review finding outside the exact range it was given is declined
and listed, but no longer holds ALLOW back; /security-review's still does.
- A4: the manifest must_contain recipe uses grep -E -e ... -- so a pattern
like --sep is not read as an option.
- A5: indented sub-points belong to their criterion (counted), and a
criterion with sub-points and no command warns BRIEF_SC_SUBPOINTS.
- A6: a brief requiring bash 3.2 whose criterion runs plain bash warns
BRIEF_SC_BASH_NOT_PINNED; /trekbrief proposes /bin/bash.
- A7: the plan template's and /trekplan's checkpoints stage the step's files.
Tests fail on 5f9faee first (tests/lib/trial2-fixes.test.mjs, 11 of 14).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
8094b00861
commit
7c11c2aa9f
18 changed files with 300 additions and 39 deletions
|
|
@ -332,7 +332,9 @@ sentence with your recommended answer first.
|
|||
testable conditions — commands to run, observations, or metrics."
|
||||
A command the review can run is one test-runner call (`npm test`,
|
||||
`node --test <file>`, `bash tests/<script>.sh`); propose a script under
|
||||
`tests/` for anything else (the product's own binary, a pipe, `jq`).
|
||||
`tests/` for anything else (the product's own binary, a pipe, `jq`). When
|
||||
the brief requires bash 3.2, propose `/bin/bash tests/<script>.sh`: plain
|
||||
`bash` is whichever bash comes first on PATH, which may be newer.
|
||||
- _Sharpen_ (criterion is vague): "'{quoted criterion}' is subjective.
|
||||
Which command, observation, or metric would prove this is met?"
|
||||
- _Quantify_ (performance/quality claim): "You mentioned it should be
|
||||
|
|
@ -830,7 +832,11 @@ criterion whose command `/trekreview` will never run (the product's own
|
|||
binary, `jq`, `grep`, a pipe). Name each one to the user in plain words and
|
||||
offer to rewrite it as a script under `tests/` that the criterion runs
|
||||
(`bash tests/<script>.sh` exits 0) before the brief is approved; if the user
|
||||
keeps it, it stays a criterion a human judges, and the report says so.
|
||||
keeps it, it stays a criterion a human judges, and the report says so. The
|
||||
same goes for `BRIEF_SC_SUBPOINTS` (a criterion with sub-points and no command
|
||||
of its own: offer one script that checks them all, or one criterion per
|
||||
sub-point) and `BRIEF_SC_BASH_NOT_PINNED` (the brief requires bash 3.2 but a
|
||||
criterion runs plain `bash`: offer `/bin/bash`).
|
||||
|
||||
**Build the operator-annotation HTML, then print the report.** After the
|
||||
brief is validated, run `scripts/annotate.mjs` to produce a self-contained
|
||||
|
|
|
|||
|
|
@ -1521,7 +1521,8 @@ holds, regardless of Verify exit code.
|
|||
`.sh` file that appears in `git diff --name-only {start_sha}` (safety
|
||||
net for unlisted scripts), run `bash -n {path}`. Non-zero exit = fail.
|
||||
5. **must_contain patterns:** for each `{path, pattern}` pair, run
|
||||
`grep -E "{pattern}" {path}`. No match = fail.
|
||||
`grep -E -e "{pattern}" -- {path}` (`-e` and `--`: a pattern such as `--sep`
|
||||
is never read as an option). No match = fail.
|
||||
|
||||
**On manifest failure:**
|
||||
|
||||
|
|
|
|||
|
|
@ -629,7 +629,7 @@ Do NOT write this synthesis to disk. It is internal working context only.
|
|||
> - Pattern: `src/middleware/cors.test.ts` (follow this style)
|
||||
> - **Verify:** `npm test -- jwt.test.ts` → expected: `3 passing`
|
||||
> - **On failure:** revert — `git checkout -- src/middleware/jwt.ts src/middleware/jwt.test.ts`
|
||||
> - **Checkpoint:** `git commit -m "feat(auth): add JWT verification middleware"`
|
||||
> - **Checkpoint:** `git add src/middleware/jwt.ts src/middleware/jwt.test.ts && git commit -m "feat(auth): add JWT verification middleware"`
|
||||
> - **Manifest:**
|
||||
> ```yaml
|
||||
> manifest:
|
||||
|
|
@ -704,7 +704,7 @@ Write the plan following the template structure. The plan MUST include:
|
|||
each pinned to a test; and any Global Constraint the step could break,
|
||||
quoted verbatim.
|
||||
- **On failure:** — recovery action (revert/retry/skip/escalate)
|
||||
- **Checkpoint:** — git commit command after success
|
||||
- **Checkpoint:** — `git add` of the step's Files and test files, then the git commit, after success
|
||||
|
||||
Steps are vertical slices: each cuts a narrow but complete path through the
|
||||
layers it touches and is verifiable on its own. Never "all the tests, then
|
||||
|
|
|
|||
|
|
@ -307,8 +307,11 @@ code, never posts anywhere, and never uploads the repo to a cloud review.
|
|||
later in this conversation. Do not start Phase 6 before both engines have
|
||||
delivered or have been recorded as not run. For each engine that delivered,
|
||||
transcribe every finding — file, line (0 when none), the one-sentence summary,
|
||||
the failure scenario, the category if given, and a narrower catalogue
|
||||
`rule_key` only when one plainly fits — into
|
||||
the failure scenario, the category if given (it decides the tier:
|
||||
`simplification`, `efficiency`, `style` and `readability` enter as MINOR), the
|
||||
fix the engine proposes (`fix`) only when its text states one — never write one
|
||||
the engine did not give — and a narrower catalogue `rule_key` only when one
|
||||
plainly fits — into
|
||||
`{project_dir}/.review-engine-{code-review|security-review}.json` as
|
||||
`{"engine": "...", "findings": [...]}`. Transcribe; never re-judge, merge or
|
||||
drop an engine finding on the way. Then convert it:
|
||||
|
|
@ -325,9 +328,12 @@ Exit 0 prints a reviewer payload (`claude-code-review` /
|
|||
`claude-security-review`) for Phase 6 plus `out_of_scope`: engine findings on
|
||||
files outside the review's SHA range (`/security-review` diffs against
|
||||
origin's default branch, which can be wider). Pass `out_of_scope` to the
|
||||
coordinator as well: each one is `unverified` (`engine:out-of-scope`), is
|
||||
listed under `## Declined to Judge` and in Coverage, and forbids `ALLOW` —
|
||||
never silently dropped. Absolute paths under the repo root are made relative
|
||||
coordinator as well: each one is listed under `## Declined to Judge`
|
||||
(`engine:out-of-scope`) and in Coverage — never silently dropped. One from
|
||||
`/security-review` is `unverified` and forbids `ALLOW`: its base is wider than
|
||||
this review and nobody here judged it. One from `/code-review` does not hold
|
||||
`ALLOW` back: that engine was given exactly this range, so a citation outside
|
||||
it is a remark about code this review did not change. Absolute paths under the repo root are made relative
|
||||
before the scope match (`--repo-root`). Exit 1 = the transcription is
|
||||
malformed, or the scope list is empty: fix it from the engine's own text (or
|
||||
the scope from Phase 3) and re-run; never feed a payload that did not pass.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue