Commit graph

11 commits

Author SHA1 Message Date
878871a6ad
fix(review): the Arm B workflow keeps the contract's row on a trimmed title and a tie
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 15:06:23 +01:00
af90e82794
refactor: simplify pass (trekexecute)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 22:22:18 +02:00
a5fcb5e8b4
fix(review): group a triplet's findings title first, so arrival order never changes the rows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 18:30:52 +02:00
7175783ce5
fix(review): a past-the-end line makes a finding unverified, never refuted, and the Declined to Judge copy-through is pinned
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 17:13:03 +02:00
8677836060
fix: close the gaps the review of the trial-5 fixes found
- park-attempt-files: parks in the git dir (outside the work tree, so Sub-step C.2 never registers a parked file again), reads a path with a space, names a new file under a header it cannot read, parks a symlink, never overwrites a parked file, and exits 1 while a file the attempt made is still in the tree; the upward diff is saved in one form whatever the git config.

- brief-validator: BRIEF_SC_NOT_RUNNABLE only when a later backticked span is a command; a criterion that only names a flag or a path stays prose (Handover 1 says so now).

- review ids: counted over the final list (runContract), as fill-finding-ids counts review.md; trimmed_count matches rows by id; engine-findings and fidelity-diff use computeFindingIds; the --workflow dedup keeps one source's different findings apart, the decision-23 put-back stays one copy per triplet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 20:11:08 +02:00
b47cabdf30
fix(review): one reviewer's two findings on one triplet keep a row each
Pass 1 merged two different /code-review findings on (bin/slug.mjs, 46, CODE_REVIEW_DEFECT) and the second lost its row and its action. A reviewer never duplicates itself: its second finding on a triplet under another title stays its own row, with the id over rule_key#2 (the form critic-diff already uses); fill-finding-ids counts the same way (voyage-lekerepo trial 5, finding 7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 19:50:20 +02:00
40f6ae1177
fix(review): close the gate's fail-open paths an independent review found
An adversarial read of d88245c reproduced 15 problems; all fixed here, each
with a test that fails on the old code.

- verdict-gate: a greener verdict must be backed by the LAST fence being a
  `json` block of the right shape, with no BLOCKER in the JSON (any case),
  in `## Findings (BLOCKER)`, or a differing finding-id list; two verdict
  lines and more than one file argument are exit 2. ({}, null, lowercase
  "blocker", a trailing ```jsonc and a later json fence all gave exit 0.)
- engine-findings: a transcribed "narrower" key can no longer lower an
  engine tier; absolute paths under --repo-root match the scope; "12" lines
  parse; an empty scope is an error; out-of-scope engine findings are
  unverified (`engine:out-of-scope`) and forbid ALLOW.
- recurrence: the catch-all engine keys and keys normalised from unknown
  ones are never proposed; the same unfixed finding in two iterations is not
  a recurrence; proposals are validated before they are offered; \r/U+2028
  and damaged markers are refused; writes go through a symlinked CLAUDE.md.
  Prose points --repo at the session directory, not the git toplevel.
- Workflow substrate: engines are counted from what arrived, and the
  verdict's floor is recomputed from the coordinator's findings; now tested
  by running the script with stubbed Workflow globals. Two literal NUL bytes
  written as \u0000 (git saw the file as binary).
- Trimming keeps Pass 1's "Also flagged by" provenance; `## Declined to
  Judge` and remediation steps are required from review_version 1.1 only
  (N-1); the zero-diff review now carries the JSON block the gate needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:03:16 +02:00
d88245c06e
feat(review): the review is a gate - verdict in the file, CC engine, project memory
Ordre 5 (wave 1, P11 + P5), review_version 1.1.

- K7: `verdict: ALLOW | WARN | BLOCK` is required in review.md (strict
  error, soft warning for 1.0 reviews). New CLI lib/review/verdict-gate.mjs:
  0 = ALLOW/WARN, 1 = BLOCK, 2 = could not decide (never green); a verdict
  greener than the file's own BLOCKERs or JSON block is exit 2. Realpath
  main guard (the old file:// guard exits 0 under a symlink - measured).
- No finding is struck for length: succinctness trims only MINOR/SUGGESTION
  text (catalogue tier), keeps BLOCKER/MAJOR whole, counts trims; the
  coordinator lists what it declined to judge.
- Claude Code's /code-review and /security-review run as the engine via the
  Skill tool (never --fix/--comment/--post/ultra); lib/review/engine-findings.mjs
  turns their findings into a reviewer payload; two catalogue keys
  (CODE_REVIEW_DEFECT, SECURITY_REVIEW_VULNERABILITY), catalogue 12 -> 14.
  An engine that did not run forbids ALLOW.
- Every BLOCKER is written as a numbered `### Step R<n>` with its source
  finding (strict error REVIEW_MISSING_REMEDIATION_STEP).
- P5: lib/review/recurrence.mjs - a finding class in two review.md files
  (current + git history, no new store) becomes one proposal, routed by the
  catalogue's memory_route; written only between Voyage markers in
  .claude/rules/voyage-review-lessons.md under a size budget, CLAUDE.md gets
  a pointer only; the operator approves every line (Phase 8.7).
- Workflow substrate: catalogue enum in step; a missing reviewer or engine
  forbids ALLOW there too (it returned ALLOW on zero findings before).
- Fixtures review-run-A/B said WARN over a BLOCKER; now BLOCK.

Borrowed and credited in code: Superpowers (declined-to-judge), ai-native-sdlc
(cap only nits; mistake/fix/check), Spec Kit (managed block; converge steps),
Pocock (retro routing), Kiro (verdict file + gate; evidence per lesson),
Cherny (engine -> brief control order; review -> memory; budget).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:45:52 +02:00
20cdc22803 fix(review): an anonymous invalid payload is unattributable, not a reviewer named "unnamed reviewer"
Follow-up defect in the reviewer accounting added by e2aec01, found by review
and confirmed by probe before fixing.

validateFindings only WARNS on a missing `reviewer` field, so a payload can
fail schema while carrying no name. ingest then records `reviewer: null`, and
runContract turned that null into the literal reviewer name "unnamed reviewer".

MEASURED before the fix:
  runContract([{findings:[{file:'x.mjs',line:1,rule_key:'NOPE',severity:'MAJOR'}]}],
              {expectedReviewers:['code-correctness-reviewer']})
  -> missing_reviewers = ["unnamed reviewer", "code-correctness-reviewer"]
One failure, two entries, one of them an agent nobody launched. The
`reported.delete(s.reviewer)` line was also inert for that case, since a null
name was never in the set to begin with.

Fix: skipped payloads are split by whether they carry a name. Named ones go to
missing_reviewers as before; anonymous ones increment the new
`unattributable_payloads` count, which forbids ALLOW on its own - so stripping
a reviewer name from a payload cannot restore ALLOW, and the floor does not
depend on the caller passing expectedReviewers. `allow_blocked_by` reports the
two facts separately: `missing-reviewer:<name>` and `unattributable-payload (n)`.

The old behaviour never produced a false ALLOW - it failed in the safe
direction - but it named a reviewer that did not exist, which is the kind of
output an operator would chase.

Iron Law: two failing tests first (double entry; anonymous-payload-alone must
forbid ALLOW), then the fix.

Also verified in this pass, by temporarily adding a fake reason to
UNVERIFIED_REASONS: the prose-vocabulary pin does go red when a reason is
declared in the lib but missing from agents/review-coordinator.md. A pin that
cannot fail is not a pin.

Suite 1034 (1032/0/2) -> 1036 (1034/0/2), 0 failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 22:54:31 +02:00
e2aec019ac fix(review): fail-closed verdicts - an unsubstantiated finding can no longer yield ALLOW
computeVerdict counted only the findings handed to it (reasoned.kept), so a
finding removed by Pass 2 or Pass 3, and a reviewer whose payload was thrown
away or never arrived, were arithmetically identical to a finding that never
existed. All three pushed the verdict toward ALLOW.

Measured before the fix (probes, 2026-09-01):
  - a BLOCKER with a 101-character title -> ALLOW (Pass 2 succinctness)
  - a payload with one ad-hoc rule_key is skipped WHOLE at ingest, taking a
    valid BLOCKER sibling with it -> ALLOW
  - a reviewer that never reported -> ALLOW
Pass 3's own no-citation / unknown-rule_key branches turned out unreachable
through runContract (validateFindings rejects those payloads first), so the
reachable exposure was Pass 2 plus the skipped/absent reviewer.

THE OPEN DESIGN DECISION, and why it went against the order's default.
The order proposed: indeterminate file-existence YES, plain succinctness NO
("a too-long finding is not an uncertain finding"). I kept the first and
overrode the second, on one principle:

  A removal is `dropped` only when the test REFUTED the finding as a claim
  about this codebase. Every other removal is `unverified`.

Succinctness and actionability read a `.length`. They never examine the claim,
so they cannot establish the finding is unreal - and dropping a BLOCKER for a
101-character title is precisely the fail-open shape being fixed. Three things
settled it:

1. Under the order's default the fix would have been almost inert. Pass 3's
   drop branches are unreachable via runContract, so leaving Pass 2 out would
   have left the only reachable finding-level exposure open.
2. Cost asymmetry, priced rather than asserted: the verdict is not a gate.
   Handover 6 feeds `findings` filtered to BLOCKER+MAJOR into /trekplan
   (commands/trekplan.md:218); `verdict` is optional metadata
   (docs/HANDOVER-CONTRACTS.md:353). Nothing loops or re-plans on WARN. So a
   false `unverified` costs WARN plus a printed reason; a false drop costs a
   silent ALLOW over a live BLOCKER.
3. unknown-rule_key joins them for the same reason: an ad-hoc key is a real
   defect wearing the wrong label, and v5.1.1 high-effort mode already KEEPS
   those, normalised to PLAN_EXECUTE_DRIFT. Refuting them at normal effort
   while keeping them at high effort would be incoherent.

no-citation stays a drop: a finding whose file is empty or whose line is
negative names no location, so it makes no checkable claim at all - the one
deterministic refutation, and what the Pass 3 prose already said it was.

Iron Law: tests/lib/coordinator-contract.test.mjs first, red (missing export +
the three measured ALLOWs), then production code. Two existing assertions were
updated AFTER implementation as contract changes, not to make the red pass.
A known-positive control pins that ALLOW is still reachable - without it,
"no ALLOW" is not a fail-closed contract, only a broken one.

lib/review/coordinator-contract.mjs
  + classifySuppression / REFUTING_REASONS / UNVERIFIED_REASONS - one
    vocabulary owned by the lib, including the tokens only the LLM
    coordinator emits (accuracy:refuted, file-existence:refuted/indeterminate),
    so prose and lib cannot drift. Unclassified reasons default to unverified:
    the default fails closed.
  ~ judgeFilter / reasonablenessFilter return {kept, dropped, unverified}
  ~ computeVerdict(findings, {unverified, missingReviewers}) -> + allow_blocked_by.
    Never raises a verdict, only withholds ALLOW. Unverified findings are NOT
    counted into a severity tier: their severity was never substantiated, and
    counting it would be invention.
  ~ runContract(payloads, {expectedReviewers}) -> + unverified,
    missing_reviewers, allow_blocked_by. `suppressed` stays the union of
    dropped + unverified, so existing consumers (gold-eval) keep their meaning.

agents/review-coordinator.md - Pass 2/3 tables gain a fate column, new
  "Suppression is two-valued" section, Pass 4 threshold table gains the two
  fail-closed rows, Executive Summary must state a withheld ALLOW, Suppressed
  Findings tags each line [dropped]/[unverified]. Pass 3's unknown-rule_key
  bullet explicitly says high-effort does not reach that branch, so the same
  input never has two documented fates.

commands/trekreview.md - Phase 5 "Reviewer accounting": the expected set is
  written down before the spawn, a silent reviewer gets one re-ask and then
  STOP. That extends the pattern already in the file (schema failure -> 2
  bounded re-asks -> "do not feed unvalidated findings to the coordinator") to
  the other two ways a reviewer goes missing, rather than softening it to WARN.
  The lib's missing_reviewers stays as belt-and-braces for direct callers.

docs/agent-return-channel-defect.md - the "inferred, not observed" caveat on
  the unnamed arm above 66 lines is struck: akashic-intelligence S27
  (f168630) measured 2/2 unnamed agents returning against a 4370-line plan,
  30449 B and 10989 B, both valid JSON. Recorded with akashic's own two
  caveats intact - the measurer owns the finding, and byte-identity between
  the returned string and the file on disk was not proven. The separate S25
  named-arm figures are left standing; these are two measurements, not a
  correction of one by the other.

No release, no version bump, no tag, no catalogue ref, no Workflow port.
Suite 1025 (1023/0/2) -> 1034 (1032/0/2), 0 failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 22:46:39 +02:00
971604d870 feat(eval): add review-coordinator contract reference impl + deterministic test 2026-06-26 11:54:48 +02:00