computeVerdict counted only the findings handed to it (reasoned.kept), so a
finding removed by Pass 2 or Pass 3, and a reviewer whose payload was thrown
away or never arrived, were arithmetically identical to a finding that never
existed. All three pushed the verdict toward ALLOW.
Measured before the fix (probes, 2026-09-01):
- a BLOCKER with a 101-character title -> ALLOW (Pass 2 succinctness)
- a payload with one ad-hoc rule_key is skipped WHOLE at ingest, taking a
valid BLOCKER sibling with it -> ALLOW
- a reviewer that never reported -> ALLOW
Pass 3's own no-citation / unknown-rule_key branches turned out unreachable
through runContract (validateFindings rejects those payloads first), so the
reachable exposure was Pass 2 plus the skipped/absent reviewer.
THE OPEN DESIGN DECISION, and why it went against the order's default.
The order proposed: indeterminate file-existence YES, plain succinctness NO
("a too-long finding is not an uncertain finding"). I kept the first and
overrode the second, on one principle:
A removal is `dropped` only when the test REFUTED the finding as a claim
about this codebase. Every other removal is `unverified`.
Succinctness and actionability read a `.length`. They never examine the claim,
so they cannot establish the finding is unreal - and dropping a BLOCKER for a
101-character title is precisely the fail-open shape being fixed. Three things
settled it:
1. Under the order's default the fix would have been almost inert. Pass 3's
drop branches are unreachable via runContract, so leaving Pass 2 out would
have left the only reachable finding-level exposure open.
2. Cost asymmetry, priced rather than asserted: the verdict is not a gate.
Handover 6 feeds `findings` filtered to BLOCKER+MAJOR into /trekplan
(commands/trekplan.md:218); `verdict` is optional metadata
(docs/HANDOVER-CONTRACTS.md:353). Nothing loops or re-plans on WARN. So a
false `unverified` costs WARN plus a printed reason; a false drop costs a
silent ALLOW over a live BLOCKER.
3. unknown-rule_key joins them for the same reason: an ad-hoc key is a real
defect wearing the wrong label, and v5.1.1 high-effort mode already KEEPS
those, normalised to PLAN_EXECUTE_DRIFT. Refuting them at normal effort
while keeping them at high effort would be incoherent.
no-citation stays a drop: a finding whose file is empty or whose line is
negative names no location, so it makes no checkable claim at all - the one
deterministic refutation, and what the Pass 3 prose already said it was.
Iron Law: tests/lib/coordinator-contract.test.mjs first, red (missing export +
the three measured ALLOWs), then production code. Two existing assertions were
updated AFTER implementation as contract changes, not to make the red pass.
A known-positive control pins that ALLOW is still reachable - without it,
"no ALLOW" is not a fail-closed contract, only a broken one.
lib/review/coordinator-contract.mjs
+ classifySuppression / REFUTING_REASONS / UNVERIFIED_REASONS - one
vocabulary owned by the lib, including the tokens only the LLM
coordinator emits (accuracy:refuted, file-existence:refuted/indeterminate),
so prose and lib cannot drift. Unclassified reasons default to unverified:
the default fails closed.
~ judgeFilter / reasonablenessFilter return {kept, dropped, unverified}
~ computeVerdict(findings, {unverified, missingReviewers}) -> + allow_blocked_by.
Never raises a verdict, only withholds ALLOW. Unverified findings are NOT
counted into a severity tier: their severity was never substantiated, and
counting it would be invention.
~ runContract(payloads, {expectedReviewers}) -> + unverified,
missing_reviewers, allow_blocked_by. `suppressed` stays the union of
dropped + unverified, so existing consumers (gold-eval) keep their meaning.
agents/review-coordinator.md - Pass 2/3 tables gain a fate column, new
"Suppression is two-valued" section, Pass 4 threshold table gains the two
fail-closed rows, Executive Summary must state a withheld ALLOW, Suppressed
Findings tags each line [dropped]/[unverified]. Pass 3's unknown-rule_key
bullet explicitly says high-effort does not reach that branch, so the same
input never has two documented fates.
commands/trekreview.md - Phase 5 "Reviewer accounting": the expected set is
written down before the spawn, a silent reviewer gets one re-ask and then
STOP. That extends the pattern already in the file (schema failure -> 2
bounded re-asks -> "do not feed unvalidated findings to the coordinator") to
the other two ways a reviewer goes missing, rather than softening it to WARN.
The lib's missing_reviewers stays as belt-and-braces for direct callers.
docs/agent-return-channel-defect.md - the "inferred, not observed" caveat on
the unnamed arm above 66 lines is struck: akashic-intelligence S27
(f168630) measured 2/2 unnamed agents returning against a 4370-line plan,
30449 B and 10989 B, both valid JSON. Recorded with akashic's own two
caveats intact - the measurer owns the finding, and byte-identity between
the returned string and the file on disk was not proven. The separate S25
named-arm figures are left standing; these are two measurements, not a
correction of one by the other.
No release, no version bump, no tag, no catalogue ref, no Workflow port.
Suite 1025 (1023/0/2) -> 1034 (1032/0/2), 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
akashic-intelligence lost /trekplan Phase 9 twice to reviewer agents that
never returned. Reproduced here and measured: the cause is the Agent tool's
`name` parameter, not the agent definitions.
Passing `name` does not label a subagent - it changes its kind. The spawn is
recorded as taskKind "in_process_teammate" (spawnDepth 0) instead of a real
subagent (spawnDepth 1). A teammate's final assistant text is not a return
value; it reaches the parent only if the teammate itself calls
SendMessage(to: "main"). plan-critic and scope-guardian declare
tools: [Read, Glob, Grep] - no SendMessage - so as teammates they are
structurally incapable of returning, whatever the prompt says.
Denominators: named 0/5 returned; named + explicit SendMessage 1/1; unnamed
3/3 (plan-critic and scope-guardian each returned full findings + the JSON
block in ~110s). Model override 2/2 non-returning, so the override is not
the variable. All 5 named agents produced correct final text in their
transcripts - only delivery failed, and that output is recoverable on disk.
The defect is a harness behaviour, so it is documented rather than silently
worked around: docs/agent-return-channel-defect.md carries the mechanism,
every denominator, the two broken queries that nearly became facts, the
unmeasured cells, and a recommended working shape for consumers (use agents,
drop `name`; do not fall back to inline review, which costs the dedup step).
Prevention is pinned, TDD red->green: the four spawning commands each state
the rule and name the mechanism, and doc-consistency derives the spawning set
from the command files so the pin cannot go vacuous.
Suite 1013 (1011/0/2), +5 from baseline 1008.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014MfB5Ecp8vvGyou8uFGxBV
Make the bake-off-validated Workflow substrate (Arm B) reachable behind an
opt-in --workflow flag for /trekreview. Default Phase 5-6 path stays prose
to preserve the lower portability floor; --workflow raises the consumer
floor to Claude Code 2.1.154+ (the Workflow tool).
- arg-parser: --workflow added to trekreview boolean flags
- commands/trekreview.md: flag row + Phase 5 substrate-routing gate +
new section 'Phase 5-6 via the Workflow substrate' (invocation contract,
S10 gotchas, bake-off citation, auto/bypass residual as Known limitation)
- docs/command-modes.md: --workflow row in /trekreview table
- routes to existing scripts/trekreview-armB.workflow.mjs (byte-identical to
the S10 part-B POSITIVE build); integration is pure routing, no script change
TDD: 8 new tests (arg-parser flag recognition + combine; command/doc prose
pins for route, opt-in posture, 2.1.154+ floor, bake-off evidence).
Suite 662 -> 670 (668 pass / 2 skip / 0 fail). plugin validate clean modulo
known root-CLAUDE.md warning. Resolves W1-narrow-wins-plan.md S11.
Codify the /trekreview Phase 5 reviewer JSON contract as a validated schema so
main validates each reviewer's output instead of merely JSON.parse-ing it, and
re-asks on schema failure (not just parse failure). Retires the fragility at
trekreview.md:202-204. No Workflow dependency (S8 tier 1, ships regardless).
New lib/review/findings-schema.mjs (3-layer Content -> Raw-text -> CLI shim,
reusing result.mjs error-shape + rule-catalogue RULE_KEYS/SEVERITY_VALUES):
- validateFindings(payload): hard errors on load-bearing fields the dedup
triplet + verdict depend on — file, rule_key (in catalogue), severity (enum),
line (integer >= 0); accumulates all errors; per-finding location.
- extractFindingsBlock(text): last fenced ```json block (the :202 contract).
- validateReviewerOutput(text): extract + parse + schema, unifying parse and
schema failures under one bounded re-ask path.
Stable codes: FINDINGS_NOT_OBJECT/_NOT_ARRAY/_NO_JSON_BLOCK/_PARSE_ERROR (top),
FINDING_MISSING_FILE/_MISSING_RULE_KEY/_UNKNOWN_RULE_KEY/_BAD_SEVERITY/_BAD_LINE
(per-finding). Descriptive fields + unknown keys tolerated (forward-compat).
Phase 5 prose: replace "parse last json block; on parse error re-emit" with
"validate against findings-schema; on failure re-ask conforming JSON, bounded
N=2; never feed unvalidated findings to the coordinator".
TDD: 27 failing tests first, then minimal code to pass. Suite 606 -> 647
(645 pass / 2 skip / 0 fail). claude plugin validate clean (only the known
root-CLAUDE.md warning). Plan: docs/W1-narrow-wins-plan.md S9.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
v5.0.0 added a read-only HTML render. v5.0.1 deleted that and pointed at
/playground document-critique, which pre-generates Claude's suggestions
and asks the operator to approve/reject them. The operator asked for the
opposite — a surface where THEY drive every annotation. v5.0.2 lands it.
scripts/annotate.mjs (~430 lines, zero deps) takes any artifact .md and
writes a self-contained HTML next to it. The HTML renders the document
with line numbers, lets the operator click any line to add their own
note (inline textarea, save with Cmd+Enter or button), keeps a sidebar
of all notes (editable + deletable + persisted in localStorage per
artifact path), and exposes Copy Prompt to gather every note into one
structured prompt. Operator copies, pastes back, Claude revises the .md.
The three producing commands now run annotate.mjs at their last step and
print the file:// link with explicit "Click any line to add YOUR OWN note"
instructions. The v5.0.1 /playground document-critique line is gone.
npm test green: 516 tests, 514 pass, 0 fail, 2 skipped.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The v5.0.0 stop-gap had /trekbrief, /trekplan, and /trekreview each render
a read-only {artifact}.html (via scripts/render-artifact.mjs) AND print a
vague "run the /playground plugin" instruction. In practice the read-only
HTML was redundant with what /playground produces and the instruction
wasn't copy-paste-ready — the operator had to guess the right invocation.
v5.0.1 deletes scripts/render-artifact.mjs + its test + npm run render,
and makes each producing command end with a single boxed, literal,
copy-paste-ready line:
/playground build a document-critique playground for {artifact_path}
One paste from the operator launches the official playground skill's
document-critique template, which builds an interactive HTML — artifact
on the left, per-line Approve/Reject/Comment cards on the right, Copy
Prompt button at the bottom. Mark suggestions, click Copy Prompt, paste
back, Claude revises the .md. Doc-consistency test pins the literal
invocation so the prose cannot soften back into vagueness.
npm test green: 503 tests, 501 pass, 0 fail, 2 skipped.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The v4.2/v4.3 bespoke playground SPA (~388 KB), the /trekrevise command,
Handover 8 (annotation → revision), the supporting lib/ modules
(anchor-parser, annotation-digest, markdown-write, revision-guard), the
Playwright e2e suite, and the @playwright/test / @axe-core/playwright
devDeps are removed. A browser walkthrough found the playground borderline
unusable, and it duplicated the official /playground plugin's
document-critique / diff-review templates.
In their place: scripts/render-artifact.mjs — a small, zero-dependency
renderer that turns a brief/plan/review .md into a self-contained,
design-system-styled, zero-network .html (frontmatter folded into a
<details> block). /trekbrief, /trekplan, and /trekreview call it on their
last step and print the file:// link; to annotate, run /playground
(document-critique) on the .md and paste the generated prompt back.
Resolves the v4.3.1-deferred findings as moot (their target files are
deleted). npm test green: 509 tests, 507 pass, 0 fail, 2 skipped.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
