Round 7 (516dffa..45aa9e1, WARN: 12 MAJOR, 0 BLOCKER) and the fence defect
found while writing its review.md. Each fix began with a test that failed on
0f400d3 (v6.3.0); suite 2208 (2206/0/2), 14 tests more.
- Tests-only (operator 2026-10-11): the plan's Files: alone decides, and the
text says so (red-proof, trekexecute Sub-step B, step-implementer,
HANDOVER-CONTRACTS). red-proof no longer reads the step's diff, so a change
already in the tree when the step began no longer counts against it, and a
plan placed outside .claude/ reads the same. The round-6 test that had the
diff overrule the plan is rewritten to the new rule.
- The Files: reader keeps a plain file beside a backticked path
(", Dockerfile", "the CLI entry src/cli.mjs", "and Makefile"). Measured on
the 520 Files: fields in 182 plans: no field reads differently.
- Red proof reports a command its timeout killed as killed after the timeout,
not "could not start"; the release check tells a timeout from output past
its buffer and from another signal. One reading of a stop
(criteria-runner stoppedBy) for red proof, the criteria runner and the
release check.
- A trailing `# comment` is dropped in a backticked check too, and in a
command passed as written, so the words screened are the words run.
- The release check reads a relative --review in --repo.
- The 15-minute timeout is defined once (criteria-runner.mjs).
- The findings check reads each cited file once per validation.
- One isMain helper (lib/util/is-main.mjs) for every CLI: the 31 copies of
the realpath guard and the 20 modules still on the file:// guard, which
exited 0 having done nothing under a symlink or a path with a space.
- The Bash guard writes its no-one-to-ask refusal once, on a direct condition.
- The findings check ends a ```json block only at a fence on its own line, so
three backticks inside a JSON string no longer end it (also in 6.2.0).
- Brief-conformance notes: a test for test-lock green on exit 127 (it
passed at once; it fails when 127 is read as could-not-run); the prose now
names the git revision `@~1` as the one leading-@ exception.
Not changed: the example plan's steps 1-2 keep their tests-only warning
(operator decision 2026-10-09; the r5 test records it). Brief note 4 (a
Regression: whose runner is not installed goes to the fix rounds) is a
routing question for the operator.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run the full T2 §5 prose-vs-Workflow /trekreview bake-off (operator GO,
choice "a"): 3 runs/arm on a rich-finding JWT-auth fixture, resolving the
smoke's 0-finding limitation.
Deliverables:
- tests/fixtures/bakeoff-rich/ — JWT-auth brief + diff with 5 seeded blatant,
brief-traceable issues (varied severity/rule_key, one dual-flaggable).
- scripts/bakeoff-armA-merge.mjs — Arm A (prose) validate (NW1) + triplet-dedup,
matching Arm B's dedup exactly.
- scripts/bakeoff-fidelity.mjs — cross-arm + within-arm + granularity-ladder
fidelity analysis over the structured arm outputs.
- docs/T2-bakeoff-results.md §Full run — the T2 §5 verdict.
Result (3 runs/arm, both arms ran the coordinator):
- Verdict fidelity EQUIVALENT — all 6 runs BLOCK, cross-arm verdict-match 1.0.
- Finding-set: substrate is fidelity-neutral. Cross-arm jaccard 0.41 (triplet)
→ 0.71 (file,rule_key) → 1.0 (file); cross-arm ≈ within-arm at every
granularity. Issue coverage 5/5 in 6/6 runs. Low triplet jaccard is
line-citation noise shared by both arms, not a substrate effect.
- Token +4.4% (Arm B vs A; <=+15%). Classifier interference 0 at 9-agent
concurrency. JSON-robustness: Arm B schema-forced; Arm A 6/6 valid via NW1.
- VERDICT POSITIVE → S11 proceeds with opt-in --workflow flag.
Caveat (per plan posture): strict triplet-jaccard>=0.7 flag is 0/9, a
metric-calibration artifact (both arms sub-0.7 against themselves), not a
regression. Residual: F4 auto/bypass explicit-mode check (mode not settable
in-session).
Suite green (662/660 pass, 2 skip); plugin validate clean (modulo the
pre-existing root-CLAUDE.md warning). No production code changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
