Case 4 (plan-halts-without-phase-signals) measured on dc9b480, 6 runs:
5 of 6. The failing run called only intent-approval.mjs, halted on
BRIEF_INTENT_NOT_APPROVED, and so failed validator-ran and
halt-names-signals: the fixture has no intent marker, so the case can
measure the wrong gate. PM: 6 of 12 on dc9b480, 7 of 10 on 66e1fa1.
New tests: the case-4 fixture must pass intent-approval --check and fail
only BRIEF_V51_MISSING_SIGNALS; a not_contains grader must fail the case
on any BRIEF_INTENT_* code in the trace. says-fail / says-pass must take
FAILED / PASSED as the verdict word (v2's word boundary dropped them).
Red: 3 of 9 in tests/lib/proevesett.test.mjs fail, each for its own reason.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Offline structure tests for evals/ (the suite itself needs headless runs):
- no case reaches the steg 1 intent gate (PM: 0 traces touched it). The
new case's scaffold is in: its brief passes brief-validator --soft
(2.1 WITH phase_signals) and --check gives BRIEF_INTENT_NOT_APPROVED —
that test is green, so the fixture reaches the gate.
- review-requires-project: trekreview.md composes 'Error: --project <dir>
is required.' in prose (8/10 in the PM run, backticks broke the regex),
and its arg-parser line passes "$@", which the Bash tool never has.
- no-error-code misses REVIEW_WRONG_TYPE; PASS/FAIL graders are raw
substrings; no-write cannot see a write through Bash.
7 tests, 6 red.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
