voyage/evals/plan-halts-without-intent-approval
Kjell Tore Guttormsen 348aa95639
test(proevesett): red — no case for the intent gate, case 5 composed in prose, weak graders
Offline structure tests for evals/ (the suite itself needs headless runs):
- no case reaches the steg 1 intent gate (PM: 0 traces touched it). The
  new case's scaffold is in: its brief passes brief-validator --soft
  (2.1 WITH phase_signals) and --check gives BRIEF_INTENT_NOT_APPROVED —
  that test is green, so the fixture reaches the gate.
- review-requires-project: trekreview.md composes 'Error: --project <dir>
  is required.' in prose (8/10 in the PM run, backticks broke the regex),
  and its arg-parser line passes "$@", which the Bash tool never has.
- no-error-code misses REVIEW_WRONG_TYPE; PASS/FAIL graders are raw
  substrings; no-write cannot see a write through Bash.
7 tests, 6 red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 09:26:31 +02:00
..
case.yaml test(proevesett): red — no case for the intent gate, case 5 composed in prose, weak graders 2026-09-23 09:26:31 +02:00
scaffold.sh test(proevesett): red — no case for the intent gate, case 5 composed in prose, weak graders 2026-09-23 09:26:31 +02:00