voyage/evals
Kjell Tore Guttormsen 687710a975
fix(proevesett): says-pass/says-fail read the verdict in any letter case
Case 8 failed 2 of 12 in the PM re-measurement of ea66a98 on correct replies
that said "passed" without PASS. The patterns now spell each letter as a
class ([Pp][Aa][Ss][Ss]...) instead of relying on a regex flag, since a grader
file carries only `pattern`. The verb forms pass/passes/passed and
fail/fails/failed are verdicts; a preceding "not " or "n't " still refuses.

Measured on the PM's recorded replies (distinct runs): old says-pass 10/12 on
case 8 (reproduces the finding), new 12/12; says-pass on case 6 replies 0/8
both, says-fail 8/8 on case 6 and 0/12 on case 8 both.

Suite after git add: 1235 tests, 1233/0/2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:29:39 +02:00
..
plan-halts-without-intent-approval test(proevesett): a case for the intent gate; case 5's error comes from code; graders see what they claim 2026-09-23 09:28:38 +02:00
plan-halts-without-phase-signals fix(proevesett): case 4 reads the sequencing gate's own code from the trace 2026-09-23 10:50:48 +02:00
plan-project-not-initialized test(evals): Voyage's fixed test set as a plugin eval suite, expectations written before the first run 2026-09-23 07:52:34 +02:00
plan-rejects-unknown-export test(proevesett): a case for the intent gate; case 5's error comes from code; graders see what they claim 2026-09-23 09:28:38 +02:00
plan-requires-brief test(proevesett): a case for the intent gate; case 5's error comes from code; graders see what they claim 2026-09-23 09:28:38 +02:00
review-requires-project test(proevesett): a case for the intent gate; case 5's error comes from code; graders see what they claim 2026-09-23 09:28:38 +02:00
review-validate-flags-bad-finding-id fix(proevesett): says-pass/says-fail read the verdict in any letter case 2026-09-23 11:29:39 +02:00
review-validate-passes-clean-review fix(proevesett): says-pass/says-fail read the verdict in any letter case 2026-09-23 11:29:39 +02:00