voyage/evals/review-validate-flags-bad-finding-id/graders/says-fail.md
Kjell Tore Guttormsen 687710a975
fix(proevesett): says-pass/says-fail read the verdict in any letter case
Case 8 failed 2 of 12 in the PM re-measurement of ea66a98 on correct replies
that said "passed" without PASS. The patterns now spell each letter as a
class ([Pp][Aa][Ss][Ss]...) instead of relying on a regex flag, since a grader
file carries only `pattern`. The verb forms pass/passes/passed and
fail/fails/failed are verdicts; a preceding "not " or "n't " still refuses.

Measured on the PM's recorded replies (distinct runs): old says-pass 10/12 on
case 8 (reproduces the finding), new 12/12; says-pass on case 6 replies 0/8
both, says-fail 8/8 on case 6 and 0/12 on case 8 both.

Suite after git add: 1235 tests, 1233/0/2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 11:29:39 +02:00

113 B
Raw Blame History

type pattern
regex (?<![Nn][Oo][Tt]\s)(?<![Nn][\x27’][Tt]\s)\b[Ff][Aa][Ii][Ll](?:[Ss]|[Ee][Dd])?\b