feat(routing): every eval case names its event; the P10 eval gate is exempted with its reason (A1)

P10's model-free part (wave 4, order A1, decision 63). The routing test now
checks every case in evals/ for from=, not only the skill-tagged ones (R1):
a case with no skill tag is not routed, only checked, and the report gives
both counts ("45 cases checked for from=, 37 routed"). The eight test-set
cases get from=proevesett-eval-suite (d311f3b) or
from=proevesett-intent-gate (3892a83), the commits that added them.

The release gate that reads the last test-set run is written into the
runbook as exempted with its reason (it waits for the fixed test set to run
again) and still says where it goes (scripts/release-check.mjs).

review-630's prototype-tag stub gets from=test: under R1 an unrouted case is
still checked for from=, and the stub tests that a prototype name is never
routed, which stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-19 18:15:00 +02:00
commit cf08df9d55
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
17 changed files with 145 additions and 26 deletions

View file

@ -1,7 +1,7 @@
---
description: "/trekplan on a brief that passes the sequencing gate but carries no intent approval halts at the intent gate (veikart steg 1)."
expected_outcome: "intent-approval.mjs --check runs and reports BRIEF_INTENT_NOT_APPROVED; /trekplan halts and names /trekbrief --approve; no Agent call (no exploration swarm), no plan.md."
tags: [proevesett]
tags: [proevesett, from=proevesett-intent-gate]
runs: 1
max_turns: 15
timeout_seconds: 300

View file

@ -1,7 +1,7 @@
---
description: "/trekplan on a brief_version 2.1 brief without phase_signals halts at the sequencing gate."
expected_outcome: "brief-validator runs, reports BRIEF_V51_MISSING_SIGNALS, and /trekplan halts with the phase_signals hint; no Agent call (no exploration swarm), no plan.md."
tags: [proevesett]
tags: [proevesett, from=proevesett-eval-suite]
runs: 1
max_turns: 15
timeout_seconds: 300

View file

@ -1,7 +1,7 @@
---
description: "/trekplan --project on a missing folder stops before any exploration."
expected_outcome: "'Error: project directory not initialized. Run /trekbrief to create it.'; no Agent call, no plan.md."
tags: [proevesett]
tags: [proevesett, from=proevesett-eval-suite]
runs: 1
max_turns: 8
timeout_seconds: 300

View file

@ -1,7 +1,7 @@
---
description: "/trekplan --export with a removed format names the format and stops."
expected_outcome: "'Error: export format 'pr' is not supported.'; no Agent call, no file written."
tags: [proevesett]
tags: [proevesett, from=proevesett-eval-suite]
runs: 1
max_turns: 8
timeout_seconds: 300

View file

@ -1,7 +1,7 @@
---
description: "/trekplan with no arguments prints usage and stops."
expected_outcome: "Usage block ending in 'A brief is required. Produce one with /trekbrief first.'; no Agent call, no file written."
tags: [proevesett]
tags: [proevesett, from=proevesett-eval-suite]
runs: 1
max_turns: 8
timeout_seconds: 300

View file

@ -1,7 +1,7 @@
---
description: "/trekreview with no arguments demands --project and stops."
expected_outcome: "'Error: --project <dir> is required.' plus usage; no Agent call, no file written."
tags: [proevesett]
tags: [proevesett, from=proevesett-eval-suite]
runs: 1
max_turns: 8
timeout_seconds: 300

View file

@ -1,7 +1,7 @@
---
description: "KNOWN-POSITIVE. /trekreview --validate on a review.md whose finding ID is not 40-char hex fails and names the rule."
expected_outcome: "review-validator runs and reports REVIEW_BAD_FINDING_ID; the one-line summary says FAIL; no Agent call, no file written."
tags: [proevesett]
tags: [proevesett, from=proevesett-eval-suite]
runs: 1
max_turns: 10
timeout_seconds: 300

View file

@ -1,7 +1,7 @@
---
description: "KNOWN-NEGATIVE. /trekreview --validate on a valid review.md passes and raises no error code."
expected_outcome: "review-validator runs; the one-line summary says PASS; no REVIEW_* error code appears anywhere in the trace; no Agent call, no file written."
tags: [proevesett]
tags: [proevesett, from=proevesett-eval-suite]
runs: 1
max_turns: 10
timeout_seconds: 300