feat(routing): every eval case names its event; the P10 eval gate is exempted with its reason (A1)
P10's model-free part (wave 4, order A1, decision 63). The routing test now
checks every case in evals/ for from=, not only the skill-tagged ones (R1):
a case with no skill tag is not routed, only checked, and the report gives
both counts ("45 cases checked for from=, 37 routed"). The eight test-set
cases get from=proevesett-eval-suite (d311f3b) or
from=proevesett-intent-gate (3892a83), the commits that added them.
The release gate that reads the last test-set run is written into the
runbook as exempted with its reason (it waits for the fixed test set to run
again) and still says where it goes (scripts/release-check.mjs).
review-630's prototype-tag stub gets from=test: under R1 an unrouted case is
still checked for from=, and the stub tests that a prototype name is never
routed, which stands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
1b613421eb
commit
cf08df9d55
17 changed files with 145 additions and 26 deletions
|
|
@ -673,7 +673,9 @@ The workshop behind them runs in `npm test`: the form check `lib/skills/skill-li
|
|||
"Use when ...", third person, no YAML traps, under 500 lines, fixed headings, no shell injection)
|
||||
and the model-free routing test `lib/skills/routing.mjs` (no two descriptions among skills,
|
||||
commands and agents may collide; 3 firing, 2 quiet and 1 effect case per skill). The eval cases sit
|
||||
in `evals/skill-*` for the operator's own eval run. How to write one, starting with a baseline the
|
||||
in `evals/skill-*` for the operator's own eval run. Every case in `evals/`, the fixed test set's
|
||||
too, names the event it came from (`from=`); the routing test fails a case without it and reports
|
||||
how many cases it checked for `from=` and how many it routed. How to write one, starting with a baseline the
|
||||
helper fails without the skill: [docs/skills-authoring.md](docs/skills-authoring.md).
|
||||
|
||||
Borrowed, with credit: the form check and routing test from addyosmani/agent-skills (MIT); the red
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue