feat(routing): every eval case names its event; the P10 eval gate is exempted with its reason (A1)

P10's model-free part (wave 4, order A1, decision 63). The routing test now
checks every case in evals/ for from=, not only the skill-tagged ones (R1):
a case with no skill tag is not routed, only checked, and the report gives
both counts ("45 cases checked for from=, 37 routed"). The eight test-set
cases get from=proevesett-eval-suite (d311f3b) or
from=proevesett-intent-gate (3892a83), the commits that added them.

The release gate that reads the last test-set run is written into the
runbook as exempted with its reason (it waits for the fixed test set to run
again) and still says where it goes (scripts/release-check.mjs).

review-630's prototype-tag stub gets from=test: under R1 an unrouted case is
still checked for from=, and the stub tests that a prototype name is never
routed, which stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## Unreleased — The annotating skill, the run coupling and P10's model-free part
+
+### Doors
+
+- two-way: every case in `evals/` names its event, and the routing test checks every case (P10, R1) — a revert lets a case without `from=` through again; the eight tags added to the fixed test set stay harmless.
+
+### Changed
+
+- **P10's model-free part is done; the eval gate (P10) is exempted until the fixed test set runs again.** Every case in `evals/` now names the event it came from (`from=`), the eight test-set cases included (`from=proevesett-eval-suite`, `from=proevesett-intent-gate`). The routing test (`lib/skills/routing.mjs`) fails any case without it, routed or not, and its report gives both counts: cases checked for `from=` and cases routed. The release gate that would read the last test-set run is written into `docs/release-runbook.md` as exempted with its reason, not built: a gate that can never fire is not built.
+
 ## v6.5.0 — 2026-10-18 — Three skills for brief, criteria and review, English labels, and three trial-5 fixes
 
 Wave 3 adds three skills, each one source for a discipline that was copied

CLAUDE.md

@@ -139,7 +139,9 @@ Full flag reference for each command (modes, `--gates`, `--profile`, breaking ch
 | `voyage:reviewing-adversarially` | No praise; no finding without a source (`file:line` or quote); no downgrade without a reason; summary-only is said, not judged; a Declined to Judge list; claims checked against the diff, not the report; axes kept separate. Replaces the per-agent "Interaction Awareness" block | the 3 review agents + `scope-guardian` `skills:` (preload); `/trekreview` names it at spawn |
 | `voyage:writing-criteria` | Each criterion / `Verify:` / `Regression:` / `End-to-end:` is one command the criteria runner accepts, as the line's first backticked span; anything more is `bash tests/<name>.sh`; it must be able to go red (runner deterministic, go-red prose only) | `brief-reviewer` + `plan-critic` `skills:` (preload); `/trekplan` names it |
 
-> **Commands are the trunk** (operator, 06.10). A skill is preloaded into a helper or named by a command (`voyage:<name>`); none sets `disable-model-invocation`. Form = S1, checked by `lib/skills/skill-lint.mjs`; `lib/skills/routing.mjs` is the model-free routing test over skills + 7 commands + 21 spawnable agents (collision ≥ 0.75 fails; thresholds pinned, never lowered). Eval cases `evals/skill-*` (3 fire / 2 quiet / 1 effect) are written, not run. Method (red baseline first) → `docs/skills-authoring.md`.
+> **Commands are the trunk** (operator, 06.10). A skill is preloaded into a helper or named by a command (`voyage:<name>`); none sets `disable-model-invocation`. Form = S1, checked by `lib/skills/skill-lint.mjs`; `lib/skills/routing.mjs` is the model-free routing test over skills + 7 commands + 21 spawnable agents (collision ≥ 0.75 fails; thresholds pinned, never lowered). Eval cases `evals/skill-*` (3 fire / 2 quiet / 1 effect) are written, not run.
+> Every case in `evals/` (the test set too) carries `from=<event>`; the routing test fails one without it and reports both counts, checked for `from=` and routed
+> (P10's model-free part, R1; the release eval gate is exempted until the test set runs again). Method (red baseline first) → `docs/skills-authoring.md`.
 
 ## Reference docs (read on demand)
 

README.md

@@ -673,7 +673,9 @@ The workshop behind them runs in `npm test`: the form check `lib/skills/skill-li
 "Use when ...", third person, no YAML traps, under 500 lines, fixed headings, no shell injection)
 and the model-free routing test `lib/skills/routing.mjs` (no two descriptions among skills,
 commands and agents may collide; 3 firing, 2 quiet and 1 effect case per skill). The eval cases sit
-in `evals/skill-*` for the operator's own eval run. How to write one, starting with a baseline the
+in `evals/skill-*` for the operator's own eval run. Every case in `evals/`, the fixed test set's
+too, names the event it came from (`from=`); the routing test fails a case without it and reports
+how many cases it checked for `from=` and how many it routed. How to write one, starting with a baseline the
 helper fails without the skill: [docs/skills-authoring.md](docs/skills-authoring.md).
 
 Borrowed, with credit: the form check and routing test from addyosmani/agent-skills (MIT); the red

docs/release-runbook.md

@@ -225,14 +225,18 @@ changes introduced by commits that are not ancestors of the previously
 reverted merge." Revert the revert commit instead, and release it as a new
 version.
 
-## Not built yet: the eval gate (P10)
+## Exempted: the eval gate (P10)
 
 The local gate before a release that reads the date and score of the last run
 of the fixed test set and compares it with the last change in the
 configuration paths (commands, agents, skills, hooks, and a changed model alias
-counts as a change). It waits for the test set to run again; a gate that can
+counts as a change). It is exempted from the release, with a reason: it waits
-never fire is not built. When it comes, it is one more check in
+for the fixed test set to run again, and a gate that can never fire is not
-`scripts/release-check.mjs`, before the door check, and one more step in § 1.
+built. P10's model-free part is in place: every case in `evals/` names the
+event it came from (`from=`), and the routing test (`lib/skills/routing.mjs`,
+run by `npm test`) fails a case without it. When the gate comes, it is one more
+check in `scripts/release-check.mjs`, before the door check, and one more step
+in § 1.
 
 ## Credits
 

docs/skills-authoring.md

@@ -72,6 +72,10 @@ tags: [skill-effect, skill=test-first, from=baseline-tfd-crash-red]
 ```
 
 `from=` names the event the case came from: a measured failure, a review finding, an analysis.
+It is required on every case in `evals/`, not only the skill cases: a case of the fixed test set
+(`tags: [proevesett, from=proevesett-eval-suite]`) has no skill tag, so it is not routed, but the
+routing test still fails it without `from=`. Its report gives both counts, cases checked for
+`from=` and cases routed.
 
 - A **skill-trigger** case has one grader: `type: tool_used`, `tool: Skill`, with an `input_match`
   that accepts `voyage:<name>` and the bare name.

evals/plan-halts-without-intent-approval/prompt.md

@@ -1,7 +1,7 @@
 ---
 description: "/trekplan on a brief that passes the sequencing gate but carries no intent approval halts at the intent gate (veikart steg 1)."
 expected_outcome: "intent-approval.mjs --check runs and reports BRIEF_INTENT_NOT_APPROVED; /trekplan halts and names /trekbrief --approve; no Agent call (no exploration swarm), no plan.md."
-tags: [proevesett]
+tags: [proevesett, from=proevesett-intent-gate]
 runs: 1
 max_turns: 15
 timeout_seconds: 300

evals/plan-halts-without-phase-signals/prompt.md

@@ -1,7 +1,7 @@
 ---
 description: "/trekplan on a brief_version 2.1 brief without phase_signals halts at the sequencing gate."
 expected_outcome: "brief-validator runs, reports BRIEF_V51_MISSING_SIGNALS, and /trekplan halts with the phase_signals hint; no Agent call (no exploration swarm), no plan.md."
-tags: [proevesett]
+tags: [proevesett, from=proevesett-eval-suite]
 runs: 1
 max_turns: 15
 timeout_seconds: 300

evals/plan-project-not-initialized/prompt.md

@@ -1,7 +1,7 @@
 ---
 description: "/trekplan --project on a missing folder stops before any exploration."
 expected_outcome: "'Error: project directory not initialized. Run /trekbrief to create it.'; no Agent call, no plan.md."
-tags: [proevesett]
+tags: [proevesett, from=proevesett-eval-suite]
 runs: 1
 max_turns: 8
 timeout_seconds: 300

evals/plan-rejects-unknown-export/prompt.md

@@ -1,7 +1,7 @@
 ---
 description: "/trekplan --export with a removed format names the format and stops."
 expected_outcome: "'Error: export format 'pr' is not supported.'; no Agent call, no file written."
-tags: [proevesett]
+tags: [proevesett, from=proevesett-eval-suite]
 runs: 1
 max_turns: 8
 timeout_seconds: 300

evals/plan-requires-brief/prompt.md

@@ -1,7 +1,7 @@
 ---
 description: "/trekplan with no arguments prints usage and stops."
 expected_outcome: "Usage block ending in 'A brief is required. Produce one with /trekbrief first.'; no Agent call, no file written."
-tags: [proevesett]
+tags: [proevesett, from=proevesett-eval-suite]
 runs: 1
 max_turns: 8
 timeout_seconds: 300

evals/review-requires-project/prompt.md

@@ -1,7 +1,7 @@
 ---
 description: "/trekreview with no arguments demands --project and stops."
 expected_outcome: "'Error: --project <dir> is required.' plus usage; no Agent call, no file written."
-tags: [proevesett]
+tags: [proevesett, from=proevesett-eval-suite]
 runs: 1
 max_turns: 8
 timeout_seconds: 300

evals/review-validate-flags-bad-finding-id/prompt.md

@@ -1,7 +1,7 @@
 ---
 description: "KNOWN-POSITIVE. /trekreview --validate on a review.md whose finding ID is not 40-char hex fails and names the rule."
 expected_outcome: "review-validator runs and reports REVIEW_BAD_FINDING_ID; the one-line summary says FAIL; no Agent call, no file written."
-tags: [proevesett]
+tags: [proevesett, from=proevesett-eval-suite]
 runs: 1
 max_turns: 10
 timeout_seconds: 300

evals/review-validate-passes-clean-review/prompt.md

@@ -1,7 +1,7 @@
 ---
 description: "KNOWN-NEGATIVE. /trekreview --validate on a valid review.md passes and raises no error code."
 expected_outcome: "review-validator runs; the one-line summary says PASS; no REVIEW_* error code appears anywhere in the trace; no Agent call, no file written."
-tags: [proevesett]
+tags: [proevesett, from=proevesett-eval-suite]
 runs: 1
 max_turns: 10
 timeout_seconds: 300

lib/skills/routing.mjs

@@ -20,7 +20,9 @@
 // A case is tagged in its prompt.md frontmatter:
 //   tags: [skill-trigger | skill-no-trigger | skill-effect, skill=<name>,
 //          owner=<command|agent|skill name> (negatives), from=<event>]
-// `from=` names the event the case came from (P10: every case is marked).
+// `from=` names the event the case came from (P10, R1): it is required on every
+// case in evals/, and a case with no skill tag (the fixed test set) is not
+// routed, only checked for `from=`.
 //
 // What this cannot see: how Claude Code really chooses. A ranking over word
 // overlap is a proxy; the operator's eval run is the measurement.
@@ -180,8 +182,12 @@ export function loadEntries(root) {
 
 const KIND_TAGS = { 'skill-trigger': 'trigger', 'skill-no-trigger': 'no-trigger', 'skill-effect': 'effect' };
 
-/** The routing-tagged eval cases of a plugin root (evals/<case>/prompt.md). */
+/**
-export function loadCases(root) {
+ * Every eval case of a plugin root (evals/<case>/prompt.md), routed or not.
+ * `kind` is null for a case with no skill tag (the fixed test set): it is not
+ * routed, only checked for `from=` (P10, R1).
+ */
+export function loadAllCases(root) {
   const d = join(root, 'evals');
   if (!existsSync(d)) return [];
   const out = [];
@@ -192,14 +198,18 @@ export function loadCases(root) {
     const tagsField = readFrontmatterFields(text).fields.tags;
     const tags = tagsField && Array.isArray(tagsField.value) ? tagsField.value.map(String) : [];
     const kindTag = tags.find((t) => Object.hasOwn(KIND_TAGS, t));
-    if (!kindTag) continue;
     const val = (k) => { const t = tags.find((x) => x.startsWith(`${k}=`)); return t ? t.slice(k.length + 1) : null; };
     const body = text.replace(/^---\n[\s\S]*?\n---\n?/, '').trim();
-    out.push({ case: name, kind: KIND_TAGS[kindTag], skill: val('skill'), owner: val('owner'), from: val('from'), prompt: body });
+    out.push({ case: name, kind: kindTag ? KIND_TAGS[kindTag] : null, skill: val('skill'), owner: val('owner'), from: val('from'), prompt: body });
   }
   return out;
 }
 
+/** The routing-tagged eval cases of a plugin root. */
+export function loadCases(root) {
+  return loadAllCases(root).filter((c) => c.kind);
+}
+
 // ---- the check ----
 
 /**
@@ -214,7 +224,8 @@ export function checkRouting(root, t = THRESHOLDS) {
   // name must not take its place. An owner may be any kind; a skill wins a tie.
   const skillByName = new Map(skills.map((e) => [e.name, e]));
   const byName = new Map([...entries.filter((e) => e.kind !== 'skill'), ...skills].map((e) => [e.name, e]));
-  const cases = loadCases(root);
+  const all = loadAllCases(root);
+  const cases = all.filter((c) => c.kind);
   const errors = [];
   const warnings = [];
   let positives = 0; let rank1 = 0; let negatives = 0;
@@ -225,8 +236,11 @@ export function checkRouting(root, t = THRESHOLDS) {
     (c.level === 'error' ? errors : warnings).push(entry);
   }
 
-  for (const c of cases) {
+  for (const c of all) {
     if (!c.from) errors.push({ code: 'ROUTING_CASE_NO_SOURCE', case: c.case, message: `${c.case}: tag the event the case came from (from=...)` });
+  }
+
+  for (const c of cases) {
     const self = c.skill && skillByName.get(c.skill);
     if (!self || self.kind !== 'skill') {
       errors.push({ code: 'ROUTING_UNKNOWN_SKILL', case: c.case, message: `${c.case}: skill=${c.skill} is not a skill in skills/` });
@@ -276,7 +290,7 @@ export function checkRouting(root, t = THRESHOLDS) {
   if (positives && rate < t.MIN_RANK1_RATE) {
     errors.push({ code: 'ROUTING_RANK1_FLOOR', message: `${rank1} of ${positives} positives rank their skill first (${(rate * 100).toFixed(0)}%); the floor is ${(t.MIN_RANK1_RATE * 100).toFixed(0)}%` });
   }
-  return { errors, warnings, stats: { entries: entries.length, skills: skills.length, cases: cases.length, positives, negatives, rank1 } };
+  return { errors, warnings, stats: { entries: entries.length, skills: skills.length, sourced: all.length, cases: cases.length, positives, negatives, rank1 } };
 }
 
 // ---- CLI ----
@@ -294,7 +308,7 @@ if (isMain(import.meta.url)) {
   else {
     for (const e of r.errors) process.stdout.write(`FAIL ${e.code}: ${e.message}\n`);
     for (const w of r.warnings) process.stdout.write(`warn ${w.code}: ${w.message}\n`);
-    process.stdout.write(`${r.stats.entries} descriptions, ${r.stats.cases} cases, ${r.stats.rank1}/${r.stats.positives} positives first; ${r.errors.length} finding(s)\n`);
+    process.stdout.write(`${r.stats.entries} descriptions, ${r.stats.sourced} cases checked for from=, ${r.stats.cases} routed, ${r.stats.rank1}/${r.stats.positives} positives first; ${r.errors.length} finding(s)\n`);
   }
   process.exit(r.errors.length ? 1 : 0);
 }

tests/lib/p10-model-free.test.mjs

@@ -0,0 +1,82 @@
+// tests/lib/p10-model-free.test.mjs
+// P10's model-free part (wave 4, order A1). Every case in evals/ names the
+// event it came from (`from=`), not only the skill cases; the routing check
+// covers every case and says how many it checked for `from=` and how many it
+// routed; and the release gate that would read the last test-set run is
+// written down as exempted with a reason, not built.
+
+import { test } from 'node:test';
+import { strict as assert } from 'node:assert';
+import { spawnSync } from 'node:child_process';
+import { mkdtempSync, mkdirSync, writeFileSync, readdirSync, readFileSync, existsSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { dirname, join } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { readFrontmatterFields } from '../../lib/skills/skill-frontmatter.mjs';
+import { checkRouting } from '../../lib/skills/routing.mjs';
+
+const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
+const CLI = join(ROOT, 'lib', 'skills', 'routing.mjs');
+
+const tagsOf = (file) => {
+  const f = readFrontmatterFields(readFileSync(file, 'utf8')).fields.tags;
+  return f && Array.isArray(f.value) ? f.value.map(String) : [];
+};
+
+test('P10: every case in evals/ names the event it came from (from=)', () => {
+  const dirs = readdirSync(join(ROOT, 'evals')).filter((d) => existsSync(join(ROOT, 'evals', d, 'prompt.md'))).sort();
+  assert.ok(dirs.length >= 45, `${dirs.length} cases`);
+  const missing = dirs.filter((d) => !tagsOf(join(ROOT, 'evals', d, 'prompt.md')).some((t) => /^from=\S+$/.test(t)));
+  assert.deepEqual(missing, [], `${missing.length} of ${dirs.length} cases have no from=: ${missing.join(', ')}`);
+});
+
+/** A plugin root with one skill, its 3/2/1 cases, and the extra cases given. */
+function stubRoot(extra) {
+  const root = mkdtempSync(join(tmpdir(), 'p10-model-free-'));
+  mkdirSync(join(root, 'skills', 'tidy-imports'), { recursive: true });
+  writeFileSync(join(root, 'skills', 'tidy-imports', 'SKILL.md'),
+    '---\nname: tidy-imports\ndescription: Sorts the import lines of a source file. Use when imports are out of order.\n---\n');
+  mkdirSync(join(root, 'commands'), { recursive: true });
+  writeFileSync(join(root, 'commands', 'rotate-logs.md'), '---\nname: rotate-logs\ndescription: Compresses and deletes old log files on disk.\n---\n');
+  const cases = {
+    p1: ['skill-trigger, skill=tidy-imports, from=test', 'sort the import lines'],
+    p2: ['skill-trigger, skill=tidy-imports, from=test', 'import lines are out of order'],
+    p3: ['skill-trigger, skill=tidy-imports, from=test', 'sort imports in the source file'],
+    n1: ['skill-no-trigger, skill=tidy-imports, owner=rotate-logs, from=test', 'delete old log files'],
+    n2: ['skill-no-trigger, skill=tidy-imports, owner=rotate-logs, from=test', 'compress the logs on disk'],
+    e1: ['skill-effect, skill=tidy-imports, from=test', 'x'],
+    ...extra,
+  };
+  for (const [n, [tags, prompt]] of Object.entries(cases)) {
+    mkdirSync(join(root, 'evals', n), { recursive: true });
+    writeFileSync(join(root, 'evals', n, 'prompt.md'), `---\ndescription: "case ${n}"\ntags: [${tags}]\nruns: 1\n---\n\n${prompt}\n`);
+  }
+  return root;
+}
+
+test('P10 (R1): a case with no skill tag and no from= is a finding of the routing check', () => {
+  const root = stubRoot({ 'plain-gate': ['proevesett', '/voyage:trekplan'], 'plain-sourced': ['proevesett, from=test', '/voyage:trekplan'] });
+  try {
+    const r = checkRouting(root);
+    const src = r.errors.filter((e) => e.code === 'ROUTING_CASE_NO_SOURCE').map((e) => e.case);
+    assert.deepEqual(src, ['plain-gate'], JSON.stringify(r.errors));
+    assert.ok(!r.errors.some((e) => e.case === 'plain-sourced'), 'a sourced case with no skill tag is not routed');
+  } finally { rmSync(root, { recursive: true, force: true }); }
+});
+
+test('P10 (R1): the report shows the denominators: cases checked for from= and cases routed', () => {
+  const root = stubRoot({ 'plain-sourced': ['proevesett, from=test', '/voyage:trekplan'] });
+  try {
+    const r = checkRouting(root);
+    assert.deepEqual([r.stats.sourced, r.stats.cases], [7, 6], JSON.stringify(r.stats));
+    const out = spawnSync('node', [CLI, '--root', root], { encoding: 'utf8' });
+    assert.match(out.stdout, /7 cases checked for from=, 6 routed/, out.stdout + out.stderr);
+  } finally { rmSync(root, { recursive: true, force: true }); }
+});
+
+// Found by its words, not its heading: the section title is set at the release.
+test('P10: the CHANGELOG says the eval gate is exempted, with the reason', () => {
+  const md = readFileSync(join(ROOT, 'CHANGELOG.md'), 'utf8');
+  assert.ok(/^- [^\n]*eval gate \(P10\)[^\n]*exempted[^\n]*test set runs again/m.test(md),
+    'a CHANGELOG line must say the eval gate (P10) is exempted until the test set runs again');
+});

tests/lib/release-prose-pins.test.mjs

@@ -118,8 +118,9 @@ test('runbook: the release check exits as the script does — 0 ready, 1 not rea
   assert.match(before, /0 = ready, 1 = not ready[^.]*, 2 = could not run — never green/);
 });
 
-test('runbook: the eval gate (P10) is named as not built, with where it goes', () => {
+test('runbook: the eval gate (P10) is exempted with its reason, and says where it goes', () => {
-  const p10 = part(read(RUNBOOK), 'Not built yet: the eval gate (P10)');
+  const p10 = part(read(RUNBOOK), 'Exempted: the eval gate (P10)').replace(/\s+/g, ' ');
+  assert.match(p10, /waits for the fixed test set to run again/, 'P10 must say why the gate is exempted');
   assert.ok(p10.includes('scripts/release-check.mjs'), 'P10 must say which file the gate goes into');
 });
 

tests/lib/review-630.test.mjs

@@ -384,7 +384,7 @@ test('review 630 r3: routing reads a kind tag only from its own table, never Obj
     for (const d of ['skills', 'commands', 'agents', 'evals']) cpSync(join(ROOT, d), join(dir, d), { recursive: true });
     const c = join(dir, 'evals', 'zz-proto');
     cpSync(join(ROOT, 'evals', 'skill-test-first-quiet-1'), c, { recursive: true });
-    writeFileSync(join(c, 'prompt.md'), '---\ntags: [constructor, owner=nobody]\n---\n\nWhat time is it?\n');
+    writeFileSync(join(c, 'prompt.md'), '---\ntags: [constructor, owner=nobody, from=test]\n---\n\nWhat time is it?\n');
     const { checkRouting } = awaitImportSync();
     const r = checkRouting(dir);
     assert.ok(!r.errors.some((e) => String(e.message ?? '').includes('zz-proto') || e.case === 'zz-proto'), JSON.stringify(r.errors));