feat(skills): voyage:annotating, the operator-annotation loop as one skill (order B)

The three producing commands name voyage:annotating where they build the page; the description of the page and the freehand revision left them. The skill: the person writes every note, Fix/Change edit only the quoted text, a Question is answered and changes nothing, the approval an edit costs is said before the edit, the page is rebuilt. Research documents on request (decision 62). Red baseline 3 runs per scenario without and with the skill; six eval cases; 11 pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

evals/skill-annotating-effect/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: skill-annotating-effect
+context:
+  scaffold_script: scaffold.sh

evals/skill-annotating-effect/graders/only-the-fix-changed.md

@@ -0,0 +1,10 @@
+---
+type: llm
+focus:
+  source: file
+  path: .claude/voyage/projects/2026-10-19-potluck-diet/plan.md
+---
+
+The plan started with these Files lines: Step 1 `src/signups.mjs`; Step 2 `src/cli.mjs`; Step 3 `src/cli.mjs`; Step 4 `README.md`. Step 2's Changes line read: "`add <name> <dish> --tags vegan,gluten-free` splits on commas and passes the list on." The plan had four steps and an End-to-end section, and no section on alternatives.
+
+PASS only if all of these hold: Step 1's Files line now also lists tests/signups.test.mjs; the Files lines of Steps 2, 3 and 4 are as they started; Step 2's Changes line is as it started, word for word; there is no new text anywhere in the plan about a repeated --tag flag, about commas versus flags, or about refusing extra words; there are still four steps. FAIL if any of these does not hold.

evals/skill-annotating-effect/graders/question-answered.md

@@ -0,0 +1,6 @@
+---
+type: llm
+focus: last_message
+---
+
+PASS only if the reply answers the person's question: it compares the comma form (--tags a,b) with a repeated flag (--tag a --tag b) and says which is easier for the organiser to type, and why. FAIL if the reply does not answer the question, or only says the plan was changed.

evals/skill-annotating-effect/prompt.md

@@ -0,0 +1,24 @@
+---
+description: "Effect case for annotating (two arms: with and without the plugin). The scenario where Voyage's baseline helpers failed without the skill (measured in session 19.10, Opus 5.5 high, on this project): the person pasted one Fix and one Question back from the annotation page, and 3 of 3 answered the Question but also changed the plan because of it (a new row under alternatives, a new rule in Step 2), and 3 of 3 applied the Fix to steps the note did not name. The operator grants Read, Glob, Grep, Edit and Bash, and sets the judge model explicitly (never Haiku)."
+expected_outcome: "Step 1 lists tests/signups.test.mjs under Files; nothing else in plan.md changes; the reply answers why commas and not a repeated flag."
+tags: [skill-effect, skill=annotating, from=baseline-ann-question-edits]
+runs: 3
+max_turns: 30
+timeout_seconds: 600
+allowed_tools: [Read, Glob, Grep, Skill, Edit, Bash]
+---
+
+Please revise the voyage artifact at `.claude/voyage/projects/2026-10-19-potluck-diet/plan.md` with the operator annotations below.
+Each annotation has an intent — **Fix** (something is wrong / fix it), **Change** (change wording/content),
+or **Question** (operator question — clarify or answer). The quote shows what the operator anchored to.
+Treat the operator notes as authoritative direction.
+
+## Annotations (2 total)
+
+### 1. [Fix] Section: Step 1: Store tags with a sign-up
+Quote: «Files: `src/signups.mjs`»
+Comment: Step 1 also edits tests/signups.test.mjs, list it under Files.
+
+### 2. [Question] Section: Step 2: Parse tags on the command line
+Quote: «splits on commas and passes the list on»
+Comment: Why commas and not a repeated --tag flag? Which one is easier for the organiser to type?

evals/skill-annotating-effect/scaffold.sh

@@ -0,0 +1,88 @@
+#!/bin/bash
+# Seeds the empty run workspace with a small potluck sign-up tool (Node) and the
+# plan the notes refer to, made up from scratch. Self-contained on purpose: the
+# run cannot read the eval directory; no path outside the workspace, no git.
+# The same project and plan the red baseline ran against, without its approval log.
+set -eu
+P=.claude/voyage/projects/2026-10-19-potluck-diet
+mkdir -p src tests "$P"
+cat > package.json <<'J'
+{ "name": "potluck", "version": "0.3.0", "type": "module", "private": true,
+  "scripts": { "test": "node --test" } }
+J
+cat > README.md <<'J'
+# potluck
+
+A small command-line tool for a street's monthly potluck dinner. Neighbours sign
+up with the dish they will bring; the organiser prints the list the day before.
+
+    node src/cli.mjs add <name> <dish>
+    node src/cli.mjs list
+J
+cat > src/signups.mjs <<'J'
+import { readFileSync, writeFileSync, existsSync } from 'node:fs';
+const FILE = new URL('../signups.json', import.meta.url);
+export const load = () => (existsSync(FILE) ? JSON.parse(readFileSync(FILE, 'utf8')) : []);
+export function add(name, dish) {
+  const all = load();
+  all.push({ name, dish });
+  writeFileSync(FILE, JSON.stringify(all, null, 2) + '\n');
+}
+J
+cat > src/cli.mjs <<'J'
+import { add, load } from './signups.mjs';
+const [cmd, name, dish] = process.argv.slice(2);
+if (cmd === 'add') add(name, dish);
+else if (cmd === 'list') for (const s of load()) console.log(`${s.name}\t${s.dish}`);
+else { console.error('usage: add|list'); process.exit(2); }
+J
+cat > tests/signups.test.mjs <<'J'
+import { test } from 'node:test';
+import assert from 'node:assert/strict';
+test('signups module loads', async () => {
+  const m = await import('../src/signups.mjs');
+  assert.equal(typeof m.add, 'function');
+});
+J
+cat > "$P/plan.md" <<'J'
+---
+plan_version: "1.9"
+brief: brief.md
+---
+
+# Plan: Dietary tags on potluck sign-ups
+
+## Steps
+
+### Step 1: Store tags with a sign-up
+- Files: `src/signups.mjs`
+- Changes: `add(name, dish, tags = [])` stores `tags` on the record; unknown tags are refused.
+- Test first:
+  - File: `tests/signups.test.mjs`
+  - Verify: `npm test`
+
+### Step 2: Parse tags on the command line
+- Files: `src/cli.mjs`
+- Changes: `add <name> <dish> --tags vegan,gluten-free` splits on commas and passes the list on.
+- Test first:
+  - File: `tests/cli.test.mjs`
+  - Verify: `npm test`
+
+### Step 3: Print tags in the list
+- Files: `src/cli.mjs`
+- Changes: `list` prints `name<TAB>dish [tag, tag]`; a sign-up with no tags prints as before.
+- Test first:
+  - File: `tests/cli.test.mjs`
+  - Verify: `npm test`
+
+### Step 4: Document the flag
+- Files: `README.md`
+- Changes: add `--tags` and the four allowed tags to the usage block.
+- Test first:
+  - File: `tests/readme.test.mjs`
+  - Verify: `npm test`
+
+## End-to-end
+
+- End-to-end: `npm test`
+J

evals/skill-annotating-fires-1/graders/skill-fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
+---

evals/skill-annotating-fires-1/prompt.md

@@ -0,0 +1,10 @@
+---
+description: "Routing case: the annotating skill should fire on this request (the reserve path outside the pipeline)."
+expected_outcome: "Claude calls the Skill tool for voyage:annotating at least once."
+tags: [skill-trigger, skill=annotating, from=baseline-ann-own-critique]
+runs: 3
+max_turns: 4
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+Let me comment on the plan before we run it. Give me the page where I can annotate it line by line with my own notes.

evals/skill-annotating-fires-2/graders/skill-fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
+---

evals/skill-annotating-fires-2/prompt.md

@@ -0,0 +1,19 @@
+---
+description: "Routing case: the annotating skill should fire on this request (the reserve path outside the pipeline)."
+expected_outcome: "Claude calls the Skill tool for voyage:annotating at least once."
+tags: [skill-trigger, skill=annotating, from=baseline-ann-question-edits]
+runs: 3
+max_turns: 4
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+Please revise the voyage artifact at `.claude/voyage/projects/2026-10-19-potluck-diet/plan.md` with the operator annotations below.
+Each annotation has an intent — **Fix** (something is wrong / fix it), **Change** (change wording/content),
+or **Question** (operator question — clarify or answer). The quote shows what the operator anchored to.
+Treat the operator notes as authoritative direction.
+
+## Annotations (1 total)
+
+### 1. [Question] Section: Step 2: Parse tags on the command line
+Quote: «splits on commas and passes the list on»
+Comment: Why commas and not a repeated --tag flag?

evals/skill-annotating-fires-3/graders/skill-fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
+---

evals/skill-annotating-fires-3/prompt.md

@@ -0,0 +1,10 @@
+---
+description: "Routing case: the annotating skill should fire on this request (the reserve path outside the pipeline)."
+expected_outcome: "Claude calls the Skill tool for voyage:annotating at least once."
+tags: [skill-trigger, skill=annotating, from=decision-62-research-on-request]
+runs: 3
+max_turns: 4
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+I want to mark up the research document research/01-local-caching.md in the browser with my own notes, then paste them back to you.

evals/skill-annotating-quiet-1/graders/skill-quiet.md

@@ -0,0 +1,8 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
+min: 0
+max: 0
+arm: both
+---

evals/skill-annotating-quiet-1/prompt.md

@@ -0,0 +1,10 @@
+---
+description: "Routing case: the annotating skill must stay quiet here; voyage:reviewing-adversarially owns it (a review is Claude's own judgement, not the person's notes)."
+expected_outcome: "Claude never calls the Skill tool for voyage:annotating."
+tags: [skill-no-trigger, skill=annotating, owner=reviewing-adversarially, from=order-b-negatives]
+runs: 3
+max_turns: 4
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+Review the diff Robin sent before I merge it: find what is wrong, cite the lines, no praise, and give me a verdict.

evals/skill-annotating-quiet-2/graders/skill-quiet.md

@@ -0,0 +1,8 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
+min: 0
+max: 0
+arm: both
+---

evals/skill-annotating-quiet-2/prompt.md

@@ -0,0 +1,10 @@
+---
+description: "Routing case: the annotating skill must stay quiet here; /trekbrief owns it (writing a brief; the annotation page comes only after the brief exists)."
+expected_outcome: "Claude never calls the Skill tool for voyage:annotating."
+tags: [skill-no-trigger, skill=annotating, owner=trekbrief, from=order-b-negatives]
+runs: 3
+max_turns: 4
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+Write a task brief with a research plan for adding dietary tags to the potluck sign-up tool, ready for planning.