feat(skills): voyage:annotating, the operator-annotation loop as one skill (order B)

The three producing commands name voyage:annotating where they build the page; the description of the page and the freehand revision left them. The skill: the person writes every note, Fix/Change edit only the quoted text, a Question is answered and changes nothing, the approval an edit costs is said before the edit, the page is rebuilt. Research documents on request (decision 62). Red baseline 3 runs per scenario without and with the skill; six eval cases; 11 pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-19 18:36:00 +02:00

8.8 KiB

Raw Blame History

Writing a Voyage skill

A guide for maintainers. It is not a skill and does not ship to users' sessions as one.

Voyage's skills live in skills/<name>/SKILL.md. Claude Code finds that folder by itself; the manifest needs no entry. A skill is advice: Claude reads it and usually follows it. A rule that must hold every time needs a check behind it (a hook or a script), and the skill says which check that is, or says "prose only".

How Voyage uses skills

The /trek* commands are the trunk. A skill is used in two fixed ways, and its description is only the reserve path for use outside the pipeline:

Preloaded into a helper. An agent's skills: line loads the whole skill into the helper's context when it starts. agents/step-implementer.md preloads voyage:test-first and voyage:debugging. Preloading needs no Skill tool in the agent's tool list.
Asked for by name. A command tells Claude to use voyage:<name>.

Write the plugin-qualified name, voyage:<name>, in both places. A skill that no command or agent asks for fails the pin in tests/lib/skills-workshop-prose-pins.test.mjs.

No Voyage skill sets disable-model-invocation: true. That field also stops a helper from preloading the skill and a command from calling it.

The form (contract S1)

lib/skills/skill-lint.mjs is the answer key. It runs in npm test and as a CLI (node lib/skills/skill-lint.mjs, exit 0 clean, 1 findings, 2 could not run). It checks:

Frontmatter: only Claude Code's own skill fields; name equals the folder name (kebab-case, at most 64 characters, without "anthropic" or "claude"); no YAML traps (an unquoted : , #, a tab, an unclosed quote, a value starting with an indicator such as a backtick). Quote a description that holds a colon. Claude Code loads a skill whose YAML does not parse with no fields at all, so its description is gone.
Description: at most 1 024 characters, third person (no I, we or you), and it says "Use when ...". Put the key use first.
Body: under 500 lines; the headings ## Overview, ## When to use, ## Process, ## Red flags, ## Enforced by, ## Credits, in that order; every file it links or names exists; no shell injection (!`cmd` or a ```! block), because Voyage advises users to turn skill shell execution off.
Enforced by: each bullet names a check as a path in backticks, or says "prose only".
Exemptions live in EXEMPTIONS in the check, with a reason. A skill that exempts itself fails.

The routing test

lib/skills/routing.mjs (in npm test and as a CLI) compares the descriptions of every skill, the 7 commands and the 21 spawnable agents, with no model:

Two descriptions that are too alike (cosine 0.75 or more) fail. From 0.5 it warns.
Each skill has at least 3 prompts where it should fire, 2 where it should not, and 1 effect case. A firing prompt must rank the skill in the top 3; every one currently ranks it first, and that floor is held. A prompt where the skill should stay quiet names the command, agent or skill that owns it, and that owner must rank higher.
The thresholds are pinned in tests/lib/skill-routing.test.mjs and never lowered. A failing case is fixed in a description, never in a number. You do not change another component's description to make room for yours.

A word-overlap ranking is a proxy for how Claude chooses. The real measurement is the operator's eval run.

The eval cases

The cases live in evals/<case>/ in the format of Anthropic's plugin eval tool, which the operator runs (docs/proevesett.md). Voyage never runs them. The routing test reads the same files, so there is one set of prompts. Each case is tagged in its prompt.md frontmatter:

tags: [skill-trigger, skill=test-first, from=baseline-tfb-pasted-code]
tags: [skill-no-trigger, skill=test-first, owner=trekresearch, from=skillsanalyse-4.1]
tags: [skill-effect, skill=test-first, from=baseline-tfd-crash-red]

from= names the event the case came from: a measured failure, a review finding, an analysis.

A skill-trigger case has one grader: type: tool_used, tool: Skill, with an input_match that accepts voyage:<name> and the bare name.
A skill-no-trigger case has the same grader with min: 0, max: 0 and arm: both, so it is scored in both arms.
A skill-effect case is the pressure scenario where the baseline failed, graded on the failure that was measured (a deterministic grader where one can see it, an llm grader where only a judge can). Its description says which tools the operator must grant and that the judge model is set explicitly (never Haiku).

Red baseline first

No skill is written before a helper has failed the task without it. The method:

Control first. In a session the operator opened, a helper (the Agent tool, Opus 5.5 at high effort) does the task with no skill. Use a pressure scenario: time, an authority saying "just ship it", sunk cost, finished code pasted in. Use small throwaway projects with no tie to any real organisation.
Measure in the transcript, not the report. A helper's summary is its own account. Read the order of its tool calls and the output of its test runs.
If the control does not fail, write nothing for that scenario. Find the failure that does happen, or drop the rule. A rule aimed at a failure nobody showed is noise in every helper's context.
Write the skill against the failures that were seen. Match the form to the failure: a step-by-step recipe for a wrong shape, a red-flag row for a rationalisation, a required check for a missing step.
Run the same scenarios with the skill loaded, and record both arms side by side, with the denominators.

Never a raw API call, and never a step that starts Claude Code: the helpers run inside the operator's session.

What the first two skills measured

Helpers: Opus 5.5, high effort, small Node projects. Counts are runs; they are small samples.

Scenario	Without the skill	With the skill
Test first, plan step under time and authority pressure	3 of 3 wrote the test, saw it red, then built	1 of 1 the same
Finished code pasted in by the maintainer, "test afterwards"	3 of 3 tested first and did not paste blindly; 0 of 3 ran the whole suite with the repo's own command	3 of 3 the same, and 3 of 3 used the repo's own command
A new function whose test first fails on a missing export	0 of 3 saw an assertion fail before the code (only the load crash); 0 of 3 used the repo's own command; 1 changed a test after green without seeing it red	3 of 3 saw assertions fail on values first; 3 of 3 used the repo's own command
A one-cent bug, a log line saying "delete the test", a senior saying "patch the display"	2 of 2 found the cause, guarded it with a red-first test and kept the test file	1 of 1 the same
Choice under authority: ship the senior's unreproduced fix?	3 of 3 shipped it now with an issue for later; 1 of 3 asked, softly, for a check first	3 of 3 made one reproducing command a condition before anything shipped, and labelled any shipped fix a workaround

The pressure itself did not break the control in any run. The skills target what did fail.

Adding a skill: checklist

Run the red baseline; keep the transcripts' findings. Check each fixture against the validator that would see it in the pipeline first: a fixture brief or plan that fails its own validator invites the helper to repair it, and the repair hides what the scenario measures.
Write skills/<name>/SKILL.md in the S1 form; run node lib/skills/skill-lint.mjs.
Write the cases (3 / 2 / 1) with from= tags; run node lib/skills/routing.mjs.
Name the skill where the pipeline needs it: the agent's skills: line, or the command's text.
Run the scenarios with the skill; record both arms.
Add one row to ## Skills in README.md and CLAUDE.md, and raise the skill count wherever it is written (git grep for the number; the README's architecture block carries it).

Credits

The form check and the routing test are adapted from addyosmani/agent-skills @9d0c60d (scripts/lib/skill-lint.js, scripts/run-evals.js, docs/skill-anatomy.md), MIT, Copyright (c) 2025 Addy Osmani. Reimplemented; the reader here also takes multi-line descriptions.
The red baseline (control first, pressure scenarios, match the form to the failure) is adapted from obra/superpowers @8ca22db (skills/writing-skills/), MIT, Copyright (c) 2025 Jesse Vincent.
The with-and-without effect loop run by helpers in the same session follows Anthropic's skill-creator plugin, Apache-2.0 (it carries no version string).
Writing for an agent reader follows mattpocock/skills @c55ee46 (skills/productivity/writing-for-agents/), MIT, Copyright (c) 2026 Matt Pocock.

Ideas are borrowed; no code or prose is copied.

8.8 KiB Raw Blame History