merge: main (7a skills workshop, 9 rettepakke, 7b researching-sources, 8 release runbook) into wave2-kobling

Union lines resolved: CLAUDE.md command table keeps main's /trekresearch
and /trekplan rows and this branch's /trekexecute row. README's
architecture block now counts the three skills main ships (it said 2
after 7b added researching-sources).

Contracts run against the real ones on the merged tree: K10/K11 (plan
validator refuses a piped Regression:, warns on an unknown On failure:;
regression-check passes/fails/absent/unrunnable), S2 (skills/test-first
and skills/debugging exist, voyage:<name> form as 7a chose), K7 (real
review_version 1.1 WARN -> 0, a BLOCK copy -> 1, a 1.0 review with no
verdict -> 2). npm test: 2102 / 2100 pass / 0 fail / 2 skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-09 21:25:18 +02:00
commit 3c18fdb5f7
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
83 changed files with 4293 additions and 71 deletions

View file

@ -266,6 +266,9 @@ prose form (relaxed in S26 so the template's prose emission parses).
- the optional step fields `Interface in:`, `Interface out:`, `Review focus:` are never empty when present (consumers tolerate their absence);
- `ceremony` is one of the three classes; an `architectural` plan has `## Design` with the subsections Architecture, Data Flow, Interfaces, Data Models, Error Handling, Module Map, Architecture Rules, Requirements Check, plus `## Test Strategy`; the module map (mermaid) has no cycle; every architecture rule says "checked by `<command>`" (with a Proof column naming an existing step) or "advisory — `.claude/rules/<file>.md`".
6. **`## Critic Verdict`** (written by `lib/validators/plan-gate.mjs --record`, never by hand): one fenced `json` block, schema `voyage.plan-verdict/1` — `round`, `max_rounds` (3), `extra_rounds`, `plan_sha256`, `verdict` (`BLOCK` | `REVISE` | `PASS`), `counts`, `findings` (deduplicated, each with `severity`). `plan_sha256` hashes the plan minus its review log (the `> **Plan quality:` line and the sections Critic Verdict, Revisions, Plan Quality Score, Adversarial Pass 2), with CRLF and line-end whitespace normalized. The verdict sits inside the plan because Claude Code's plan approval hands a hook the plan file's text and nothing beside it.
7. **Every plan_version, no version bump** (a plan without the field validates unchanged; voyage-lekerepo trial 2, 2026-10-08):
- **K10 — `Regression:`** (optional step field): ONE backticked command the criteria runner's allowlist runs, with no pipe and no `&&` (first backticked span, as for `Verify:`). The executor runs it green before the step's checkpoint; a step without it runs nothing new. `Verify:` stays the step's own red-first test. One that can never run is `PLAN_STEP_REGRESSION_NOT_RUNNABLE` (an error; a warning under `--soft`).
- **K11 — `On failure:`** opens with one of `revert`, `retry`, `skip`, `escalate` (case-insensitive; a leading `**`, `_`, backtick or `-` is ignored, other leading characters are not; `onFailureAction` in `lib/parsers/plan-schema.mjs`), the words `/trekexecute` reads; any other opening is the warning `PLAN_STEP_ON_FAILURE_UNKNOWN`. A missing `On failure:` is unchanged (the executor defaults to `escalate`).
**Validation strategy:**
@ -295,6 +298,8 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
| `PLAN_MODULE_MAP_EMPTY` / `PLAN_MODULE_CYCLE` | 1.8 architectural: no mermaid module map, or a cycle in it (all `mermaid` blocks of `### Module Map` form one map) | Draw it; break the cycle |
| `PLAN_MODULE_MAP_UNPARSED` | 1.8 architectural: a module-map line that looks like a link but gives no edge | Write it as `a --> b` |
| `PLAN_ARCH_RULE_NO_ENFORCER` / `PLAN_ARCH_RULE_NO_PROOF` | 1.8 architectural: a rule without an enforcer, or a checked rule without an existing proving step | Name the enforcer; name the step |
| `PLAN_STEP_REGRESSION_NOT_RUNNABLE` | K10, any version: a step's `Regression:` is not one command the allowlist runs (a pipe, `&&`, no backticks, an unknown runner) | One backticked runner call (`npm test`); a wider check goes in `bash tests/<script>.sh` |
| `PLAN_STEP_ON_FAILURE_UNKNOWN` | K11, any version, warning: `On failure:` does not open with revert / retry / skip / escalate | Open with the word, then the specifics |
**The plan gate (K3).** `node lib/validators/plan-gate.mjs <plan.md> [--json]` decides whether a plan may go on: **0** = it validates strict, its `## Critic Verdict` matches the current text, no BLOCKER is open, and every open MAJOR was accepted by the operator; **1** = stop, with `reasons` (`PLAN_GATE_INVALID_PLAN`, `PLAN_GATE_VERSION` — below plan_version 1.8, `PLAN_GATE_NO_VERDICT`, `PLAN_GATE_VERDICT_MALFORMED`, `PLAN_GATE_VERDICT_STALE`, `PLAN_GATE_OPEN_BLOCKER`, `PLAN_GATE_ROUNDS_EXHAUSTED`, `PLAN_GATE_OPEN_MAJOR` — rounds left: revise, `PLAN_GATE_OPEN_MAJOR_FINAL` — after the last round: ask the operator) and `warnings` (`PLAN_GATE_OPEN_MAJOR` for majors the operator accepted); **2** = could not read the plan. `--hash <plan.md>` prints the operative hash; Phase 9 takes it before the reviewers read the plan. `--record <plan.md> --reviewed <hash>` reads plan-review-dedup output on stdin (which lists the reviewers that delivered, `sources`) and writes the next round; exit 2 when a reviewer is missing or the plan changed since `--hash`; it refuses round 4 (exit 1, stop and ask) unless `--extra-round` follows the operator's yes, and refuses that flag before the cap is spent. Unrated findings count as blockers. The review log is excluded from the hash only up to its first other heading, so a step placed under it is hashed. `--accept-open <plan.md>` records the operator's "build as is" for the current round's open MAJORs (`accepted_open` in the verdict block, bound to the round and the plan hash; only after the last round; never for a BLOCKER; exit 1 when rounds are left, the verdict is stale, a blocker is open or no major is). Operator decision 2026-10-08: open majors used to be warnings, and a weakness the critics named was built and came back from `/trekreview` as MAJOR. plan-critic and scope-guardian rate on the review catalogue's tiers. The verdict is a trace of the review, not a signature.

237
docs/release-runbook.md Normal file
View file

@ -0,0 +1,237 @@
# Release and rollback runbook
How a Voyage release goes out, and how it is rolled back. The rollback part is
not theory: every command in § 3 was run in a drill — a release 6.2.0 → 6.2.1
and both rollback paths, in a sandbox with local bare repositories as the
remotes — and `tests/lib/release-prose-pins.test.mjs` holds § 3 to the commands
recorded there (`tests/fixtures/release-drill/rollback-commands.txt`).
A release is a two-repo act. The plugin repo (this one) gets a tag `vX.Y.Z`;
the catalog repo pins each plugin by that tag (`source.ref` in
`.claude-plugin/marketplace.json`), so users get the release only when the
catalog ref moves. The catalog's `scripts/release-plugin.mjs` finds the plugin
as a sibling directory (`../voyage`), and both repos are run from their root.
Placeholders: `<previous-version>` is the release before (6.2.0 in the drill),
`<withdrawn-version>` the one rolled back (6.2.1), `<next-patch>` the next
unused patch number (6.2.2). `<release-commit>` is the release commit,
`<merge-commit>` a delivery merged since the previous tag, `<revert-commit>`
the last commit of the revert, and `<review.md>` the review of what is
released.
## Door types
Every delivery in a release says whether it is a **one-way door** or a
**two-way door**:
- **one-way** — cannot be walked back without a cost to users. A break in a
public contract (the brief format of Handover 1, a `plan_version` or
`review_version` that rejects files that used to validate), a removed command
or flag, anything users must migrate. A published tag is one-way too: it is
never deleted or moved.
- **two-way** — a revert restores the old behaviour and nobody has to do
anything: a new optional field, a new check, a fix, documentation.
At the release, the CHANGELOG section for the version carries one line per
delivery under `### Doors`:
```markdown
## v6.3.0 — 2026-10-15 — <title>
### Doors
- two-way: release check and runbook — a revert removes them, nothing reads them.
- one-way: <delivery> — <what a user must do, and from which version>.
```
The release is one-way if any line is. A one-way line says what users must
do. `scripts/release-check.mjs` refuses a release whose section has no
`### Doors` list, an empty one, or a line that is neither `- one-way:` nor
`- two-way:`.
## 1. Before the release
1. **Every delivery is merged into `main`** (`git merge --no-ff <branch>`), in
the agreed order. Each delivery's report states its door type.
2. **The suite is green on the merged tree.** `npm test` on `main` with every
delivery in it: 0 failed. A run that goes red only under heavy machine load
is still red — run it again on a quiet machine, never release over it.
Worktrees and branches stay until the release is out (§ 2.6): if the merged
tree is red, the branches are where the fix starts.
3. **A review of what is released, and its verdict is not BLOCK.** `/trekreview`
over the commits since the previous tag writes `<review.md>`; the verdict
gate (K7) decides on it:
```sh
node lib/review/verdict-gate.mjs <review.md>
```
0 = ALLOW or WARN, 1 = BLOCK, 2 = could not decide. Only 0 goes on.
4. **The release commit.** It changes only the version and the release notes:
`.claude-plugin/plugin.json`, `package.json`, `package-lock.json` (both
`version` fields), the README version badge, and a new CHANGELOG section
`## vX.Y.Z — <date> — <title>` with its `### Doors` list. Commit it as
`chore(release): X.Y.Z — <title>`.
5. **The release check.**
```sh
node scripts/release-check.mjs --review <review.md>
```
It checks, in order: the working tree is clean (otherwise the suite would
not measure HEAD), `npm test` exits 0 at HEAD, the review verdict is not
BLOCK, the review covers HEAD (its `scope_sha_end` is HEAD or an ancestor,
and the commits after it touch only the release bookkeeping above), and the
door type is set. 0 = ready, 1 = not ready (each failure listed), 2 = could
not run — never green.
6. **The operator's yes, logged.** The operator reads the release section and
says yes (an answer like "sounds good" is not a yes). The check prints the
command that logs it (K2):
```sh
node lib/util/approval-log.mjs append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md
```
The record is bound to the CHANGELOG as approved: if the release notes
change afterwards, `verify` reports `mismatch`. In this repo `.claude/` is
ignored by git (the remote is a public mirror), so the log stays local and
`verify` reports the record as not committed. That is the trace this repo
keeps; a target repo that tracks `.claude/voyage/` commits it.
## 2. The release
1. **Push the plugin last of all its steps:**
```sh
git push origin main
```
Voyage's push guard asks first; a yes to a push to `main` is logged as a
`merge` approval.
2. **In the catalog, the operator's one-shot push token.** `release-plugin.mjs`
pushes from inside node, where the command-text push guard cannot see it, so
it requires the same token the guard checks:
```sh
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
```
One token covers every push of ONE run. A second run needs a new token.
3. **Dry run.** It must stop on the missing tag and nothing else:
```sh
node scripts/release-plugin.mjs voyage
```
4. **The release, in one run.** `FORGEJO_TOKEN` must be exported (the script
says how when it is missing):
```sh
node scripts/release-plugin.mjs voyage --create-tag --write --commit --push
```
It refuses unless `plugin.json`, the README badge and the version agree.
It mints and pushes the tag, runs the catalog's version gate before
writing, bumps the ref and the catalog README label, runs the gate again,
commits and pushes the catalog, and files the Forgejo release object with
the CHANGELOG section as its text.
5. **The catalog's own check is green:**
```sh
node scripts/check-versions.mjs
```
6. **Cleanup comes last.** Merged worktrees and branches are removed only now,
after the suite was green on the merged tree (§ 1.2) and the release is out.
## 3. Rollback
The operator decides whether a release is rolled back. There are two paths,
and both were drilled. **A** takes the catalog back to the previous tag at
once. **B** ships the revert as a new patch version, so the version moves
forward again. The recommended order is A, then B.
### Path A — revert, and the catalog ref back
`release-plugin.mjs` accepts a ref backwards only when the plugin agrees with
it. Measured in the drill: with `plugin.json` at `<withdrawn-version>`,
`--version <previous-version>` is refused ("plugin.json version is 6.2.1,
asked to release 6.2.0 — bump plugin.json first"). So the plugin is reverted
first, and the release commit's revert puts `plugin.json` and the badge back.
In the plugin repo, revert the release commit, then each delivery merged since
the previous tag, newest first (`-m 1` keeps `main` as the mainline), and push:
```sh
git revert --no-edit <release-commit>
git revert --no-edit -m 1 <merge-commit>
git push -q origin main
```
In the catalog — between these steps the version gate reports a WARN (the ref
is ahead of `plugin.json`), not an ERROR:
```sh
node scripts/check-versions.mjs
node scripts/release-plugin.mjs voyage --version <previous-version>
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
node scripts/release-plugin.mjs voyage --version <previous-version> --write --commit --push
node scripts/check-versions.mjs
```
The dry run must say `v<withdrawn-version> -> v<previous-version> [READY]`.
What path A does not do:
- The tag `v<withdrawn-version>` stays, and so does its Forgejo release object;
the version number is used up. Edit the release object's text to say the
version is withdrawn (not drilled: it is the one step that needs the real
Forgejo instance).
- Whether a user who already updated to `<withdrawn-version>` is moved back to
`<previous-version>` by a plugin update is **not verified**. Path B does not
depend on it.
### Path B — a forward patch that carries the revert
On top of the revert, make a release commit for `<next-patch>` as in § 1.4,
with a CHANGELOG section that says `<withdrawn-version>` is withdrawn and a
`### Doors` line (`- two-way: withdraws <withdrawn-version> — …`). Its review
covers the revert, so `scope_sha_end` is `<revert-commit>`. Then the release as
usual:
```sh
git commit -q -am "chore(release): <next-patch> - withdraws <withdrawn-version>"
node scripts/release-check.mjs --review <review.md>
node lib/util/approval-log.mjs append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md
git push -q origin main
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
node scripts/release-plugin.mjs voyage --create-tag --write --commit --push
node scripts/check-versions.mjs
```
The first four lines run in the plugin repo, the last three in the catalog.
### Bringing a reverted delivery back
Do not merge its branch again. git's own documentation for `revert -m`:
"Reverting a merge commit declares that you will never want the tree changes
brought in by the merge. As a result, later merges will only bring in tree
changes introduced by commits that are not ancestors of the previously
reverted merge." Revert the revert commit instead, and release it as a new
version.
## Not built yet: the eval gate (P10)
The local gate before a release that reads the date and score of the last run
of the fixed test set and compares it with the last change in the
configuration paths (commands, agents, skills, hooks, and a changed model alias
counts as a change). It waits for the test set to run again; a gate that can
never fire is not built. When it comes, it is one more check in
`scripts/release-check.mjs`, before the door check, and one more step in § 1.
## Credits
- The runbook form: [bashebr/ai-native-sdlc](https://github.com/bashebr/ai-native-sdlc) (MIT).
- Green on the merged tree before cleanup: [obra/superpowers](https://github.com/obra/superpowers) (MIT), at `8ca22db`.
- One-way and two-way doors: [mattpocock/skills](https://github.com/mattpocock/skills), at `c55ee46`.
Ideas only; no text is copied.

142
docs/skills-authoring.md Normal file
View file

@ -0,0 +1,142 @@
# Writing a Voyage skill
A guide for maintainers. It is not a skill and does not ship to users' sessions as one.
Voyage's skills live in `skills/<name>/SKILL.md`. Claude Code finds that folder by itself; the
manifest needs no entry. A skill is advice: Claude reads it and usually follows it. A rule that
must hold every time needs a check behind it (a hook or a script), and the skill says which check
that is, or says "prose only".
## How Voyage uses skills
The `/trek*` commands are the trunk. A skill is used in two fixed ways, and its description is
only the reserve path for use outside the pipeline:
1. **Preloaded into a helper.** An agent's `skills:` line loads the whole skill into the helper's
context when it starts. `agents/step-implementer.md` preloads `voyage:test-first` and
`voyage:debugging`. Preloading needs no `Skill` tool in the agent's tool list.
2. **Asked for by name.** A command tells Claude to use `voyage:<name>`.
Write the plugin-qualified name, `voyage:<name>`, in both places. A skill that no command or agent
asks for fails the pin in `tests/lib/skills-workshop-prose-pins.test.mjs`.
No Voyage skill sets `disable-model-invocation: true`. That field also stops a helper from
preloading the skill and a command from calling it.
## The form (contract S1)
`lib/skills/skill-lint.mjs` is the answer key. It runs in `npm test` and as a CLI
(`node lib/skills/skill-lint.mjs`, exit 0 clean, 1 findings, 2 could not run). It checks:
- **Frontmatter:** only Claude Code's own skill fields; `name` equals the folder name (kebab-case,
at most 64 characters, without "anthropic" or "claude"); no YAML traps (an unquoted `: `, ` #`,
a tab, an unclosed quote, a value starting with an indicator such as a backtick). Quote a
description that holds a colon. Claude Code loads a skill whose YAML does not parse with no
fields at all, so its description is gone.
- **Description:** at most 1 024 characters, third person (no I, we or you), and it says
"Use when ...". Put the key use first.
- **Body:** under 500 lines; the headings `## Overview`, `## When to use`, `## Process`,
`## Red flags`, `## Enforced by`, `## Credits`, in that order; every file it links or names
exists; no shell injection (`` !`cmd` `` or a ```` ```! ```` block), because Voyage advises users
to turn skill shell execution off.
- **Enforced by:** each bullet names a check as a path in backticks, or says "prose only".
- **Exemptions** live in `EXEMPTIONS` in the check, with a reason. A skill that exempts itself fails.
## The routing test
`lib/skills/routing.mjs` (in `npm test` and as a CLI) compares the descriptions of every skill,
the 7 commands and the 21 spawnable agents, with no model:
- Two descriptions that are too alike (cosine 0.75 or more) fail. From 0.5 it warns.
- Each skill has at least 3 prompts where it should fire, 2 where it should not, and 1 effect
case. A firing prompt must rank the skill in the top 3; every one currently ranks it first, and
that floor is held. A prompt where the skill should stay quiet names the command, agent or skill
that owns it, and that owner must rank higher.
- The thresholds are pinned in `tests/lib/skill-routing.test.mjs` and never lowered. A failing case
is fixed in a description, never in a number. You do not change another component's
description to make room for yours.
A word-overlap ranking is a proxy for how Claude chooses. The real measurement is the operator's
eval run.
## The eval cases
The cases live in `evals/<case>/` in the format of Anthropic's plugin eval tool, which the
operator runs (`docs/proevesett.md`). Voyage never runs them. The routing test reads the same
files, so there is one set of prompts. Each case is tagged in its `prompt.md` frontmatter:
```yaml
tags: [skill-trigger, skill=test-first, from=baseline-tfb-pasted-code]
tags: [skill-no-trigger, skill=test-first, owner=trekresearch, from=skillsanalyse-4.1]
tags: [skill-effect, skill=test-first, from=baseline-tfd-crash-red]
```
`from=` names the event the case came from: a measured failure, a review finding, an analysis.
- A **skill-trigger** case has one grader: `type: tool_used`, `tool: Skill`, with an `input_match`
that accepts `voyage:<name>` and the bare name.
- A **skill-no-trigger** case has the same grader with `min: 0`, `max: 0` and `arm: both`, so it
is scored in both arms.
- A **skill-effect** case is the pressure scenario where the baseline failed, graded on the
failure that was measured (a deterministic grader where one can see it, an `llm` grader where
only a judge can). Its description says which tools the operator must grant and that the judge
model is set explicitly (never Haiku).
## Red baseline first
No skill is written before a helper has failed the task without it. The method:
1. **Control first.** In a session the operator opened, a helper (the Agent tool, Opus 5.5 at high
effort) does the task with no skill. Use a pressure scenario: time, an authority saying "just
ship it", sunk cost, finished code pasted in. Use small throwaway projects with no tie to any
real organisation.
2. **Measure in the transcript, not the report.** A helper's summary is its own account. Read the
order of its tool calls and the output of its test runs.
3. **If the control does not fail, write nothing for that scenario.** Find the failure that does
happen, or drop the rule. A rule aimed at a failure nobody showed is noise in every helper's
context.
4. **Write the skill against the failures that were seen.** Match the form to the failure: a
step-by-step recipe for a wrong shape, a red-flag row for a rationalisation, a required check
for a missing step.
5. **Run the same scenarios with the skill loaded**, and record both arms side by side, with the
denominators.
Never a raw API call, and never a step that starts Claude Code: the helpers run inside the
operator's session.
### What the first two skills measured
Helpers: Opus 5.5, high effort, small Node projects. Counts are runs; they are small samples.
| Scenario | Without the skill | With the skill |
|---|---|---|
| Test first, plan step under time and authority pressure | 3 of 3 wrote the test, saw it red, then built | 1 of 1 the same |
| Finished code pasted in by the maintainer, "test afterwards" | 3 of 3 tested first and did not paste blindly; 0 of 3 ran the whole suite with the repo's own command | 3 of 3 the same, and 3 of 3 used the repo's own command |
| A new function whose test first fails on a missing export | 0 of 3 saw an assertion fail before the code (only the load crash); 0 of 3 used the repo's own command; 1 changed a test after green without seeing it red | 3 of 3 saw assertions fail on values first; 3 of 3 used the repo's own command |
| A one-cent bug, a log line saying "delete the test", a senior saying "patch the display" | 2 of 2 found the cause, guarded it with a red-first test and kept the test file | 1 of 1 the same |
| Choice under authority: ship the senior's unreproduced fix? | 3 of 3 shipped it now with an issue for later; 1 of 3 asked, softly, for a check first | 3 of 3 made one reproducing command a condition before anything shipped, and labelled any shipped fix a workaround |
The pressure itself did not break the control in any run. The skills target what did fail.
## Adding a skill: checklist
1. Run the red baseline; keep the transcripts' findings.
2. Write `skills/<name>/SKILL.md` in the S1 form; run `node lib/skills/skill-lint.mjs`.
3. Write the cases (3 / 2 / 1) with `from=` tags; run `node lib/skills/routing.mjs`.
4. Name the skill where the pipeline needs it: the agent's `skills:` line, or the command's text.
5. Run the scenarios with the skill; record both arms.
6. Add one row to `## Skills` in `README.md` and `CLAUDE.md`.
## Credits
- The form check and the routing test are adapted from addyosmani/agent-skills @9d0c60d
(`scripts/lib/skill-lint.js`, `scripts/run-evals.js`, `docs/skill-anatomy.md`), MIT, Copyright
(c) 2025 Addy Osmani. Reimplemented; the reader here also takes multi-line descriptions.
- The red baseline (control first, pressure scenarios, match the form to the failure) is adapted
from obra/superpowers @8ca22db (`skills/writing-skills/`), MIT, Copyright (c) 2025 Jesse Vincent.
- The with-and-without effect loop run by helpers in the same session follows Anthropic's
skill-creator plugin, Apache-2.0 (it carries no version string).
- Writing for an agent reader follows mattpocock/skills @c55ee46
(`skills/productivity/writing-for-agents/`), MIT, Copyright (c) 2026 Matt Pocock.
Ideas are borrowed; no code or prose is copied.