P10's model-free part (wave 4, order A1, decision 63). The routing test now
checks every case in evals/ for from=, not only the skill-tagged ones (R1):
a case with no skill tag is not routed, only checked, and the report gives
both counts ("45 cases checked for from=, 37 routed"). The eight test-set
cases get from=proevesett-eval-suite (d311f3b) or
from=proevesett-intent-gate (3892a83), the commits that added them.
The release gate that reads the last test-set run is written into the
runbook as exempted with its reason (it waits for the fixed test set to run
again) and still says where it goes (scripts/release-check.mjs).
review-630's prototype-tag stub gets from=test: under R1 an unrouted case is
still checked for from=, and the stub tests that a prototype name is never
routed, which stands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
247 lines
11 KiB
Markdown
247 lines
11 KiB
Markdown
# Release and rollback runbook
|
|
|
|
How a Voyage release goes out, and how it is rolled back. The rollback part is
|
|
not theory: every command in § 3 was run in a drill — a release 6.2.0 → 6.2.1
|
|
and both rollback paths, in a sandbox with local bare repositories as the
|
|
remotes — and `tests/lib/release-prose-pins.test.mjs` holds § 3 to the commands
|
|
recorded there (`tests/fixtures/release-drill/rollback-commands.txt`).
|
|
|
|
A release is a two-repo act. The plugin repo (this one) gets a tag `vX.Y.Z`;
|
|
the catalog repo pins each plugin by that tag (`source.ref` in
|
|
`.claude-plugin/marketplace.json`), so users get the release only when the
|
|
catalog ref moves. The catalog's `scripts/release-plugin.mjs` finds the plugin
|
|
as a sibling directory (`../voyage`), and both repos are run from their root.
|
|
|
|
Placeholders: `<previous-version>` is the release before (6.2.0 in the drill),
|
|
`<withdrawn-version>` the one rolled back (6.2.1), `<next-patch>` the next
|
|
unused patch number (6.2.2). `<release-commit>` is the release commit,
|
|
`<merge-commit>` a delivery merged since the previous tag, `<revert-commit>`
|
|
the last commit of the revert, and `<review.md>` the review of what is
|
|
released.
|
|
|
|
## Door types
|
|
|
|
Every delivery in a release says whether it is a **one-way door** or a
|
|
**two-way door**:
|
|
|
|
- **one-way** — cannot be walked back without a cost to users. A break in a
|
|
public contract (the brief format of Handover 1, a `plan_version` or
|
|
`review_version` that rejects files that used to validate), a removed command
|
|
or flag, anything users must migrate. A published tag is one-way too: it is
|
|
never deleted or moved.
|
|
- **two-way** — a revert restores the old behaviour and nobody has to do
|
|
anything: a new optional field, a new check, a fix, documentation.
|
|
|
|
At the release, the CHANGELOG section for the version carries one line per
|
|
delivery under `### Doors`:
|
|
|
|
```markdown
|
|
## v6.3.0 — 2026-10-15 — <title>
|
|
|
|
### Doors
|
|
|
|
- two-way: release check and runbook — a revert removes them, nothing reads them.
|
|
- one-way: <delivery> — <what a user must do, and from which version>.
|
|
```
|
|
|
|
The release is one-way if any line is. A one-way line says what users must
|
|
do. `scripts/release-check.mjs` refuses a release whose section has no
|
|
`### Doors` list, an empty one, or a line that is neither `- one-way:` nor
|
|
`- two-way:`.
|
|
|
|
## 1. Before the release
|
|
|
|
1. **Every delivery is merged into `main`** (`git merge --no-ff <branch>`), in
|
|
the agreed order. Each delivery's report states its door type.
|
|
2. **The suite is green on the merged tree.** `npm test` on `main` with every
|
|
delivery in it: 0 failed. A run that goes red only under heavy machine load
|
|
is still red — run it again on a quiet machine, never release over it.
|
|
Worktrees and branches stay until the release is out (§ 2.6): if the merged
|
|
tree is red, the branches are where the fix starts.
|
|
3. **A review of what is released, and its verdict is not BLOCK.** `/trekreview`
|
|
over the commits since the previous tag writes `<review.md>`; the verdict
|
|
gate (K7) decides on it:
|
|
|
|
```sh
|
|
node lib/review/verdict-gate.mjs <review.md>
|
|
```
|
|
|
|
0 = ALLOW or WARN, 1 = BLOCK, 2 = could not decide. Only 0 goes on.
|
|
4. **The release commit.** It changes only the version and the release notes:
|
|
`.claude-plugin/plugin.json`, `package.json`, `package-lock.json` (both
|
|
`version` fields), the README version badge, and a new CHANGELOG section
|
|
`## vX.Y.Z — <date> — <title>` with its `### Doors` list. Commit it as
|
|
`chore(release): X.Y.Z — <title>`.
|
|
5. **The release check.**
|
|
|
|
```sh
|
|
node scripts/release-check.mjs --review <review.md>
|
|
```
|
|
|
|
From another directory, add `--repo <dir>`: a relative `--review` is then
|
|
read in that directory, like every other path the check reads, or from
|
|
where you stand when only that file exists. When both exist and differ,
|
|
the check does not guess: exit 2, pass an absolute path.
|
|
|
|
It checks, in order: the working tree is clean (otherwise the suite would
|
|
not measure HEAD), `npm test` exits 0 at HEAD, the review verdict is not
|
|
BLOCK, the review covers HEAD (its `scope_sha_end` is HEAD or an ancestor,
|
|
and the commits after it touch only the release bookkeeping above and
|
|
Voyage's own artifacts under `.claude/projects/` and `.claude/voyage/`, such
|
|
as the review itself and the approval log), and the door type is set. 0 = ready, 1 = not ready (each failure listed), 2 = could
|
|
not run — never green.
|
|
6. **The operator's yes, logged.** The operator reads the release section and
|
|
says yes (an answer like "sounds good" is not a yes). The check prints the
|
|
command that logs it (K2):
|
|
|
|
```sh
|
|
node lib/util/approval-log.mjs append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md
|
|
```
|
|
|
|
The record is bound to the CHANGELOG as approved: if the release notes
|
|
change afterwards, `verify` reports `mismatch`. In this repo `.claude/` is
|
|
ignored by git (the remote is a public mirror), so the log stays local and
|
|
`verify` reports the record as not committed. That is the trace this repo
|
|
keeps; a target repo that tracks `.claude/voyage/` commits it.
|
|
|
|
## 2. The release
|
|
|
|
1. **Push the plugin last of all its steps:**
|
|
|
|
```sh
|
|
git push origin main
|
|
```
|
|
|
|
Voyage's push guard asks first; a yes to a push to `main` is logged as a
|
|
`merge` approval.
|
|
2. **In the catalog, the operator's one-shot push token.** `release-plugin.mjs`
|
|
pushes from inside node, where the command-text push guard cannot see it, so
|
|
it requires the same token the guard checks:
|
|
|
|
```sh
|
|
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
|
|
```
|
|
|
|
One token covers every push of ONE run. A second run needs a new token.
|
|
3. **Dry run.** It must stop on the missing tag and nothing else:
|
|
|
|
```sh
|
|
node scripts/release-plugin.mjs voyage
|
|
```
|
|
|
|
4. **The release, in one run.** `FORGEJO_TOKEN` must be exported (the script
|
|
says how when it is missing):
|
|
|
|
```sh
|
|
node scripts/release-plugin.mjs voyage --create-tag --write --commit --push
|
|
```
|
|
|
|
It refuses unless `plugin.json`, the README badge and the version agree.
|
|
It mints and pushes the tag, runs the catalog's version gate before
|
|
writing, bumps the ref and the catalog README label, runs the gate again,
|
|
commits and pushes the catalog, and files the Forgejo release object with
|
|
the CHANGELOG section as its text.
|
|
5. **The catalog's own check is green:**
|
|
|
|
```sh
|
|
node scripts/check-versions.mjs
|
|
```
|
|
|
|
6. **Cleanup comes last.** Merged worktrees and branches are removed only now,
|
|
after the suite was green on the merged tree (§ 1.2) and the release is out.
|
|
|
|
## 3. Rollback
|
|
|
|
The operator decides whether a release is rolled back. There are two paths,
|
|
and both were drilled. **A** takes the catalog back to the previous tag at
|
|
once. **B** ships the revert as a new patch version, so the version moves
|
|
forward again. The recommended order is A, then B.
|
|
|
|
### Path A — revert, and the catalog ref back
|
|
|
|
`release-plugin.mjs` accepts a ref backwards only when the plugin agrees with
|
|
it. Measured in the drill: with `plugin.json` at `<withdrawn-version>`,
|
|
`--version <previous-version>` is refused ("plugin.json version is 6.2.1,
|
|
asked to release 6.2.0 — bump plugin.json first"). So the plugin is reverted
|
|
first, and the release commit's revert puts `plugin.json` and the badge back.
|
|
|
|
In the plugin repo, revert the release commit, then each delivery merged since
|
|
the previous tag, newest first (`-m 1` keeps `main` as the mainline), and push:
|
|
|
|
```sh
|
|
git revert --no-edit <release-commit>
|
|
git revert --no-edit -m 1 <merge-commit>
|
|
git push -q origin main
|
|
```
|
|
|
|
In the catalog — between these steps the version gate reports a WARN (the ref
|
|
is ahead of `plugin.json`), not an ERROR:
|
|
|
|
```sh
|
|
node scripts/check-versions.mjs
|
|
node scripts/release-plugin.mjs voyage --version <previous-version>
|
|
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
|
|
node scripts/release-plugin.mjs voyage --version <previous-version> --write --commit --push
|
|
node scripts/check-versions.mjs
|
|
```
|
|
|
|
The dry run must say `v<withdrawn-version> -> v<previous-version> [READY]`.
|
|
|
|
What path A does not do:
|
|
- The tag `v<withdrawn-version>` stays, and so does its Forgejo release object;
|
|
the version number is used up. Edit the release object's text to say the
|
|
version is withdrawn (not drilled: it is the one step that needs the real
|
|
Forgejo instance).
|
|
- Whether a user who already updated to `<withdrawn-version>` is moved back to
|
|
`<previous-version>` by a plugin update is **not verified**. Path B does not
|
|
depend on it.
|
|
|
|
### Path B — a forward patch that carries the revert
|
|
|
|
On top of the revert, make a release commit for `<next-patch>` as in § 1.4,
|
|
with a CHANGELOG section that says `<withdrawn-version>` is withdrawn and a
|
|
`### Doors` line (`- two-way: withdraws <withdrawn-version> — …`). Its review
|
|
covers the revert, so `scope_sha_end` is `<revert-commit>`. Then the release as
|
|
usual:
|
|
|
|
```sh
|
|
git commit -q -am "chore(release): <next-patch> - withdraws <withdrawn-version>"
|
|
node scripts/release-check.mjs --review <review.md>
|
|
node lib/util/approval-log.mjs append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md
|
|
git push -q origin main
|
|
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
|
|
node scripts/release-plugin.mjs voyage --create-tag --write --commit --push
|
|
node scripts/check-versions.mjs
|
|
```
|
|
|
|
The first four lines run in the plugin repo, the last three in the catalog.
|
|
|
|
### Bringing a reverted delivery back
|
|
|
|
Do not merge its branch again. git's own documentation for `revert -m`:
|
|
"Reverting a merge commit declares that you will never want the tree changes
|
|
brought in by the merge. As a result, later merges will only bring in tree
|
|
changes introduced by commits that are not ancestors of the previously
|
|
reverted merge." Revert the revert commit instead, and release it as a new
|
|
version.
|
|
|
|
## Exempted: the eval gate (P10)
|
|
|
|
The local gate before a release that reads the date and score of the last run
|
|
of the fixed test set and compares it with the last change in the
|
|
configuration paths (commands, agents, skills, hooks, and a changed model alias
|
|
counts as a change). It is exempted from the release, with a reason: it waits
|
|
for the fixed test set to run again, and a gate that can never fire is not
|
|
built. P10's model-free part is in place: every case in `evals/` names the
|
|
event it came from (`from=`), and the routing test (`lib/skills/routing.mjs`,
|
|
run by `npm test`) fails a case without it. When the gate comes, it is one more
|
|
check in `scripts/release-check.mjs`, before the door check, and one more step
|
|
in § 1.
|
|
|
|
## Credits
|
|
|
|
- The runbook form: [bashebr/ai-native-sdlc](https://github.com/bashebr/ai-native-sdlc) (MIT).
|
|
- Green on the merged tree before cleanup: [obra/superpowers](https://github.com/obra/superpowers) (MIT), at `8ca22db`.
|
|
- One-way and two-way doors: [mattpocock/skills](https://github.com/mattpocock/skills), at `c55ee46`.
|
|
|
|
Ideas only; no text is copied.
|