open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

merge(wave2): 8 release check and rollback runbook

Browse source 
Order 8 of wave 2: scripts/release-check.mjs (tree, suite, review
verdict, review coverage, door type; 0 ready / 1 not ready / 2 could not
run) and docs/release-runbook.md with both rollback paths drilled in a
sandbox. Branch suite 2085 (2083/0/2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-09 19:55:06 +02:00
commit ecf305014b
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
7 changed files with 985 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
CLAUDE.md
View file

@ -135,3 +135,6 @@ Full flag reference for each command (modes, `--gates`, `--profile`, breaking ch
- **Profile system (`--profile economy/balanced/premium/fable`), lookup order, custom profiles:** `docs/operations.md`
- **Observability (Stop hook, OTLP/textfile export, SSRF mitigation):** `docs/operations.md`
- **Handover contracts (the 7 pipeline handovers):** `docs/HANDOVER-CONTRACTS.md`
- **Release and rollback (P13):** `docs/release-runbook.md` — the drilled runbook (before / release / rollback).
`scripts/release-check.mjs --review <review.md>` decides before a release: 0 ready, 1 not ready, 2 never green.
Every delivery states its door type (one-way / two-way) under `### Doors` in the CHANGELOG section.

4
README.md
View file

@ -858,6 +858,10 @@ An opt-in upstream architect plugin (not bundled) can insert a Claude-Code-speci
Migration notes for breaking changes (v1.x → v2.0, v2.x → v3.0) live in [CHANGELOG.md](CHANGELOG.md) and [MIGRATION.md](MIGRATION.md). v3.x non-breaking — minor bumps within v3 add features without changing pipeline contracts.
## Releasing and rolling back
A Voyage release follows [docs/release-runbook.md](docs/release-runbook.md): before the release, the release itself, and the rollback. `node scripts/release-check.mjs --review <review.md>` decides before anything is pushed — the suite is green at HEAD, the review verdict for what is released is not BLOCK and covers it, and every delivery says whether it is a one-way or a two-way door (`### Doors` in the CHANGELOG section). 0 = ready, 1 = not ready, 2 = could not run (never green). The rollback (revert, then the catalog ref back, then a forward patch) was drilled in a sandbox before it was written down.
## Quality infrastructure (since v3.1.0)
The plugin ships with `node:test`-based unit tests and a `lib/` directory of pure-JS validators wired into the commands. Forking the plugin for internal use? Run `npm test` to confirm the parsers, validators, and doc-consistency invariants still hold:

237
docs/release-runbook.md Normal file
View file

@ -0,0 +1,237 @@
# Release and rollback runbook
How a Voyage release goes out, and how it is rolled back. The rollback part is
not theory: every command in § 3 was run in a drill — a release 6.2.0 → 6.2.1
and both rollback paths, in a sandbox with local bare repositories as the
remotes — and `tests/lib/release-prose-pins.test.mjs` holds § 3 to the commands
recorded there (`tests/fixtures/release-drill/rollback-commands.txt`).
A release is a two-repo act. The plugin repo (this one) gets a tag `vX.Y.Z`;
the catalog repo pins each plugin by that tag (`source.ref` in
`.claude-plugin/marketplace.json`), so users get the release only when the
catalog ref moves. The catalog's `scripts/release-plugin.mjs` finds the plugin
as a sibling directory (`../voyage`), and both repos are run from their root.
Placeholders: `<previous-version>` is the release before (6.2.0 in the drill),
`<withdrawn-version>` the one rolled back (6.2.1), `<next-patch>` the next
unused patch number (6.2.2). `<release-commit>` is the release commit,
`<merge-commit>` a delivery merged since the previous tag, `<revert-commit>`
the last commit of the revert, and `<review.md>` the review of what is
released.
## Door types
Every delivery in a release says whether it is a **one-way door** or a
**two-way door**:
- **one-way** — cannot be walked back without a cost to users. A break in a
public contract (the brief format of Handover 1, a `plan_version` or
`review_version` that rejects files that used to validate), a removed command
or flag, anything users must migrate. A published tag is one-way too: it is
never deleted or moved.
- **two-way** — a revert restores the old behaviour and nobody has to do
anything: a new optional field, a new check, a fix, documentation.
At the release, the CHANGELOG section for the version carries one line per
delivery under `### Doors`:
```markdown
## v6.3.0 — 2026-10-15 — <title>
### Doors
- two-way: release check and runbook — a revert removes them, nothing reads them.
- one-way: <delivery> — <what a user must do, and from which version>.
```
The release is one-way if any line is. A one-way line says what users must
do. `scripts/release-check.mjs` refuses a release whose section has no
`### Doors` list, an empty one, or a line that is neither `- one-way:` nor
`- two-way:`.
## 1. Before the release
1. **Every delivery is merged into `main`** (`git merge --no-ff <branch>`), in
the agreed order. Each delivery's report states its door type.
2. **The suite is green on the merged tree.** `npm test` on `main` with every
delivery in it: 0 failed. A run that goes red only under heavy machine load
is still red — run it again on a quiet machine, never release over it.
Worktrees and branches stay until the release is out (§ 2.6): if the merged
tree is red, the branches are where the fix starts.
3. **A review of what is released, and its verdict is not BLOCK.** `/trekreview`
over the commits since the previous tag writes `<review.md>`; the verdict
gate (K7) decides on it:
```sh
node lib/review/verdict-gate.mjs <review.md>
```
0 = ALLOW or WARN, 1 = BLOCK, 2 = could not decide. Only 0 goes on.
4. **The release commit.** It changes only the version and the release notes:
`.claude-plugin/plugin.json`, `package.json`, `package-lock.json` (both
`version` fields), the README version badge, and a new CHANGELOG section
`## vX.Y.Z — <date> — <title>` with its `### Doors` list. Commit it as
`chore(release): X.Y.Z — <title>`.
5. **The release check.**
```sh
node scripts/release-check.mjs --review <review.md>
```
It checks, in order: the working tree is clean (otherwise the suite would
not measure HEAD), `npm test` exits 0 at HEAD, the review verdict is not
BLOCK, the review covers HEAD (its `scope_sha_end` is HEAD or an ancestor,
and the commits after it touch only the release bookkeeping above), and the
door type is set. 0 = ready, 1 = not ready (each failure listed), 2 = could
not run — never green.
6. **The operator's yes, logged.** The operator reads the release section and
says yes (an answer like "sounds good" is not a yes). The check prints the
command that logs it (K2):
```sh
node lib/util/approval-log.mjs append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md
```
The record is bound to the CHANGELOG as approved: if the release notes
change afterwards, `verify` reports `mismatch`. In this repo `.claude/` is
ignored by git (the remote is a public mirror), so the log stays local and
`verify` reports the record as not committed. That is the trace this repo
keeps; a target repo that tracks `.claude/voyage/` commits it.
## 2. The release
1. **Push the plugin last of all its steps:**
```sh
git push origin main
```
Voyage's push guard asks first; a yes to a push to `main` is logged as a
`merge` approval.
2. **In the catalog, the operator's one-shot push token.** `release-plugin.mjs`
pushes from inside node, where the command-text push guard cannot see it, so
it requires the same token the guard checks:
```sh
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
```
One token covers every push of ONE run. A second run needs a new token.
3. **Dry run.** It must stop on the missing tag and nothing else:
```sh
node scripts/release-plugin.mjs voyage
```
4. **The release, in one run.** `FORGEJO_TOKEN` must be exported (the script
says how when it is missing):
```sh
node scripts/release-plugin.mjs voyage --create-tag --write --commit --push
```
It refuses unless `plugin.json`, the README badge and the version agree.
It mints and pushes the tag, runs the catalog's version gate before
writing, bumps the ref and the catalog README label, runs the gate again,
commits and pushes the catalog, and files the Forgejo release object with
the CHANGELOG section as its text.
5. **The catalog's own check is green:**
```sh
node scripts/check-versions.mjs
```
6. **Cleanup comes last.** Merged worktrees and branches are removed only now,
after the suite was green on the merged tree (§ 1.2) and the release is out.
## 3. Rollback
The operator decides whether a release is rolled back. There are two paths,
and both were drilled. **A** takes the catalog back to the previous tag at
once. **B** ships the revert as a new patch version, so the version moves
forward again. The recommended order is A, then B.
### Path A — revert, and the catalog ref back
`release-plugin.mjs` accepts a ref backwards only when the plugin agrees with
it. Measured in the drill: with `plugin.json` at `<withdrawn-version>`,
`--version <previous-version>` is refused ("plugin.json version is 6.2.1,
asked to release 6.2.0 — bump plugin.json first"). So the plugin is reverted
first, and the release commit's revert puts `plugin.json` and the badge back.
In the plugin repo, revert the release commit, then each delivery merged since
the previous tag, newest first (`-m 1` keeps `main` as the mainline), and push:
```sh
git revert --no-edit <release-commit>
git revert --no-edit -m 1 <merge-commit>
git push -q origin main
```
In the catalog — between these steps the version gate reports a WARN (the ref
is ahead of `plugin.json`), not an ERROR:
```sh
node scripts/check-versions.mjs
node scripts/release-plugin.mjs voyage --version <previous-version>
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
node scripts/release-plugin.mjs voyage --version <previous-version> --write --commit --push
node scripts/check-versions.mjs
```
The dry run must say `v<withdrawn-version> -> v<previous-version> [READY]`.
What path A does not do:
- The tag `v<withdrawn-version>` stays, and so does its Forgejo release object;
the version number is used up. Edit the release object's text to say the
version is withdrawn (not drilled: it is the one step that needs the real
Forgejo instance).
- Whether a user who already updated to `<withdrawn-version>` is moved back to
`<previous-version>` by a plugin update is **not verified**. Path B does not
depend on it.
### Path B — a forward patch that carries the revert
On top of the revert, make a release commit for `<next-patch>` as in § 1.4,
with a CHANGELOG section that says `<withdrawn-version>` is withdrawn and a
`### Doors` line (`- two-way: withdraws <withdrawn-version> — …`). Its review
covers the revert, so `scope_sha_end` is `<revert-commit>`. Then the release as
usual:
```sh
git commit -q -am "chore(release): <next-patch> - withdraws <withdrawn-version>"
node scripts/release-check.mjs --review <review.md>
node lib/util/approval-log.mjs append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md
git push -q origin main
mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
node scripts/release-plugin.mjs voyage --create-tag --write --commit --push
node scripts/check-versions.mjs
```
The first four lines run in the plugin repo, the last three in the catalog.
### Bringing a reverted delivery back
Do not merge its branch again. git's own documentation for `revert -m`:
"Reverting a merge commit declares that you will never want the tree changes
brought in by the merge. As a result, later merges will only bring in tree
changes introduced by commits that are not ancestors of the previously
reverted merge." Revert the revert commit instead, and release it as a new
version.
## Not built yet: the eval gate (P10)
The local gate before a release that reads the date and score of the last run
of the fixed test set and compares it with the last change in the
configuration paths (commands, agents, skills, hooks, and a changed model alias
counts as a change). It waits for the test set to run again; a gate that can
never fire is not built. When it comes, it is one more check in
`scripts/release-check.mjs`, before the door check, and one more step in § 1.
## Credits
- The runbook form: [bashebr/ai-native-sdlc](https://github.com/bashebr/ai-native-sdlc) (MIT).
- Green on the merged tree before cleanup: [obra/superpowers](https://github.com/obra/superpowers) (MIT), at `8ca22db`.
- One-way and two-way doors: [mattpocock/skills](https://github.com/mattpocock/skills), at `c55ee46`.
Ideas only; no text is copied.

265
scripts/release-check.mjs Normal file
View file

@ -0,0 +1,265 @@
#!/usr/bin/env node
// scripts/release-check.mjs
// P13 — the check before a release (docs/release-runbook.md § Before the release).
// One command answers "may this HEAD be released?", so the answer is an exit
// code, never a summary:
//
// node scripts/release-check.mjs --review <review.md> [--repo <dir>] [--version X.Y.Z] [--json]
// 0 = ready
// 1 = not ready — every failed check is listed
// 2 = could not run: not a git repo, no version to release, the suite could
// not be started, or the review is missing or one the verdict gate (K7)
// cannot decide. Exit 2 is NEVER green.
//
// The five checks, in the order they are reported:
// tree no uncommitted changes — otherwise the suite measures something
// other than HEAD
// suite `npm test` exits 0 at HEAD (run for real, never read from a log)
// review lib/review/verdict-gate.mjs on the release's review.md: BLOCK
// stops (1), undecided stops (2)
// coverage the review is of THIS history: its `scope_sha_end` is HEAD or an
// ancestor of HEAD, and the commits after it touch only the release
// bookkeeping (version files, CHANGELOG, the README version badge)
// door the CHANGELOG section for the version has a `### Doors` list with
// one `- one-way: …` or `- two-way: …` line per delivery
//
// The version is --version, else `.claude-plugin/plugin.json`. The repo is
// --repo, else the working directory. Nothing here writes, tags or pushes.
// A ready run prints the K2 command that logs the operator's yes; it does not
// run it — the approval is the operator's.
//
// Door types (a one-way door cannot be walked back without a cost to users; a
// two-way door can) are credited to mattpocock/skills@c55ee46; the green suite
// on the merged tree to obra/superpowers@8ca22db. Ideas only, no text copied.
import { existsSync, readFileSync, realpathSync } from 'node:fs';
import { spawnSync } from 'node:child_process';
import { dirname, join, relative, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { decideVerdict } from '../lib/review/verdict-gate.mjs';
import { parseDocument } from '../lib/util/frontmatter.mjs';
const PLUGIN_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
export const EXIT = Object.freeze({ READY: 0, NOT_READY: 1, UNDECIDED: 2 });
export const CHECK_IDS = Object.freeze(['tree', 'suite', 'review', 'coverage', 'door']);
// Files a release commit may change after the reviewed commit. README.md is
// allowed only for lines that carry the version badge.
export const RELEASE_BOOKKEEPING = Object.freeze([
'CHANGELOG.md', 'package.json', 'package-lock.json', '.claude-plugin/plugin.json', 'README.md',
]);
const BADGE_RE = /badge\/version-/;
const DOOR_LINE_RE = /^- (one-way|two-way): \S/;
const git = (cwd, args) => spawnSync('git', args, { cwd, encoding: 'utf8' });
/**
* The CHANGELOG section for `version`, heading line excluded, or null. Matches
* `## v1.2.3 …`, `## [1.2.3] …` and `## 1.2.3 …` exactly: 1.1.0 is not 1.10.0
* and not 1.1.0-pre.
*/
export function releaseSection(text, version) {
if (typeof text !== 'string' || !version) return null;
const lines = text.split('\n');
const headingVersion = (line) => {
const m = line.match(/^##\s+\[?v?([^\]\s(]+)\]?/);
return m ? m[1] : null;
};
let start = -1;
for (let i = 0; i < lines.length; i++) {
if (/^##\s/.test(lines[i]) && headingVersion(lines[i]) === version) { start = i + 1; break; }
}
if (start === -1) return null;
let end = lines.length;
for (let i = start; i < lines.length; i++) {
if (/^##\s/.test(lines[i])) { end = i; break; }
}
return lines.slice(start, end).join('\n');
}
/**
* The `### Doors` list of a release section. Top-level `- ` lines must read
* `- one-way: …` or `- two-way: …`; indented lines continue the entry above.
* @returns {{ found: boolean, entries: Array<{door: string, text: string}>, bad: string[] }}
*/
export function parseDoors(section) {
const lines = String(section ?? '').split('\n');
const start = lines.findIndex((l) => /^###\s+Doors\s*$/.test(l));
if (start === -1) return { found: false, entries: [], bad: [] };
const entries = [];
const bad = [];
for (let i = start + 1; i < lines.length; i++) {
const line = lines[i];
if (/^#{2,3}\s/.test(line)) break;
if (!line.startsWith('- ')) continue;
const m = line.match(DOOR_LINE_RE);
if (m) entries.push({ door: m[1], text: line.slice(m[0].length - 1).trim() });
else bad.push(line);
}
return { found: true, entries, bad };
}
function checkTree(repo) {
const r = git(repo, ['status', '--porcelain']);
if (r.status !== 0) return { id: 'tree', status: 'error', detail: `git status failed: ${r.stderr.trim()}` };
const changed = r.stdout.split('\n').filter(Boolean);
if (changed.length > 0) {
return { id: 'tree', status: 'fail', detail: `${changed.length} uncommitted change(s) — commit or remove them; the suite must measure HEAD`, items: changed };
}
return { id: 'tree', status: 'pass', detail: 'working tree clean' };
}
function checkSuite(repo, treeClean) {
if (!treeClean) return { id: 'suite', status: 'fail', detail: 'suite not run: the tree has uncommitted changes, so it would not measure HEAD' };
const r = spawnSync('npm', ['test'], { cwd: repo, encoding: 'utf8', maxBuffer: 256 * 1024 * 1024 });
if (r.error) return { id: 'suite', status: 'error', detail: `npm test could not start: ${r.error.message} — not measured` };
const out = `${r.stdout ?? ''}\n${r.stderr ?? ''}`;
const summary = out.split('\n').filter((l) => /^ℹ (tests|pass|fail|skipped) /.test(l)).map((l) => l.slice(2));
const tail = summary.length ? ` (${summary.join(', ')})` : '';
if (r.status !== 0) return { id: 'suite', status: 'fail', detail: `npm test exited ${r.status} at HEAD${tail}` };
return { id: 'suite', status: 'pass', detail: `npm test exited 0 at HEAD${tail}` };
}
function checkReview(reviewPath) {
if (!reviewPath) return { check: { id: 'review', status: 'error', detail: 'no --review given — the release has no review verdict to read (K7)' }, text: null };
if (!existsSync(reviewPath)) return { check: { id: 'review', status: 'error', detail: `${reviewPath} does not exist — no review, no release` }, text: null };
let text;
try { text = readFileSync(reviewPath, 'utf8'); } catch (e) {
return { check: { id: 'review', status: 'error', detail: `${reviewPath} cannot be read: ${e.message}` }, text: null };
}
const v = decideVerdict(text);
if (v.exit === 2) return { check: { id: 'review', status: 'error', detail: `verdict gate could not decide: ${v.reason}` }, text: null };
if (v.exit === 1) return { check: { id: 'review', status: 'fail', detail: `review verdict BLOCK — ${v.reason}` }, text };
return { check: { id: 'review', status: 'pass', detail: `review verdict ${v.verdict}` }, text };
}
function checkCoverage(repo, reviewText) {
if (reviewText === null) return { id: 'coverage', status: 'skip', detail: 'not checked: no readable review' };
let sha;
try { sha = parseDocument(reviewText).parsed?.frontmatter?.scope_sha_end; } catch { sha = undefined; }
sha = sha === undefined || sha === null ? '' : String(sha).trim();
if (!sha) return { id: 'coverage', status: 'fail', detail: 'review.md has no scope_sha_end — cannot tell what it reviewed' };
const resolved = git(repo, ['rev-parse', '--verify', '--quiet', `${sha}^{commit}`]);
if (resolved.status !== 0) return { id: 'coverage', status: 'fail', detail: `the reviewed commit ${sha} is not in the history of HEAD (unknown commit)` };
const full = resolved.stdout.trim();
const anc = git(repo, ['merge-base', '--is-ancestor', full, 'HEAD']);
if (anc.status !== 0) return { id: 'coverage', status: 'fail', detail: `the reviewed commit ${full.slice(0, 12)} is not in the history of HEAD` };
const names = git(repo, ['diff', '--name-only', full, 'HEAD']);
if (names.status !== 0) return { id: 'coverage', status: 'error', detail: `git diff failed: ${names.stderr.trim()}` };
const changed = names.stdout.split('\n').filter(Boolean);
const outside = changed.filter((f) => !RELEASE_BOOKKEEPING.includes(f));
if (changed.includes('README.md')) {
const d = git(repo, ['diff', '-U0', full, 'HEAD', '--', 'README.md']);
const moved = d.stdout.split('\n').filter((l) => /^[+-]/.test(l) && !/^(\+\+\+|---)( |$)/.test(l));
if (moved.some((l) => !BADGE_RE.test(l))) outside.push('README.md (lines other than the version badge)');
}
if (outside.length > 0) {
return { id: 'coverage', status: 'fail', detail: `changed after the reviewed commit ${full.slice(0, 12)} and not covered by the review: ${outside.join(', ')}`, items: outside };
}
const n = changed.length;
return { id: 'coverage', status: 'pass', detail: n === 0 ? `review is of HEAD (${full.slice(0, 12)})` : `review covers HEAD except release bookkeeping (${changed.join(', ')})` };
}
function checkDoor(repo, version) {
const p = join(repo, 'CHANGELOG.md');
if (!existsSync(p)) return { id: 'door', status: 'fail', detail: 'no CHANGELOG.md — the door type is not set' };
const section = releaseSection(readFileSync(p, 'utf8'), version);
if (section === null) return { id: 'door', status: 'fail', detail: `CHANGELOG.md has no section for ${version} — the door type is not set` };
const d = parseDoors(section);
if (!d.found) return { id: 'door', status: 'fail', detail: `the ${version} section has no "### Doors" list — the door type is not set` };
if (d.bad.length > 0) return { id: 'door', status: 'fail', detail: `door lines that are neither one-way nor two-way: ${d.bad.join(' | ')}`, items: d.bad };
if (d.entries.length === 0) return { id: 'door', status: 'fail', detail: `the ${version} "### Doors" list has no door entries` };
const oneWay = d.entries.filter((e) => e.door === 'one-way');
const n = `${d.entries.length} ${d.entries.length === 1 ? 'delivery' : 'deliveries'}`;
const detail = oneWay.length
? `${n}, ${oneWay.length} one-way: ${oneWay.map((e) => e.text).join(' | ')}`
: `${n}, all two-way`;
return { id: 'door', status: 'pass', detail, oneWay: oneWay.length };
}
function resolveVersion(repo, explicit) {
if (explicit) return String(explicit).replace(/^v/, '');
try {
const v = JSON.parse(readFileSync(join(repo, '.claude-plugin', 'plugin.json'), 'utf8')).version;
return typeof v === 'string' && v ? v : null;
} catch { return null; }
}
/**
* Run every check. Pure of process exit; the CLI below prints and exits.
* @returns {{ exit: 0|1|2, version: string|null, repo: string, checks: object[], reason?: string }}
*/
export function releaseCheck({ repo, review, version: explicitVersion }) {
const top = git(repo, ['rev-parse', '--show-toplevel']);
if (top.error || top.status !== 0) return { exit: EXIT.UNDECIDED, version: null, repo, checks: [], reason: `${repo} is not a git repository` };
const root = top.stdout.trim();
const version = resolveVersion(root, explicitVersion);
if (!version) return { exit: EXIT.UNDECIDED, version: null, repo: root, checks: [], reason: 'no version to release: pass --version or set .claude-plugin/plugin.json version' };
const tree = checkTree(root);
const suite = checkSuite(root, tree.status === 'pass');
const { check: rev, text } = checkReview(review ? resolve(review) : null);
const coverage = checkCoverage(root, text);
const door = checkDoor(root, version);
const checks = [tree, suite, rev, coverage, door];
let exit = EXIT.READY;
if (checks.some((c) => c.status === 'error')) exit = EXIT.UNDECIDED;
else if (checks.some((c) => c.status === 'fail')) exit = EXIT.NOT_READY;
return { exit, version, repo: root, checks };
}
function approvalCommand(root) {
const cli = join(PLUGIN_ROOT, 'lib', 'util', 'approval-log.mjs');
const rel = relative(root, cli);
const shown = rel && !rel.startsWith('..') ? rel : cli;
return `node ${shown} append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md`;
}
function render(result) {
const mark = { pass: '✓', fail: '✗', error: '?', skip: '·' };
const out = [];
if (result.reason) {
out.push(`release-check: could not run — ${result.reason}`);
out.push('NOT MEASURED (exit 2) — never green.');
return out.join('\n');
}
out.push(`release-check: v${result.version} in ${result.repo}`);
for (const c of result.checks) out.push(` ${mark[c.status]} ${c.id.padEnd(8)} ${c.detail}`);
if (result.exit === EXIT.READY) {
out.push('READY (exit 0). Before anything is pushed, the operator reads the release section and, if yes, logs it from the repo root:');
out.push(` ${approvalCommand(result.repo)}`);
} else if (result.exit === EXIT.NOT_READY) {
out.push('NOT READY (exit 1) — fix every ✗ above, commit, and run the check again.');
} else {
out.push('NOT MEASURED (exit 2) — at least one check could not run (?). Never green.');
}
return out.join('\n');
}
function parseArgs(argv) {
const out = { repo: process.cwd(), review: null, version: null, json: false };
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === '--repo') out.repo = argv[++i];
else if (a === '--review') out.review = argv[++i];
else if (a === '--version') out.version = argv[++i];
else if (a === '--json') out.json = true;
}
return out;
}
function isMain() {
try {
return Boolean(process.argv[1]) && realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url));
} catch { return false; }
}
if (isMain()) {
const args = parseArgs(process.argv.slice(2));
const result = releaseCheck({ repo: resolve(args.repo), review: args.review, version: args.version });
process.stdout.write((args.json ? JSON.stringify(result, null, 2) : render(result)) + '\n');
process.exit(result.exit);
}

27
tests/fixtures/release-drill/rollback-commands.txt vendored Normal file
View file

@ -0,0 +1,27 @@
# Commands run in the release + rollback drill (wave-2 order 8, P13), phases
# ROLLBACK A and ROLLBACK B, copied from the drill log of 2026-10-08T19:27:15Z.
# The drill ran in a sandbox: local bare repos as the remotes, a HOME of its own,
# no FORGEJO_TOKEN. Versions are written as the runbook's placeholders:
# 6.2.0 = <previous-version>, 6.2.1 = <withdrawn-version>, 6.2.2 = <next-patch>
# Commits appear as <release-commit>, <merge-commit>, <revert-commit>; the review
# file as <review.md>; drill-only helpers as <drill-helper>. [label] = the repo it ran in.
# tests/lib/release-prose-pins.test.mjs: every command in docs/release-runbook.md § 3
# must be one of these lines.
[voyage] git revert --no-edit <release-commit>
[voyage] git revert --no-edit -m 1 <merge-commit>
[voyage] node -p require('./.claude-plugin/plugin.json').version
[voyage] git push -q origin main
[catalog] node scripts/check-versions.mjs
[catalog] node scripts/release-plugin.mjs voyage --version <previous-version>
[catalog] mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
[catalog] node scripts/release-plugin.mjs voyage --version <previous-version> --write --commit --push
[catalog] node scripts/check-versions.mjs
[sandbox] git -C $SANDBOX/remotes/catalog.git show main:.claude-plugin/marketplace.json
[voyage] node <drill-helper>/bump.mjs . <previous-version> <next-patch> $SANDBOX/section-<next-patch>.md
[voyage] git commit -q -am "chore(release): <next-patch> - withdraws <withdrawn-version>"
[voyage] node scripts/release-check.mjs --review <review.md>
[voyage] node lib/util/approval-log.mjs append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md
[voyage] git push -q origin main
[catalog] mkdir -p "$HOME/.claude/runtime/push-approvals" && touch "$HOME/.claude/runtime/push-approvals/$(pwd | sed 's|/|_|g')"
[catalog] node scripts/release-plugin.mjs voyage --create-tag --write --commit --push
[catalog] node scripts/check-versions.mjs

146
tests/lib/release-prose-pins.test.mjs Normal file
View file

@ -0,0 +1,146 @@
// tests/lib/release-prose-pins.test.mjs
// Prose pins owned by the release process (wave-2 order 8, P13). Counted as
// doc pins by lib/util/test-census.mjs (PIN_FILE_RE matches `prose-pins.test.mjs`).
//
// The load-bearing pin: the rollback part of docs/release-runbook.md may only
// hold commands that were run in the release + rollback drill. The drill's
// rollback commands are recorded in tests/fixtures/release-drill/rollback-commands.txt
// (copied from the drill log, versions and commits written as the runbook's
// placeholders). A command written into § 3 that the drill never ran fails here.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { parseDoors, releaseSection } from '../../scripts/release-check.mjs';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const read = (rel) => readFileSync(join(ROOT, rel), 'utf8');
const RUNBOOK = 'docs/release-runbook.md';
const FIXTURE = 'tests/fixtures/release-drill/rollback-commands.txt';
const APPROVAL = 'node lib/util/approval-log.mjs append --kind release --subject CHANGELOG.md --sha256-of CHANGELOG.md';
/** The lines from `## <heading>` to the next `## ` heading outside a fenced block. */
function part(md, heading) {
const lines = md.split('\n');
const start = lines.findIndex((l) => l === `## ${heading}`);
assert.ok(start !== -1, `${RUNBOOK} must have "## ${heading}"`);
let fenced = false;
let end = lines.length;
for (let i = start + 1; i < lines.length; i++) {
if (lines[i].startsWith('```')) fenced = !fenced;
if (!fenced && lines[i].startsWith('## ')) { end = i; break; }
}
return lines.slice(start, end).join('\n');
}
/** Every command line in the ```sh blocks of a markdown part. */
function shCommands(md) {
const out = [];
const re = /```sh\n([\s\S]*?)```/g;
let m;
while ((m = re.exec(md)) !== null) {
for (const line of m[1].split('\n')) {
const t = line.trim();
if (t && !t.startsWith('#')) out.push(t);
}
}
return out;
}
function drillCommands() {
return read(FIXTURE).split('\n')
.filter((l) => l.trim() && !l.startsWith('#'))
.map((l) => l.replace(/^\[(voyage|catalog)\] /, '').trim());
}
test('runbook: the three parts, in order — before, release, rollback', () => {
const md = read(RUNBOOK);
const at = ['## 1. Before the release', '## 2. The release', '## 3. Rollback'].map((h) => md.indexOf(`\n${h}`));
for (const i of at) assert.ok(i !== -1, `${RUNBOOK} must have each of the three parts`);
assert.ok(at[0] < at[1] && at[1] < at[2], 'the parts must come in the order before → release → rollback');
});
test('runbook: every command in the rollback part was run in the drill', () => {
const ran = new Set(drillCommands());
const written = shCommands(part(read(RUNBOOK), '3. Rollback'));
assert.ok(written.length >= 10, `the rollback part must carry the drilled commands (found ${written.length})`);
const unrun = written.filter((c) => !ran.has(c));
assert.deepEqual(unrun, [], `commands in § 3 that the drill never ran:\n${unrun.join('\n')}`);
});
test('drill fixture: both rollback paths were run — revert + ref back, and the forward patch', () => {
const ran = drillCommands();
for (const c of [
'git revert --no-edit <release-commit>',
'git revert --no-edit -m 1 <merge-commit>',
'node scripts/release-plugin.mjs voyage --version <previous-version> --write --commit --push',
'node scripts/release-plugin.mjs voyage --create-tag --write --commit --push',
'node scripts/check-versions.mjs',
]) assert.ok(ran.includes(c), `the drill fixture must record: ${c}`);
});
test('runbook § 1: the verdict gate (K7), the release check and the logged yes (K2)', () => {
const before = part(read(RUNBOOK), '1. Before the release');
assert.ok(before.includes('node lib/review/verdict-gate.mjs <review.md>'), '§ 1 must run the K7 verdict gate');
assert.ok(before.includes('node scripts/release-check.mjs --review <review.md>'), '§ 1 must run the release check');
assert.ok(before.includes(APPROVAL), '§ 1 must log the release approval with the K2 command the check prints');
assert.match(before, /npm test/, '§ 1 must require the suite on the merged tree');
});
test('runbook § 2: push last, then one release-plugin run, then the catalog check, then cleanup', () => {
const rel = shCommands(part(read(RUNBOOK), '2. The release'));
const order = [
'git push origin main',
'node scripts/release-plugin.mjs voyage --create-tag --write --commit --push',
'node scripts/check-versions.mjs',
].map((c) => rel.indexOf(c));
for (const i of order) assert.ok(i !== -1, '§ 2 must carry the push, the release run and the catalog check');
assert.ok(order[0] < order[1] && order[1] < order[2], '§ 2 order: git push, release-plugin, check-versions');
assert.match(part(read(RUNBOOK), '2. The release'), /Cleanup comes last/);
});
test('runbook: the door-type example parses as a release section with one-way and two-way lines', () => {
const md = part(read(RUNBOOK), 'Door types');
const block = md.match(/```markdown\n([\s\S]*?)```/);
assert.ok(block, 'the door types part must show the CHANGELOG form');
const section = releaseSection(block[1], '6.3.0');
assert.ok(section !== null, 'the example must be a release section');
const doors = parseDoors(section);
assert.deepEqual(doors.entries.map((e) => e.door).sort(), ['one-way', 'two-way']);
assert.deepEqual(doors.bad, []);
});
test('runbook: the release check exits as the script does — 0 ready, 1 not ready, 2 never green', () => {
const before = part(read(RUNBOOK), '1. Before the release').replace(/\s+/g, ' ');
assert.match(before, /0 = ready, 1 = not ready[^.]*, 2 = could not run — never green/);
});
test('runbook: the eval gate (P10) is named as not built, with where it goes', () => {
const p10 = part(read(RUNBOOK), 'Not built yet: the eval gate (P10)');
assert.ok(p10.includes('scripts/release-check.mjs'), 'P10 must say which file the gate goes into');
});
test('runbook: public — no local paths', () => {
const md = read(RUNBOOK);
for (const bad of ['/Users/', '/private/', '/tmp/', '~/repos', 'scratchpad']) {
assert.ok(!md.includes(bad), `${RUNBOOK} must not carry a local path (${bad})`);
}
});
test('README: a section on releasing and rolling back that points to the runbook', () => {
const md = read('README.md');
const start = md.indexOf('\n## Releasing and rolling back');
assert.ok(start !== -1, 'README must have "## Releasing and rolling back"');
const sec = md.slice(start, md.indexOf('\n## ', start + 1));
assert.ok(sec.includes('docs/release-runbook.md'), 'the README section must link the runbook');
assert.ok(sec.includes('scripts/release-check.mjs'), 'the README section must name the release check');
});
test('CLAUDE.md: the pointer to the runbook and the release check', () => {
const md = read('CLAUDE.md');
assert.ok(md.includes('docs/release-runbook.md'), 'CLAUDE.md must point to the runbook');
assert.ok(md.includes('scripts/release-check.mjs'), 'CLAUDE.md must name the release check');
});

303
tests/scripts/release-check.test.mjs Normal file
View file

@ -0,0 +1,303 @@
// tests/scripts/release-check.test.mjs
// P13 — the check before a release (wave-2 order 8). One command answers "may
// this HEAD be released?": the suite is green at HEAD, the review verdict for
// what is released is not BLOCK (K7) and covers it, and every delivery in the
// release says whether it is a one-way or a two-way door.
// 0 = ready, 1 = not ready (listed), 2 = could not run (never green).
//
// Every test builds a throwaway git repo shaped like a release: one commit with
// the work, then a release commit that only bumps versions and the CHANGELOG.
// The repo's own `npm test` is a one-liner that exits 0 or 1, so the suite
// check runs for real without running Voyage's suite inside its own suite.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { spawnSync } from 'node:child_process';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
const CHECK = join(ROOT, 'scripts', 'release-check.mjs');
const mod = existsSync(CHECK) ? await import('../../scripts/release-check.mjs') : {};
const git = (cwd, ...args) => {
const r = spawnSync('git', args, { cwd, encoding: 'utf8' });
if (r.status !== 0) throw new Error(`git ${args.join(' ')}: ${r.stderr}`);
return r.stdout.trim();
};
function reviewText({ verdict = 'ALLOW', sha }) {
const id = 'a'.repeat(40);
const severity = verdict === 'BLOCK' ? 'BLOCKER' : 'MINOR';
const fmVerdict = verdict === null ? '' : `verdict: ${verdict}\n`;
const json = JSON.stringify({
verdict: verdict ?? undefined,
counts: { BLOCKER: severity === 'BLOCKER' ? 1 : 0, MAJOR: 0, MINOR: severity === 'MINOR' ? 1 : 0, SUGGESTION: 0 },
allow_blocked_by: [],
findings: [{ id, severity, rule_key: 'MISSING_TEST', file: 'a.mjs', line: 1, title: 't', detail: 'd', recommended_action: 'Add a test.' }],
}, null, 2);
return `---
type: trekreview
review_version: "1.1"
task: "t"
slug: s
project_dir: .claude/projects/2026-10-08-s/
brief_path: .claude/projects/2026-10-08-s/brief.md
scope_sha_end: ${sha}
reviewed_files_count: 1
${fmVerdict}findings:
- ${id}
---
# Review: t
## Executive Summary
x
## Coverage
| File | Treatment | Reason |
|------|-----------|--------|
| a.mjs | deep-review | default |
## Remediation Summary
x
\`\`\`json
${json}
\`\`\`
`;
}
const DOORS = `### Doors
- two-way: the fixture change — revert it and release a new patch.
`;
/**
* A repo with the work in one commit and a release commit for 1.0.1 on top.
* Options switch one thing off at a time; the default is a ready release.
*/
function fixture({
suiteExit = 0, doors = DOORS, verdict = 'ALLOW', writeReview = true,
reviewSha = 'work', afterRelease = null, dirty = false, version = '1.0.1',
} = {}) {
const base = mkdtempSync(join(tmpdir(), 'release-check-'));
const repo = join(base, 'repo');
mkdirSync(join(repo, '.claude-plugin'), { recursive: true });
git(repo, 'init', '-q', '-b', 'main');
git(repo, 'config', 'user.email', 't@example.invalid');
git(repo, 'config', 'user.name', 't');
git(repo, 'config', 'commit.gpgsign', 'false');
const pkg = (v) => JSON.stringify({ name: 'fx', version: v, scripts: { test: `node -e "process.exit(${suiteExit})"` } }, null, 2) + '\n';
writeFileSync(join(repo, 'package.json'), pkg('1.0.0'));
writeFileSync(join(repo, '.claude-plugin', 'plugin.json'), JSON.stringify({ name: 'fx', version: '1.0.0' }) + '\n');
writeFileSync(join(repo, 'README.md'), '# fx\n\n![Version](https://img.shields.io/badge/version-1.0.0-blue)\n\nText.\n');
writeFileSync(join(repo, 'CHANGELOG.md'), '# Changelog\n\n## v1.0.0 — 2026-10-01 — first\n\n- first\n');
writeFileSync(join(repo, 'a.mjs'), 'export const a = 1;\n');
git(repo, 'add', '-A');
git(repo, 'commit', '-q', '-m', 'feat: first');
writeFileSync(join(repo, 'a.mjs'), 'export const a = 2;\n');
git(repo, 'add', 'a.mjs');
git(repo, 'commit', '-q', '-m', 'fix: the work');
const workSha = git(repo, 'rev-parse', 'HEAD');
if (version !== null) {
writeFileSync(join(repo, 'package.json'), pkg(version));
writeFileSync(join(repo, '.claude-plugin', 'plugin.json'), JSON.stringify({ name: 'fx', version }) + '\n');
} else {
writeFileSync(join(repo, '.claude-plugin', 'plugin.json'), JSON.stringify({ name: 'fx' }) + '\n');
writeFileSync(join(repo, 'package.json'), JSON.stringify({ name: 'fx', scripts: { test: 'node -e "process.exit(0)"' } }) + '\n');
}
writeFileSync(join(repo, 'README.md'), `# fx\n\n![Version](https://img.shields.io/badge/version-${version ?? '1.0.0'}-blue)\n\nText.\n`);
writeFileSync(join(repo, 'CHANGELOG.md'),
`# Changelog\n\n## v${version ?? '9.9.9'} — 2026-10-08 — fixture\n\n${doors ?? ''}\n### Fixed\n\n- the work\n\n## v1.0.0 — 2026-10-01 — first\n\n- first\n`);
git(repo, 'add', '-A');
git(repo, 'commit', '-q', '-m', 'chore(release): fixture');
if (afterRelease) {
for (const [rel, text] of Object.entries(afterRelease)) writeFileSync(join(repo, rel), text);
git(repo, 'add', '-A');
git(repo, 'commit', '-q', '-m', 'after the review');
}
if (dirty) writeFileSync(join(repo, 'a.mjs'), 'export const a = 3;\n');
const review = join(base, 'review.md');
const sha = reviewSha === 'work' ? workSha : reviewSha;
if (writeReview) writeFileSync(review, reviewText({ verdict, sha }));
return { base, repo, review, cleanup: () => rmSync(base, { recursive: true, force: true }) };
}
function run(fx, extra = []) {
return spawnSync(process.execPath, [CHECK, '--repo', fx.repo, '--review', fx.review, ...extra], { encoding: 'utf8' });
}
test('release-check — the CLI exists', () => {
assert.ok(existsSync(CHECK), 'scripts/release-check.mjs must exist (P13)');
});
test('release-check — everything in place → 0 (ready)', () => {
const fx = fixture();
try {
const r = run(fx);
assert.equal(r.status, 0, r.stdout + r.stderr);
assert.match(r.stdout, /READY/);
} finally { fx.cleanup(); }
});
test('release-check — a ready release prints the release approval command (K2)', () => {
const fx = fixture();
try {
const r = run(fx);
assert.equal(r.status, 0, r.stdout + r.stderr);
assert.match(r.stdout, /approval-log\.mjs append --kind release --subject CHANGELOG\.md --sha256-of CHANGELOG\.md/);
} finally { fx.cleanup(); }
});
test('release-check — review verdict BLOCK → 1', () => {
const fx = fixture({ verdict: 'BLOCK' });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /BLOCK/);
} finally { fx.cleanup(); }
});
test('release-check — no review.md → 2 (never green)', () => {
const fx = fixture({ writeReview: false });
try {
const r = run(fx);
assert.equal(r.status, 2, r.stdout + r.stderr);
} finally { fx.cleanup(); }
});
test('release-check — a review the verdict gate cannot decide (no verdict) → 2', () => {
const fx = fixture({ verdict: null });
try {
const r = run(fx);
assert.equal(r.status, 2, r.stdout + r.stderr);
} finally { fx.cleanup(); }
});
test('release-check — no door type in the release section → 1', () => {
const fx = fixture({ doors: null });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /door/i);
} finally { fx.cleanup(); }
});
test('release-check — a door line that is neither one-way nor two-way → 1', () => {
const fx = fixture({ doors: '### Doors\n\n- two-way: fine\n- maybe: not a door type\n' });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /maybe/);
} finally { fx.cleanup(); }
});
test('release-check — an empty Doors section → 1', () => {
const fx = fixture({ doors: '### Doors\n\n' });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /no door entries/i);
} finally { fx.cleanup(); }
});
test('release-check — the suite red at HEAD → 1', () => {
const fx = fixture({ suiteExit: 1 });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /suite/i);
} finally { fx.cleanup(); }
});
test('release-check — uncommitted changes → 1 (the suite would not measure HEAD)', () => {
const fx = fixture({ dirty: true });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /uncommitted/i);
} finally { fx.cleanup(); }
});
test('release-check — a code change after the reviewed commit → 1 (the review does not cover it)', () => {
const fx = fixture({ afterRelease: { 'a.mjs': 'export const a = 99;\n' } });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /a\.mjs/);
} finally { fx.cleanup(); }
});
test('release-check — a README prose change after the reviewed commit → 1 (only the badge may move)', () => {
const fx = fixture({ afterRelease: { 'README.md': '# fx\n\n![Version](https://img.shields.io/badge/version-1.0.1-blue)\n\nNew text.\n' } });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /README\.md/);
} finally { fx.cleanup(); }
});
test('release-check — a review of a commit that is not in this history → 1', () => {
const fx = fixture({ reviewSha: 'f'.repeat(40) });
try {
const r = run(fx);
assert.equal(r.status, 1, r.stdout + r.stderr);
assert.match(r.stdout, /not in the history of HEAD/);
} finally { fx.cleanup(); }
});
test('release-check — no version to release (no --version, no plugin.json version) → 2', () => {
const fx = fixture({ version: null });
try {
assert.equal(run(fx).status, 2);
} finally { fx.cleanup(); }
});
test('release-check — not a git repo → 2', () => {
const dir = mkdtempSync(join(tmpdir(), 'release-check-nogit-'));
try {
const r = spawnSync(process.execPath, [CHECK, '--repo', dir, '--review', join(dir, 'review.md'), '--version', '1.0.0'], { encoding: 'utf8' });
assert.equal(r.status, 2, r.stdout + r.stderr);
} finally { rmSync(dir, { recursive: true, force: true }); }
});
test('release-check — --json reports every check and the exit', () => {
const fx = fixture({ verdict: 'BLOCK' });
try {
const r = run(fx, ['--json']);
assert.equal(r.status, 1);
const out = JSON.parse(r.stdout);
assert.equal(out.exit, 1);
assert.equal(out.version, '1.0.1');
assert.deepEqual(out.checks.map((c) => c.id), ['tree', 'suite', 'review', 'coverage', 'door']);
assert.equal(out.checks.find((c) => c.id === 'review').status, 'fail');
} finally { fx.cleanup(); }
});
test('release-check — parseDoors reads one-way and two-way entries and names the bad ones', () => {
assert.equal(typeof mod.parseDoors, 'function');
const section = '### Doors\n\n- one-way: plan 1.9 rejects plan 1.7 files\n (users regenerate their plans)\n- two-way: a new check\n- maybe: x\n\n### Added\n\n- one-way: not in Doors\n';
const d = mod.parseDoors(section);
assert.deepEqual(d.entries.map((e) => e.door), ['one-way', 'two-way']);
assert.deepEqual(d.bad, ['- maybe: x']);
assert.equal(d.found, true);
});
test('release-check — releaseSection matches the CHANGELOG heading dialects exactly', () => {
assert.equal(typeof mod.releaseSection, 'function');
const text = '# C\n\n## v1.10.0 — x\n\nten\n\n## [1.1.0] - y\n\none\n\n## v1.1.0-pre\n\npre\n';
assert.match(mod.releaseSection(text, '1.1.0'), /one/);
assert.doesNotMatch(mod.releaseSection(text, '1.1.0'), /ten|pre/);
assert.match(mod.releaseSection(text, '1.10.0'), /ten/);
assert.equal(mod.releaseSection(text, '2.0.0'), null);
});