voyage/skills/writing-criteria/SKILL.md

name

description

writing-criteria

Criteria discipline for the checks Voyage runs itself: each brief success criterion, each plan Verify, Regression and End-to-end line and each Verification bullet is one command the criteria runner accepts, written as its first backticked span (a known test runner, plain words, no pipe, no &&, no shell), and a check that needs more becomes a script declared as bash tests/<name>.sh. Each criterion must be able to go red: a command that exits 0 whatever the code does is no criterion. Use when writing or reviewing success criteria, a Verify, Regression or End-to-end line or a Verification section, or when a validator reports a criterion as not runnable (BRIEF_SC_NOT_RUNNABLE, PLAN_STEP_VERIFY_NOT_RUNNABLE).

Writing criteria

Overview

Voyage runs the checks a brief or a plan declares, and it decides "done" from their exit codes. lib/verification/criteria-runner.mjs runs each one with no shell, as its words, and only when it is a known test-runner invocation. Anything else is NOT RUN: never executed, never green. In /trekexecute Phase 7 an unrunnable plan criterion stops the run; in /trekreview an unrunnable success criterion is an absent measurement. A criterion that runs but cannot fail measures nothing either. This skill is the one place that says how to write both right.

When to use

• Writing a brief's ## Success Criteria.
• Writing a plan step's Verify:, Regression: or End-to-end: line, or the plan's ## Verification.
• Reviewing them: brief-reviewer and plan-critic preload this skill.
• A validator reports BRIEF_SC_NOT_RUNNABLE, PLAN_VERIFICATION_NOT_RUNNABLE, PLAN_STEP_VERIFY_NOT_RUNNABLE, PLAN_STEP_REGRESSION_NOT_RUNNABLE or PLAN_END_TO_END_NOT_RUNNABLE.
• Not for: writing the test itself and seeing it fail (that is voyage:test-first), or starting the app to look at a change by hand (Claude Code's own run).

Process

1. One criterion, one command. No &&, ;, |, >, <, $( or backtick inside it. Two things to check are two criteria, or one script (step 5). A line that chains the suite, a setup and three checks is refused whole, and then none of it is measured.

2. The command is the first backticked span on the line. The runner reads the first span as the command. Name the feature in plain words, not in backticks before the command: - shelf count prints the number of books: `bash tests/count.sh` exits 0. A line that opens with the feature in backticks (`shelf count` prints ...) declares shelf count as its command.

3. Use a runner the runner knows. These forms run:

   Form	Note
   npm test, npm run <script>	the script must be in package.json when the check runs; a plan may add it
   node --test <files>	a glob only on Node 21 and later; elsewhere name the files
   vitest, jest, pytest, python -m pytest, uv run pytest	with a short list of read-only options
   cargo test, go test, make test	nothing may follow make test
   bash tests/<name>.sh, /bin/bash tests/<name>.sh	a script under tests/; the second pins the system bash
   git status, log, diff, show, ls-files	read-only only; git grep is not on the list

4. Plain words. No shell reads the line, so a word that means something in a shell is refused: a quote, a backslash, $, a brace, a leading ~, a word starting with @ (a git revision such as @~1 is the one exception), a path that climbs out with .. or starts at /, an environment prefix (CI=1 npm test), and any option not on the runner's read-only list. A trailing # comment is dropped, unless it holds a quote, $, backtick or backslash; then the line is refused.

5. A check that needs more is a script. The product's own binary, a temporary folder, an environment variable, comparing output, jq, grep, curl: put it in a script under tests/ that exits non-zero with a message when the check fails (set -eu, an explicit exit 1), and declare bash tests/<name>.sh. The script goes in the Files: of the step that builds the behaviour. This is how an End-to-end: line runs the product the way a user does.

6. It must be able to go red. Before writing a criterion, name the wrong result that would make it fail. The runner decides on the exit code alone: the → expected: text after the command is for the reader and is never compared. So a command that exits 0 whatever the code does is no criterion: git status --porcelain "expected: no output" exits 0 on a dirty tree too, git log and git diff --stat report and exit 0, and npm test alone proves no new behaviour unless a test for that behaviour is in it. A check on output goes in a script that compares it and exits 1. A step's Verify: is ONE command, the step's own new test, seen red before the step; a Verify: that runs a test file which already passes can never be red. The suite the step must not break goes in Regression:.

7. Check it before handing it on. node ${CLAUDE_PLUGIN_ROOT}/lib/validators/plan-validator.mjs --strict --json <plan> and lib/validators/brief-validator.mjs name every unrunnable criterion. A refused criterion is rewritten (steps 1 to 5), never kept with a note.

Examples

Each line marked Runs: is accepted by the criteria runner, each Refused: is refused, and each Never red: is accepted but exits 0 whatever the code does (a pin in npm test holds all three).

• Runs: npm test
• Runs: npm run typecheck
• Runs: node --test tests/count.test.mjs
• Runs: bash tests/count.sh
• Runs: /bin/bash tests/e2e.sh
• Runs: pytest -q tests/test_count.py
• Runs: cargo test --workspace
• Refused: npm test && npm run lint (two commands: write two criteria)
• Refused: node bin/shelf.mjs count (the product's binary is no test runner: bash tests/count.sh)
• Refused: node bin/shelf.mjs list --json | jq length (a pipe: move the check into a script)
• Refused: SHELF_FILE=tmp/s.json node --test tests/cli.test.mjs (an environment prefix: set it in the test)
• Refused: node -e "process.exit(0)" (node, but not node --test)
• Refused: npx tsc --noEmit (not a runner: npm run typecheck)
• Refused: curl -s localhost:8080/health (not a runner: a script under tests/ that calls it)
• Refused: git grep -q count -- src (git grep is not a read-only subcommand on the list)
• Refused: make test lint (nothing may follow make test)
• Never red: git status --porcelain
• Never red: git log --oneline -2

Red flags

Thought	What it means
"One line with && checks it all"	The runner refuses the whole line; none of it is measured. Two criteria, or one script.
"Name the feature in backticks, then the command"	The first backticked span is the command. The feature goes in plain words.
"A user runs node bin/app.mjs, so that is the check"	It is still no test runner. A script under tests/ runs it.
"A quick node -e one-liner checks the JSON"	Refused. The same code in a test file runs as node --test.
"npm test exits 0, so the feature is done"	It exited 0 before the change too, unless a test for the feature is in it.
"git status --porcelain → expected: no output fails on a dirty tree"	The runner reads the exit code, never the expected text. It exits 0 either way: it can never go red.
"Verify: runs the step's test plus a CLI check"	Verify: is one command, the step's new test. The rest is a script or Regression:.
"NOT RUN is not a failure, so it can stay"	NOT RUN is never green: an unrunnable plan criterion stops /trekexecute Phase 7.

Enforced by

• lib/verification/criteria-runner.mjs: deterministic. It runs only what its allowlist accepts, with no shell, and reports everything else NOT RUN, never green.
• lib/validators/plan-validator.mjs and lib/validators/brief-validator.mjs: the same allowlist when the plan or brief is written (PLAN_*_NOT_RUNNABLE are errors in a plan, BRIEF_SC_NOT_RUNNABLE a warning in a brief).
• prose only: that a criterion can go red and that it tests what the brief asks for. brief-reviewer and plan-critic judge that by reading; no check runs it.

Credits

• Voyage's own content; nothing is borrowed. The rules restate lib/verification/criteria-runner.mjs. The red flags come from Voyage's own baseline runs without this skill (docs/skills-authoring.md).