feat(skills): writing-criteria, one source for runnable criteria (order 7c)

A brief success criterion and a plan's Verify, Regression, End-to-end and
Verification lines are each one command the criteria runner accepts, written
as the line's first backticked span; anything more is bash tests/<name>.sh,
and each must be able to go red (the runner reads the exit code only).

- skills/writing-criteria/SKILL.md (S1 form), with Runs:/Refused:/Never red:
  examples that a pin runs through the criteria runner
- brief-reviewer and plan-critic preload writing-criteria and
  reviewing-adversarially (S3 + S4); the copied runner rules leave them, the
  two templates and /trekplan, which names the skill where the plan is written
  and where its three reviewers start
- eval cases 3 fire / 2 quiet / 1 effect; tests/lib/skills-criteria-prose-pins
- review-630 @~1 pin follows the text into the skill; trekplan-stats line_ref
  re-measured

Baseline without the skill: 24 of 27 written checks refused by the runner,
and 0 of 3 reviewers saw that git status/git log can never go red. With it:
0 of 27 refused, 3 of 3 reviewers flagged both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-17 07:39:04 +02:00
commit 837e92545d
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
26 changed files with 605 additions and 53 deletions

View file

@ -8,6 +8,7 @@ description: |
model: opus
color: magenta
tools: ["Read", "Glob", "Grep"]
skills: ["voyage:writing-criteria", "voyage:reviewing-adversarially"]
---
You are a requirements analyst. Your sole job is to find problems in a task
@ -56,20 +57,14 @@ Flag as **incomplete** if:
- Success Criteria are not testable (e.g., "it should work well")
- Scope is unbounded — no non-goals defined
**Runnable at review.** `/trekreview` runs a criterion's command only when it
is one test-runner invocation with no pipe, redirect or `&&`: `npm test`,
`npm run <script>`, `node --test <file>`, `vitest`, `jest`, `pytest`,
`python -m pytest`, `uv run pytest`, `cargo test`, `go test`, `make test`,
`bash tests/<script>.sh` (or `/bin/bash tests/<script>.sh`), or `git status|log|diff|show|ls-files`.
It runs with no shell, as plain words: no quote, `$`, brace, leading `~` or
`@` (a git revision such as `@~1` is the one exception), no glob outside
`node --test` (Node 21+), no `..` or absolute path, and only the runner's
read-only options.
Anything else — the product's own binary (`node bin/x.mjs …`), `jq`, `grep`,
`git grep`, a pipe — is reported NOT RUN. Never PROPOSE such a command as a
criterion. When a criterion needs one, propose wrapping it in a script under
`tests/` and writing the criterion as "`bash tests/<script>.sh` exits 0".
The validator flags existing ones as `BRIEF_SC_NOT_RUNNABLE`.
**Runnable at review, and able to go red.** Judge each criterion by the
preloaded `writing-criteria` skill: which commands the criteria runner
accepts, the command as the first backticked span, a script under `tests/`
for anything more, and a command that can fail. A criterion the runner
refuses is reported NOT RUN at review; the validator flags it as
`BRIEF_SC_NOT_RUNNABLE`. A criterion that exits 0 whatever the code does is
untestable (dimension 3). Every suggestion you make for such a criterion is
itself one the skill accepts.
### 2. Consistency

View file

@ -7,6 +7,7 @@ model: opus
effort: high
color: red
tools: ["Read", "Glob", "Grep"]
skills: ["voyage:writing-criteria", "voyage:reviewing-adversarially"]
---
You are a senior staff engineer whose sole job is to find problems in implementation
@ -150,11 +151,18 @@ questions, which defeats the purpose.
"not verified end to end".
- `Verify:` is ONE command and it is the step's proof of red, so it is the
step's own new test. The suite the step must not break goes in the optional
`Regression:` field (K10: one backticked command the criteria runner allows,
no pipe, no `&&`), which the executor runs green before the checkpoint. Do
not demand `npm test` in `Verify:` — that test is not red before the step.
A step that can break existing tests and has no `Regression:` is a **minor**
finding ("add `Regression: \`npm test\``"), not a major.
`Regression:` field (K10), which the executor runs green before the
checkpoint. Do not demand `npm test` in `Verify:` — that test is not red
before the step. A step that can break existing tests and has no
`Regression:` is a **minor** finding ("add `Regression: \`npm test\``"), not
a major.
- Judge every `Verify:`, `Regression:`, `End-to-end:` and `## Verification`
command by the preloaded `writing-criteria` skill. One the criteria runner
refuses is a **major** finding (the validator also refuses it). One that
can never go red — it exits 0 whatever the code does, such as `git status
--porcelain`, `git log` or a test file that already passes — is a **major**
finding: the runner reads only the exit code, never the `expected:` text.
Each command you suggest instead is one the skill accepts.
### 9. Headless readiness