feat(skills): writing-criteria, one source for runnable criteria (order 7c)
A brief success criterion and a plan's Verify, Regression, End-to-end and Verification lines are each one command the criteria runner accepts, written as the line's first backticked span; anything more is bash tests/<name>.sh, and each must be able to go red (the runner reads the exit code only). - skills/writing-criteria/SKILL.md (S1 form), with Runs:/Refused:/Never red: examples that a pin runs through the criteria runner - brief-reviewer and plan-critic preload writing-criteria and reviewing-adversarially (S3 + S4); the copied runner rules leave them, the two templates and /trekplan, which names the skill where the plan is written and where its three reviewers start - eval cases 3 fire / 2 quiet / 1 effect; tests/lib/skills-criteria-prose-pins - review-630 @~1 pin follows the text into the skill; trekplan-stats line_ref re-measured Baseline without the skill: 24 of 27 written checks refused by the runner, and 0 of 3 reviewers saw that git status/git log can never go red. With it: 0 of 27 refused, 3 of 3 reviewers flagged both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
31c77fd8d6
commit
837e92545d
26 changed files with 605 additions and 53 deletions
|
|
@ -444,9 +444,14 @@ unchanged since /trekbrief's review"), and continue to Phase 5. Exit 1
|
|||
pass, or a brief from another producer) or 2 (could not run): review it as
|
||||
below (operator decision 2026-10-08).
|
||||
|
||||
Launch the **brief-reviewer** agent:
|
||||
Launch the **brief-reviewer** agent. It has `voyage:writing-criteria` and
|
||||
`voyage:reviewing-adversarially` preloaded through its `skills:` line; name
|
||||
both in the prompt anyway, so the rules hold even where preloading did not
|
||||
happen.
|
||||
Prompt: "Review this task brief for quality: {brief_path}. Check completeness,
|
||||
consistency, testability, scope clarity, and research-plan validity.
|
||||
Follow the writing-criteria skill (each success criterion is one command the
|
||||
criteria runner accepts, and it can go red) and the reviewing-adversarially skill.
|
||||
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
|
||||
|
||||
Handle the verdict:
|
||||
|
|
@ -709,7 +714,7 @@ Do NOT write this synthesis to disk. It is internal working context only.
|
|||
>
|
||||
> **Two step rules at every plan_version** (no version bump; a plan without
|
||||
> the field is unchanged): the optional `Regression:` field (K10) holds ONE
|
||||
> backticked command the criteria runner allows, with no pipe and no `&&` —
|
||||
> backticked command the criteria runner accepts (`voyage:writing-criteria`) —
|
||||
> usually `npm test`, the suite the step must not break. `Verify:` stays the
|
||||
> step's own test, the one that is red first; `Regression:` is what the
|
||||
> executor runs green before the checkpoint, so a step whose red test is a
|
||||
|
|
@ -824,24 +829,13 @@ Write the plan following the template structure. The plan MUST include:
|
|||
npm test
|
||||
node --test tests/lib/x.test.mjs
|
||||
```
|
||||
The command must also be one the criteria runner is allowed to run: a test
|
||||
runner (`npm test`, `npm run <script>`, `node --test <file>`, `vitest`,
|
||||
`jest`, `pytest`, `python -m pytest`, `uv run pytest`, `cargo test`,
|
||||
`go test`, `make test`), `bash <script under tests/>` (also
|
||||
`/bin/bash <script under tests/>`, to pin the system shell), or a read-only
|
||||
`git` subcommand (`status`, `log`, `diff`, `show`, `ls-files` — not `git
|
||||
grep`) — with no pipe, redirect or `&&`. It runs with no shell: its words
|
||||
reach the runner as written, so a word with a quote, backslash, `$`,
|
||||
brace, or a leading `~` or `@` is refused (a git revision such as `@~1`
|
||||
is the one exception), a glob only reaches
|
||||
`node --test` on Node 21 and later (which expands its own patterns), no
|
||||
path climbs out with `..` or starts at `/` (git included), each runner
|
||||
takes a short list of read-only options, and nothing may follow
|
||||
`make test`. A trailing `# comment` is dropped, on a line of a shell fence and in a backticked command alike;
|
||||
in a backticked command it may not hold a quote, `$`, backtick or backslash (the executor passes the
|
||||
span on inside double quotes), and such a comment is refused, not dropped. Anything else is reported NOT
|
||||
RUN, never executed and never failed. A check that needs more than that is
|
||||
wrapped in a script under `tests/` and declared as `bash tests/<script>.sh`.
|
||||
Each command is written by the `voyage:writing-criteria` skill (use it by
|
||||
name before you write this section and the steps' `Verify:`, `Regression:`
|
||||
and `End-to-end:` lines): one command the criteria runner accepts, in plain
|
||||
words, a script under `tests/` (`bash tests/<script>.sh`) for anything
|
||||
more, and a command that can go red. Anything the runner refuses is
|
||||
reported NOT RUN, never executed and never failed, and an unrunnable plan
|
||||
criterion stops `/trekexecute` Phase 7.
|
||||
9. **Estimated Scope** — File counts and complexity rating.
|
||||
|
||||
### Quality standards
|
||||
|
|
@ -962,11 +956,20 @@ Then launch two review agents **in parallel — emit both Agent tool calls in a
|
|||
single assistant message turn** (same pattern as Phase 5 exploration). They
|
||||
have zero data dependencies; serializing them wastes 30–60 seconds per run.
|
||||
|
||||
Both reviewers are asked for `voyage:writing-criteria` and
|
||||
`voyage:reviewing-adversarially` by name. `plan-critic` has both preloaded;
|
||||
`scope-guardian` has `voyage:reviewing-adversarially` preloaded and gets the
|
||||
writing-criteria rule in one sentence in its prompt, for the brief's criteria
|
||||
it traces into `## Verification`.
|
||||
|
||||
**plan-critic** — adversarial review of the plan.
|
||||
Prompt: "Review this implementation plan for the task: {task}.
|
||||
Plan file: {plan path}. Read it and find every problem — missing steps,
|
||||
wrong ordering, fragile assumptions, missing error handling, scope creep,
|
||||
underspecified steps. Rate each finding as blocker, major, or minor.
|
||||
Follow the writing-criteria skill (every check is one command the criteria
|
||||
runner accepts, and one that can never go red is a finding) and the
|
||||
reviewing-adversarially skill.
|
||||
End your response with the REQUIRED machine-readable `json` findings block
|
||||
(schema in `agents/plan-critic.md`) so the orchestrator can pipe it to the
|
||||
dedup helper. You have no Write tool — return the block inline, do not write a file.
|
||||
|
|
@ -980,7 +983,10 @@ Task: {task}. Brief file: {brief_path}. Plan file: {plan path}.
|
|||
Find scope creep (plan does more than the brief requires) and scope gaps
|
||||
(plan misses brief requirements). Check that referenced files and functions
|
||||
exist. Verify that every Success Criterion in the brief is covered by the
|
||||
plan's Verification section. End your response with the REQUIRED machine-readable
|
||||
plan's Verification section. Follow the writing-criteria skill for those
|
||||
criteria (each is one command the criteria runner accepts, and it can go red)
|
||||
and the reviewing-adversarially skill.
|
||||
End your response with the REQUIRED machine-readable
|
||||
`json` findings block (schema in `agents/scope-guardian.md`). You have no Write
|
||||
tool — return the block inline, do not write a file.
|
||||
Read local files only inside the repo under exploration, plus paths this prompt gives you (the brief, the plan, and templates it names)."
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue