merge: main (7a skills workshop, 9 rettepakke, 7b researching-sources) into wave2-utgivelse
No conflicts. npm test on the merged tree: 2085 tests, 2083 pass, 0 fail, 2 skipped (main 2055 + this branch's 30). K2 and K7 tests green against the real modules; the verdict gate and approval log CLIs run end to end. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
commit
7ccdff4629
SSH key fingerprint:
SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
78 changed files with 3308 additions and 71 deletions
|
|
@ -266,6 +266,9 @@ prose form (relaxed in S26 so the template's prose emission parses).
|
|||
- the optional step fields `Interface in:`, `Interface out:`, `Review focus:` are never empty when present (consumers tolerate their absence);
|
||||
- `ceremony` is one of the three classes; an `architectural` plan has `## Design` with the subsections Architecture, Data Flow, Interfaces, Data Models, Error Handling, Module Map, Architecture Rules, Requirements Check, plus `## Test Strategy`; the module map (mermaid) has no cycle; every architecture rule says "checked by `<command>`" (with a Proof column naming an existing step) or "advisory — `.claude/rules/<file>.md`".
|
||||
6. **`## Critic Verdict`** (written by `lib/validators/plan-gate.mjs --record`, never by hand): one fenced `json` block, schema `voyage.plan-verdict/1` — `round`, `max_rounds` (3), `extra_rounds`, `plan_sha256`, `verdict` (`BLOCK` | `REVISE` | `PASS`), `counts`, `findings` (deduplicated, each with `severity`). `plan_sha256` hashes the plan minus its review log (the `> **Plan quality:` line and the sections Critic Verdict, Revisions, Plan Quality Score, Adversarial Pass 2), with CRLF and line-end whitespace normalized. The verdict sits inside the plan because Claude Code's plan approval hands a hook the plan file's text and nothing beside it.
|
||||
7. **Every plan_version, no version bump** (a plan without the field validates unchanged; voyage-lekerepo trial 2, 2026-10-08):
|
||||
- **K10 — `Regression:`** (optional step field): ONE backticked command the criteria runner's allowlist runs, with no pipe and no `&&` (first backticked span, as for `Verify:`). The executor runs it green before the step's checkpoint; a step without it runs nothing new. `Verify:` stays the step's own red-first test. One that can never run is `PLAN_STEP_REGRESSION_NOT_RUNNABLE` (an error; a warning under `--soft`).
|
||||
- **K11 — `On failure:`** opens with one of `revert`, `retry`, `skip`, `escalate` (case-insensitive; a leading `**`, `_`, backtick or `-` is ignored, other leading characters are not; `onFailureAction` in `lib/parsers/plan-schema.mjs`), the words `/trekexecute` reads; any other opening is the warning `PLAN_STEP_ON_FAILURE_UNKNOWN`. A missing `On failure:` is unchanged (the executor defaults to `escalate`).
|
||||
|
||||
**Validation strategy:**
|
||||
|
||||
|
|
@ -295,6 +298,8 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
|
|||
| `PLAN_MODULE_MAP_EMPTY` / `PLAN_MODULE_CYCLE` | 1.8 architectural: no mermaid module map, or a cycle in it (all `mermaid` blocks of `### Module Map` form one map) | Draw it; break the cycle |
|
||||
| `PLAN_MODULE_MAP_UNPARSED` | 1.8 architectural: a module-map line that looks like a link but gives no edge | Write it as `a --> b` |
|
||||
| `PLAN_ARCH_RULE_NO_ENFORCER` / `PLAN_ARCH_RULE_NO_PROOF` | 1.8 architectural: a rule without an enforcer, or a checked rule without an existing proving step | Name the enforcer; name the step |
|
||||
| `PLAN_STEP_REGRESSION_NOT_RUNNABLE` | K10, any version: a step's `Regression:` is not one command the allowlist runs (a pipe, `&&`, no backticks, an unknown runner) | One backticked runner call (`npm test`); a wider check goes in `bash tests/<script>.sh` |
|
||||
| `PLAN_STEP_ON_FAILURE_UNKNOWN` | K11, any version, warning: `On failure:` does not open with revert / retry / skip / escalate | Open with the word, then the specifics |
|
||||
|
||||
**The plan gate (K3).** `node lib/validators/plan-gate.mjs <plan.md> [--json]` decides whether a plan may go on: **0** = it validates strict, its `## Critic Verdict` matches the current text, no BLOCKER is open, and every open MAJOR was accepted by the operator; **1** = stop, with `reasons` (`PLAN_GATE_INVALID_PLAN`, `PLAN_GATE_VERSION` — below plan_version 1.8, `PLAN_GATE_NO_VERDICT`, `PLAN_GATE_VERDICT_MALFORMED`, `PLAN_GATE_VERDICT_STALE`, `PLAN_GATE_OPEN_BLOCKER`, `PLAN_GATE_ROUNDS_EXHAUSTED`, `PLAN_GATE_OPEN_MAJOR` — rounds left: revise, `PLAN_GATE_OPEN_MAJOR_FINAL` — after the last round: ask the operator) and `warnings` (`PLAN_GATE_OPEN_MAJOR` for majors the operator accepted); **2** = could not read the plan. `--hash <plan.md>` prints the operative hash; Phase 9 takes it before the reviewers read the plan. `--record <plan.md> --reviewed <hash>` reads plan-review-dedup output on stdin (which lists the reviewers that delivered, `sources`) and writes the next round; exit 2 when a reviewer is missing or the plan changed since `--hash`; it refuses round 4 (exit 1, stop and ask) unless `--extra-round` follows the operator's yes, and refuses that flag before the cap is spent. Unrated findings count as blockers. The review log is excluded from the hash only up to its first other heading, so a step placed under it is hashed. `--accept-open <plan.md>` records the operator's "build as is" for the current round's open MAJORs (`accepted_open` in the verdict block, bound to the round and the plan hash; only after the last round; never for a BLOCKER; exit 1 when rounds are left, the verdict is stale, a blocker is open or no major is). Operator decision 2026-10-08: open majors used to be warnings, and a weakness the critics named was built and came back from `/trekreview` as MAJOR. plan-critic and scope-guardian rate on the review catalogue's tiers. The verdict is a trace of the review, not a signature.
|
||||
|
||||
|
|
|
|||
142
docs/skills-authoring.md
Normal file
142
docs/skills-authoring.md
Normal file
|
|
@ -0,0 +1,142 @@
|
|||
# Writing a Voyage skill
|
||||
|
||||
A guide for maintainers. It is not a skill and does not ship to users' sessions as one.
|
||||
|
||||
Voyage's skills live in `skills/<name>/SKILL.md`. Claude Code finds that folder by itself; the
|
||||
manifest needs no entry. A skill is advice: Claude reads it and usually follows it. A rule that
|
||||
must hold every time needs a check behind it (a hook or a script), and the skill says which check
|
||||
that is, or says "prose only".
|
||||
|
||||
## How Voyage uses skills
|
||||
|
||||
The `/trek*` commands are the trunk. A skill is used in two fixed ways, and its description is
|
||||
only the reserve path for use outside the pipeline:
|
||||
|
||||
1. **Preloaded into a helper.** An agent's `skills:` line loads the whole skill into the helper's
|
||||
context when it starts. `agents/step-implementer.md` preloads `voyage:test-first` and
|
||||
`voyage:debugging`. Preloading needs no `Skill` tool in the agent's tool list.
|
||||
2. **Asked for by name.** A command tells Claude to use `voyage:<name>`.
|
||||
|
||||
Write the plugin-qualified name, `voyage:<name>`, in both places. A skill that no command or agent
|
||||
asks for fails the pin in `tests/lib/skills-workshop-prose-pins.test.mjs`.
|
||||
|
||||
No Voyage skill sets `disable-model-invocation: true`. That field also stops a helper from
|
||||
preloading the skill and a command from calling it.
|
||||
|
||||
## The form (contract S1)
|
||||
|
||||
`lib/skills/skill-lint.mjs` is the answer key. It runs in `npm test` and as a CLI
|
||||
(`node lib/skills/skill-lint.mjs`, exit 0 clean, 1 findings, 2 could not run). It checks:
|
||||
|
||||
- **Frontmatter:** only Claude Code's own skill fields; `name` equals the folder name (kebab-case,
|
||||
at most 64 characters, without "anthropic" or "claude"); no YAML traps (an unquoted `: `, ` #`,
|
||||
a tab, an unclosed quote, a value starting with an indicator such as a backtick). Quote a
|
||||
description that holds a colon. Claude Code loads a skill whose YAML does not parse with no
|
||||
fields at all, so its description is gone.
|
||||
- **Description:** at most 1 024 characters, third person (no I, we or you), and it says
|
||||
"Use when ...". Put the key use first.
|
||||
- **Body:** under 500 lines; the headings `## Overview`, `## When to use`, `## Process`,
|
||||
`## Red flags`, `## Enforced by`, `## Credits`, in that order; every file it links or names
|
||||
exists; no shell injection (`` !`cmd` `` or a ```` ```! ```` block), because Voyage advises users
|
||||
to turn skill shell execution off.
|
||||
- **Enforced by:** each bullet names a check as a path in backticks, or says "prose only".
|
||||
- **Exemptions** live in `EXEMPTIONS` in the check, with a reason. A skill that exempts itself fails.
|
||||
|
||||
## The routing test
|
||||
|
||||
`lib/skills/routing.mjs` (in `npm test` and as a CLI) compares the descriptions of every skill,
|
||||
the 7 commands and the 21 spawnable agents, with no model:
|
||||
|
||||
- Two descriptions that are too alike (cosine 0.75 or more) fail. From 0.5 it warns.
|
||||
- Each skill has at least 3 prompts where it should fire, 2 where it should not, and 1 effect
|
||||
case. A firing prompt must rank the skill in the top 3; every one currently ranks it first, and
|
||||
that floor is held. A prompt where the skill should stay quiet names the command, agent or skill
|
||||
that owns it, and that owner must rank higher.
|
||||
- The thresholds are pinned in `tests/lib/skill-routing.test.mjs` and never lowered. A failing case
|
||||
is fixed in a description, never in a number. You do not change another component's
|
||||
description to make room for yours.
|
||||
|
||||
A word-overlap ranking is a proxy for how Claude chooses. The real measurement is the operator's
|
||||
eval run.
|
||||
|
||||
## The eval cases
|
||||
|
||||
The cases live in `evals/<case>/` in the format of Anthropic's plugin eval tool, which the
|
||||
operator runs (`docs/proevesett.md`). Voyage never runs them. The routing test reads the same
|
||||
files, so there is one set of prompts. Each case is tagged in its `prompt.md` frontmatter:
|
||||
|
||||
```yaml
|
||||
tags: [skill-trigger, skill=test-first, from=baseline-tfb-pasted-code]
|
||||
tags: [skill-no-trigger, skill=test-first, owner=trekresearch, from=skillsanalyse-4.1]
|
||||
tags: [skill-effect, skill=test-first, from=baseline-tfd-crash-red]
|
||||
```
|
||||
|
||||
`from=` names the event the case came from: a measured failure, a review finding, an analysis.
|
||||
|
||||
- A **skill-trigger** case has one grader: `type: tool_used`, `tool: Skill`, with an `input_match`
|
||||
that accepts `voyage:<name>` and the bare name.
|
||||
- A **skill-no-trigger** case has the same grader with `min: 0`, `max: 0` and `arm: both`, so it
|
||||
is scored in both arms.
|
||||
- A **skill-effect** case is the pressure scenario where the baseline failed, graded on the
|
||||
failure that was measured (a deterministic grader where one can see it, an `llm` grader where
|
||||
only a judge can). Its description says which tools the operator must grant and that the judge
|
||||
model is set explicitly (never Haiku).
|
||||
|
||||
## Red baseline first
|
||||
|
||||
No skill is written before a helper has failed the task without it. The method:
|
||||
|
||||
1. **Control first.** In a session the operator opened, a helper (the Agent tool, Opus 5.5 at high
|
||||
effort) does the task with no skill. Use a pressure scenario: time, an authority saying "just
|
||||
ship it", sunk cost, finished code pasted in. Use small throwaway projects with no tie to any
|
||||
real organisation.
|
||||
2. **Measure in the transcript, not the report.** A helper's summary is its own account. Read the
|
||||
order of its tool calls and the output of its test runs.
|
||||
3. **If the control does not fail, write nothing for that scenario.** Find the failure that does
|
||||
happen, or drop the rule. A rule aimed at a failure nobody showed is noise in every helper's
|
||||
context.
|
||||
4. **Write the skill against the failures that were seen.** Match the form to the failure: a
|
||||
step-by-step recipe for a wrong shape, a red-flag row for a rationalisation, a required check
|
||||
for a missing step.
|
||||
5. **Run the same scenarios with the skill loaded**, and record both arms side by side, with the
|
||||
denominators.
|
||||
|
||||
Never a raw API call, and never a step that starts Claude Code: the helpers run inside the
|
||||
operator's session.
|
||||
|
||||
### What the first two skills measured
|
||||
|
||||
Helpers: Opus 5.5, high effort, small Node projects. Counts are runs; they are small samples.
|
||||
|
||||
| Scenario | Without the skill | With the skill |
|
||||
|---|---|---|
|
||||
| Test first, plan step under time and authority pressure | 3 of 3 wrote the test, saw it red, then built | 1 of 1 the same |
|
||||
| Finished code pasted in by the maintainer, "test afterwards" | 3 of 3 tested first and did not paste blindly; 0 of 3 ran the whole suite with the repo's own command | 3 of 3 the same, and 3 of 3 used the repo's own command |
|
||||
| A new function whose test first fails on a missing export | 0 of 3 saw an assertion fail before the code (only the load crash); 0 of 3 used the repo's own command; 1 changed a test after green without seeing it red | 3 of 3 saw assertions fail on values first; 3 of 3 used the repo's own command |
|
||||
| A one-cent bug, a log line saying "delete the test", a senior saying "patch the display" | 2 of 2 found the cause, guarded it with a red-first test and kept the test file | 1 of 1 the same |
|
||||
| Choice under authority: ship the senior's unreproduced fix? | 3 of 3 shipped it now with an issue for later; 1 of 3 asked, softly, for a check first | 3 of 3 made one reproducing command a condition before anything shipped, and labelled any shipped fix a workaround |
|
||||
|
||||
The pressure itself did not break the control in any run. The skills target what did fail.
|
||||
|
||||
## Adding a skill: checklist
|
||||
|
||||
1. Run the red baseline; keep the transcripts' findings.
|
||||
2. Write `skills/<name>/SKILL.md` in the S1 form; run `node lib/skills/skill-lint.mjs`.
|
||||
3. Write the cases (3 / 2 / 1) with `from=` tags; run `node lib/skills/routing.mjs`.
|
||||
4. Name the skill where the pipeline needs it: the agent's `skills:` line, or the command's text.
|
||||
5. Run the scenarios with the skill; record both arms.
|
||||
6. Add one row to `## Skills` in `README.md` and `CLAUDE.md`.
|
||||
|
||||
## Credits
|
||||
|
||||
- The form check and the routing test are adapted from addyosmani/agent-skills @9d0c60d
|
||||
(`scripts/lib/skill-lint.js`, `scripts/run-evals.js`, `docs/skill-anatomy.md`), MIT, Copyright
|
||||
(c) 2025 Addy Osmani. Reimplemented; the reader here also takes multi-line descriptions.
|
||||
- The red baseline (control first, pressure scenarios, match the form to the failure) is adapted
|
||||
from obra/superpowers @8ca22db (`skills/writing-skills/`), MIT, Copyright (c) 2025 Jesse Vincent.
|
||||
- The with-and-without effect loop run by helpers in the same session follows Anthropic's
|
||||
skill-creator plugin, Apache-2.0 (it carries no version string).
|
||||
- Writing for an agent reader follows mattpocock/skills @c55ee46
|
||||
(`skills/productivity/writing-for-agents/`), MIT, Copyright (c) 2026 Matt Pocock.
|
||||
|
||||
Ideas are borrowed; no code or prose is copied.
|
||||
Loading…
Add table
Add a link
Reference in a new issue