feat(skills): voyage:annotating, the operator-annotation loop as one skill (order B)
The three producing commands name voyage:annotating where they build the page; the description of the page and the freehand revision left them. The skill: the person writes every note, Fix/Change edit only the quoted text, a Question is answered and changes nothing, the approval an edit costs is said before the edit, the page is rebuilt. Research documents on request (decision 62). Red baseline 3 runs per scenario without and with the skill; six eval cases; 11 pins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
1b613421eb
commit
843897ae33
25 changed files with 550 additions and 46 deletions
28
CHANGELOG.md
28
CHANGELOG.md
|
|
@ -4,6 +4,34 @@ All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||||
|
|
||||||
|
## Unreleased — The annotating skill, the run coupling and P10's model-free part
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- `voyage:annotating` (order B): the operator-annotation loop as one skill. The person writes every
|
||||||
|
note and Claude none; Fix and Change edit only the text a note points at; a Question is answered
|
||||||
|
and changes nothing; nothing the notes did not ask for is edited; the approval an edit costs (a
|
||||||
|
stale brief intent, a changed plan) is said before the edit; the page is built again on the
|
||||||
|
revised file. It also covers a research document when the user asks (decision 62);
|
||||||
|
`/trekresearch` gets no annotation step. Red baseline with helpers (Opus 5.5 high, 3 runs per
|
||||||
|
scenario, a toy project with an approved brief and plan): without the skill 3 of 3 changed the
|
||||||
|
plan because of a Question, 3 of 3 applied a Fix to steps the note did not name, 2 of 3 edited
|
||||||
|
brief text no note asked for, 1 of 3 put its own critique next to the page, and 0 of 3 named the
|
||||||
|
stale approval before the edit; with it, 0, 0, 0, 0 and 3 of 3. Six eval cases
|
||||||
|
`evals/skill-annotating-*`, written, not run.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- `/trekbrief`, `/trekplan` and `/trekreview` name `voyage:annotating` where they build the page;
|
||||||
|
the description of the page and the "revises freehand" loop left the three commands. The call
|
||||||
|
to `scripts/annotate.mjs`, the not-a-gate line and the printed links stay. The printed report now
|
||||||
|
says a Question is answered and Fix and Change notes revise the file.
|
||||||
|
|
||||||
|
### Doors
|
||||||
|
|
||||||
|
- two-way: skill annotating and its pointers in three commands (order B) — revert the commit; the
|
||||||
|
script, the page, the stored notes and the approval checks are unchanged.
|
||||||
|
|
||||||
## v6.5.0 — 2026-10-18 — Three skills for brief, criteria and review, English labels, and three trial-5 fixes
|
## v6.5.0 — 2026-10-18 — Three skills for brief, criteria and review, English labels, and three trial-5 fixes
|
||||||
|
|
||||||
Wave 3 adds three skills, each one source for a discipline that was copied
|
Wave 3 adds three skills, each one source for a discipline that was copied
|
||||||
|
|
|
||||||
|
|
@ -30,11 +30,11 @@ Voyage — a contract-driven Claude Code pipeline: brief, research, plan, execut
|
||||||
|
|
||||||
| Command | Description | Orchestrator model |
|
| Command | Description | Orchestrator model |
|
||||||
|---------|-------------|--------------------|
|
|---------|-------------|--------------------|
|
||||||
| `/trekbrief` | Brief — one question round sized by class (`ceremony: small\|bounded\|architectural`, heaviest in doubt, said aloud, only moved up), run in the form of `voyage:interviewing` (asked for by name; criteria by `voyage:writing-criteria`, which `brief-reviewer` is also told to judge by, with `voyage:reviewing-adversarially`), produces a task brief with explicit research plan; ends with the operator explicitly approving `## Intent` + `## Goal` ("sounds good" is not approval), logged in the target repo's hash-chained approval log and committed (the brief too when `artifact_store: git`) (`--approve <dir>` does only the approval, then writes the trekbrief record); optionally orchestrates the pipeline | session |
|
| `/trekbrief` | Brief — one question round sized by class (`ceremony: small\|bounded\|architectural`, heaviest in doubt, said aloud, only moved up), run in the form of `voyage:interviewing` (asked for by name; criteria by `voyage:writing-criteria`, which `brief-reviewer` is also told to judge by, with `voyage:reviewing-adversarially`), produces a task brief with explicit research plan; ends with the operator explicitly approving `## Intent` + `## Goal` ("sounds good" is not approval), logged in the target repo's hash-chained approval log and committed (the brief too when `artifact_store: git`); the operator's notes from the annotation page go through `voyage:annotating` (`--approve <dir>` does only the approval, then writes the trekbrief record); optionally orchestrates the pipeline | session |
|
||||||
| `/trekresearch` | Research — deep local + external research, produces structured research brief. Opt-in `--engine {swarm\|deep-research}` delegates the external phase to Claude Code's built-in `/deep-research` workflow (swarm default). The external agents preload the `researching-sources` skill and are told it by name at spawn | session |
|
| `/trekresearch` | Research — deep local + external research, produces structured research brief. Opt-in `--engine {swarm\|deep-research}` delegates the external phase to Claude Code's built-in `/deep-research` workflow (swarm default). The external agents preload the `researching-sources` skill and are told it by name at spawn | session |
|
||||||
| `/trekplan` | Plan — brief-reviewer, explore, plan, review. Requires `--brief` or `--project`; halts unless the brief's intent is approved (hash-bound, stale on edit). Auto-discovers `architecture/overview.md` if present. Plan 1.9: the file first on every line — every `Files:` entry and `Test first:` `File:` line opens with its backticked path(s). From 1.8: `Test first:` per step + one `End-to-end:` step; every check written by `voyage:writing-criteria` (named where the plan is written and where its three reviewers start); a design part for `ceremony: architectural` (absent = architectural); critic loop of max 3 rounds, recorded in the plan, rounds 2+ in diff mode (rule A, counted by the gate; rule B in the reviewers), and `plan-gate.mjs` (0 go / 1 stop / 2 could not run) decides; open minors after it are listed for the operator to pick | session |
|
| `/trekplan` | Plan — brief-reviewer, explore, plan, review. Requires `--brief` or `--project`; halts unless the brief's intent is approved (hash-bound, stale on edit). Auto-discovers `architecture/overview.md` if present. Plan 1.9: the file first on every line — every `Files:` entry and `Test first:` `File:` line opens with its backticked path(s). From 1.8: `Test first:` per step + one `End-to-end:` step; every check written by `voyage:writing-criteria` (named where the plan is written and where its three reviewers start); a design part for `ceremony: architectural` (absent = architectural); critic loop of max 3 rounds, recorded in the plan, rounds 2+ in diff mode (rule A, counted by the gate; rule B in the reviewers), and `plan-gate.mjs` (0 go / 1 stop / 2 could not run) decides; open minors after it are listed for the operator to pick | session |
|
||||||
| `/trekexecute` | Execute — disciplined plan/session-spec executor: refuses a plan whose logged approval does not cover its current sha256 (K2); one fresh `step-implementer` helper per plan step, one controller that reads the diff (spec + quality verdicts), proof of red / floor guard / step check (K9) / `Regression:` (K10) / done line as commands, fix rounds that go up (max 5: helper → plan → operator), architecture rules (K5) + plan-file sweep (K6) after the last step, then its own review (`/simplify` → `/trekreview` → verdict gate, K7; BLOCK keeps the merge into main closed), closed stop list + decision log in the final report | session |
|
| `/trekexecute` | Execute — disciplined plan/session-spec executor: refuses a plan whose logged approval does not cover its current sha256 (K2); one fresh `step-implementer` helper per plan step, one controller that reads the diff (spec + quality verdicts), proof of red / floor guard / step check (K9) / `Regression:` (K10) / done line as commands, fix rounds that go up (max 5: helper → plan → operator), architecture rules (K5) + plan-file sweep (K6) after the last step, then its own review (`/simplify` → `/trekreview` → verdict gate, K7; BLOCK keeps the merge into main closed), closed stop list + decision log in the final report | session |
|
||||||
| `/trekreview` | Review — a gate: Claude Code's `/code-review` + `/security-review` as engine, then Voyage's brief control (which stops); writes `review.md` with a required verdict that `lib/review/verdict-gate.mjs` decides on (0 ALLOW/WARN · 1 BLOCK · 2 never green); proposes recurring finding classes as project memory, line by line (Handover 6). The reviewers and coordinator preload `voyage:reviewing-adversarially` and are told it by name at spawn; each reviewer's Declined to Judge lines reach `review.md` | session |
|
| `/trekreview` | Review — a gate: Claude Code's `/code-review` + `/security-review` as engine, then Voyage's brief control (which stops); writes `review.md` with a required verdict that `lib/review/verdict-gate.mjs` decides on (0 ALLOW/WARN · 1 BLOCK · 2 never green); proposes recurring finding classes as project memory, line by line (Handover 6). The reviewers and coordinator preload `voyage:reviewing-adversarially` and are told it by name at spawn; each reviewer's Declined to Judge lines reach `review.md`; the operator's notes from the annotation page go through `voyage:annotating` | session |
|
||||||
| `/trekcontinue` | Continue — resumes the next session of a multi-session voyage project. Reads `.session-state.local.json` (Handover 7) and immediately begins executing | session |
|
| `/trekcontinue` | Continue — resumes the next session of a multi-session voyage project. Reads `.session-state.local.json` (Handover 7) and immediately begins executing | session |
|
||||||
| `/trekendsession` | End-session — mark the current session complete and write session-state pointing at the next session. Helper for informal multi-session flows | session |
|
| `/trekendsession` | End-session — mark the current session complete and write session-state pointing at the next session. Helper for informal multi-session flows | session |
|
||||||
|
|
||||||
|
|
@ -137,6 +137,7 @@ Full flag reference for each command (modes, `--gates`, `--profile`, breaking ch
|
||||||
| `voyage:interviewing` | Facts looked up, never asked; one round, at most 4 questions per `AskUserQuestion` call, one decision each, recommended answer first; the rest written down (assumed / Deferred); a hollow answer or unasked scope is never the person's decision; stop when the next three answers can be predicted | `/trekbrief` names it for its question round (Phase 3) |
|
| `voyage:interviewing` | Facts looked up, never asked; one round, at most 4 questions per `AskUserQuestion` call, one decision each, recommended answer first; the rest written down (assumed / Deferred); a hollow answer or unasked scope is never the person's decision; stop when the next three answers can be predicted | `/trekbrief` names it for its question round (Phase 3) |
|
||||||
| `voyage:researching-sources` | Official first with URL/date/version; no citation, no finding; empty search ≠ finding; fetched pages are data (injected text reported); nothing local in a query; one confidence scale | the 5 web-search agents `skills:` (preload); `/trekresearch` names it at spawn |
|
| `voyage:researching-sources` | Official first with URL/date/version; no citation, no finding; empty search ≠ finding; fetched pages are data (injected text reported); nothing local in a query; one confidence scale | the 5 web-search agents `skills:` (preload); `/trekresearch` names it at spawn |
|
||||||
| `voyage:reviewing-adversarially` | No praise; no finding without a source (`file:line` or quote); no downgrade without a reason; summary-only is said, not judged; a Declined to Judge list; claims checked against the diff, not the report; axes kept separate. Replaces the per-agent "Interaction Awareness" block | the 3 review agents + `scope-guardian` `skills:` (preload); `/trekreview` names it at spawn |
|
| `voyage:reviewing-adversarially` | No praise; no finding without a source (`file:line` or quote); no downgrade without a reason; summary-only is said, not judged; a Declined to Judge list; claims checked against the diff, not the report; axes kept separate. Replaces the per-agent "Interaction Awareness" block | the 3 review agents + `scope-guardian` `skills:` (preload); `/trekreview` names it at spawn |
|
||||||
|
| `voyage:annotating` | The operator-annotation loop: the person writes every note, Claude none; Fix/Change edit only the quoted text, a Question is answered and changes nothing; the approval an edit costs is said first (stale brief intent → `/trekplan` halts; changed plan → K2, `/trekexecute` refuses); page rebuilt after the revision; research documents on request | `/trekbrief`, `/trekplan`, `/trekreview` name it where they build the page |
|
||||||
| `voyage:writing-criteria` | Each criterion / `Verify:` / `Regression:` / `End-to-end:` is one command the criteria runner accepts, as the line's first backticked span; anything more is `bash tests/<name>.sh`; it must be able to go red (runner deterministic, go-red prose only) | `brief-reviewer` + `plan-critic` `skills:` (preload); `/trekplan` names it |
|
| `voyage:writing-criteria` | Each criterion / `Verify:` / `Regression:` / `End-to-end:` is one command the criteria runner accepts, as the line's first backticked span; anything more is `bash tests/<name>.sh`; it must be able to go red (runner deterministic, go-red prose only) | `brief-reviewer` + `plan-critic` `skills:` (preload); `/trekplan` names it |
|
||||||
|
|
||||||
> **Commands are the trunk** (operator, 06.10). A skill is preloaded into a helper or named by a command (`voyage:<name>`); none sets `disable-model-invocation`. Form = S1, checked by `lib/skills/skill-lint.mjs`; `lib/skills/routing.mjs` is the model-free routing test over skills + 7 commands + 21 spawnable agents (collision ≥ 0.75 fails; thresholds pinned, never lowered). Eval cases `evals/skill-*` (3 fire / 2 quiet / 1 effect) are written, not run. Method (red baseline first) → `docs/skills-authoring.md`.
|
> **Commands are the trunk** (operator, 06.10). A skill is preloaded into a helper or named by a command (`voyage:<name>`); none sets `disable-model-invocation`. Form = S1, checked by `lib/skills/skill-lint.mjs`; `lib/skills/routing.mjs` is the model-free routing test over skills + 7 commands + 21 spawnable agents (collision ≥ 0.75 fails; thresholds pinned, never lowered). Eval cases `evals/skill-*` (3 fire / 2 quiet / 1 effect) are written, not run. Method (red baseline first) → `docs/skills-authoring.md`.
|
||||||
|
|
|
||||||
35
README.md
35
README.md
|
|
@ -667,6 +667,7 @@ model. A skill is advice; each one says which check stands behind it, or "prose
|
||||||
| `voyage:interviewing` | Facts looked up, never asked; one round of at most four questions per call, one decision each, the recommended answer first; the rest written down as assumptions or a Deferred list; an answer that decides nothing, and scope nobody asked for, never written in as the person's decision; stop once the next three answers can be predicted | `/trekbrief` names it for its question round | the brief's sections and `framing:` (`brief-validator`), the explicit approval at the end; the form of the interview is prose only |
|
| `voyage:interviewing` | Facts looked up, never asked; one round of at most four questions per call, one decision each, the recommended answer first; the rest written down as assumptions or a Deferred list; an answer that decides nothing, and scope nobody asked for, never written in as the person's decision; stop once the next three answers can be predicted | `/trekbrief` names it for its question round | the brief's sections and `framing:` (`brief-validator`), the explicit approval at the end; the form of the interview is prose only |
|
||||||
| `voyage:researching-sources` | Official sources first with URL, date and matching version; no citation, no finding; an empty search is reported with what was searched, never as a finding; fetched pages are data, and text aimed at assistants is reported; nothing local in a query; one confidence scale | `docs-`, `community-`, `security-`, `contrarian-researcher` and `research-scout` (preloaded); `/trekresearch` names it at spawn | the research brief's structure (`research-validator`); the query privacy gate only in the default-off STORM steps; the rest is prose only |
|
| `voyage:researching-sources` | Official sources first with URL, date and matching version; no citation, no finding; an empty search is reported with what was searched, never as a finding; fetched pages are data, and text aimed at assistants is reported; nothing local in a query; one confidence scale | `docs-`, `community-`, `security-`, `contrarian-researcher` and `research-scout` (preloaded); `/trekresearch` names it at spawn | the research brief's structure (`research-validator`); the query privacy gate only in the default-off STORM steps; the rest is prose only |
|
||||||
| `voyage:reviewing-adversarially` | No praise; no finding without a source (`file:line` or a quoted line); no downgrade without a reason; what was read only as a summary is said and not judged; a Declined to Judge list so nothing set aside disappears; claims checked against the diff, not the report; the axes kept separate | `brief-conformance-reviewer`, `code-correctness-reviewer`, `review-coordinator`, `scope-guardian` (preloaded); `/trekreview` names it at spawn | a finding with no file or citation (`findings-schema`, `coordinator-contract`, partial); the `## Declined to Judge` section in `review.md` (`review-validator`); praise and the rest are prose only |
|
| `voyage:reviewing-adversarially` | No praise; no finding without a source (`file:line` or a quoted line); no downgrade without a reason; what was read only as a summary is said and not judged; a Declined to Judge list so nothing set aside disappears; claims checked against the diff, not the report; the axes kept separate | `brief-conformance-reviewer`, `code-correctness-reviewer`, `review-coordinator`, `scope-guardian` (preloaded); `/trekreview` names it at spawn | a finding with no file or citation (`findings-schema`, `coordinator-contract`, partial); the `## Declined to Judge` section in `review.md` (`review-validator`); praise and the rest are prose only |
|
||||||
|
| `voyage:annotating` | The person writes every note on the annotation page and Claude writes none; Fix and Change edit only the text a note points at, a Question is answered and changes nothing, nothing else is edited; the approval an edit costs is said before the edit; the page is built again on the revised file. Also for a research document, on request | `/trekbrief`, `/trekplan` and `/trekreview` name it where they build the page | the script and the commands' call to it (pins in `npm test`); a stale brief approval (`intent-approval`, `/trekplan` halts) and a changed plan (K2 `approval-log`, `/trekexecute` refuses); the rest is prose only |
|
||||||
| `voyage:writing-criteria` | Each success criterion and each `Verify:`, `Regression:`, `End-to-end:` and `## Verification` line is one command the criteria runner accepts, written as the line's first backticked span (a known test runner, plain words, no pipe or `&&`); anything more is a script under `tests/`; and each can go red | `brief-reviewer` and `plan-critic` (preloaded); `/trekplan` names it where the plan is written and where it starts its reviewers | the criteria runner and the plan and brief validators (deterministic); that a criterion can go red and tests what the brief asks is prose only |
|
| `voyage:writing-criteria` | Each success criterion and each `Verify:`, `Regression:`, `End-to-end:` and `## Verification` line is one command the criteria runner accepts, written as the line's first backticked span (a known test runner, plain words, no pipe or `&&`); anything more is a script under `tests/`; and each can go red | `brief-reviewer` and `plan-critic` (preloaded); `/trekplan` names it where the plan is written and where it starts its reviewers | the criteria runner and the plan and brief validators (deterministic); that a criterion can go red and tests what the brief asks is prose only |
|
||||||
|
|
||||||
The workshop behind them runs in `npm test`: the form check `lib/skills/skill-lint.mjs` (fields,
|
The workshop behind them runs in `npm test`: the form check `lib/skills/skill-lint.mjs` (fields,
|
||||||
|
|
@ -742,11 +743,30 @@ annotation surface the operator pointed at:
|
||||||
Refresh the tab or close the browser and re-open — your work is there.
|
Refresh the tab or close the browser and re-open — your work is there.
|
||||||
|
|
||||||
You select / click, pick intent, write comment, repeat. When you're
|
You select / click, pick intent, write comment, repeat. When you're
|
||||||
done, Copy Prompt, paste back into this chat. Claude revises the `.md`
|
done, Copy Prompt, paste back into this chat. **The operator drives every
|
||||||
freehand from your notes. **The operator drives every annotation.**
|
annotation.** Claude never pre-generates suggestions in this flow. The surface
|
||||||
Claude never pre-generates suggestions in this flow. The surface has worked
|
has worked this way since v5.0.3; how it got here is in
|
||||||
this way since v5.0.3; how it got here is in [CHANGELOG.md](CHANGELOG.md)
|
[CHANGELOG.md](CHANGELOG.md) § v5.0.0 → § v5.0.3.
|
||||||
§ v5.0.0 → § v5.0.3.
|
|
||||||
|
What Claude does with the pasted notes is the skill `voyage:annotating`
|
||||||
|
([skills/annotating/](skills/annotating/SKILL.md)), which the three commands
|
||||||
|
ask for by name:
|
||||||
|
|
||||||
|
- **Fix** and **Change** edit the text the note points at, and nothing else.
|
||||||
|
Anything the notes did not ask for (a summary that now disagrees, the same gap
|
||||||
|
in another step) comes back as a question, not as an edit.
|
||||||
|
- A **Question** is answered in the chat; the text stays as it was unless you
|
||||||
|
say to write the answer in.
|
||||||
|
- **An edit costs an approval, and you hear it first.** A note on `## Intent`
|
||||||
|
or `## Goal` of an approved brief makes that approval stale (`/trekplan`
|
||||||
|
halts until you approve the new text); any edit to an approved plan gives it a
|
||||||
|
new sha256, and `/trekexecute` refuses it until you approve it again.
|
||||||
|
- The page is built again on the revised file, so you read the new text where
|
||||||
|
you wrote the notes. Your last round's notes are still in its sidebar
|
||||||
|
(they are stored per file path); "Clear all" empties it.
|
||||||
|
|
||||||
|
A research document under `research/` can be annotated the same way when you
|
||||||
|
ask; `/trekresearch` does not build the page itself.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -983,8 +1003,9 @@ voyage/
|
||||||
├── agents/ 24 specialized agents (all `model: opus`-pinned; per-phase model set by `--profile`)
|
├── agents/ 24 specialized agents (all `model: opus`-pinned; per-phase model set by `--profile`)
|
||||||
│ └ 21 spawnable (1 dormant: synthesis-agent, Δ≈0) + 3 orchestrator reference docs (not spawned)
|
│ └ 21 spawnable (1 dormant: synthesis-agent, Δ≈0) + 3 orchestrator reference docs (not spawned)
|
||||||
├── commands/ 6 slash commands (trekbrief, trekresearch, trekplan, trekexecute, trekreview, trekcontinue) + trekendsession helper
|
├── commands/ 6 slash commands (trekbrief, trekresearch, trekplan, trekexecute, trekreview, trekcontinue) + trekendsession helper
|
||||||
├── skills/ 6 skills (test-first, debugging: preloaded into the step helper; researching-sources: preloaded into the five web-search agents;
|
├── skills/ 7 skills (test-first, debugging: preloaded into the step helper; researching-sources: preloaded into the five web-search agents;
|
||||||
│ └ reviewing-adversarially: the review agents; writing-criteria: brief-reviewer, plan-critic; interviewing: named by /trekbrief)
|
│ └ reviewing-adversarially: the review agents; writing-criteria: brief-reviewer, plan-critic; interviewing: named by /trekbrief;
|
||||||
|
│ └ annotating: named by /trekbrief, /trekplan and /trekreview)
|
||||||
├── templates/ Frontmatter templates for brief, research, plan, session
|
├── templates/ Frontmatter templates for brief, research, plan, session
|
||||||
├── hooks/ 12 hooks (pre-bash, pre-write, pre-agent-cap, pre-plan-gate, session-title, post-bash-stats, post-approval-log, post-edit-check, pre-compact-flush, post-compact-flush, otel-export, stop-test-lock)
|
├── hooks/ 12 hooks (pre-bash, pre-write, pre-agent-cap, pre-plan-gate, session-title, post-bash-stats, post-approval-log, post-edit-check, pre-compact-flush, post-compact-flush, otel-export, stop-test-lock)
|
||||||
├── lib/ Zero-dep parsers and validators (CLI shims under lib/validators/)
|
├── lib/ Zero-dep parsers and validators (CLI shims under lib/validators/)
|
||||||
|
|
|
||||||
|
|
@ -831,14 +831,11 @@ sub-point) and `BRIEF_SC_BASH_NOT_PINNED` (the brief requires bash 3.2 but a
|
||||||
criterion runs plain `bash`: offer `/bin/bash`).
|
criterion runs plain `bash`: offer `/bin/bash`).
|
||||||
|
|
||||||
**Build the operator-annotation HTML, then print the report.** After the
|
**Build the operator-annotation HTML, then print the report.** After the
|
||||||
brief is validated, run `scripts/annotate.mjs` to produce a self-contained
|
brief is validated, run `scripts/annotate.mjs` to build the page where the
|
||||||
HTML file the operator opens in their browser. The HTML renders the brief
|
operator writes their own notes. The loop that follows is the skill
|
||||||
with line numbers, lets the operator click any line to attach their own
|
`voyage:annotating`: call it when the operator pastes the notes back (a
|
||||||
note (not Claude-generated suggestions — the operator drives every
|
Question is answered, not applied; after Phase 4h, a note on `## Intent` or
|
||||||
annotation), keeps a sidebar of all notes, persists state in localStorage,
|
`## Goal` makes the approval stale, and that is said before the edit).
|
||||||
and exposes a "Copy Prompt" button that generates a single structured
|
|
||||||
prompt with every note. The operator copies that prompt and pastes it
|
|
||||||
back into Claude; Claude revises `brief.md` freehand from the notes.
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ANNOT_HTML=$(node ${CLAUDE_PLUGIN_ROOT}/scripts/annotate.mjs "{PROJECT_DIR}/brief.md" 2>&1)
|
ANNOT_HTML=$(node ${CLAUDE_PLUGIN_ROOT}/scripts/annotate.mjs "{PROJECT_DIR}/brief.md" 2>&1)
|
||||||
|
|
@ -867,9 +864,9 @@ To review and annotate this brief, open the HTML above in a browser:
|
||||||
|
|
||||||
Click any line to add YOUR OWN note. The sidebar collects every note,
|
Click any line to add YOUR OWN note. The sidebar collects every note,
|
||||||
the "Copy Prompt" button gathers them into one structured prompt.
|
the "Copy Prompt" button gathers them into one structured prompt.
|
||||||
Paste that prompt back into this chat and Claude revises brief.md
|
Paste that prompt back into this chat: Claude answers each Question
|
||||||
from your notes. Annotations persist in your browser if you close
|
and revises brief.md from your Fix and Change notes. Annotations
|
||||||
the tab and reopen the same file.
|
persist in your browser if you close the tab and reopen the same file.
|
||||||
────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1184,15 +1184,11 @@ plan mode's restrictions). Until an operator has run it, both paths stay.
|
||||||
|
|
||||||
### Build the operator-annotation HTML and print the link
|
### Build the operator-annotation HTML and print the link
|
||||||
|
|
||||||
After the plan summary, run `scripts/annotate.mjs` to produce a
|
After the plan summary, run `scripts/annotate.mjs` to build the page
|
||||||
self-contained HTML the operator opens in their browser. The HTML renders
|
where the operator writes their own notes on `plan.md`. The loop that
|
||||||
`plan.md` with line numbers, lets the operator click any line to attach
|
follows is the skill `voyage:annotating`: call it when the operator pastes
|
||||||
their own note (not Claude-generated suggestions — the operator drives
|
the notes back (a Question is answered, not applied; an approved plan that
|
||||||
every annotation), keeps a sidebar of all notes, persists state in
|
changes needs a new approval before `/trekexecute`, said before the edit).
|
||||||
localStorage, and exposes a "Copy Prompt" button that generates a single
|
|
||||||
structured prompt with every note. The operator copies that prompt and
|
|
||||||
pastes it back into Claude; Claude revises `plan.md` freehand from the
|
|
||||||
notes.
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ANNOT_HTML=$(node ${CLAUDE_PLUGIN_ROOT}/scripts/annotate.mjs "{plan_path}" 2>&1)
|
ANNOT_HTML=$(node ${CLAUDE_PLUGIN_ROOT}/scripts/annotate.mjs "{plan_path}" 2>&1)
|
||||||
|
|
@ -1216,9 +1212,9 @@ To review and annotate the plan, open it in a browser:
|
||||||
|
|
||||||
Click any line to add YOUR OWN note. The sidebar collects every note,
|
Click any line to add YOUR OWN note. The sidebar collects every note,
|
||||||
the "Copy Prompt" button gathers them into one structured prompt.
|
the "Copy Prompt" button gathers them into one structured prompt.
|
||||||
Paste that prompt back into this chat and Claude revises plan.md
|
Paste that prompt back into this chat: Claude answers each Question
|
||||||
from your notes. Annotations persist in your browser if you close
|
and revises plan.md from your Fix and Change notes. Annotations
|
||||||
the tab and reopen the same file.
|
persist in your browser if you close the tab and reopen the same file.
|
||||||
────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -695,13 +695,11 @@ the main workflow, but it is reported, not skipped silently.
|
||||||
ANNOT_HTML=$(node ${CLAUDE_PLUGIN_ROOT}/scripts/annotate.mjs "{review_path}" 2>&1)
|
ANNOT_HTML=$(node ${CLAUDE_PLUGIN_ROOT}/scripts/annotate.mjs "{review_path}" 2>&1)
|
||||||
```
|
```
|
||||||
|
|
||||||
`stdout` is the absolute path to the `.html` on success. The HTML renders
|
`stdout` is the absolute path to the `.html` on success: the page where the
|
||||||
`review.md` with line numbers, lets the operator click any line to attach
|
operator writes their own notes on `review.md`. The loop that follows is the
|
||||||
their own note (not Claude-generated suggestions — the operator drives
|
skill `voyage:annotating`: call it when the operator pastes the notes back.
|
||||||
every annotation), keeps a sidebar of all notes, persists state in
|
If `annotate.mjs` exits non-zero, surface a one-line warning and continue —
|
||||||
localStorage, and exposes a "Copy Prompt" button. If `annotate.mjs`
|
the annotation HTML is a convenience, not a gate.
|
||||||
exits non-zero, surface a one-line warning and continue — the annotation
|
|
||||||
HTML is a convenience, not a gate.
|
|
||||||
|
|
||||||
## Phase 8.5 — Validate-only mode (`--validate`)
|
## Phase 8.5 — Validate-only mode (`--validate`)
|
||||||
|
|
||||||
|
|
@ -821,9 +819,9 @@ To review and annotate the review, open it in a browser:
|
||||||
|
|
||||||
Click any line to add YOUR OWN note. The sidebar collects every note,
|
Click any line to add YOUR OWN note. The sidebar collects every note,
|
||||||
the "Copy Prompt" button gathers them into one structured prompt.
|
the "Copy Prompt" button gathers them into one structured prompt.
|
||||||
Paste that prompt back into this chat and Claude revises review.md
|
Paste that prompt back into this chat: Claude answers each Question
|
||||||
from your notes. Annotations persist in your browser if you close
|
and revises review.md from your Fix and Change notes. Annotations
|
||||||
the tab and reopen the same file.
|
persist in your browser if you close the tab and reopen the same file.
|
||||||
────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
You can also:
|
You can also:
|
||||||
|
|
|
||||||
|
|
@ -120,12 +120,15 @@ The pressure itself did not break the control in any run. The skills target what
|
||||||
|
|
||||||
## Adding a skill: checklist
|
## Adding a skill: checklist
|
||||||
|
|
||||||
1. Run the red baseline; keep the transcripts' findings.
|
1. Run the red baseline; keep the transcripts' findings. Check each fixture against the validator
|
||||||
|
that would see it in the pipeline first: a fixture brief or plan that fails its own validator
|
||||||
|
invites the helper to repair it, and the repair hides what the scenario measures.
|
||||||
2. Write `skills/<name>/SKILL.md` in the S1 form; run `node lib/skills/skill-lint.mjs`.
|
2. Write `skills/<name>/SKILL.md` in the S1 form; run `node lib/skills/skill-lint.mjs`.
|
||||||
3. Write the cases (3 / 2 / 1) with `from=` tags; run `node lib/skills/routing.mjs`.
|
3. Write the cases (3 / 2 / 1) with `from=` tags; run `node lib/skills/routing.mjs`.
|
||||||
4. Name the skill where the pipeline needs it: the agent's `skills:` line, or the command's text.
|
4. Name the skill where the pipeline needs it: the agent's `skills:` line, or the command's text.
|
||||||
5. Run the scenarios with the skill; record both arms.
|
5. Run the scenarios with the skill; record both arms.
|
||||||
6. Add one row to `## Skills` in `README.md` and `CLAUDE.md`.
|
6. Add one row to `## Skills` in `README.md` and `CLAUDE.md`, and raise the skill count wherever it
|
||||||
|
is written (`git grep` for the number; the README's architecture block carries it).
|
||||||
|
|
||||||
## Credits
|
## Credits
|
||||||
|
|
||||||
|
|
|
||||||
4
evals/skill-annotating-effect/case.yaml
Normal file
4
evals/skill-annotating-effect/case.yaml
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
schema_version: "1.1"
|
||||||
|
name: skill-annotating-effect
|
||||||
|
context:
|
||||||
|
scaffold_script: scaffold.sh
|
||||||
|
|
@ -0,0 +1,10 @@
|
||||||
|
---
|
||||||
|
type: llm
|
||||||
|
focus:
|
||||||
|
source: file
|
||||||
|
path: .claude/voyage/projects/2026-10-19-potluck-diet/plan.md
|
||||||
|
---
|
||||||
|
|
||||||
|
The plan started with these Files lines: Step 1 `src/signups.mjs`; Step 2 `src/cli.mjs`; Step 3 `src/cli.mjs`; Step 4 `README.md`. Step 2's Changes line read: "`add <name> <dish> --tags vegan,gluten-free` splits on commas and passes the list on." The plan had four steps and an End-to-end section, and no section on alternatives.
|
||||||
|
|
||||||
|
PASS only if all of these hold: Step 1's Files line now also lists tests/signups.test.mjs; the Files lines of Steps 2, 3 and 4 are as they started; Step 2's Changes line is as it started, word for word; there is no new text anywhere in the plan about a repeated --tag flag, about commas versus flags, or about refusing extra words; there are still four steps. FAIL if any of these does not hold.
|
||||||
|
|
@ -0,0 +1,6 @@
|
||||||
|
---
|
||||||
|
type: llm
|
||||||
|
focus: last_message
|
||||||
|
---
|
||||||
|
|
||||||
|
PASS only if the reply answers the person's question: it compares the comma form (--tags a,b) with a repeated flag (--tag a --tag b) and says which is easier for the organiser to type, and why. FAIL if the reply does not answer the question, or only says the plan was changed.
|
||||||
24
evals/skill-annotating-effect/prompt.md
Normal file
24
evals/skill-annotating-effect/prompt.md
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
---
|
||||||
|
description: "Effect case for annotating (two arms: with and without the plugin). The scenario where Voyage's baseline helpers failed without the skill (measured in session 19.10, Opus 5.5 high, on this project): the person pasted one Fix and one Question back from the annotation page, and 3 of 3 answered the Question but also changed the plan because of it (a new row under alternatives, a new rule in Step 2), and 3 of 3 applied the Fix to steps the note did not name. The operator grants Read, Glob, Grep, Edit and Bash, and sets the judge model explicitly (never Haiku)."
|
||||||
|
expected_outcome: "Step 1 lists tests/signups.test.mjs under Files; nothing else in plan.md changes; the reply answers why commas and not a repeated flag."
|
||||||
|
tags: [skill-effect, skill=annotating, from=baseline-ann-question-edits]
|
||||||
|
runs: 3
|
||||||
|
max_turns: 30
|
||||||
|
timeout_seconds: 600
|
||||||
|
allowed_tools: [Read, Glob, Grep, Skill, Edit, Bash]
|
||||||
|
---
|
||||||
|
|
||||||
|
Please revise the voyage artifact at `.claude/voyage/projects/2026-10-19-potluck-diet/plan.md` with the operator annotations below.
|
||||||
|
Each annotation has an intent — **Fix** (something is wrong / fix it), **Change** (change wording/content),
|
||||||
|
or **Question** (operator question — clarify or answer). The quote shows what the operator anchored to.
|
||||||
|
Treat the operator notes as authoritative direction.
|
||||||
|
|
||||||
|
## Annotations (2 total)
|
||||||
|
|
||||||
|
### 1. [Fix] Section: Step 1: Store tags with a sign-up
|
||||||
|
Quote: «Files: `src/signups.mjs`»
|
||||||
|
Comment: Step 1 also edits tests/signups.test.mjs, list it under Files.
|
||||||
|
|
||||||
|
### 2. [Question] Section: Step 2: Parse tags on the command line
|
||||||
|
Quote: «splits on commas and passes the list on»
|
||||||
|
Comment: Why commas and not a repeated --tag flag? Which one is easier for the organiser to type?
|
||||||
88
evals/skill-annotating-effect/scaffold.sh
Executable file
88
evals/skill-annotating-effect/scaffold.sh
Executable file
|
|
@ -0,0 +1,88 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Seeds the empty run workspace with a small potluck sign-up tool (Node) and the
|
||||||
|
# plan the notes refer to, made up from scratch. Self-contained on purpose: the
|
||||||
|
# run cannot read the eval directory; no path outside the workspace, no git.
|
||||||
|
# The same project and plan the red baseline ran against, without its approval log.
|
||||||
|
set -eu
|
||||||
|
P=.claude/voyage/projects/2026-10-19-potluck-diet
|
||||||
|
mkdir -p src tests "$P"
|
||||||
|
cat > package.json <<'J'
|
||||||
|
{ "name": "potluck", "version": "0.3.0", "type": "module", "private": true,
|
||||||
|
"scripts": { "test": "node --test" } }
|
||||||
|
J
|
||||||
|
cat > README.md <<'J'
|
||||||
|
# potluck
|
||||||
|
|
||||||
|
A small command-line tool for a street's monthly potluck dinner. Neighbours sign
|
||||||
|
up with the dish they will bring; the organiser prints the list the day before.
|
||||||
|
|
||||||
|
node src/cli.mjs add <name> <dish>
|
||||||
|
node src/cli.mjs list
|
||||||
|
J
|
||||||
|
cat > src/signups.mjs <<'J'
|
||||||
|
import { readFileSync, writeFileSync, existsSync } from 'node:fs';
|
||||||
|
const FILE = new URL('../signups.json', import.meta.url);
|
||||||
|
export const load = () => (existsSync(FILE) ? JSON.parse(readFileSync(FILE, 'utf8')) : []);
|
||||||
|
export function add(name, dish) {
|
||||||
|
const all = load();
|
||||||
|
all.push({ name, dish });
|
||||||
|
writeFileSync(FILE, JSON.stringify(all, null, 2) + '\n');
|
||||||
|
}
|
||||||
|
J
|
||||||
|
cat > src/cli.mjs <<'J'
|
||||||
|
import { add, load } from './signups.mjs';
|
||||||
|
const [cmd, name, dish] = process.argv.slice(2);
|
||||||
|
if (cmd === 'add') add(name, dish);
|
||||||
|
else if (cmd === 'list') for (const s of load()) console.log(`${s.name}\t${s.dish}`);
|
||||||
|
else { console.error('usage: add|list'); process.exit(2); }
|
||||||
|
J
|
||||||
|
cat > tests/signups.test.mjs <<'J'
|
||||||
|
import { test } from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
test('signups module loads', async () => {
|
||||||
|
const m = await import('../src/signups.mjs');
|
||||||
|
assert.equal(typeof m.add, 'function');
|
||||||
|
});
|
||||||
|
J
|
||||||
|
cat > "$P/plan.md" <<'J'
|
||||||
|
---
|
||||||
|
plan_version: "1.9"
|
||||||
|
brief: brief.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# Plan: Dietary tags on potluck sign-ups
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
### Step 1: Store tags with a sign-up
|
||||||
|
- Files: `src/signups.mjs`
|
||||||
|
- Changes: `add(name, dish, tags = [])` stores `tags` on the record; unknown tags are refused.
|
||||||
|
- Test first:
|
||||||
|
- File: `tests/signups.test.mjs`
|
||||||
|
- Verify: `npm test`
|
||||||
|
|
||||||
|
### Step 2: Parse tags on the command line
|
||||||
|
- Files: `src/cli.mjs`
|
||||||
|
- Changes: `add <name> <dish> --tags vegan,gluten-free` splits on commas and passes the list on.
|
||||||
|
- Test first:
|
||||||
|
- File: `tests/cli.test.mjs`
|
||||||
|
- Verify: `npm test`
|
||||||
|
|
||||||
|
### Step 3: Print tags in the list
|
||||||
|
- Files: `src/cli.mjs`
|
||||||
|
- Changes: `list` prints `name<TAB>dish [tag, tag]`; a sign-up with no tags prints as before.
|
||||||
|
- Test first:
|
||||||
|
- File: `tests/cli.test.mjs`
|
||||||
|
- Verify: `npm test`
|
||||||
|
|
||||||
|
### Step 4: Document the flag
|
||||||
|
- Files: `README.md`
|
||||||
|
- Changes: add `--tags` and the four allowed tags to the usage block.
|
||||||
|
- Test first:
|
||||||
|
- File: `tests/readme.test.mjs`
|
||||||
|
- Verify: `npm test`
|
||||||
|
|
||||||
|
## End-to-end
|
||||||
|
|
||||||
|
- End-to-end: `npm test`
|
||||||
|
J
|
||||||
5
evals/skill-annotating-fires-1/graders/skill-fired.md
Normal file
5
evals/skill-annotating-fires-1/graders/skill-fired.md
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
---
|
||||||
|
type: tool_used
|
||||||
|
tool: Skill
|
||||||
|
input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
|
||||||
|
---
|
||||||
10
evals/skill-annotating-fires-1/prompt.md
Normal file
10
evals/skill-annotating-fires-1/prompt.md
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
---
|
||||||
|
description: "Routing case: the annotating skill should fire on this request (the reserve path outside the pipeline)."
|
||||||
|
expected_outcome: "Claude calls the Skill tool for voyage:annotating at least once."
|
||||||
|
tags: [skill-trigger, skill=annotating, from=baseline-ann-own-critique]
|
||||||
|
runs: 3
|
||||||
|
max_turns: 4
|
||||||
|
allowed_tools: [Read, Glob, Grep, Skill]
|
||||||
|
---
|
||||||
|
|
||||||
|
Let me comment on the plan before we run it. Give me the page where I can annotate it line by line with my own notes.
|
||||||
5
evals/skill-annotating-fires-2/graders/skill-fired.md
Normal file
5
evals/skill-annotating-fires-2/graders/skill-fired.md
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
---
|
||||||
|
type: tool_used
|
||||||
|
tool: Skill
|
||||||
|
input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
|
||||||
|
---
|
||||||
19
evals/skill-annotating-fires-2/prompt.md
Normal file
19
evals/skill-annotating-fires-2/prompt.md
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
---
|
||||||
|
description: "Routing case: the annotating skill should fire on this request (the reserve path outside the pipeline)."
|
||||||
|
expected_outcome: "Claude calls the Skill tool for voyage:annotating at least once."
|
||||||
|
tags: [skill-trigger, skill=annotating, from=baseline-ann-question-edits]
|
||||||
|
runs: 3
|
||||||
|
max_turns: 4
|
||||||
|
allowed_tools: [Read, Glob, Grep, Skill]
|
||||||
|
---
|
||||||
|
|
||||||
|
Please revise the voyage artifact at `.claude/voyage/projects/2026-10-19-potluck-diet/plan.md` with the operator annotations below.
|
||||||
|
Each annotation has an intent — **Fix** (something is wrong / fix it), **Change** (change wording/content),
|
||||||
|
or **Question** (operator question — clarify or answer). The quote shows what the operator anchored to.
|
||||||
|
Treat the operator notes as authoritative direction.
|
||||||
|
|
||||||
|
## Annotations (1 total)
|
||||||
|
|
||||||
|
### 1. [Question] Section: Step 2: Parse tags on the command line
|
||||||
|
Quote: «splits on commas and passes the list on»
|
||||||
|
Comment: Why commas and not a repeated --tag flag?
|
||||||
5
evals/skill-annotating-fires-3/graders/skill-fired.md
Normal file
5
evals/skill-annotating-fires-3/graders/skill-fired.md
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
---
|
||||||
|
type: tool_used
|
||||||
|
tool: Skill
|
||||||
|
input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
|
||||||
|
---
|
||||||
10
evals/skill-annotating-fires-3/prompt.md
Normal file
10
evals/skill-annotating-fires-3/prompt.md
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
---
|
||||||
|
description: "Routing case: the annotating skill should fire on this request (the reserve path outside the pipeline)."
|
||||||
|
expected_outcome: "Claude calls the Skill tool for voyage:annotating at least once."
|
||||||
|
tags: [skill-trigger, skill=annotating, from=decision-62-research-on-request]
|
||||||
|
runs: 3
|
||||||
|
max_turns: 4
|
||||||
|
allowed_tools: [Read, Glob, Grep, Skill]
|
||||||
|
---
|
||||||
|
|
||||||
|
I want to mark up the research document research/01-local-caching.md in the browser with my own notes, then paste them back to you.
|
||||||
8
evals/skill-annotating-quiet-1/graders/skill-quiet.md
Normal file
8
evals/skill-annotating-quiet-1/graders/skill-quiet.md
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
---
|
||||||
|
type: tool_used
|
||||||
|
tool: Skill
|
||||||
|
input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
|
||||||
|
min: 0
|
||||||
|
max: 0
|
||||||
|
arm: both
|
||||||
|
---
|
||||||
10
evals/skill-annotating-quiet-1/prompt.md
Normal file
10
evals/skill-annotating-quiet-1/prompt.md
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
---
|
||||||
|
description: "Routing case: the annotating skill must stay quiet here; voyage:reviewing-adversarially owns it (a review is Claude's own judgement, not the person's notes)."
|
||||||
|
expected_outcome: "Claude never calls the Skill tool for voyage:annotating."
|
||||||
|
tags: [skill-no-trigger, skill=annotating, owner=reviewing-adversarially, from=order-b-negatives]
|
||||||
|
runs: 3
|
||||||
|
max_turns: 4
|
||||||
|
allowed_tools: [Read, Glob, Grep, Skill]
|
||||||
|
---
|
||||||
|
|
||||||
|
Review the diff Robin sent before I merge it: find what is wrong, cite the lines, no praise, and give me a verdict.
|
||||||
8
evals/skill-annotating-quiet-2/graders/skill-quiet.md
Normal file
8
evals/skill-annotating-quiet-2/graders/skill-quiet.md
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
---
|
||||||
|
type: tool_used
|
||||||
|
tool: Skill
|
||||||
|
input_match: '"skill"\s*:\s*"(?:voyage:)?annotating"'
|
||||||
|
min: 0
|
||||||
|
max: 0
|
||||||
|
arm: both
|
||||||
|
---
|
||||||
10
evals/skill-annotating-quiet-2/prompt.md
Normal file
10
evals/skill-annotating-quiet-2/prompt.md
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
---
|
||||||
|
description: "Routing case: the annotating skill must stay quiet here; /trekbrief owns it (writing a brief; the annotation page comes only after the brief exists)."
|
||||||
|
expected_outcome: "Claude never calls the Skill tool for voyage:annotating."
|
||||||
|
tags: [skill-no-trigger, skill=annotating, owner=trekbrief, from=order-b-negatives]
|
||||||
|
runs: 3
|
||||||
|
max_turns: 4
|
||||||
|
allowed_tools: [Read, Glob, Grep, Skill]
|
||||||
|
---
|
||||||
|
|
||||||
|
Write a task brief with a research plan for adding dietary tags to the potluck sign-up tool, ready for planning.
|
||||||
98
skills/annotating/SKILL.md
Normal file
98
skills/annotating/SKILL.md
Normal file
|
|
@ -0,0 +1,98 @@
|
||||||
|
---
|
||||||
|
name: annotating
|
||||||
|
description: "The operator's own annotation loop for a Voyage brief, plan or review (a research document too, when asked): build the page with scripts/annotate.mjs so the person writes every note, take the pasted Copy Prompt back, and revise note by note. Fix and Change edit only the text the note points at; a Question is answered and changes nothing unless the person says so; nothing the notes did not ask for is edited. A note on ## Intent or ## Goal of an approved brief makes the approval stale, and a changed plan needs a new approval, said before the edit. The page is built again on the revised file. Use when someone wants to comment on, annotate or mark up a brief, plan, review or research document in the browser, or pastes back the notes from that page (Fix, Change, Question)."
|
||||||
|
---
|
||||||
|
|
||||||
|
# Annotating
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The annotation page lets the person read a Voyage artifact in the browser, click a line and write
|
||||||
|
their own note, then copy every note back as one prompt. The loop is theirs: Claude builds the page,
|
||||||
|
and later does what the notes say, no more. Two things go wrong without this skill. A Question gets
|
||||||
|
answered and then written into the text as if it were a change. And the revision spreads: the same
|
||||||
|
"fix" applied to parts no note named, a section rewritten to match, a new criterion added. Every such
|
||||||
|
edit is a decision the person did not make, and on an approved brief or plan it also throws away an
|
||||||
|
approval.
|
||||||
|
|
||||||
|
## When to use
|
||||||
|
|
||||||
|
- A `/trek*` command has just written a brief, plan or review and builds the page.
|
||||||
|
- Someone says "let me comment on the plan", "I want to mark up the brief", "annotate this".
|
||||||
|
- Someone asks to annotate a research document under `research/`. `/trekresearch` does not build
|
||||||
|
the page itself; build it on request.
|
||||||
|
- The person pastes a prompt that starts "Please revise the voyage artifact at" and lists notes as
|
||||||
|
`[Fix]`, `[Change]` or `[Question]`.
|
||||||
|
- Not for: someone who asks for Claude's own review of the artifact. That is a review
|
||||||
|
(`voyage:reviewing-adversarially`, `/trekreview`), not annotation.
|
||||||
|
|
||||||
|
## Process
|
||||||
|
|
||||||
|
1. **Build the page.** Run the script on the file and keep its stdout, the absolute path of the
|
||||||
|
page:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ANNOT_HTML=$(node ${CLAUDE_PLUGIN_ROOT}/scripts/annotate.mjs "<path to the .md>" 2>&1)
|
||||||
|
```
|
||||||
|
|
||||||
|
Print `file://<path>` and the line `open file://<path>`. If the script exits non-zero, say so in
|
||||||
|
one line and go on: the page is a convenience, never a gate.
|
||||||
|
2. **The person writes the notes. Claude writes none.** Next to the link, say only how the page works:
|
||||||
|
click a line (or select a phrase first), pick Fix, Change or Question, write the note, and press
|
||||||
|
"Copy Prompt" when done; the notes stay in the browser if the tab is closed. No list of "points I
|
||||||
|
would look at", no summary of weaknesses, no suggested notes. Those would steer the notes the
|
||||||
|
person writes. If they want Claude's view as well, they ask for a review.
|
||||||
|
3. **Read the pasted notes.** Each note has a number, a type, the section, the quoted text it is
|
||||||
|
anchored to, and the comment. Work from the quote and the comment, one note at a time.
|
||||||
|
4. **Before any edit, check what the edit costs.**
|
||||||
|
- A brief with `intent_approved_hash` in its frontmatter, and a Fix or Change in `## Intent` or
|
||||||
|
`## Goal`: say in one line, before the edit, that the edit makes the approval stale, that
|
||||||
|
`/trekplan` halts until the new text is approved, and how to approve it
|
||||||
|
(`/trekbrief --approve <project dir>`). Then edit.
|
||||||
|
- A plan the person approved: any edit gives it a new sha256, the approval log no longer covers it,
|
||||||
|
and `/trekexecute` refuses it until the plan is approved again. Say that before the edit.
|
||||||
|
- Never stamp, log or commit an approval in this loop. Only the person's explicit answer approves.
|
||||||
|
5. **Fix and Change: edit the text the note points at, and only that.** The quote marks the place.
|
||||||
|
If the comment cannot be done without touching other text (a summary that now disagrees, the same
|
||||||
|
gap in another step, a new criterion the change seems to need), do not edit that text. List it in
|
||||||
|
the reply as a question with a recommended answer. A validator error that was already there before
|
||||||
|
the notes is reported, not fixed in passing.
|
||||||
|
6. **Question: answer it in the reply. The text stays as it was.** No new row under alternatives, no
|
||||||
|
new rule, no rewording to "make the answer visible". End the answer with one line offering to write
|
||||||
|
it in if the person wants that. Only their yes turns a Question into a change.
|
||||||
|
7. **Build the page again** on the revised file, and print the new link, so the person reads the
|
||||||
|
revised text where they wrote the notes. The page keeps notes per file path, so last round's notes
|
||||||
|
still show in its sidebar: tell the person that "Clear all" empties it before the next round.
|
||||||
|
8. **Reply note by note:** for each number, "edited" (with the new wording) or "answered" (with the
|
||||||
|
answer); then the approval consequence from step 4, the open questions from step 5, and the link.
|
||||||
|
|
||||||
|
## Red flags
|
||||||
|
|
||||||
|
| Thought | What it means |
|
||||||
|
|---|---|
|
||||||
|
| "I'll add a few points of my own next to the link, to help" | The notes are the person's. Your points steer theirs. Offer a review instead. |
|
||||||
|
| "The Question shows a gap, so I'll fix the gap too" | Answer it. The gap is theirs to turn into a Change. |
|
||||||
|
| "I'll record the answer under alternatives, so it isn't lost" | That is an edit nobody asked for, and on an approved plan it costs the approval. |
|
||||||
|
| "The same fix applies to steps 2 to 4 as well" | Then ask. The note named one step. |
|
||||||
|
| "The summary no longer matches the new intent, I'll update it" | Ask, with a recommended answer. The person approves the words. |
|
||||||
|
| "The brief fails the validator anyway, I'll fix the frontmatter while I'm here" | Report it. A fix in passing is still a change the person did not ask for. |
|
||||||
|
| "I'll mention the approval is stale at the end" | Say it before the edit. It is the price of the edit. |
|
||||||
|
| "I'll re-stamp it so planning can go on" | Only the person's explicit approval stamps. |
|
||||||
|
|
||||||
|
## Enforced by
|
||||||
|
|
||||||
|
- `tests/lib/doc-consistency.test.mjs` and `tests/lib/skills-annotating-prose-pins.test.mjs` (in
|
||||||
|
`npm test`): the script `scripts/annotate.mjs` exists, the three producing commands call it, print
|
||||||
|
the `file://` link and the `open` line, and name this skill. Deterministic, but only at test time.
|
||||||
|
- `lib/validators/intent-approval.mjs` (inside `/trekplan`): an edit to `## Intent` or `## Goal`
|
||||||
|
after approval makes the approval stale, and planning halts until the new text is approved.
|
||||||
|
- `lib/util/approval-log.mjs` (inside `/trekexecute`, K2): a plan whose current sha256 the approval
|
||||||
|
log does not cover is refused. Both checks hold whether or not the edit's cost was said.
|
||||||
|
- prose only: that the person writes every note and Claude none, that a Question changes nothing,
|
||||||
|
that only what the notes ask for is edited, that the cost of the edit is said before it, and that
|
||||||
|
the page is built again.
|
||||||
|
|
||||||
|
## Credits
|
||||||
|
|
||||||
|
Voyage's own content: the page (`scripts/annotate.mjs`), the loop and its rules come from Voyage's
|
||||||
|
pipeline and from the red baseline run for this skill. Nothing here is borrowed.
|
||||||
4
tests/fixtures/jsonl-schemas.md
vendored
4
tests/fixtures/jsonl-schemas.md
vendored
|
|
@ -19,9 +19,9 @@
|
||||||
|
|
||||||
| schema_id | fields | writer_path | line_ref | v4.1 additive | PII |
|
| schema_id | fields | writer_path | line_ref | v4.1 additive | PII |
|
||||||
|-----------|--------|-------------|----------|---------------|-----|
|
|-----------|--------|-------------|----------|---------------|-----|
|
||||||
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:1128-1152 | profile, phase_models, profile_source | none |
|
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:1125-1149 | profile, phase_models, profile_source | none |
|
||||||
| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | lib/stats/stats-append.mjs (trekresearch.md Stats tracking) | trekresearch.md:782-808 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
|
| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | lib/stats/stats-append.mjs (trekresearch.md Stats tracking) | trekresearch.md:782-808 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
|
||||||
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | lib/stats/stats-append.mjs (trekplan.md Phase 12) | trekplan.md:1277-1305 | profile, phase_models, parallel_agents, profile_source | none |
|
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | lib/stats/stats-append.mjs (trekplan.md Phase 12) | trekplan.md:1273-1301 | profile, phase_models, parallel_agents, profile_source | none |
|
||||||
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2907-2929 | profile, phase_models, profile_source | none |
|
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2907-2929 | profile, phase_models, profile_source | none |
|
||||||
| trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:78-99 | payload.profile, payload.phase_models, payload.profile_source | none |
|
| trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:78-99 | payload.profile, payload.phase_models, payload.profile_source | none |
|
||||||
| trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
|
| trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
|
||||||
|
|
|
||||||
140
tests/lib/skills-annotating-prose-pins.test.mjs
Normal file
140
tests/lib/skills-annotating-prose-pins.test.mjs
Normal file
|
|
@ -0,0 +1,140 @@
|
||||||
|
// tests/lib/skills-annotating-prose-pins.test.mjs
|
||||||
|
// Wave 4, order B: the `annotating` skill and its wiring into the three
|
||||||
|
// producing commands (/trekbrief, /trekplan, /trekreview).
|
||||||
|
//
|
||||||
|
// What is pinned, and why:
|
||||||
|
// • the skill's form is not pinned here: the form check
|
||||||
|
// (lib/skills/skill-lint.mjs) and the routing test (lib/skills/routing.mjs)
|
||||||
|
// run over every skill in `npm test` and are the authority (contract S1);
|
||||||
|
// • the three commands ask for `voyage:annotating` by name (contract R3);
|
||||||
|
// • the long explanation of the page and the loop left the commands: the
|
||||||
|
// skill is the one source. What each command keeps is the call to
|
||||||
|
// scripts/annotate.mjs, the "not a gate" line and the printed report,
|
||||||
|
// which tests/lib/doc-consistency.test.mjs also pins;
|
||||||
|
// • /trekresearch gets no annotation step (decision 62): the skill says it
|
||||||
|
// can be used on a research document when the user asks;
|
||||||
|
// • the loop the skill carries, aimed at what the red baseline showed;
|
||||||
|
// • the skill says plainly what is deterministic and what is prose only.
|
||||||
|
// Counted as doc pins by lib/util/test-census.mjs (file name *-prose-pins).
|
||||||
|
|
||||||
|
import { test } from 'node:test';
|
||||||
|
import { strict as assert } from 'node:assert';
|
||||||
|
import { existsSync, readFileSync } from 'node:fs';
|
||||||
|
import { dirname, join } from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||||
|
const ROOT = join(HERE, '..', '..');
|
||||||
|
const read = (rel) => readFileSync(join(ROOT, rel), 'utf8');
|
||||||
|
const flat = (t) => t.replace(/\s+/g, ' ');
|
||||||
|
|
||||||
|
const SKILL = 'skills/annotating/SKILL.md';
|
||||||
|
const skill = () => {
|
||||||
|
assert.ok(existsSync(join(ROOT, SKILL)), `${SKILL} missing`);
|
||||||
|
return read(SKILL);
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Text from `## <heading>` up to the next `## ` heading. */
|
||||||
|
function section(text, heading) {
|
||||||
|
const start = text.indexOf(`\n## ${heading}`);
|
||||||
|
assert.ok(start >= 0, `missing ## ${heading}`);
|
||||||
|
const next = text.indexOf('\n## ', start + 4);
|
||||||
|
return text.slice(start, next < 0 ? undefined : next);
|
||||||
|
}
|
||||||
|
|
||||||
|
const PRODUCERS = ['trekbrief.md', 'trekplan.md', 'trekreview.md'];
|
||||||
|
|
||||||
|
// ---- the commands ask for the skill, and keep only the call and the report ----
|
||||||
|
|
||||||
|
test('pin: the three producing commands ask for voyage:annotating by name', () => {
|
||||||
|
for (const f of PRODUCERS) {
|
||||||
|
assert.ok(read(`commands/${f}`).includes('`voyage:annotating`'), `commands/${f} must name voyage:annotating`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pin: the explanation the skill took over is gone from the commands', () => {
|
||||||
|
// The sentences that described the page and the loop in each command
|
||||||
|
// (1b61342: trekbrief.md 833-842, trekplan.md 1189-1197, trekreview.md
|
||||||
|
// 694-699). The page is described in the skill and the README; the loop
|
||||||
|
// ("revises freehand") is what the skill replaces.
|
||||||
|
const moved = [
|
||||||
|
/lets the operator click any line to attach/i,
|
||||||
|
/keeps a sidebar of all notes/i,
|
||||||
|
/persists state in localStorage/i,
|
||||||
|
/exposes a "Copy Prompt" button/i,
|
||||||
|
/freehand from the notes/i,
|
||||||
|
];
|
||||||
|
for (const f of PRODUCERS) {
|
||||||
|
const t = flat(read(`commands/${f}`));
|
||||||
|
for (const re of moved) assert.doesNotMatch(t, re, `commands/${f} still carries: ${re}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pin: each command keeps the call, the not-a-gate line and the printed links', () => {
|
||||||
|
for (const f of PRODUCERS) {
|
||||||
|
const t = read(`commands/${f}`);
|
||||||
|
assert.match(t, /ANNOT_HTML=\$\(node \$\{CLAUDE_PLUGIN_ROOT\}\/scripts\/annotate\.mjs "/, `${f}: the call`);
|
||||||
|
assert.match(flat(t), /convenience, not a gate/, `${f}: a failed page never blocks`);
|
||||||
|
assert.match(t, /open file:\/\/\{\$ANNOT_HTML\}/, `${f}: the open line`);
|
||||||
|
assert.match(t, /YOUR OWN/, `${f}: the operator writes the notes`);
|
||||||
|
}
|
||||||
|
for (const f of ['trekbrief.md', 'trekplan.md']) {
|
||||||
|
assert.match(read(`commands/${f}`), /Annotation HTML:\s+file:\/\/\{\$ANNOT_HTML\}/, `${f}: the report line`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pin: /trekresearch has no annotation step (decision 62)', () => {
|
||||||
|
const t = read('commands/trekresearch.md');
|
||||||
|
assert.ok(!t.includes('annotate.mjs'), 'trekresearch must not run the annotation script');
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- the skill carries the loop, aimed at what the red baseline showed ----
|
||||||
|
// Baseline (order B, 3 runs per scenario, no skill): 3 of 3 changed the plan
|
||||||
|
// because of a Question; 3 of 3 applied a Fix to steps the note did not name;
|
||||||
|
// 2 of 3 edited brief text no note asked for; 1 of 3 wrote its own critique
|
||||||
|
// next to the page; 0 of 3 named the stale approval before the edit.
|
||||||
|
|
||||||
|
test('pin: annotating builds the page with the script and leaves every note to the person', () => {
|
||||||
|
const p = flat(section(skill(), 'Process'));
|
||||||
|
assert.match(p, /node \$\{CLAUDE_PLUGIN_ROOT\}\/scripts\/annotate\.mjs/);
|
||||||
|
assert.match(p, /The person writes the notes\. Claude writes none\./);
|
||||||
|
assert.match(p, /convenience, never a gate/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pin: annotating — a Question is answered and the text stays as it was', () => {
|
||||||
|
const p = flat(section(skill(), 'Process'));
|
||||||
|
assert.match(p, /\*\*Question: answer it in the reply\. The text stays as it was\.\*\*/);
|
||||||
|
assert.match(p, /Only their yes turns a Question into a change/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pin: annotating — Fix and Change edit only what the note points at; the rest is asked', () => {
|
||||||
|
const p = flat(section(skill(), 'Process'));
|
||||||
|
assert.match(p, /\*\*Fix and Change: edit the text the note points at, and only that\.\*\*/);
|
||||||
|
assert.match(p, /the same gap in another step/);
|
||||||
|
assert.match(p, /already there before the notes is reported, not fixed in passing/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pin: annotating — the approval an edit costs is said before the edit', () => {
|
||||||
|
const p = flat(section(skill(), 'Process'));
|
||||||
|
assert.match(p, /`## Intent` or `## Goal`[^.]*before the edit, that the edit makes the approval stale/);
|
||||||
|
assert.match(p, /new sha256[^.]*`\/trekexecute` refuses it until the plan is approved again\. Say that before the edit\./);
|
||||||
|
assert.match(p, /Never stamp, log or commit an approval in this loop/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pin: annotating — the page is built again, and a research document is covered on request', () => {
|
||||||
|
assert.match(flat(section(skill(), 'Process')), /\*\*Build the page again\*\* on the revised file/);
|
||||||
|
assert.match(flat(section(skill(), 'When to use')), /research document under `research\/`\. `\/trekresearch` does not build the page itself/);
|
||||||
|
assert.match(flat(skill()), /"Clear all"/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('pin: annotating says what is enforced and what is prose only', () => {
|
||||||
|
const e = flat(section(skill(), 'Enforced by'));
|
||||||
|
for (const p of ['tests/lib/doc-consistency.test.mjs', 'scripts/annotate.mjs', 'lib/validators/intent-approval.mjs', 'lib/util/approval-log.mjs']) {
|
||||||
|
assert.ok(e.includes(`\`${p}\``), `Enforced by must name ${p}`);
|
||||||
|
}
|
||||||
|
assert.match(e, /prose only: that the person writes every note and Claude none, that a Question changes nothing/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("pin: annotating is Voyage's own content", () => {
|
||||||
|
assert.match(flat(section(skill(), 'Credits')), /Voyage's own content: [\s\S]*Nothing here is borrowed\./);
|
||||||
|
});
|
||||||
Loading…
Add table
Add a link
Reference in a new issue