open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

chore(cleanup): R-b R2 — STORM loop, cap hook and the --engine research choice removed

Browse source 
/trekresearch loses Phase 4.5, the bounded Phase 5 loop (Phase 5 keeps its
single pass of at most 2 follow-ups), the --engine flag, the deep-research
adapter and its output self-check. hooks.json loses the pre-agent-cap
PreToolUse entry (README: 9 hooks = 9 scripts). The STORM env vars leave the
docs; query-privacy-gate.mjs stays, called by no command, and says so.

Files removed: 13 (4 docs, 1 hook, 1 lib module, 1 script, 5 tests, 1 fixture).
Files edited: 17. Tests: 2600 (2598 pass / 0 fail / 2 skipped) before,
2467 (2466 pass / 0 fail / 1 skipped, R3 pending) after.
Durable check: group R2 green and no longer pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-11-07 20:32:38 +01:00
commit 32621dcb63
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
30 changed files with 34 additions and 4070 deletions
Download patch file Download diff file Expand all files Collapse all files

16
CLAUDE.md
View file

@ -31,7 +31,7 @@ Voyage — a contract-driven Claude Code pipeline: brief, research, plan, execut
| Command | Description | Orchestrator model |
|---------|-------------|--------------------|
| `/trekbrief` | Brief — one question round sized by class (`ceremony: small\|bounded\|architectural`, heaviest in doubt, said aloud, only moved up), run in the form of `voyage:interviewing` (asked for by name; criteria by `voyage:writing-criteria`, which `brief-reviewer` is also told to judge by, with `voyage:reviewing-adversarially`), produces a task brief with explicit research plan; ends with the operator explicitly approving `## Intent` + `## Goal` ("sounds good" is not approval), logged in the target repo's hash-chained approval log and committed (the brief too when `artifact_store: git`); the operator's notes from the annotation page go through `voyage:annotating` (`--approve <dir>` does only the approval); optionally orchestrates the pipeline | session |
| `/trekresearch` | Research — deep local + external research, produces structured research brief. Opt-in `--engine {swarm\|deep-research}` delegates the external phase to Claude Code's built-in `/deep-research` workflow (swarm default). The external agents preload the `researching-sources` skill and are told it by name at spawn | session |
| `/trekresearch` | Research — deep local + external research, produces structured research brief. The external agents preload the `researching-sources` skill and are told it by name at spawn | session |
| `/trekplan` | Plan — brief-reviewer, explore, plan, review. Requires `--brief` or `--project`; halts unless the brief's intent is approved (hash-bound, stale on edit). Auto-discovers `architecture/overview.md` if present. Plan 1.9: the file first on every line — every `Files:` entry and `Test first:` `File:` line opens with its backticked path(s). From 1.8: `Test first:` per step + one `End-to-end:` step; every check written by `voyage:writing-criteria` (named where the plan is written and where its three reviewers start); a design part for `ceremony: architectural` (absent = architectural); critic loop of max 3 rounds, recorded in the plan, rounds 2+ in diff mode (rule A, counted by the gate; rule B in the reviewers), and `plan-gate.mjs` (0 go / 1 stop / 2 could not run) decides; open minors after it are listed for the operator to pick | session |
| `/trekexecute` | Execute — disciplined plan/session-spec executor: refuses a plan whose logged approval does not cover its current sha256 (K2); one fresh `step-implementer` helper per plan step, one controller that reads the diff (spec + quality verdicts), proof of red / floor guard / step check (K9) / `Regression:` (K10) / done line as commands, fix rounds that go up (max 5: helper → plan → operator), architecture rules (K5) + plan-file sweep (K6) after the last step, the app seen with Claude Code's built-in `run` while the `End-to-end:` command stays the verdict (`run` gives no exit code), then its own review (`/simplify` → `/trekreview` → verdict gate, K7; BLOCK keeps the merge into main closed), closed stop list + decision log in the final report | session |
| `/trekreview` | Review — a gate: Claude Code's `/code-review` + `/security-review` as engine, then Voyage's brief control (which stops); writes `review.md` with a required verdict that `lib/review/verdict-gate.mjs` decides on (0 ALLOW, or WARN with every counting MAJOR answered · 1 BLOCK · 2 never green); the operator answers each counting MAJOR (Phase 8.8; `--answers` runs only that, no new round); proposes recurring finding classes as project memory, line by line (Handover 6). The reviewers and coordinator preload `voyage:reviewing-adversarially` and are told it by name at spawn; each reviewer's Declined to Judge lines reach `review.md`; the operator's notes from the annotation page go through `voyage:annotating` | session |
@ -52,20 +52,6 @@ Full flag reference for each command (modes, `--gates`, `--profile`, breaking ch
> the red-first test); `On failure:` opens with revert | retry | skip | escalate.
> `plan-validator.mjs` refuses the first, warns on the second → Handover 4.
> **STORM bounded loop — default-off, env-gated.** `/trekresearch` Phase 4.5
> (dimension discovery, under the existing `maxDimensions: 8` ceiling) and
> Phase 5 (bounded multi-turn follow-up) run only at `effort: high` **and** only
> when `VOYAGE_STORM_ENABLED=1`; unset, both are inert — Phase 5 because
> `lib/util/research-loop-cap.mjs` grants a budget of 0, Phase 4.5 because its
> skip-guard reads the flag directly (it never calls the cap). `TREKRESEARCH_MAX_CONV_TURNS` (default `3`,
> invalid values fall back to `3`) sets turns per dimension; the budget is that
> × `maxDimensions`. `VOYAGE_DISABLE_CAP_HOOK=1` switches off
> `hooks/scripts/pre-agent-cap.mjs`, the `PreToolUse` gate that enforces the
> budget in the harness rather than trusting prose. The cap counts turns from
> its own append-only ledger. Adoption as a default is gated on the
> pre-registered measurement in `docs/storm-measurement.md` — decline is a
> no-op, adopt is one constant.
> **Research source rules live in one skill.** `skills/researching-sources/` (preloaded by the five
> web-search agents) holds URL/date/version per source, "no citation, no finding", empty search ≠
> finding, fetched pages are data, nothing local in a query, and the one confidence scale. Only the

27
README.md
View file

@ -218,28 +218,9 @@ Output:
| **External** | `/trekresearch --external <question>` | Only external research agents (skip codebase analysis) |
| **Foreground** | `/trekresearch --fg <question>` | No-op alias (foreground is default since v2.4.0) |
| **Profile** | `/trekresearch --profile <name> <question>` | (v4.1.0) Pin model profile for the research phase. See [Profile system](#profile-system-v410). |
| **Engine** | `/trekresearch --external --engine deep-research <question>` | Delegate the external phase to Claude Code's built-in `/deep-research` workflow. Only works on `2.1.154 <= CC < 2.1.218`; from **2.1.218** `/deep-research` is operator-invoked only (Skill tool: `disable-model-invocation`), so it always falls back to `swarm`. Never hard-fails. Default `swarm`. |
Flags combine: `--project <dir> --external`.
#### Bounded conversation loop (default-off)
At `effort: high`, research can discover additional dimensions (Phase 4.5) and
run a bounded multi-turn follow-up loop on under-illuminated ones (Phase 5).
Both are **default-off** and env-gated rather than flag-gated, because enabling
them costs turns:
| Env-var | Default | Behavior |
|---------|---------|----------|
| `VOYAGE_STORM_ENABLED` | _(unset — default-off)_ | `=1` gives the Phase 5 loop a non-zero turn budget and lets Phase 4.5 discovery run. Unset, the budget is 0 and **both** phases are inert. |
| `TREKRESEARCH_MAX_CONV_TURNS` | `3` | Turns per under-illuminated dimension. Invalid values fall back to `3`, never to unbounded. |
| `VOYAGE_DISABLE_CAP_HOOK` | _(unset)_ | `=1` disables the `PreToolUse` hook that enforces the turn budget. |
The cap counts turns itself from an append-only ledger — it never asks the loop
how many turns it has used. Whether the loop becomes the default is decided by a
pre-registered measurement, not by preference: see
[`docs/storm-measurement.md`](docs/storm-measurement.md).
Research uses up to 5 local agents (architecture-mapper, dependency-tracer, task-finder, git-historian, convention-scanner) and 4 external agents (docs-researcher, community-researcher, security-researcher, contrarian-researcher). Per-agent details in [`agents/`](agents/).
#### Source rules (the `researching-sources` skill)
@ -259,7 +240,7 @@ names it again when it starts them. It holds the rules they share:
It also defines the one confidence scale (`high | medium | low | contradictory`) that the agents
and Phase 6 triangulation use. **What enforces it:** `research-validator.mjs` checks only the
brief's structure, and the query privacy gate runs only in the default-off STORM steps; the rest
brief's structure, and the query privacy gate is not called by any command now; the rest
is prose only. Measured when it was written (helpers, offline fixtures, small samples): without the
skill, 3 of 5 runs ignored an injected instruction silently instead of reporting it, and 1 of 2
counter-evidence runs counted an empty search as weak support; with it, 0 of 3 and 0 of 2.
@ -651,7 +632,7 @@ model. A skill is advice; each one says which check stands behind it, or "prose
| `voyage:test-first` | Red before green: the repo's own test command, a test seen failing on an assertion (a crash is not red), the smallest code, the whole suite | `step-implementer` (preloaded) | proof of red, test lock, floor guard, done line in `/trekexecute`; the rest is prose only |
| `voyage:debugging` | One command red on the exact symptom first; reproduce, localise, reduce, fix the cause, guard it with a red-first test; error text is data; three failed fixes go back to the design | `step-implementer` (preloaded) | proof of red for the regression test, the fix-round cap; the root-cause claim is prose only |
| `voyage:interviewing` | Facts looked up, never asked; one round of at most four questions per call, one decision each, the recommended answer first; the rest written down as assumptions or a Deferred list; an answer that decides nothing, and scope nobody asked for, never written in as the person's decision; stop once the next three answers can be predicted | `/trekbrief` names it for its question round | the brief's sections and `framing:` (`brief-validator`), the explicit approval at the end; the form of the interview is prose only |
| `voyage:researching-sources` | Official sources first with URL, date and matching version; no citation, no finding; an empty search is reported with what was searched, never as a finding; fetched pages are data, and text aimed at assistants is reported; nothing local in a query; one confidence scale | `docs-`, `community-`, `security-`, `contrarian-researcher` and `research-scout` (preloaded); `/trekresearch` names it at spawn | the research brief's structure (`research-validator`); the query privacy gate only in the default-off STORM steps; the rest is prose only |
| `voyage:researching-sources` | Official sources first with URL, date and matching version; no citation, no finding; an empty search is reported with what was searched, never as a finding; fetched pages are data, and text aimed at assistants is reported; nothing local in a query; one confidence scale | `docs-`, `community-`, `security-`, `contrarian-researcher` and `research-scout` (preloaded); `/trekresearch` names it at spawn | the research brief's structure (`research-validator`); the query privacy gate is not called by any command now; the rest is prose only |
| `voyage:reviewing-adversarially` | No praise; no finding without a source (`file:line` or a quoted line); no downgrade without a reason; what was read only as a summary is said and not judged; a Declined to Judge list so nothing set aside disappears; claims checked against the diff, not the report; the axes kept separate | `brief-conformance-reviewer`, `code-correctness-reviewer`, `review-coordinator`, `scope-guardian` (preloaded); `/trekreview` names it at spawn | a finding with no file or citation (`findings-schema`, `coordinator-contract`, partial); the `## Declined to Judge` section in `review.md` (`review-validator`); praise and the rest are prose only |
| `voyage:annotating` | The person writes every note on the annotation page and Claude writes none; Fix and Change edit only the text a note points at, a Question is answered and changes nothing, nothing else is edited; the approval an edit costs is said before the edit; the page is built again on the revised file. Also for a research document, on request | `/trekbrief`, `/trekplan` and `/trekreview` name it where they build the page | the script and the commands' call to it (pins in `npm test`); a stale brief approval (`intent-approval`, `/trekplan` halts) and a changed plan (K2 `approval-log`, `/trekexecute` refuses); the rest is prose only |
| `voyage:writing-criteria` | Each success criterion and each `Verify:`, `Regression:`, `End-to-end:` and `## Verification` line is one command the criteria runner accepts, written as the line's first backticked span (a known test runner, plain words, no pipe or `&&`); anything more is a script under `tests/`; and each can go red | `brief-reviewer` and `plan-critic` (preloaded); `/trekplan` names it where the plan is written and where it starts its reviewers | the criteria runner and the plan and brief validators (deterministic); that a criterion can go red and tests what the brief asks is prose only |
@ -815,7 +796,7 @@ flowchart TB
BR --> RES --> PL --> EX --> RV
```
`* = conditional`: convention-scanner / test-strategist on medium+ codebases (50+ files); research-scout for unknown external tech; contrarian-researcher, when selected, runs in wave 2 after wave 1 returns, with the leading answer of wave 1's external findings to attack (with `--engine deep-research`, when the engine is active, after the engine returns, from the leading answer of its report), always at `effort=high`; brief-conformance-reviewer skipped under `--quick`.
`* = conditional`: convention-scanner / test-strategist on medium+ codebases (50+ files); research-scout for unknown external tech; contrarian-researcher, when selected, runs in wave 2 after wave 1 returns, with the leading answer of wave 1's external findings to attack, always at `effort=high`; brief-conformance-reviewer skipped under `--quick`.
> **Which Claude Code primitive each phase uses — and the alternatives considered (Workflow substrate, delegated orchestrator)** → see [docs/architecture.md §Primitives per step](docs/architecture.md#primitives-per-step-decision-matrix).
@ -986,7 +967,7 @@ voyage/
│ └ reviewing-adversarially: the review agents; writing-criteria: brief-reviewer, plan-critic; interviewing: named by /trekbrief;
│ └ annotating: named by /trekbrief, /trekplan and /trekreview)
├── templates/ Frontmatter templates for brief, research, plan, session
├── hooks/ 10 hooks (pre-bash, pre-write, pre-agent-cap, pre-plan-gate, session-title, post-approval-log, post-edit-check, pre-compact-flush, post-compact-flush, stop-test-lock)
├── hooks/ 9 hooks (pre-bash, pre-write, pre-plan-gate, session-title, post-approval-log, post-edit-check, pre-compact-flush, post-compact-flush, stop-test-lock)
├── lib/ Zero-dep parsers and validators (CLI shims under lib/validators/)
├── tests/ comprehensive node:test suite — `npm test` is the fork-readiness gate
├── docs/ HANDOVER-CONTRACTS.md, architecture.md, operations.md, profiles.md, command-modes.md, release-runbook.md

413
commands/trekresearch.md
View file

@ -1,7 +1,7 @@
---
name: trekresearch
description: Deep research combining local codebase analysis with external knowledge, producing structured research briefs with triangulation and confidence ratings
argument-hint: "[--project <dir>] [--quick | --local | --external | --fg] [--engine swarm|deep-research] <research question>"
argument-hint: "[--project <dir>] [--quick | --local | --external | --fg] <research question>"
allowed-tools: Agent, Read, Glob, Grep, Write, Edit, Bash, AskUserQuestion, WebSearch, WebFetch, mcp__tavily__tavily_search, mcp__tavily__tavily_research
---
@ -97,21 +97,6 @@ Supported flags:
enforcement only fires at `≥ 2.2`. Absent → no version check. See
`docs/HANDOVER-CONTRACTS.md` §Handover 1 for the pre-2.2 enforcement hole.
8. `--engine <name>` — opt-in external-research engine. Accepts `--engine <name>`
where `<name>` is `swarm` or `deep-research`. **Default: `swarm`** (unchanged
behavior). `swarm` runs Voyage's own external-research agent swarm;
`deep-research` delegates the external phase to Claude Code's built-in
`/deep-research` dynamic workflow and adapts its report into the research-brief
schema. **The delegation only works inside a closed version window:
`2.1.154 <= CC < 2.1.218`** (dynamic workflows enabled). From Claude Code
**2.1.218** `/deep-research` starts only when the operator invokes it, and the
Skill tool refuses a model invocation outright (`disable-model-invocation`), so
on 2.1.218 or newer the engine **always** falls back to `swarm`. The flag is
kept as an additive opt-in and **never hard-fails**: outside the window it
degrades to `swarm` and logs the reason. Orthogonal to
`--profile`/`phase_signals`; only affects the external phase. Set
**engine = {swarm|deep-research}** (the *requested* engine).
Flags can be combined:
- `--local` — local-only research
- `--external --quick` — external-only, lightweight
@ -119,7 +104,7 @@ Flags can be combined:
- `--quick` alone implies both local and external (lightweight)
Defaults: **scope = both**, **execution = foreground** (only mode as of
v2.4.0), **project_dir = none**, **engine = swarm**.
v2.4.0), **project_dir = none**.
After stripping flags, the remaining text is the **research question**.
@ -140,7 +125,6 @@ Modes:
--external Only external research agents (skip codebase analysis)
--fg No-op alias (foreground is the only mode as of v2.4.0)
--project Write brief into an trekbrief project folder (auto-indexed)
--engine Opt-in external-research engine: swarm (default) | deep-research
Flags can be combined: --local, --external --quick, --project <dir> --external
@ -151,7 +135,6 @@ Examples:
/trekresearch --external What are the security implications of using Redis for sessions?
/trekresearch --fg --local What patterns does this codebase use for database access?
/trekresearch --project .claude/projects/2026-04-18-jwt-auth --external What JWT library is best for Node.js?
/trekresearch --project <dir> --external --engine deep-research <research question>
```
Do not continue past this step if no question was provided.
@ -160,7 +143,6 @@ Report the detected mode:
```
Mode: {default | quick}, Scope: {both | local | external}, Execution: foreground
Project: {project_dir or "-"}
Engine (requested): {swarm | deep-research}
Question: {research question}
```
@ -324,67 +306,6 @@ For each local agent, prompt with the research question, NOT a task description:
- convention-scanner: "Discover coding conventions relevant to evaluating {question}.
What patterns would a solution need to follow?"
### Engine selection (scope = both or external)
`--engine` affects ONLY the external portion of research. The local agents
(`### Local agents` above) and Phases 6–7 (triangulation, synthesis, brief
writing) are **engine-agnostic** — they run identically regardless of engine.
`--engine` is **moot** (treated as `swarm`) whenever the external phase does not
run at all: `--local`, `--quick`, `effort == 'low'`, or a profile with
`external_research_enabled == false` (the `economy`/`balanced` auto-disable — see
Profile below). The profile's on/off switch wins. Initialize
`effective_engine = {requested engine}`.
**engine = swarm (default):** run the `### External agents` block below
unchanged. This is byte-for-byte the current path, so `--engine swarm`
changes nothing (SC1). Keep the native-swarm anchors intact ("in parallel",
"single message", and the resolved-model injection in the Launch rules).
**engine = deep-research:**
1. **Coarse pre-gate (best-effort, NOT a trust signal).** The supported window is
`2.1.154 <= version < 2.1.218`: below `2.1.154` there are no dynamic workflows,
and from `2.1.218` (the ceiling) `/deep-research` starts only when the operator
invokes it, and the Skill tool refuses a model invocation with
`disable-model-invocation` — see step 4's reason token. Voyage does not run the
Claude Code binary to read its version (it never calls Claude Code), so the
version is *indeterminate* here. If `disableWorkflows: true` /
`CLAUDE_CODE_DISABLE_WORKFLOWS=1` is set, skip to the fallback (step 4);
otherwise proceed to step 2 — do NOT hard-fail. There is no positive
availability probe (research Dim 4), so a passing pre-gate does not guarantee the
workflow runs; the post-hoc check (step 3) is the authoritative guard.
2. **Run.** Instruct Claude (in prose, this turn) to run
`/deep-research <research question>` and request per-claim citations. Note:
interactive default/acceptEdits triggers a per-run approval prompt.
3. **Post-hoc presence + provenance check (the real guard).** Verify a real, cited
`/deep-research` report actually landed in context — substantive findings with
citations, not an empty/denied/errored turn and not bare error text. This check
must be **robust to all failure manifestations** (workflow disabled, approval
denied, runtime error, empty output, or the Skill tool refusing with
`disable-model-invocation` on CC 2.1.218+), because the disabled-headless
behavior is undocumented: no recognizable cited report in context → fall back,
regardless of how the failure surfaces.
4. **On no real report (fallback):** set `effective_engine = swarm`, run the swarm
blocks below, and **log the fallback at this decision point** — print
`Engine: deep-research → swarm (fallback: <reason>)` and carry the reason into the
Phase-8 Present summary and the brief's `## Executive Summary`. Known reason
tokens: `disable-model-invocation` (CC >= 2.1.218 — the expected reason on any
current Claude Code), `workflows-disabled`, `no-cited-report` (also what a
Claude Code below the `2.1.154` floor produces, since no version is read). **NEVER fabricate
or synthesize a substitute report** — a structurally-valid-but-invented brief
passes the structure-only validator and silently poisons `/trekplan`; that is the
worst outcome of this feature.
5. **On a real report:** keep `effective_engine = deep-research`, log
`Engine: deep-research (active)`, and carry the report into Phase 6 triangulation
as the external-findings input (adapted in Phase 7 — see the Deep-research engine
adapter below); then run wave 2 (`contrarian-researcher`, see Launch rules)
from the report's leading answer.
### External agents (scope = both or external)
Launch the new research-specialized agents:
@ -412,8 +333,7 @@ ticket IDs) already removed: the agents search with the words they are given.
### Launch rules
- Launch in two waves, both in Phase 4 (before Phase 4.5 and Phase 5, so outside
the Phase 5 loop scope marker that `hooks/scripts/pre-agent-cap.mjs` reads).
- Launch in two waves, both in Phase 4.
- **Wave 1:** ALL selected agents except `contrarian-researcher` **in parallel**
in a single message.
- **Wave 2:** `contrarian-researcher`, when selected, after wave 1 returns, in a
@ -425,9 +345,7 @@ ticket IDs) already removed: the agents search with the words they are given.
repository names, ticket IDs). It searches with the words it is given, so no
local file name or project term may be in them. When wave 1's external
findings hold no leading answer, say so and give it the research question
alone, stripped the same way. With `--engine deep-research`
(`effective_engine = deep-research`), wave 2 runs after the engine returns,
from the leading answer of its report.
alone, stripped the same way.
- Inject the resolved model at every spawn: `phase_signal_result.model` (brief
signal > profile), with the agent's frontmatter `model:` as the fallback — see
Hard rules → Cost. The orchestrator itself follows the session model.
@ -445,262 +363,18 @@ ticket IDs) already removed: the agents search with the words they are given.
`docs/agent-return-channel-defect.md`.
- convention-scanner: medium+ codebases only (50+ files)
## Phase 4.5 — Dimension discovery
**Skip this phase entirely unless `phase_signal_result.effort == 'high'` AND
`VOYAGE_STORM_ENABLED=1`.** Both conditions, never either.
This phase never invokes `research-loop-cap.mjs`, so the cap's own flag check
does not cover it — the flag has to be read here. Gating on effort alone would
leave discovery mutating the dimension list at `effort: high` with the flag
unset, making `dimensions` diverge from `dimensions_baseline` and putting the
decline branch out of reach for half the mechanism. Doing nothing must leave
**both** STORM phases inert.
Phase 4 retrieves more than the interview knew to ask for. This phase mines
that surplus: findings that were **retrieved but unintegrated** — material an
agent surfaced that no interview dimension claims.
1. **Mine.** Walk the Phase-4 agent results and collect findings that map to
no existing dimension.
2. **Rerank.** Order candidates by relevance to the research question **and**
dissimilarity to the dimensions already on the list. A candidate that
restates an existing dimension is not a discovery.
3. **Augment under the existing ceiling.** Append candidates to the dimension
list only while the **whole** list (interview + discovered) stays at or
below `maxDimensions: 8` (`MAX_TOTAL_DIMENSIONS` in
`lib/util/research-loop-cap.mjs`). The ceiling is **not**
raised here, so the documented 3–8 dimension range stays true and the
README prose about it stays untouched. If the interview already produced 8
dimensions, this phase discovers nothing and says so.
**The ceiling has a reader — use it.** Once the final list is settled, run
the check below. Exit 1 means the list exceeded the ceiling: drop discovered
dimensions until it passes. Do not proceed to Phase 5 on a rejected list —
the turn budget is sized against this same ceiling, so a list over it spends
a budget that was never approved for it.
```bash
# Same VOYAGE_ROOT resolution as the per-turn protocol in Phase 5. Exit 0 =
# within the ceiling, exit 1 = rejected. JSON on stdout: {ok, count, ceiling, reason?}
node "$VOYAGE_ROOT/lib/util/research-loop-cap.mjs" --check-dimensions {final dimension count}
```
The ceiling constant is `MAX_TOTAL_DIMENSIONS` in
`lib/util/research-loop-cap.mjs` — deliberately the same constant that sizes
the Phase 5 turn budget, so the two axes of the bounded-cost NFR cannot end
up enforcing different numbers for one ceiling.
4. **Record the baseline.** Keep the interview-derived count as
`dimensions_baseline` so the discovered delta is machine-readable against
the final `dimensions`.
5. **Attest membership, not just the count.** Set
`dimensions_baseline_preserved: true` only if EVERY interview-derived
dimension is still on the final list — this phase appends, it never replaces.
Set it `false` if any was dropped, merged away, or rewritten. A count delta
cannot show this: dropping two interview dimensions and appending three
discovered ones is `+1` and still not a superset, which is exactly what the
Success Criterion forbids. `storm-measure.mjs --activation-check` reads the
field and fails when it is absent, so omitting it is not the silent default.
Every outbound query generated from a discovered dimension passes
`query-privacy-gate.mjs` before it leaves the machine — see the per-turn
protocol in Phase 5. That gate controls the **egress** risk this phase's
Independence crossing creates: local paths and identifiers travelling inside a
query. It does **not** control the **bias** risk — it inspects query content and
cannot stop a local finding from steering an external agent's question. The bias
controls are structural (blind initial swarm, append-only crossing, unconditional
`contrarian-researcher` at `effort: high`); see Hard rules → Independence.
## Phase 5 — Targeted follow-ups
Review all agent results. Identify knowledge gaps — dimensions where findings
are thin, contradictory, or missing (**under-illuminated dimensions**).
**Standard and low effort — unchanged single pass.** For each significant gap,
launch a targeted follow-up agent (model: phase_signal_result.model, else opus — a
For each significant gap, launch a targeted follow-up agent (model: phase_signal_result.model, else opus — a
follow-up may be one of Claude Code's built-in agents, which have no Voyage
frontmatter to fall back to; see Launch rules) with a narrow, specific
brief; a follow-up that searches the web names the `researching-sources` skill
in its prompt, as above. Maximum 2 follow-ups. If no gaps exist, skip: "Initial research
sufficient — no follow-ups needed." Then go to Phase 6.
**The bounded loop below runs ONLY when `phase_signal_result.effort == 'high'`**
(resolved in Phase 1; see `### High-effort behavior (v5.1.1)`). At any other
effort this whole sub-section is inert — no loop, no cap ledger, no new
counters beyond zero.
### Loop bound
**Maximum 3 turns per under-illuminated dimension.** The bound is per
dimension, not per run: the worst case is 3 turns × the whole dimension list
under the `maxDimensions: 8` ceiling (`MAX_TOTAL_DIMENSIONS`), which is what
`research-loop-cap.mjs` sizes itself against. The cap counts itself from its
own append-only ledger — it never asks this prose how many turns it has used.
The loop is **default-off**: `research-loop-cap.mjs` grants a budget of 0
unless `VOYAGE_STORM_ENABLED=1`. Doing nothing leaves the mechanism off.
### Loop scope marker
`hooks/scripts/pre-agent-cap.mjs` (PreToolUse on `WebSearch|WebFetch|Task`)
enforces the same bound in the harness rather than trusting this prose — but it
enforces **only** for a session that carries a scope marker, and allows
unconditionally for every session that does not. That is what keeps a globally
wired PreToolUse hook from denying tool calls in unrelated sessions. Write the
marker once, immediately before the first turn:
`CLAUDE_PLUGIN_DATA` is **empty in the Bash tool's process env** even in a
plugin-enabled session, so the root is resolved with the same fallback
`research-loop-cap.mjs` uses — `~/.claude/voyage`. Reader and writer must
resolve identically; a marker written where the hook does not look leaves the
hook allowing unconditionally while the docs call it enforcing.
```bash
# Arms the PreToolUse cap for THIS session only.
# CLAUDE_CODE_SESSION_ID is the same id the hook reads as `session_id`.
DATA="${CLAUDE_PLUGIN_DATA:-$HOME/.claude/voyage}"
case "$DATA" in /*) SCOPE_DIR="$DATA/trekresearch-loop-scope" ;; *) SCOPE_DIR="" ;; esac
if [ -n "$SCOPE_DIR" ] && [ -n "${CLAUDE_CODE_SESSION_ID:-}" ] && mkdir -p "$SCOPE_DIR" 2>/dev/null; then
printf '{"runId":"%s","startedAt":"%s"}\n' "{run_id}" "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
> "$SCOPE_DIR/${CLAUDE_CODE_SESSION_ID}.json"
else
echo "[voyage] scope marker not written — harness cap stays inert for this run"
fi
```
An empty `CLAUDE_CODE_SESSION_ID` is checked before the path is composed, not
after: unset, the marker becomes `.json`, which no hook lookup matches and no
TTL sweep ever cleans up.
`runId` MUST be the same `{run_id}` passed to `research-loop-cap.mjs --run-id`.
The hook counts ledger lines carrying that id, so a marker written with any
other id counts zero turns and enforces nothing.
**A failed marker write is not a reason to stop.** The hook is defence in
depth; `research-loop-cap.mjs` is the gate and stays correct on its own. Report
the failure to the operator and run the loop. The reverse — skipping the budget
gate because a marker exists — is never allowed.
**Removal belongs to every exit below, especially the exhausted one.** The hook
denies `WebSearch`/`WebFetch`/`Task` once `research-loop-cap.mjs` has denied a
turn — the gate records its own denials, so the LAST granted turn still runs its
queries and exhaustion reaches you through exit 2 of the budget gate below, not
through a blocked tool call. Once denied, the hook keeps denying for as long as
the marker is there — including Phase 6, which spawns agents. A marker that outlives the loop turns a bound on this loop into a brick
on the rest of the session. Cleanup covers the three exits and nothing else: a
crashed session runs no cleanup at all, and is covered instead by the hook's
TTL (default 2h, `VOYAGE_CAP_SCOPE_TTL_MS`), which auto-resets a stale marker.
A crash mid-loop leaves no denial record, so a resumed session is not blocked by
it; only a crash AFTER the cap denied a turn hands the resume a deny window, and
every denial prints the marker path to delete.
```bash
# Removal — idempotent, safe to repeat. Same root, same absolute-path guard as
# the write: a remove that accepts a root the write rejected (or vice versa)
# leaves markers the loop believes it cleaned up.
DATA="${CLAUDE_PLUGIN_DATA:-$HOME/.claude/voyage}"
case "$DATA" in /*) SCOPE_DIR="$DATA/trekresearch-loop-scope" ;; *) SCOPE_DIR="" ;; esac
[ -n "$SCOPE_DIR" ] && [ -n "${CLAUDE_CODE_SESSION_ID:-}" ] && \
rm -f "$SCOPE_DIR/${CLAUDE_CODE_SESSION_ID}.json"
```
### Per-turn protocol
Each turn targets exactly one under-illuminated dimension, and runs two gates
before it spends anything:
```bash
# 0. Resolve the plugin root ONCE. ${CLAUDE_PLUGIN_ROOT} is substituted in this
# command's text but is EMPTY in the Bash tool's process env, and a bare
# `node ${CLAUDE_PLUGIN_ROOT}/lib/…` then runs `node /lib/…`, which exits 1 —
# indistinguishable from a gate that said no.
# Only the literal form is substituted (a `:-` default is never replaced); the
# fallback takes the highest installed version under cache/<market>/voyage/.
VOYAGE_ROOT="${CLAUDE_PLUGIN_ROOT}"
case "$VOYAGE_ROOT" in
/*) ;;
*) VOYAGE_ROOT="$(ls -d "$HOME"/.claude/plugins/cache/*/voyage/[0-9]*/ 2>/dev/null \
| awk -F/ '{ split($(NF-1), v, "."); printf "%09d%09d%09d %s\n", v[1], v[2], v[3], $(0) }' \
| sort | tail -1 | cut -d' ' -f2-)"
VOYAGE_ROOT="${VOYAGE_ROOT%/}" ;;
esac
if [ ! -f "$VOYAGE_ROOT/lib/util/research-loop-cap.mjs" ]; then
echo "[voyage] gates could not run — plugin root unresolved (exit 2). NOT a denial:"
echo " stop the loop and report to the operator. Never proceed ungated."
exit 2
fi
# 1. Budget gate — per turn, per dimension. Exit 0 = granted, exit 1 = denied.
# JSON on stdout: {ok, used, budget, reason?}
node "$VOYAGE_ROOT/lib/util/research-loop-cap.mjs" \
--run-id {run_id} --dimension {dimension} --effort {phase_signal_result.effort}
# 2. Privacy gate — EVERY outbound query, before it leaves the machine.
# Exit 0 = send as-is; exit 1 = rewrite the query and re-gate. Never bypass.
node "$VOYAGE_ROOT/lib/validators/query-privacy-gate.mjs" "{query text}"
```
**Exit 2 is not exit 1.** A denied budget gate is an exit condition, not a
retry. A failed privacy gate is a rewrite: the hard-block tier (secret-shaped
strings) is never operator-overridable, so a query that trips it must be
reformulated, not forced through. A gate that *could not run* is neither — no
rewrite can clear it, so treat it as a hard stop and say so, rather than
rewriting a query that was never the problem.
A turn that spawns an agent passes `model: phase_signal_result.model`, else
`opus` — the agent may be one of Claude Code's built-in agents, which have no
Voyage frontmatter to fall back to; see Launch rules.
**Empty turns.** A turn that returns no findings, or findings without
citations, is marked `empty`. An empty turn is counted in `empty_turns` and
does NOT re-target the same dimension — re-asking the same question of the
same silence is how a bounded loop turns into an unbounded one. Move to the
next under-illuminated dimension, or exit.
### Exits (all three, always one of them)
1. **Converged** — the dimension carries findings with citations and no
remaining contradiction. Stop turning on it. This is the normal exit. Once
the last dimension has converged, remove the scope marker.
2. **Cap exhausted** — `research-loop-cap.mjs` denies the turn. Print the
exhaustion **visibly** to the operator, never silently:
`Loop bound reached for dimension {dimension} after {N} turns — remaining
gaps are carried into the brief as open questions.` A silent cap is
indistinguishable from convergence, and that confusion is exactly what this
phase exists to prevent. Then remove the scope marker — leaving it here is
what would block Phase 6.
3. **Operator stop** — the operator interrupts. Remove the scope marker first,
then carry whatever has been gathered into Phase 6 and record the remaining
gaps as open questions. Do not re-enter the loop after a stop.
### When the loop does not apply
**No-brief default.** Without `--project` (or with a project whose `brief.md`
is absent), there are no `phase_signals` to resolve, so `effort = 'standard'`,
the loop does not run, and all new counters (`conv_turns`, `empty_turns`) are
emitted as `0`.
**Precedence matrix — each entry independently makes the loop moot**, the same
way `--engine` is moot when the external phase does not run (see the moot gate
in Phase 4):
| Condition | Effect on the loop |
|-----------|--------------------|
| `--quick` | Moot — Phase 3.5 skips to Phase 8; the swarm never runs |
| `--local` | Moot — no outbound queries to bound |
| `external_research_enabled: false` (profile) | Moot — the profile's on/off switch wins |
**Interaction rule.** A brief that carries `effort: high` **without** a
`model`, under a cheap profile (`economy`/`balanced`): the effort signal
governs orchestration shape, so the loop is armed, but the profile still
supplies the model — and if that profile disables external research, the
matrix above wins and the loop is moot regardless of effort.
**Honesty (hard rule, restated for this loop).** More turns do not make a
finding more credible. Turn count is a cost, not evidence: report what the
citations support, and let an exhausted cap show up as open questions rather
than as confidence.
## Phase 6 — Triangulation
This is the KEY phase that makes trekresearch more than aggregation.
@ -749,48 +423,6 @@ Write the brief to the `brief_destination` computed in Phase 1:
Create the parent directory if it does not exist.
### Deep-research engine adapter (engine = deep-research only)
**Only when `effective_engine == deep-research`.** The swarm path skips this
entirely — its findings already flow through Phases 6–7 unchanged (SC1).
Transform the in-context `/deep-research` report INTO
`@${CLAUDE_PLUGIN_ROOT}/templates/research-brief-template.md` — do NOT paste the
raw report. Specifically:
- Reduce the report to ≥ 1 `### {Dimension} -- Confidence: {high|medium|low|contradictory}`
entry, each carrying **External findings** bullets with per-claim source URLs.
Local findings still come from the local agents (Phase 4) and are merged in per
dimension as usual. Wave 2's `contrarian-researcher` findings (Launch rules)
are not part of the report: they are merged in per dimension as **External
findings**, cited by URL like the report's, and where one disputes the
report's leading answer, the dimension's **Contradictions** gives both sides.
- Emit a numeric `confidence ∈ [0,1]` in frontmatter and a 3-sentence
`## Executive Summary` (answer, confidence, key caveat).
- Populate `## Sources` from the report's citations.
- **If the report lacks per-claim URLs, lower the confidence and note the gap in
`## Open Questions` — do NOT fabricate URLs.** Provenance you cannot cite is not
provenance.
- If the report is large, bound the transform to the top dimensions to avoid
context truncation.
### Output self-check (engine = deep-research only)
**Only when `effective_engine == deep-research`.** After writing to
`brief_destination`, run the output validator and repair-or-fall-back. This mirrors
the trekplan Phase-8 write→validate→repair self-check; the swarm path does NOT run
it, so swarm behavior is unchanged (SC1):
```bash
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/research-validator.mjs --json "{brief_destination}"
```
On `valid: false`, repair the brief to satisfy the reported errors and re-run the
validator. If it cannot be made valid (e.g. the report was too thin to yield even
one dimension), set `effective_engine = swarm`, fall back to the swarm engine for
this run (and log the fallback per the Engine selection step), rather than emit an
invalid brief.
## Phase 8 — Present
Present a summary to the user:
@ -802,7 +434,6 @@ Present a summary to the user:
**Mode:** {default | quick}, Scope: {both | local | external}
**Brief:** {brief_destination}
**Project:** {project_dir or "-"}
**Engine (effective):** {swarm | deep-research}{, with fallback reason if it fell back}
**Confidence:** {overall confidence 0.0-1.0}
**Dimensions:** {N} researched
**Agents:** {N} local + {N} external
@ -899,14 +530,6 @@ The former third-party second-opinion bridge pass is **unavailable,
skipped** (dropped in v5.10.1 by operator directive). High effort no
longer has a bridge arm.
High effort additionally arms the Phase 5 bounded follow-up loop (max 3
turns per under-illuminated dimension, budgeted by
`research-loop-cap.mjs`, every outbound query gated by
`query-privacy-gate.mjs`). The loop stays default-off until
`VOYAGE_STORM_ENABLED=1`, and the moot matrix in Phase 5 (`--quick`,
`--local`, `external_research_enabled: false`) overrides the effort
signal whenever the external phase does not run at all.
Standard effort (or absent): the selected agents run as in Phase 4, and
`contrarian-researcher`, when selected, runs as wave 2 after the other
agents return (see Launch rules).
@ -921,32 +544,6 @@ Low effort: inline research only, no agent swarm (existing
The full source rules are the `researching-sources` skill.
- **Independence:** Do not pre-bias external agents with local findings or vice versa.
Triangulate AFTER independent research.
**Amended (v5.10) for Phase 4.5:** dimension discovery deliberately crosses this
rule. Its candidate dimensions are mined from the Phase-4 result set, which
contains output from the five local codebase agents, so a discovered dimension
can carry local context into an external query.
The crossing creates **two distinct risks**, and they do not share a control:
- **Bias** — a local finding shapes what an external agent is asked. Its
controls are structural, not a gate: the initial external swarm stays blind
to local findings, so an **independent baseline already exists** before
anything crosses; the crossing is confined to Phase 4.5 and the Phase 5 loop
it feeds, which only ADD to that baseline and never revise it; and at
`effort: high` — the only effort at which any of this runs —
`contrarian-researcher` is forced always-on, so the brief always carries an
adversarial counter-evidence pass over the result the crossed queries fed.
Triangulation still happens AFTER independent research.
- **Egress** — local paths, repo identifiers or secret-shaped strings leave the
machine inside a query. That is what `query-privacy-gate.mjs` controls: every
outbound query is inspected before it leaves, with a hard-block tier for
secret-shaped strings that no operator flag can override.
The privacy gate was previously named as the compensating control for the
crossing as a whole. It is not: it inspects query CONTENT and cannot stop a
local finding from steering an external agent's question. Attributing the bias
risk to it left that risk with no control while the text read as though it had
one.
- **Graceful degradation:** If MCP tools are unavailable (Tavily, MS Learn),
proceed with available tools and note limitations in brief metadata.
- **Cost:** Model resolution at Agent-spawn sites is a three-layer fallback:

7
docs/architecture.md
View file

@ -12,8 +12,7 @@ Imported from `CLAUDE.md` via pointer.
- `lib/review/{verdict-gate,engine-findings,recurrence}.mjs` — the review gate (K7: 0 ALLOW, or WARN with every counting MAJOR answered · 1 BLOCK · 2 never green), Claude Code `/code-review` + `/security-review` findings → reviewer payload, recurring finding classes → operator-approved project-memory lines (review_version 1.2); `lib/review/fill-finding-ids.mjs` computes the coordinator's `@@ID_<n>@@` placeholders over `(file, line, rule_key)`; `lib/validators/brief-review-stamp.mjs` lets `/trekplan` skip a brief review `/trekbrief` already ran on the same text
- `lib/validators/{brief,research,plan,progress,session-state}-validator.mjs` — schema validators with CLI shims (`node lib/validators/X.mjs --json <path>`)
- `lib/validators/architecture-discovery.mjs` — drift-WARN external-contract discovery for `architecture/overview.md`
- `lib/util/research-loop-cap.mjs` — stateful, **default-off** turn budget for the `/trekresearch` bounded conversation loop. `allowTurn()` derives the used-turn count from its own append-only JSONL ledger; it never asks the caller how many turns it has spent, because a cap that does is not a cap. Each grant first claims a turn **slot** with `O_EXCL` under `trekresearch-loop-claims/`, so the bound survives several callers deciding at once — counting the ledger and then appending is read-then-write, and Phase 4.5/5 can spawn several agents in one message. Budget = `TREKRESEARCH_MAX_CONV_TURNS` (default `3`, invalid values fall back to `3`) × `maxDimensions` (8, `MAX_TOTAL_DIMENSIONS`). **Default-off:** grants 0 unless `VOYAGE_STORM_ENABLED=1`, which is also the second condition on Phase 4.5's skip-guard (that phase does not call this module): unset, **both** STORM phases are inert. `resolveDataRoot()` is the single root for everything the loop writes — `CLAUDE_PLUGIN_DATA` when the harness sets it, `~/.claude/voyage` when it does not (it is empty in the Bash tool's process env, which is where the loop actually runs); the cap hook resolves through the same function, so writer and reader cannot disagree. A ledger that cannot be **written** denies the turn, and one that exists but cannot be **read** denies it too — only `ENOENT` counts as zero turns spent, that being the legitimate first-turn state (fail-closed). The exported `readLedger()` is the single counting rule; the cap hook calls it rather than keeping a private copy. `checkDimensionCeiling()` is the reader for the OTHER axis of the bounded-cost NFR — the size of the whole dimension list after Phase 4.5 discovery — against the same `MAX_TOTAL_DIMENSIONS`, so the two axes cannot enforce different numbers for one ceiling; an unreadable count is rejected, not waved through. CLI shim, two modes: `--run-id ID --dimension D --effort E` (budget gate) and `--check-dimensions N` (ceiling, no run id/effort/flag required since Phase 4.5 never calls the budget gate)
- `lib/validators/query-privacy-gate.mjs` — gates **every** outbound research query before it leaves the machine; the hard-block tier (secret-shaped strings) is not operator-overridable, so a query that trips it must be reformulated rather than forced through. CLI shim: `node lib/validators/query-privacy-gate.mjs "<query>"`
- `lib/validators/query-privacy-gate.mjs` — gates an outbound research query before it leaves the machine (no command calls it now: its callers were the removed STORM steps); the hard-block tier (secret-shaped strings) is not operator-overridable, so a query that trips it must be reformulated rather than forced through. CLI shim: `node lib/validators/query-privacy-gate.mjs "<query>"`
Wiring points (replaces previous prose-grep instructions):
- `/trekbrief` Phase 4g → `brief-validator` (post-write sanity check)
@ -31,8 +30,6 @@ Doc-consistency test at `tests/lib/doc-consistency.test.mjs` pins agent-table co
`hooks/scripts/session-title.mjs` (UserPromptSubmit, CC v2.1.94+) sets `sessionTitle` to `voyage:<command>:<slug>` for voyage-command invocations. Helps parallel sessions identify themselves in process lists.
`hooks/scripts/pre-agent-cap.mjs` (PreToolUse on `WebSearch|WebFetch|Task`) enforces the `/trekresearch` Phase 5 loop bound in the harness, so the cap is not merely prose the model is asked to obey. It counts spent turns read-only from the append-only ledger `research-loop-cap.mjs` writes, through that module's own exported `readLedger()`. It denies (exit 2) once the budget **gate has denied a turn** — the primitive records its denials as tombstones, and the tombstone is the boundary rather than the count, because `allowTurn()` appends before the turn runs and so the final granted turn already shows `budget` records. A ledger showing more granted turns than the budget denies too, as a backstop. Being in scope but unable to count the ledger also denies: a budget control that cannot count must not grant. Scope key = `session_id` + a marker file only the loop writes (`<resolveDataRoot()>/trekresearch-loop-scope/<session_id>.json`, the same root the ledger uses); without a marker the hook allows unconditionally, which is what keeps a globally-wired `PreToolUse` hook from over-blocking unrelated sessions. Stale markers auto-reset on a TTL and `VOYAGE_DISABLE_CAP_HOOK=1` is the kill switch. Defence in depth only — `lib/util/research-loop-cap.mjs` must stay correct if the hook stops firing (see `docs/spike-pretooluse-subagent-reach.md`).
`hooks/scripts/post-compact-flush.mjs` (PostCompact event, v3.4.0) re-injects `.session-state.local.json` after context compaction so multi-session work survives a compaction boundary. Companion to `pre-compact-flush.mjs` (which writes the state file before compaction); together they form the rehydrate cycle that keeps `/trekcontinue` reliable across long-running multi-session work.
## Architecture
@ -94,7 +91,7 @@ Which native Claude Code primitive each pipeline step runs on today, and the alt
| **execute** | Inline step loop; multi-session via `git worktree` + session orders the user runs, wave by wave; deterministic manifest audit | CC `TaskCreate`/`TodoWrite` for progress/resume (insufficient — carries no step status / attempts / SHA / drift → own typed `progress.json` contract) |
| **review** | Inline parallel reviewers (no cross-feed) → `review-coordinator` Judge | **Workflow** substrate for Phase 5–6 (bake-off POSITIVE: +4.4 % tokens / +54 % wall-time → shipped opt-in, since removed; wholesale substrate swap declined) |
| **continue** | Inline reads `.session-state.local.json` → zero-confirm resume | CC `--resume` (transcript replay, not typed work-state → insufficient) |
| **cross-cutting** | 6 hook scripts: `pre-bash` + `pre-write` guards, `pre-agent-cap` loop-bound enforcement, `session-title`, `pre-`/`post-compact` flush | — |
| **cross-cutting** | 5 hook scripts: `pre-bash` + `pre-write` guards, `session-title`, `pre-`/`post-compact` flush | — |
¹ MCP per research agent: `docs-researcher` → Microsoft Learn + Tavily · `community-`/`security-`/`contrarian-researcher` → Tavily (+ WebSearch/WebFetch). Graceful degradation when an MCP server is absent.

17
docs/command-modes.md
View file

@ -27,26 +27,9 @@ Always interactive. Phase 3 puts the task in a size class (`ceremony:`, K1; heav
| `--gates [true\|false\|closed\|open]` | (v3.4.0) Autonomy flag: bare, `true` or `closed` = on; `false` or `open` = off; any other next token is not its value. See `## Autonomy mode` in `docs/operations.md`. |
| `--min-brief-version <ver>` | (S18) Warn — never block — if an attached `--project` brief declares a version below `<ver>` (e.g. `2.2`), i.e. sidesteps framing enforcement |
| `--profile <name>` | (v4.1.0) Model profile for the research phase. |
| `--engine {swarm\|deep-research}` | (deep-research-engine) Opt-in external-research engine; `deep-research` delegates the external phase to Claude Code's built-in `/deep-research` workflow, which only works on `2.1.154 <= CC < 2.1.218` — from **2.1.218** `/deep-research` is operator-invoked only (the Skill tool refuses with `disable-model-invocation`), so the engine always falls back to `swarm`. Never hard-fails. Default `swarm`. |
Flags combine: `--project <dir> --local`, `--external --quick`.
### Bounded conversation loop (Phase 4.5 + Phase 5) — env-vars
Dimension discovery and the multi-turn follow-up loop are **default-off** and
have no flag; they are environment-gated, because turning them on costs turns.
They run only at `effort: high` (resolved from the brief's `phase_signals`).
| Env-var | Default | Behavior |
|---------|---------|----------|
| `VOYAGE_STORM_ENABLED` | _(unset — default-off)_ | `=1` grants the Phase 5 loop a non-zero turn budget and is the second condition on Phase 4.5's skip-guard. Unset, `research-loop-cap.mjs` grants 0 turns and **both** phases are inert: doing nothing keeps the whole mechanism off. |
| `TREKRESEARCH_MAX_CONV_TURNS` | `3` | Max turns per under-illuminated dimension. Budget = this × `maxDimensions` (8, `MAX_TOTAL_DIMENSIONS` in `lib/util/research-loop-cap.mjs`). Empty, non-numeric, zero, negative, `Infinity`, or any fraction that floors below `1` (`0.5`, `0.9`) fall back to `3` — never to unbounded, and never to `0`. A fraction at or above `1` floors (`2.7` → `2`). |
| `VOYAGE_DISABLE_CAP_HOOK` | _(unset)_ | `=1` disables `hooks/scripts/pre-agent-cap.mjs`, the `PreToolUse` enforcement of the turn budget. The cap primitive still applies; only the second gate is switched off. |
Adoption of the loop as a default is gated on a pre-registered measurement —
protocol, thresholds, and the exact commands in
[`docs/storm-measurement.md`](storm-measurement.md).
## /trekplan modes
| Flag | Behavior |

62
docs/deep-research-engine-brief.md
View file

@ -1,62 +0,0 @@
---
type: trekbrief
brief_version: "2.2"
task: "Opt-in /deep-research-motor for /trekresearch ekstern-fase"
slug: deep-research-engine
framing: refine
status: ready
brief_quality: complete
research_topics: 3
research_status: complete
phase_signals_partial: true
---
# Brief — Opt-in `/deep-research`-motor for `/trekresearch`
## TL;DR
- `--engine {swarm|deep-research}` på `/trekresearch` ekstern-fase; `swarm` default (uendret oppførsel).
- `deep-research` delegerer ekstern research til Claude Codes innebygde workflow og adapterer inn i research-brief-skjemaet.
- Lokal analyse, triangulering og H2-output (`research/NN-*.md`) er motor-uavhengig.
- Feature-detekteres med auto-fallback til `swarm` — aldri hard feil.
- Surface-only: `commands/` + `agents/`, ingen nye `lib/`-avhengigheter.
## Intent
`/trekresearch` eier i dag hele den eksterne research-fasen selv (Tavily / MS-Learn / Gemini-sverm). Det er portabelt, men du vedlikeholder fan-out, kryssjekk og syntese selv. Anthropic sin innebygde `/deep-research` vedlikeholder fan-out, adversariell påstandsverifisering, sitatfiltrering og websøk for deg. Lar vi operatøren *velge* `/deep-research` for den eksterne fasen, får brukeren en vedlikeholdsfri "turbo" når den er tilgjengelig — uten at Voyage mister sin egen lokal-analyse, triangulering eller H2-kontrakt.
## Goal
`/trekresearch` får et `--engine {swarm|deep-research}`-valg på den eksterne fasen. `swarm` er default (dagens oppførsel). `deep-research` delegerer den eksterne fasen til den innebygde workflowen og adapterer resultatet inn i research-brief-skjemaet. Lokal analyse, triangulering og H2-output (`research/NN-*.md`) er uendret uansett motor.
## Non-Goals
- Bygge om `/trekresearch` på et eget dynamic-workflow-substrat (vurdert og forkastet, Δ≈0).
- Multi-provider (Perplexity / Google) — hører til en senere egen `workflow`-motor, ikke denne.
- Endre den lokale svermen, handover-kontraktene eller validator-skjemaene.
- Gjøre `/deep-research` til default.
## Constraints / NFR
- `/deep-research` krever Claude Code v2.1.154+ og må være på (Pro: via `/config`). Må feature-detekteres med auto-fallback til `swarm` — aldri hard feil.
- Output må passere `research-validator.mjs` (confidence ∈ [0,1], dimensions ≥ 1, påkrevde body-seksjoner).
- Ingen nye `lib/`-avhengigheter; overflate-endring i `commands/` + `agents/`.
- Doc-consistency: oppdater command-tabellen i `CLAUDE.md` + `README.md`, kjør `npm test`.
## Success Criteria
1. `/trekresearch --external --engine swarm "<q>"` gir identisk oppførsel som før (regresjon grønn).
2. `/trekresearch --external --engine deep-research "<q>"` produserer en `research/NN-*.md` der `research-validator.mjs --json` rapporterer `valid: true`.
3. Med dynamic workflows avskrudd faller `--engine deep-research` automatisk tilbake til `swarm` og logger valget — ingen exception.
4. `npm test` (inkl. doc-consistency) er grønn.
## Research Plan
1. **Programmatisk trigging av `/deep-research`** — Kan en plugin-kommando starte den innebygde workflowen og fange artefakten, eller må `commands/trekresearch.md` instruere Claude til å kjøre den? Scope: external · Konfidens: høy · Kost: lav.
`/trekresearch --external "Kan en Claude Code plugin-kommando programmatisk starte /deep-research og hente rapporten?"`
2. **Output-adapter** — Hvordan mappe `/deep-research` sin siterte rapport til research-brief-skjemaet (dimensjoner, confidence, sitater)? Scope: both · Konfidens: høy · Kost: lav.
`/trekresearch --project <dir> --local "Hva krever research-validator.mjs av research/NN-*.md?"`
3. **Feature-deteksjon + fallback** — Hvordan oppdage om workflows er på, og hvor i fasen fallback-grenen bør sitte? Scope: local · Konfidens: middels · Kost: lav.
> **Research-status (2026-06-30, operatør-beslutning «option A»):** Topic 1 (eneste ekte eksterne ukjente) er undersøkt → `docs/deep-research-engine-research.md` (validator-grønn). Funn: `/deep-research` er en innebygd **dynamic workflow** (ikke skill) → trigging KUN via prosa-instruksjon, output **inline i kontekst** (ingen on-disk-artefakt) → motoren må være instruksjons-basert + in-context transform. **SC3 er korrekt som skrevet** (`/deep-research` ER en dynamic workflow). Topic 2 (validator-skjema: `type/created/question` + `## Executive Summary`/`## Dimensions`, `dimensions ≥ 1`) og topic 3 (fallback-plassering) er lokale kode-spørsmål reklassifisert til `/trekplan`-utforskning. `research_status: complete` reflekterer denne beslutningen.
## Open Questions / Assumptions
- Antar at `/deep-research`-rapporten kan reduseres til ≥ 1 dimensjon med per-påstand-sitater uten å bryte trianguleringen. Verifiseres i topic 2.
- Uavklart om delegering skjer via instruksjon (trygt) eller programmatisk API (raskere) — topic 1 avgjør.
## Prior Attempts
- Dvalende synthesis-agent bygget, målt (Δ≈0) og forkastet → samme lærdom gjelder substrat-bytte: mål før du adopterer.
- Workflow-substratet er allerede dokumentert som vurdert-og-valgt-bort i `docs/architecture.md` §Primitives per step.

213
docs/deep-research-engine-research.md
View file

@ -1,213 +0,0 @@
---
type: trekresearch-brief
created: 2026-06-30
question: "Can a Claude Code plugin command programmatically trigger the built-in /deep-research workflow and capture its report, or must it instruct Claude to run it?"
confidence: 0.85
dimensions: 4
mcp_servers_used: []
local_agents_used: [claude-code-guide]
external_agents_used: []
slug: deep-research-engine
feeds_brief: docs/deep-research-engine-brief.md
research_topic: 1
---
# Research — Programmatic trigging of `/deep-research` from a plugin command
> Targeted single-topic research for the `deep-research-engine` brief (Topic 1 of
> the brief's Research Plan). Topics 2 & 3 are local Voyage-code questions folded
> into `/trekplan` exploration; only Topic 1 was a genuine external unknown.
> Method: `claude-code-guide` agent (Anthropic docs + CHANGELOG, cited) +
> direct inspection of this machine (CC 2.1.196 binary, a real local `/deep-research`
> run). Not run through the `/trekresearch` swarm — see brief reconcile note.
## Research Question
Can a Claude Code **plugin slash-command** (markdown under `commands/`)
**programmatically** start the built-in `/deep-research` workflow and **capture its
report artifact** for adaptation into the research-brief schema — or must the command
instead **instruct** Claude (in prose) to run `/deep-research` and then transform the
in-context result?
## Executive Summary
Programmatic trigger + file-based capture is **not feasible**: `/deep-research` is a
built-in **dynamic workflow** (not a skill), deliberately outside the Skill-tool
allowlist, and it returns its report **inline into conversation context with no
documented on-disk report artifact**. The **only reliable path is instruction-based
delegation** — the command's prose tells Claude to run `/deep-research <q>`, then
transforms the in-context report in the same turn. Confidence **high** (Anthropic docs +
CHANGELOG + local run), with one residual gap: there is no positive "is it enabled?"
probe, so feature-detection must lean on the documented *disable* switches + version
floor + a `swarm` default.
## Dimensions
### 1. Origin + gating -- Confidence: high
**External findings:**
- `/deep-research` is a **built-in dynamic workflow**, not a command and not a bundled
skill. `commands.md` marks the `/deep-research <question>` row as **[Workflow]**;
`workflows.md`: "Claude Code includes `/deep-research` as a built-in workflow."
[VERIFIED — code.claude.com/docs/en/commands.md, code.claude.com/docs/en/workflows.md]
- Gating: available on **all paid plans** (pro/max/team/enterprise) + API/Bedrock/Vertex/
Foundry. **On Pro it must be turned on** in the *Dynamic workflows* row of `/config`.
Also requires the **WebSearch tool** to be available.
[VERIFIED — workflows.md, commands.md bundled-workflows row]
- Version floor: dynamic workflows were **introduced in CC 2.1.154**; "Dynamic workflows
require Claude Code v2.1.154 or later." The brief's `v2.1.154+` + `(Pro: via /config)`
constraints are **both correct**. The exact version that first shipped the *named*
`/deep-research` workflow is **[NOT DOCUMENTED]** (only a 2.1.196 bugfix mentions it by
name) — treat 2.1.154 as the substrate floor, not a proven introduction point.
[VERIFIED — workflows.md + CHANGELOG 2.1.154]
**Local findings:**
- This machine runs **CC 2.1.196** (`claude --version`) — substrate floor satisfied.
- The exact skill-description string lives **compiled into the binary**
(`/Users/ktg/.local/share/claude/versions/2.1.196`, Mach-O 235 MB); there is **no
`SKILL.md`** for it anywhere under `~/.claude` (system-wide `find`/`grep` — only hits
are this brief + unrelated harness notes). Confirms "Anthropic-bundled, not user skill."
### 2. Invocation mechanism (programmatic vs instruction) -- Confidence: high
**External findings:**
- **Not** via the `Skill` tool. The Skill-tool built-in allowlist is closed: only
`/init`, `/review`, `/security-review` are reachable; "Other built-in commands such as
`/compact` are not." `/deep-research` is a Workflow and is not on that list.
[VERIFIED — code.claude.com/docs/en/skills.md]
- **Instruction-based delegation is the documented mechanism.** Workflows launch when the
user types the command, or **when Claude is asked in natural language** ("use a
workflow" / "run a workflow") or via the dedicated opt-in keyword CC ships for
multi-agent orchestration (CC 2.1.160; spelled out in
`docs/cc-upgrade-2.1.181-decision-matrix.md`, which `verify.sh`'s SC1 excludes
precisely because it legitimately cites CC keywords). A plugin command whose
markdown instructs Claude to run `/deep-research <q>` is therefore the supported path.
[VERIFIED — workflows.md "Have Claude write a workflow"]
- **Approval gate caveat:** launching a workflow triggers a per-run approval prompt —
*every run* in default/acceptEdits; *first launch only* in auto; **never in `claude -p`
/ Agent SDK / bypass-permissions** ("the run starts immediately"). Voyage's headless
surface (`claude -p`) thus delegates without an interactive gate; interactive sessions
hit a prompt. [VERIFIED — workflows.md "Behavior and limits"]
- No documented blanket "commands/skills cannot nest" prohibition beyond the Skill-tool
allowlist + workflow runtime limits (no mid-run user input; 16 concurrent agents;
1000 agents/run). [VERIFIED — workflows.md; NOT DOCUMENTED for a general nesting ban]
### 3. Output capture -- Confidence: high
**External findings:**
- "When the run finishes, **the report lands in your session**"; "Claude's context holds
only the final answer." The report is **in-context**, not a file.
[VERIFIED — workflows.md]
- What *is* written to disk is the orchestration **script**, not the report: "Every run
writes its script to a file under your session's directory in `~/.claude/projects/`."
[VERIFIED — workflows.md "How a workflow runs"]
**Local findings:**
- A real `/deep-research` run on this machine left exactly one file —
`~/.claude/projects/<session>/workflows/scripts/deep-research-wf_<id>.js` — and **no
`.md` report** beside it. [VERIFIED — local filesystem inspection by claude-code-guide]
- **Consequence:** a calling command cannot `grep` a results file off disk (none is
documented to exist). It can only consume the report **as it sits in conversation
context in the same turn** — Claude reads its own prior output and transforms it into
research-brief schema. [INFERENCE — file-based capture not feasible; in-context
transform is the only avenue]
### 4. Feature detection + fallback -- Confidence: medium
**External findings:**
- **No positive enumeration / "is-enabled" API or flag is documented.** `claude --help`
exposes `--disable-slash-commands` but no "list skills/workflows" or "is-feature-on"
flag. [VERIFIED — local `claude --help`; NOT DOCUMENTED for any positive probe]
- The documented signals are the **off-switches**, read defensively: `/config` Dynamic-
workflows off; `disableWorkflows: true` / `disableBundledSkills: true` in settings.json;
`CLAUDE_CODE_DISABLE_WORKFLOWS=1` / `CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1`.
[VERIFIED — workflows.md "Turn workflows off"; CHANGELOG 2.1.x]
- **The gap:** the Pro `/config` *on*-state is the very thing you most need to detect, and
only the *disable* keys are documented; the persisted key/value for the Pro enable-state
is **[NOT DOCUMENTED]**, so it cannot be reliably grepped. A failed/disabled
`/deep-research` is also **not documented** to raise a signal a sibling command can
catch. [VERIFIED gap]
**Local findings:**
- On this machine both disable keys are unset/absent — workflows are not disabled.
## External Knowledge
### Best Practice
The "Have Claude write a workflow" + "Behavior and limits" sections of `workflows.md`
establish that workflows are operator/Claude-launched, run isolated, and return one
in-context report. The Skill-tool allowlist (`skills.md`) is the authoritative statement
that only three built-ins are tool-invocable.
### Known Issues
Per-run approval prompts outside `-p`/SDK/bypass mean an interactive `/trekresearch
--engine deep-research` will pause for operator approval on each launch — acceptable, but
worth documenting in the command UX. The absence of a positive availability probe is the
single biggest design constraint (see Dimension 4).
## Synthesis
Three cross-cutting insights that only emerge from combining the docs with Voyage's brief:
1. **The brief's SC3 is correct as written — an earlier review note was wrong.** Because
`/deep-research` *is itself* a dynamic workflow, "med dynamic workflows avskrudd faller
`--engine deep-research` tilbake til swarm" is the right feature-detection axis. A
prior brief-review remark that SC3 "conflated `/deep-research` (skill) with dynamic
workflows" was based on a wrong premise (that `/deep-research` was a skill) and is
retracted. **No SC3 brief edit is needed.**
2. **The engine must be instruction-based + in-context, never file-based.** Topic 1's
open question ("instruction (safe) vs. programmatic API (faster) — topic 1 decides")
resolves decisively to **instruction-based**: there is no programmatic API and no
on-disk report. The `deep-research` engine path in `commands/trekresearch.md` must
(a) instruct Claude to run `/deep-research <q>`, then (b) transform the in-context
report into the `research/NN-*.md` schema in the same turn. This is surface-only
(`commands/` prose), matching the brief's "ingen nye `lib/`-avhengigheter."
3. **SC3's "ingen exception" cannot rest on runtime detection — pin it to a default.**
Since no positive availability probe exists, robust fallback = version-floor check
(≥ 2.1.154) + disable-key heuristic (`disableWorkflows` / env) + **`--engine` default
of `swarm`** (explicit opt-in). The fallback is "graceful degradation by design,"
not "catch an exception at runtime." This refines, but does not contradict, SC3.
## Open Questions
- **Adapter fidelity (brief Topic 2, partly answered locally):** `research-validator.mjs`
requires `type: trekresearch-brief` + `created` + `question`; validates `confidence ∈
[0,1]` and `dimensions ≥ 1` if present; body must carry `## Executive Summary` +
`## Dimensions`. So the `/deep-research` report **can** be reduced to ≥ 1 dimension with
per-claim citations and a confidence number — the brief's assumption holds. The
remaining open part (how cleanly the in-context report maps to per-dimension
local/external splits) is a `/trekplan` exploration concern, not an external unknown.
- **Exact intro version of the *named* `/deep-research` workflow** — not documented; the
2.1.154 dynamic-workflows floor is the safe pin.
## Recommendation
Build the `deep-research` engine as **instruction-based delegation with in-context
adaptation**, not a programmatic trigger:
1. `--engine deep-research` makes `commands/trekresearch.md`'s external phase instruct
Claude to run `/deep-research <q>` and transform the returned in-context report into
`research/NN-*.md` (validator-conformant: `## Executive Summary` + `## Dimensions`,
`confidence`, `dimensions ≥ 1`).
2. Feature-detect by **graceful degradation**: version floor + disable-key heuristic +
`--engine` default `swarm`. Do not depend on a positive availability probe (none
exists). Log the chosen engine. This satisfies SC3 without a runtime exception.
3. Keep `swarm` the default (brief's Non-Goal: "ikke default-bytte"). Document the
per-run approval prompt for interactive (non-`-p`) sessions.
Confidence in the recommendation: **high** for the mechanism (instruction-based +
in-context), **medium** for the exact fallback-detection ergonomics (the one documented
gap). This is sufficient to green-light `/trekplan` with Topic 1 resolved.
## Sources
| # | Source | Type | Quality | Used in |
|---|--------|------|---------|---------|
| 1 | code.claude.com/docs/en/workflows.md | official | high | Dim 1,2,3,4 + Synthesis |
| 2 | code.claude.com/docs/en/commands.md | official | high | Dim 1 (Workflow classification, WebSearch req) |
| 3 | code.claude.com/docs/en/skills.md | official | high | Dim 2 (Skill-tool allowlist) |
| 4 | github.com/anthropics/claude-code CHANGELOG (2.1.154, 2.1.x, 2.1.196) | official | high | Dim 1,4 (version floor, disable keys) |
| 5 | Local: CC 2.1.196 binary inspection (`find`/`grep`, `claude --version`) | codebase | high | Dim 1 (bundled, no SKILL.md) |
| 6 | Local: real `/deep-research` run — only `.js` script written, no `.md` report | codebase | high | Dim 3 (no on-disk artifact) |
| 7 | Local: `lib/validators/research-validator.mjs` schema | codebase | high | Open Questions (adapter feasibility / Topic 2) |

2
docs/sdlc-playbook-gap.md
View file

@ -63,7 +63,7 @@ goes to it under MIT, and to the Anthropic article it is itself derived from.
| Candidate | Does Voyage already have it | Verdict |
|---|---|---|
| Hash-chained, tamper-evident approval ledger | No. No hash chain exists anywhere; `lib/util/research-loop-cap.mjs` is append-only but unchained | **Partly.** The idea answers "did we build what we agreed", but a new ledger engine is out of scope; the cheap form is a `prev_hash` field on the event ledger that already exists |
| Hash-chained, tamper-evident approval ledger | No. No hash chain exists anywhere | **Partly.** The idea answers "did we build what we agreed", but a new ledger engine is out of scope; the cheap form is a `prev_hash` field on the event ledger that already exists |
| Plan as a machine-readable file list with a lock | Yes — `plan_version 1.7` manifest, `plan-validator.mjs --strict`, and the independent Phase 7.5 audit | **Borrow the boundary only:** their named process-file exclusion list, so a state or docs update does not need a manifest entry |
| Short intent and spec templates | Partly — the brief already requires `## Intent`, `## Goal` and `## Success Criteria` | **No** to a separate spec artifact (the operator is both orderer and builder). **Yes** to their "Gotchas" section: one fixed place for where two constraints collided and who decided |
| A rule → enforcement table ("advisory or enforced, and where") | No such table; the doc-pins are that idea applied narrowly | **Yes, cheap.** It makes visible which of Voyage's own rules are only prose |

160
docs/spike-pretooluse-subagent-reach.md
View file

@ -1,160 +0,0 @@
# Spike: does a plugin `PreToolUse` hook reach sub-agent tool calls?
**Date:** 2026-08-09
**Claude Code version:** 2.1.226
**Plugin:** voyage 5.9.1 (installed from `ktg-plugin-marketplace`)
**Gates:** Step 10 of `plan.md` (`2026-06-30-trekresearch-storm-upgrade`)
## Question
Step 10 wants to enforce the conversation-turn cap in a `PreToolUse` hook. That
is only viable if a **plugin** `PreToolUse` hook fires on tool calls made
*inside a sub-agent*. If it does not, the cap must be enforced somewhere else
and Step 10 becomes a documented downgrade instead.
The question is genuinely open, not answerable from docs alone: the current
[hooks reference](https://code.claude.com/docs/en/hooks) states that sub-agent
tool calls fire the same hooks and carry `agent_id` / `agent_type`, while
[issue #34692](https://github.com/anthropics/claude-code/issues/34692) reported
the exact opposite behaviour. The answer is therefore version-dependent and had
to be measured on the version actually in use.
## Method
A one-shot probe hook was registered for the `WebSearch` matcher, and a
**headless child session** was launched to exercise it. The child was used
because hooks are resolved when a session starts — a matcher added mid-session
cannot be observed by the session that added it.
Two deviations from the step as originally written, both forced and both
verified not to affect the result:
1. **The matcher was injected into the installed plugin's `hooks.json`, not the
repository's.** The plan assumed the repo working tree *is* the active plugin
root. It is not: `~/.claude/plugins/cache/ktg-plugin-marketplace/voyage/5.9.1/`
is a plain directory holding its own copy, and that copy is what loads.
Editing `hooks/hooks.json` in the repo would have measured nothing. The cache
file was backed up, modified, and restored — verified byte-identical to the
repo file afterwards.
2. **The probe script lives under the session scratchpad, not `${TMPDIR}`.** A
pathguard hook refuses writes to `${TMPDIR}`. The load-bearing property was
only that the script sit **outside `hooks/scripts/`**, which
`tests/lib/doc-consistency.test.mjs:75-84` counts via `readdirSync`; the
scratchpad satisfies that just as well. The directory still holds 7 scripts.
### Probe hook
Logged every invocation as one JSON line (`tool_name`, `agent_id`,
`agent_type`, plus the untouched stdin) and always exited `0`, so it could not
alter the child's behaviour.
### Commands
```bash
# 1. inject the temporary matcher into the INSTALLED plugin
node -e '...push {matcher:"WebSearch", ...voyage-spike-hook.mjs} into hooks.PreToolUse...'
# 2. exercise it from a fresh child session
claude -p "Spawn exactly one sub-agent via the Agent tool (subagent_type: general-purpose).
Instruct that sub-agent to perform exactly ONE WebSearch for the query
'claude code hooks reference' and report back the first result title.
You MUST NOT call WebSearch yourself in the main context - only the
sub-agent may call it. When the sub-agent returns, reply with the word DONE." \
--allowedTools "Agent,Task,WebSearch" \
--max-turns 15
# 3. restore
cp "${TMPDIR}voyage-hooks-backup.json" <cache>/hooks/hooks.json
```
The child returned `DONE`.
> An earlier attempt additionally passed `--permission-mode bypassPermissions`
> and was refused by the auto-mode classifier. The flag was dropped;
> `--allowedTools` alone was sufficient.
## Raw observation
The log contains **exactly one** record — so the main context did not call
`WebSearch` itself, and the single entry is unambiguously the sub-agent's call:
```json
{"at":"2026-08-09T12:07:47.796Z","tool_name":"WebSearch",
"agent_id":"aa6d19525a4680fe0","agent_type":"general-purpose",
"raw_stdin":"{\"session_id\":\"b126fd6a-...\",\"cwd\":\"/Users/ktg/repos/ktg-plugin-marketplace/voyage\",
\"permission_mode\":\"auto\",\"agent_id\":\"aa6d19525a4680fe0\",\"agent_type\":\"general-purpose\",
\"effort\":{\"level\":\"xhigh\"},\"hook_event_name\":\"PreToolUse\",\"tool_name\":\"WebSearch\",
\"tool_input\":{\"query\":\"claude code hooks reference\"},\"tool_use_id\":\"toolu_01Ka4...\"}"}
```
Both `agent_id` and `agent_type` are populated, matching the documented
common-input fields for sub-agent-originated tool events. A main-context call
would have carried neither.
## Consequence for Step 10
A plugin `PreToolUse` hook **does** observe sub-agent tool calls on CC 2.1.226,
and can attribute them via `agent_id` / `agent_type`. Step 10 may therefore take
the enforcement branch rather than the documented-downgrade branch.
Two limits worth carrying forward, neither of which changes the verdict:
- This measures `WebSearch` on one CC version. The behaviour regressed once
before (#34692), so the hook must fail **open**, never assume it is the only
gate, and the cap must remain correct if the hook silently stops firing.
- The probe only establishes *reach*. Whether a **blocking** (exit 2) decision
from inside a sub-agent propagates usefully was not measured — the probe
always exited 0 by design.
RESULT: FIRES
## Enforcement outcome
Step 10 took the **enforcement branch**: `hooks/scripts/pre-agent-cap.mjs`
(PreToolUse, matcher `WebSearch|WebFetch|Task`), pinned by
`tests/hooks/agent-cap.test.mjs`.
What it does: counts turns spent by a run — read-only, from the append-only
ledger that `lib/util/research-loop-cap.mjs` writes — and exits 2 once
`turns_used >= max_conv_turns × maxDimensions`. It never appends to the
ledger; a cap that recorded its own enforcement would count itself.
Scope key, the part that makes a globally-wired `PreToolUse` hook safe:
`session_id` **+** a marker file only the Phase 5 loop writes, at
`${CLAUDE_PLUGIN_DATA}/trekresearch-loop-scope/<session_id>.json`:
```json
{ "runId": "<run id>", "startedAt": "<ISO-8601>" }
```
No marker for the calling session ⇒ out of scope ⇒ allow, unconditionally.
An unrelated session is never denied because some other run spent its budget.
Fail-open and fail-closed are split deliberately:
| Condition | Outcome | Why |
|---|---|---|
| No marker / no `session_id` / unparsable stdin | allow | Not evidence of a loop turn |
| Marker older than TTL (default 2h, `VOYAGE_CAP_SCOPE_TTL_MS`) | allow + auto-reset | A crashed run must not deny tool calls forever, and `--resume` keeps the same `session_id` |
| `VOYAGE_DISABLE_CAP_HOOK=1` | allow | Kill switch |
| `VOYAGE_STORM_ENABLED` ≠ `1` | allow | Default-off: no loop runs, nothing to enforce |
| In scope, `CLAUDE_PLUGIN_DATA` absent | **deny** | A budget control that cannot count must not grant — same stance as `research-loop-cap.mjs` |
| In scope, budget spent | **deny (exit 2)** | The bound |
Both limits recorded above still hold and are not closed by this step. Reach
was measured on one CC version for one tool, and blocking-propagation from
inside a sub-agent was never measured — so this hook is **defence in depth**,
and `research-loop-cap.mjs` must remain correct on its own if the hook
silently stops firing.
**Follow-up, closed (S79).** Step 10 shipped the hook correct but **latent** —
nothing wrote the marker, and it enforces exactly when a marker exists.
`commands/trekresearch.md` Phase 5 now writes it (`### Loop scope marker`,
keyed by `CLAUDE_CODE_SESSION_ID`, carrying the same `run_id` the ledger is
counted under) and removes it on all three exits. Cleanup covers exits only; a
crashed session is covered by the TTL above. Verified end-to-end: the snippet
as shipped arms the hook, the hook denies at `8/8` turns, and the removal
snippet returns it to allow so Phase 6 can still spawn agents. Pinned by
`tests/lib/doc-consistency.test.mjs` (STORM marker), which derives the
directory name from `SCOPE_DIRNAME` in the hook, so renaming either side
fails.

136
docs/storm-measurement.md
View file

@ -1,136 +0,0 @@
# STORM adoption gate — pre-registered measurement protocol
**Status:** thresholds registered, **no measurement run yet.** This document is
committed *before* the first measurement by design: a threshold chosen after
seeing the numbers is not a threshold. **Harness:** `scripts/storm-measure.mjs`
(pinned by `tests/scripts/storm-measure.test.mjs`). **Decides:** whether the
bounded Phase 5 conversation loop (`commands/trekresearch.md` §Loop bound) is
worth turning on by default.
---
## 1. What this gate measures — and what it does not
The gate measures **source and coverage breadth**:
| Metric | Definition | Arm |
|---|---|---|
| `unique_sources` | distinct external sources cited by the run | between-arm median gain |
| dimensions over baseline | `(dimensions − dimensions_baseline) / dimensions_baseline` | within-run median across the treatment arm |
It does **not** measure outline quality, answer correctness, synthesis
usefulness, or operator satisfaction. A breadth win is a *necessary* condition
for adoption, never a sufficient one. If the loop widens coverage by 40% and
the resulting briefs read worse, the correct action is still decline — the
number does not overrule a reading of the artifacts.
The second metric is a within-run delta by construction (`dimensions_baseline`
is the interview dimension count, `dimensions` the post-Phase-4.5 list), so it
needs no control arm; the control arm's value is 0 because the loop is inert
below `effort: high`.
## 2. Pre-registered thresholds
Either metric clearing the bar is enough. The brief pre-registers "median
forbedring ≥ 30 % på (a) eller (b) → adopt. < 15 % → decline" — a breadth win
on one axis counts, because either axis widening is the effect the loop claims.
Adopt is evaluated first, so a run that clears the adopt bar on one metric is
an adopt even when the other metric sits under the decline bar.
| Median gain | Verdict | Action |
|---|---|---|
| ≥ 30% on **either** metric | **adopt** | Flip the `VOYAGE_STORM_ENABLED` default (see §5) |
| < 15% on **either** metric (and no adopt) | **decline** | Leave the mechanism default-off. This is a **no-op**: nothing is rolled back |
| both metrics in 15% – 30% | **inconclusive** | Keep default-off, gather more runs, re-measure |
| either arm empty | **insufficient-data** | Not a decline — measure more |
`ADOPT_THRESHOLD = 0.30` and `DECLINE_THRESHOLD = 0.15` are exported constants
in `scripts/storm-measure.mjs` and pinned by the test suite. Changing them is a
deliberate, reviewable act, not a tuning knob to be nudged toward a result.
## 3. Excluded runs (the honest denominator)
Runs with `empty_turns > 0` are **excluded from the gain and reported as a
count**. An empty turn is one that spent budget and returned no findings, or
findings without citations. A run whose `empty_turns` cannot be **read** as a
number is excluded on the same footing: a field that says `"many"` is not
evidence of zero empty turns, and treating it as one would let the least
trustworthy run back into the denominator in the direction that flatters
adoption. Including those runs decides adoption on a broken
denominator — the loop looks cheap because its failures are averaged into its
successes. The harness prints the excluded count on every invocation; if that
count is a large fraction of the treatment arm, the finding is about the loop's
reliability, and it should be read before the gain figure is read at all.
Rows predating the Step 9 measurement fields carry no `effort` and are dropped
as `legacy` with a count. A stats file where *no* row carries `effort` is a hard
error, not an empty treatment group: a schema gap must never present itself as
"no gain".
## 4. The measurement runs (operator-run, outside this plan)
The measurement itself is **not** part of the implementation plan that built
this harness. It is an operator-run gate between that plan and any adopt commit.
Protocol:
- **n ≥ 5 runs per arm.** Fewer, and the median is an anecdote.
- **The same question set in both arms.** Two briefs, run as `--project` runs.
- The arms differ in exactly one thing: whether the loop is enabled.
- Both arms append to the same `trekresearch-stats.jsonl`; `effort` is the
grouping key that separates them.
Each run is a Claude Code session the operator opens (Voyage starts none of
them; changed 2026-09-27 before any run — the thresholds above are untouched):
```text
# Control arm (loop inert) — n >= 5, in a session opened with VOYAGE_STORM_ENABLED unset
/trekresearch --project .claude/projects/<brief-standard-effort>
# Treatment arm (loop live, effort: high in the brief's phase_signals) — n >= 5,
# in a session opened with VOYAGE_STORM_ENABLED=1 in its environment
/trekresearch --project .claude/projects/<brief-high-effort>
```
```bash
# The gate
node scripts/storm-measure.mjs --stats "${CLAUDE_PLUGIN_DATA}/trekresearch-stats.jsonl"
# Machine-readable, for a decision record
node scripts/storm-measure.mjs --json --stats "${CLAUDE_PLUGIN_DATA}/trekresearch-stats.jsonl"
# SC activation check — BOTH halves: did the latest high-effort run discover at
# least one dimension, AND is its final list a true SUPERSET of the interview
# ones? The second half is read from the run's own `dimensions_baseline_preserved`
# attestation, because the dimension NAMES that would show it directly are prose
# the exporter allowlist denies. A run that does not attest it FAILS — dropping
# two interview dimensions and appending three discovered ones is a +1 count
# delta and not a superset. Exit 0 = both halves hold.
node scripts/storm-measure.mjs --activation-check --stats "${CLAUDE_PLUGIN_DATA}/trekresearch-stats.jsonl"
```
## 5. What "adopt" concretely means
Adoption is **one constant**: `isStormEnabled()` in
`lib/util/research-loop-cap.mjs` currently requires `VOYAGE_STORM_ENABLED === '1'`.
Adopt = make the loop's budget non-zero without that opt-in, in a commit that
cites the measurement output.
This asymmetry is deliberate and was designed in before any code was written:
- **decline costs nothing** — the mechanism ships default-off, so declining is
doing nothing. No revert, no removal from a command file two other steps
already rewrote.
- **adopt costs one constant** — plus the enforcement hook
(`hooks/scripts/pre-agent-cap.mjs`) already in place to bound what gets turned
on, and the operator-visible cap-exhaustion message already required by
Phase 5's exit conditions.
## 6. Reading the result honestly
- The gate measures breadth. Say "breadth" in the decision record, not "quality".
- Report the excluded count alongside the gain, always. A 35% gain computed
after excluding 6 of 10 treatment runs is a finding about instability.
- `insufficient-data` is not a decline. Do not resolve it by lowering n.
- A verdict computed from a stats file mixing several question sets measures the
question sets, not the loop.

10
hooks/hooks.json
View file

@ -21,16 +21,6 @@
}
]
},
{
"matcher": "WebSearch|WebFetch|Task|Agent",
"hooks": [
{
"type": "command",
"command": "node",
"args": ["${CLAUDE_PLUGIN_ROOT}/hooks/scripts/pre-agent-cap.mjs"]
}
]
},
{
"matcher": "ExitPlanMode",
"hooks": [

219
hooks/scripts/pre-agent-cap.mjs
View file

@ -1,219 +0,0 @@
#!/usr/bin/env node
// Hook: pre-agent-cap.mjs
// Event: PreToolUse (WebSearch | WebFetch | Task | Agent)
// Purpose: Enforce the /trekresearch Phase 5 loop bound at the harness level,
// so the cap is a reader that fells rather than prose the model obeys.
// The subagent tool is `Agent` since Claude Code 2.1.63; the docs keep `Task`
// as an alias only "in settings and agent definitions", so the matcher names
// both (tests/hooks/storm-matcher.test.mjs).
//
// Precondition: Voyage's guards are deterministic only when no installed mod
// handles `tool.check`. A mod that does answers after the PreToolUse hooks
// and can replace their answer (code.claude.com/docs/en/permissions section Extend
// permissions with hooks). A hook cannot see whether such a mod is installed.
//
// Why this exists: the Phase 5 budget gate (lib/util/research-loop-cap.mjs) is
// invoked BY the loop. A gate the caller chooses to consult is advice. The
// spike in docs/spike-pretooluse-subagent-reach.md (RESULT: FIRES) established
// that a plugin PreToolUse hook does observe tool calls made INSIDE sub-agents
// on CC 2.1.226, which is what makes a second, non-optional gate possible.
//
// Two limits carried over from that spike, neither of which changes the design:
// - Reach was measured on one CC version and regressed once before (#34692),
// so this hook is defence in depth, never the only gate. research-loop-cap
// must stay correct if this hook silently stops firing.
// - Whether a blocking (exit 2) decision from inside a sub-agent propagates
// usefully was NOT measured — the probe always exited 0 by design.
//
// Scope key — the property that makes this safe to wire globally:
// session_id + a scope marker file that only the Phase 5 loop writes, at
// <data root>/trekresearch-loop-scope/<session_id>.json, where the data root
// comes from research-loop-cap.mjs's resolveDataRoot() — the same function
// the writer resolves through, because a writer and a reader that resolve
// the root separately are a hook that enforces nothing while reporting that
// it does:
// { "runId": "<run id>", "startedAt": "<ISO-8601>" }
// No marker for this session => out of scope => allow, unconditionally. An
// unrelated session must never be denied because some other run spent its
// budget; a PreToolUse hook that over-blocks breaks every session on the box.
//
// Stated limit, because the guarantee above is about OTHER sessions and reads
// as broader than it is: resuming a session (`--resume`) keeps the same session_id, so a
// resumed session is the same session by this key. If a run reached its cap
// and then died before removing the marker, the resume inherits the remainder
// of the TTL, for any WebSearch/WebFetch/Task — research or not. Three things
// bound it rather than close it: only an EXHAUSTED run denies at all (a
// part-spent crash leaves no tombstone and is allowed), the TTL is 2h rather
// than a working day, and every denial prints the marker path to delete. A
// liveness check would close it properly, but a PreToolUse hook has nothing
// trustworthy to check liveness against — the marker's writer is a shell
// snippet whose $$ is a subshell, not the session.
//
// Fail-open vs fail-closed, deliberately split:
// - Out of scope (no marker, no session_id, unparsable stdin, stale marker,
// kill switch, STORM off) => exit 0. Fail OPEN.
// - In scope and over budget => exit 2. Fail CLOSED, mirroring
// research-loop-cap.mjs's own stance: a budget control that cannot count
// must not grant. (The former "CLAUDE_PLUGIN_DATA absent" deny is gone —
// the root now always resolves, so that branch could no longer fire.)
// - In scope and the ledger cannot be counted (EISDIR, EACCES, EIO — anything
// but ENOENT) => exit 2, same reason. This branch used to ALLOW: the hook
// carried a private countTurns() whose catch returned 0, so an unreadable
// ledger read as "no turns spent". Counting now goes through the
// primitive's exported readLedger(), so reader and writer cannot hold
// different rules about what an unreadable ledger means.
//
// Counting is read-only. The ledger is append-only and written solely by
// research-loop-cap.mjs's allowTurn(); if this hook appended, the cap would
// count its own enforcement.
//
// Kill switch: VOYAGE_DISABLE_CAP_HOOK=1 disables enforcement entirely.
import { readFileSync, existsSync, rmSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const HERE = dirname(fileURLToPath(import.meta.url));
const { resolveLedgerPath, resolveDataRoot, resolveMaxConvTurns, isStormEnabled, readLedger, MAX_TOTAL_DIMENSIONS } =
await import(join(HERE, '..', '..', 'lib', 'util', 'research-loop-cap.mjs'));
const SCOPE_DIRNAME = 'trekresearch-loop-scope';
// 2h — comfortably longer than any real research run (a 24-turn loop at a couple
// of minutes a turn is under an hour), and short enough that debris does not own
// the rest of the working day. The TTL is measured from marker.startedAt rather
// than from last activity, and resuming a session (`--resume`) keeps the same session_id, so
// this window is what a resumed session can inherit from a run that died holding
// the marker. It was 6h; nothing needed six.
const DEFAULT_TTL_MS = 2 * 60 * 60 * 1000;
const env = process.env;
function allow() {
process.exit(0);
}
function deny(message) {
process.stderr.write(`[voyage] BLOCKED: trekresearch loop cap\n${message}\n`);
process.exit(2);
}
// 1. Kill switch.
if (env.VOYAGE_DISABLE_CAP_HOOK === '1') allow();
// 2. Default-off: no loop runs unless STORM is enabled, so nothing to enforce.
if (!isStormEnabled(env)) allow();
// 3. Parse stdin. Unparsable input is not evidence of a loop turn.
let input;
try {
input = JSON.parse(readFileSync(0, 'utf-8'));
} catch {
allow();
}
const sessionId = input?.session_id;
if (!sessionId || typeof sessionId !== 'string') allow();
// 4. Resolve the scope marker through the writer's own root resolution.
// VOYAGE_CAP_SCOPE_DIR stays as a test/override seam; unset, this lands on
// exactly the directory the Phase 5 snippet writes into.
const scopeDir = env.VOYAGE_CAP_SCOPE_DIR || resolveDataRoot(env);
const markerPath = join(scopeDir, SCOPE_DIRNAME, `${sessionId}.json`);
if (!existsSync(markerPath)) allow();
let marker;
try {
marker = JSON.parse(readFileSync(markerPath, 'utf-8'));
} catch {
allow(); // A marker we cannot read cannot tell us which run we are in.
}
if (!marker?.runId) allow();
// 5. TTL / auto-reset. A marker left behind by a crashed run must not deny
// tool calls for the rest of the machine's life.
const ttlRaw = Number(env.VOYAGE_CAP_SCOPE_TTL_MS);
const ttlMs = Number.isFinite(ttlRaw) && ttlRaw > 0 ? ttlRaw : DEFAULT_TTL_MS;
const startedAt = Date.parse(marker.startedAt ?? '');
if (!Number.isFinite(startedAt) || Date.now() - startedAt > ttlMs) {
try { rmSync(markerPath, { force: true }); } catch { /* best effort */ }
allow();
}
// --- In scope from here on. ---
// 6. The ledger is the only source of truth for turns spent, and it is counted
// through the primitive's OWN readLedger(). This hook used to carry a
// private copy of the counting rule whose read error returned 0 — so an
// unreadable ledger read as "no turns spent" and ALLOWED, in the one branch
// where this hook is supposed to fail closed.
const ledgerPath = resolveLedgerPath(env);
// 7. Same bound the primitive uses: turns-per-dimension × the whole dimension
// list under the maxDimensions: 8 ceiling.
const budget = resolveMaxConvTurns(env) * MAX_TOTAL_DIMENSIONS;
let ledger;
try {
ledger = readLedger(ledgerPath, marker.runId);
} catch (e) {
deny(
` Run ${marker.runId} is in scope, but its turn ledger could not be read:\n` +
` ${e.message}\n` +
` A budget control that cannot count must not grant. Fix or remove the\n` +
` ledger, or set VOYAGE_DISABLE_CAP_HOOK=1 to disable enforcement.`,
);
}
const toolLine =
` Tool: ${input?.tool_name ?? 'unknown'}${input?.agent_type ? ` (agent: ${input.agent_type})` : ''}\n`;
// Every denial names the marker. If this run is over and the marker outlived it
// — the loop's own cleanup covers its three exits, but a crash between the
// exhaustion record and the removal runs no cleanup at all — deleting this file
// is the remedy, and a resumed session (same session_id) would otherwise sit out
// the remaining TTL for work that has nothing to do with research.
const remedyLines =
` If this loop is not running, the marker is debris — delete it:\n` +
` ${markerPath}\n` +
` It also auto-resets ${Math.round(ttlMs / 3600000)}h after the run started (VOYAGE_CAP_SCOPE_TTL_MS).\n`;
// 8. The boundary is the TOMBSTONE, not the count.
//
// allowTurn() appends before the turn runs, so during the final granted turn the
// ledger already holds `budget` records. Denying at `granted >= budget` blocked
// that turn's own tool calls — the primitive granted B turns and this hook
// permitted B-1 — and it forced every exhausted run out through an exit-2 tool
// denial rather than the graceful "cap exhausted" exit, the only exit the prose
// at commands/trekresearch.md teaches the model to handle.
//
// Moving the boundary to `granted > budget` alone would have made this hook
// unable to fire at all once the claim mechanism made a breached ledger
// impossible — a deny branch that cannot be reached is a dead security claim,
// not a backstop. So the primitive records its own denials, and the case this
// hook exists for is the one it now catches: the gate said no and a tool call
// arrived anyway.
if (ledger.exhausted > 0) {
deny(
` Run ${marker.runId} was already denied a turn by the budget gate\n` +
` (${ledger.granted}/${budget} loop turns spent), and this call came after it.\n` +
toolLine +
` Remaining gaps belong in the brief as open questions, not in another turn.\n` +
remedyLines +
` Raise TREKRESEARCH_MAX_CONV_TURNS deliberately, or set VOYAGE_DISABLE_CAP_HOOK=1.`,
);
}
// 9. Backstop for a ledger that exceeded the bound however it managed to.
if (ledger.granted > budget) {
deny(
` Run ${marker.runId} shows ${ledger.granted} granted turns against a budget of ${budget}.\n` +
toolLine +
` The ledger has been breached; the loop is over regardless of cause.\n` +
remedyLines +
` Raise TREKRESEARCH_MAX_CONV_TURNS deliberately, or set VOYAGE_DISABLE_CAP_HOOK=1.`,
);
}
allow();

351
lib/util/research-loop-cap.mjs
View file

@ -1,351 +0,0 @@
// lib/util/research-loop-cap.mjs
// Stateful, default-off cost cap for the /trekresearch bounded conversation
// loop (Phase 4.5 dimension discovery + Phase 5 loop turns).
//
// Three properties the plan review required:
// (a) Default-off — VOYAGE_STORM_ENABLED must be '1'; otherwise the budget
// is 0 regardless of effort. This IS the decline branch: doing nothing
// leaves the mechanism off, and adopt is flipping this one constant.
// (b) The cap counts itself — allowTurn() derives used-turn count from an
// append-only JSONL ledger, never from a caller-supplied number. A cap
// that asks the caller how many turns it has used is not a cap. Each
// grant additionally claims a turn SLOT with O_EXCL, so the bound holds
// when several callers decide at once instead of only when they queue.
// (c) Correct size bound — worst case is max_conv_turns × max_total_dimensions,
// where max_total_dimensions is the WHOLE list (interview + discovered)
// under the maxDimensions cap of 8 — not × discovered-only.
//
// CLAUDE_PLUGIN_DATA absent => fall back to ~/.claude/voyage. The variable is
// EMPTY in the Bash tool's process env (measured in a live plugin-enabled
// session), and the Phase 5 bash snippet is this module's only caller — so
// denying on its absence denied turn 1 of every real run. The root is resolved
// in code rather than demanded of the environment, and hooks/scripts/
// pre-agent-cap.mjs resolves it through the SAME function, so the writer and
// the reader can never disagree about where the ledger lives.
//
// The fail-closed stance covers both directions of ledger IO: a ledger that
// cannot be WRITTEN denies the turn, and a ledger that exists but cannot be
// READ denies it too. Only ENOENT counts as zero turns spent, because that is
// the legitimate first-turn state. This module is a budget control, not
// telemetry.
//
// CLI shim, two modes:
// node lib/util/research-loop-cap.mjs --run-id ID --dimension D --effort E
// → JSON: { ok, used, budget, reason? } (exit 0 = granted, exit 1 = denied)
//
// node lib/util/research-loop-cap.mjs --check-dimensions N
// → JSON: { ok, count, ceiling, reason? } (exit 0 = within, exit 1 = rejected)
// The second axis of the bounded-cost NFR. Phase 4.5 never calls the budget
// gate, so this mode requires no run id, effort or STORM flag — but it reads
// the SAME MAX_TOTAL_DIMENSIONS the budget is sized against.
import { existsSync, mkdirSync, appendFileSync, readFileSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { homedir } from 'node:os';
import { isMain } from './is-main.mjs';
export const MAX_CONV_TURNS = 3;
export const MAX_TOTAL_DIMENSIONS = 8; // maxDimensions — whole list, not discovered-only
const LEDGER_FILENAME = 'trekresearch-loop-ledger.jsonl';
const CLAIM_DIRNAME = 'trekresearch-loop-claims';
export function isStormEnabled(env = process.env) {
return env.VOYAGE_STORM_ENABLED === '1';
}
/**
* The OTHER cost ceiling: how large the whole dimension list may get after
* Phase 4.5 discovery has appended to it.
*
* The bounded-cost NFR asks for explicit ceilings on both axes. The turn axis
* had a constant, a ledger-backed reader and a PreToolUse enforcer; the
* discovery axis had only a sentence in Phase 4.5 prose — a cap nothing reads,
* which is the failure mode the operator decision on brief_reviewer_iter_cap
* warned about. This is the reader.
*
* The ceiling is MAX_TOTAL_DIMENSIONS on purpose: the value that sizes the turn
* budget IS maxDimensions, and a second constant for the same
* number is how two readers end up enforcing different bounds.
*
* Accepts the dimension list or its count, because Phase 4.5 has the list and
* the CLI has a number. A count that cannot be read is REJECTED — a cost ceiling
* that waves through what it cannot measure is not a ceiling.
*
* @param {string[]|number|string} dimensions
* @param {{ceiling?: number}} [opts]
* @returns {{ok: boolean, count: number|null, ceiling: number, reason?: string}}
*/
export function checkDimensionCeiling(dimensions, opts = {}) {
const ceiling = Number.isFinite(opts.ceiling) ? opts.ceiling : MAX_TOTAL_DIMENSIONS;
const count = Array.isArray(dimensions) ? dimensions.length : Number(dimensions);
if (dimensions === null || dimensions === undefined || !Number.isInteger(count) || count < 0) {
return { ok: false, count: null, ceiling, reason: 'unreadable_dimension_count' };
}
if (count > ceiling) {
return { ok: false, count, ceiling, reason: 'ceiling_exceeded' };
}
return { ok: true, count, ceiling };
}
/**
* Coerce TREKRESEARCH_MAX_CONV_TURNS. NaN, empty, negative, zero, Infinity, or
* any fraction that floors below 1 all fall back to MAX_CONV_TURNS — never to
* unbounded, and never to 0.
*
* The floor is applied BEFORE the `<= 0` guard, not after. Flooring afterwards
* let '0.5' and '0.9' clear a guard written against the raw value and then
* become 0, making the budget 0 × MAX_TOTAL_DIMENSIONS = 0: every turn denied
* and the loop silently dead rather than bounded. A cap of 0 is not a narrower
* cap, it is an off switch that the documented fallback promises not to be.
*/
export function resolveMaxConvTurns(env = process.env) {
const raw = env.TREKRESEARCH_MAX_CONV_TURNS;
if (raw === undefined || raw === null || raw === '') return MAX_CONV_TURNS;
const n = Math.floor(Number(raw));
if (!Number.isFinite(n) || n <= 0) return MAX_CONV_TURNS;
return n;
}
/**
* The one data root for everything this loop writes: the turn ledger and the
* PreToolUse scope marker. CLAUDE_PLUGIN_DATA when the harness provides it,
* ~/.claude/voyage when it does not — which is the case in every Bash tool
* invocation today.
*/
export function resolveDataRoot(env = process.env) {
const dir = env.CLAUDE_PLUGIN_DATA;
if (dir && typeof dir === 'string' && dir.length > 0) return dir;
const home = env.HOME && env.HOME.length > 0 ? env.HOME : homedir();
return join(home, '.claude', 'voyage');
}
export function resolveLedgerPath(env = process.env) {
return join(resolveDataRoot(env), LEDGER_FILENAME);
}
/**
* Where the per-turn claim files live. A claim is the ATOMIC record that a turn
* slot is taken; the ledger is the readable record of what that turn was for.
*
* The claim exists because the ledger alone cannot bound the loop. Counting the
* ledger and then appending is read-then-write: N callers that all observe
* `used == budget - 1` all decide to grant, and the bound is exceeded by N-1 —
* precisely the concurrent case (several agents spawned in one message) that
* allowTurn's own comment named as the reason it had to be append-only.
*
* Claim files are empty, at most `budget` per run, and never cleaned up — the
* same standing as the ledger itself, which also grows for the life of the data
* root. Two consequences worth stating rather than discovering: reusing a
* runId across runs finds its slots already taken and denies, and two runIds
* that collide after filename sanitisation block each other. Both err toward
* denying a turn, which is the safe direction for a budget control.
*/
export function resolveClaimDir(env = process.env) {
return join(resolveDataRoot(env), CLAIM_DIRNAME);
}
function claimFileName(runId, slot) {
return `${String(runId).replace(/[^A-Za-z0-9._-]/g, '_')}-${slot}.claim`;
}
/**
* Try to take turn slot `slot` for `runId`. `wx` is O_CREAT|O_EXCL: the kernel
* decides the winner, so exactly one caller can ever create a given slot file.
*
* @returns {boolean} true when this caller took the slot, false when it was already taken
* @throws on any IO error other than EEXIST — the caller turns that into a denial
*/
function claimSlot(claimDir, runId, slot) {
try {
writeFileSync(join(claimDir, claimFileName(runId, slot)), '', { flag: 'wx' });
return true;
} catch (e) {
if (e && e.code === 'EEXIST') return false;
throw e;
}
}
/**
* Read one run's turn count off the append-only ledger.
*
* ENOENT is 0 turns spent — the legitimate first-turn state, and the reason
* this cannot simply throw on every read failure. Every OTHER read error
* (EISDIR, EACCES, EIO) THROWS, because returning 0 from an unreadable ledger
* re-granted the full budget on every call: unbounded, and the exact
* silently-grant-unlimited failure this module's header argues against three
* lines above the code that did it. The missing-directory case already failed
* closed; this makes the unreadable-file case agree with it.
*
* The `existsSync` pre-check is deliberately gone: readFileSync's own ENOENT
* carries the same information without a second syscall that can disagree with
* the read that follows it.
*
* Exported so hooks/scripts/pre-agent-cap.mjs counts through this exact
* function. A reader and a writer with private copies of the counting rule are
* how a hook ends up enforcing a different bound than the gate it backs.
*
* Two counts, deliberately separate. `granted` is turns handed out. `exhausted`
* is tombstones — records this gate wrote when it DENIED a turn. A tombstone is
* not a turn and must never consume budget; it exists so the PreToolUse hook can
* tell "turn B is in flight" (granted == budget, no tombstone) apart from "the
* gate already said no and something kept going" (tombstone present).
*
* @param {string} ledgerPath
* @param {string} runId
* @returns {{granted: number, exhausted: number}}
* @throws when the ledger exists but cannot be read
*/
export function readLedger(ledgerPath, runId) {
let text;
try {
text = readFileSync(ledgerPath, 'utf-8');
} catch (e) {
if (e && e.code === 'ENOENT') return { granted: 0, exhausted: 0 };
const err = new Error(`ledger unreadable at ${ledgerPath}: ${e.message}`);
err.code = 'VOYAGE_LEDGER_UNREADABLE';
throw err;
}
let granted = 0;
let exhausted = 0;
for (const line of text.split('\n')) {
if (!line) continue;
try {
const rec = JSON.parse(line);
if (rec.runId !== runId) continue;
if (rec.exhausted === true) exhausted++;
else granted++;
} catch { /* skip malformed lines */ }
}
return { granted, exhausted };
}
/**
* Record that this run has been denied a turn for budget.
*
* Best effort on purpose: the denial itself is already the correct answer, so a
* ledger that cannot take the tombstone must not turn a denial into a grant. The
* tombstone only strengthens the harness-level backstop.
*/
function markExhausted(ledgerPath, runId, now) {
try {
appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, exhausted: true }) + '\n');
} catch { /* best effort — see above */ }
}
/**
* Decide whether one more research-loop turn may run.
*
* Phase 4.5/5 may spawn several agents in a single message, so the decision has
* to survive concurrent callers. It does that by CLAIMING a turn slot with
* O_EXCL (see claimSlot) and only then appending to the ledger. The comment
* that used to sit here asserted "append-only: never read-modify-write" as if
* appending were itself the concurrency guarantee — but the decision path was
* count-then-append, which is read-then-write, so the claim was unsupported by
* the code beneath it. The kernel now picks the winner for each slot.
*
* @param {{runId: string, dimension: string, effort: string}} args
* @param {{env?: object, now?: Date}} [opts]
* @returns {{ok: boolean, used: number, budget: number, reason?: string}}
*/
export function allowTurn({ runId, dimension, effort } = {}, opts = {}) {
const env = opts.env || process.env;
const now = opts.now || new Date();
if (!isStormEnabled(env)) {
return { ok: false, used: 0, budget: 0, reason: 'storm_disabled' };
}
if (effort !== 'high') {
return { ok: false, used: 0, budget: 0, reason: 'effort_not_high' };
}
if (!runId || !dimension) {
return { ok: false, used: 0, budget: 0, reason: 'missing_args' };
}
const maxConvTurns = resolveMaxConvTurns(env);
const budget = maxConvTurns * MAX_TOTAL_DIMENSIONS;
const ledgerPath = resolveLedgerPath(env);
let ledger;
try {
ledger = readLedger(ledgerPath, runId);
} catch (e) {
return { ok: false, used: 0, budget, reason: `ledger-read-failed: ${e.message}` };
}
const used = ledger.granted;
// Already tombstoned: this run is over. Short-circuit so a hammered gate
// neither walks every slot again nor appends a second tombstone.
if (ledger.exhausted > 0) {
return { ok: false, used, budget, reason: 'budget_exhausted' };
}
// Claim a turn SLOT before spending anything. The ledger count only says
// where to start looking; the claim is what makes the grant exclusive. Slot
// numbers are bounded by `budget`, and each can be created exactly once, so
// the total number of grants for a run can never exceed the budget however
// many callers arrive at once.
const claimDir = resolveClaimDir(env);
let slot = used + 1;
let claimed = false;
try {
mkdirSync(claimDir, { recursive: true });
while (slot <= budget) {
if (claimSlot(claimDir, runId, slot)) { claimed = true; break; }
slot++;
}
} catch (e) {
return { ok: false, used, budget, reason: `claim-failed: ${e.message}` };
}
if (!claimed) {
markExhausted(ledgerPath, runId, now);
return { ok: false, used, budget, reason: 'budget_exhausted' };
}
try {
const dir = dirname(ledgerPath);
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, dimension, effort, slot }) + '\n');
} catch (e) {
return { ok: false, used, budget, reason: `ledger-write-failed: ${e.message}` };
}
return { ok: true, used: slot, budget };
}
// ---- CLI shim ----------------------------------------------------------------
function parseArgs(argv) {
const out = {};
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === '--run-id') out.runId = argv[++i];
else if (a === '--dimension') out.dimension = argv[++i];
else if (a === '--effort') out.effort = argv[++i];
else if (a === '--check-dimensions') out.checkDimensions = argv[++i];
}
return out;
}
if (isMain(import.meta.url)) {
const args = parseArgs(process.argv.slice(2));
// The dimension ceiling is a Phase 4.5 concern, and Phase 4.5 never calls the
// budget gate — so this branch must not inherit the gate's preconditions
// (run id, effort, STORM flag). It is a pure bound on list size.
if (args.checkDimensions !== undefined) {
const result = checkDimensionCeiling(args.checkDimensions);
process.stdout.write(JSON.stringify(result) + '\n');
process.exit(result.ok ? 0 : 1);
}
if (!args.runId || !args.dimension || !args.effort) {
process.stdout.write(JSON.stringify({
ok: false,
reason: 'usage: research-loop-cap.mjs --run-id ID --dimension D --effort standard|high|low',
}) + '\n');
process.exit(1);
}
const result = allowTurn(args);
process.stdout.write(JSON.stringify(result) + '\n');
process.exit(result.ok ? 0 : 1);
}

6
lib/validators/query-privacy-gate.mjs
View file

@ -1,8 +1,6 @@
// lib/validators/query-privacy-gate.mjs
// Inspect an outbound research query before it leaves the machine. Called
// only from the new high-effort steps (Phase 4.5 dimension discovery + the
// bounded Phase 5 loop turns) — the existing single-pass Phase 5 path is
// unchanged (Step 6, plan-v2).
// Inspect an outbound research query before it leaves the machine. No
// command calls it now: its callers were the removed STORM steps.
//
// Two-tier:
// - WARN tier — absolute filesystem paths, repo-internal identifiers.

343
scripts/storm-measure.mjs
View file

@ -1,343 +0,0 @@
#!/usr/bin/env node
// scripts/storm-measure.mjs
// Step 11 — the STORM adoption gate: deterministic Δ accounting over
// ${CLAUDE_PLUGIN_DATA}/trekresearch-stats.jsonl.
//
// What this decides: whether the bounded Phase 5 conversation loop buys enough
// extra source/coverage breadth to justify flipping VOYAGE_STORM_ENABLED on by
// default in lib/util/research-loop-cap.mjs. Nothing else. The thresholds live
// in docs/storm-measurement.md and were committed BEFORE any measurement run —
// that pre-registration is the whole point, so this script never invents them.
//
// What this does NOT measure: outline quality, answer correctness, or operator
// satisfaction. It measures breadth (distinct sources, dimensions covered).
// A breadth win is necessary for adoption, not sufficient on its own.
//
// Honesty properties, both load-bearing:
// - Runs with empty_turns > 0 are EXCLUDED from the gain and REPORTED. An
// empty turn means the loop spent budget and returned nothing; leaving those
// in decides adoption on a broken denominator.
// - A stats file carrying no `effort` field is a loud error. The silent
// failure this prevents is an empty treatment group reading as "no gain",
// which would decline the mechanism for a schema reason.
//
// Zero deps. Node stdlib only.
import { readFileSync, existsSync } from 'node:fs';
import { join } from 'node:path';
import { isMain } from '../lib/util/is-main.mjs';
// Pre-registered in docs/storm-measurement.md. Do not tune to fit a result.
export const ADOPT_THRESHOLD = 0.30;
export const DECLINE_THRESHOLD = 0.15;
const STATS_FILENAME = 'trekresearch-stats.jsonl';
// ---- pure core (unit-tested) -------------------------------------------------
/** @param {number[]} xs @returns {number|null} null for an empty list — 0 would read as a measurement. */
export function median(xs) {
if (!Array.isArray(xs) || xs.length === 0) return null;
const s = [...xs].sort((a, b) => a - b);
const mid = s.length >> 1;
return s.length % 2 ? s[mid] : (s[mid - 1] + s[mid]) / 2;
}
/**
* Parse a trekresearch-stats.jsonl body into effort-carrying records.
*
* Rows predating Step 9 have no `effort` field; they are dropped and counted as
* `legacy` rather than silently pooled into the control arm. A file where NO row
* carries `effort` throws — see the header note on silent failure.
*
* @param {string} text
* @returns {{records: object[], malformed: number, legacy: number}}
*/
export function parseStats(text) {
const lines = String(text ?? '').split('\n');
const records = [];
let malformed = 0;
let legacy = 0;
let parsed = 0;
for (const line of lines) {
if (!line.trim()) continue;
let rec;
try {
rec = JSON.parse(line);
} catch {
malformed++;
continue;
}
parsed++;
if (typeof rec?.effort !== 'string' || rec.effort.length === 0) {
legacy++;
continue;
}
records.push(rec);
}
if (parsed === 0) {
throw new Error(`storm-measure: no records in stats file (${malformed} malformed line(s)).`);
}
if (records.length === 0) {
throw new Error(
`storm-measure: no records carry an \`effort\` field (${legacy} legacy row(s)). ` +
`The gate groups on \`effort\`; without it there is no treatment arm to measure. ` +
`Re-run the measurement set on a build that emits the Step 9 fields.`,
);
}
return { records, malformed, legacy };
}
/**
* Split off the runs that must not count toward a gain.
*
* A value that cannot be READ as a turn count is excluded, not treated as zero.
* `Number('many')` is NaN, and testing `Number.isFinite(empty) && empty > 0`
* sent NaN down the eligible branch — so a garbage field silently re-entered the
* denominator, in the direction that flatters adoption: the run whose bookkeeping
* broke is the run whose numbers deserve the least trust. Absent and null stay
* eligible via `?? 0`, because a field that was never written is a genuine zero
* on any run where the loop did not arm.
*
* @param {object[]} records
* @returns {{eligible: object[], excluded: number}}
*/
export function partitionEligible(records) {
const eligible = [];
let excluded = 0;
for (const r of records) {
const empty = Number(r.empty_turns ?? 0);
if (!Number.isFinite(empty) || empty > 0) excluded++;
else eligible.push(r);
}
return { eligible, excluded };
}
function numbers(records, pick) {
return records.map(pick).filter((n) => Number.isFinite(n));
}
/** Relative gain (treatment − control)/control. null when control is absent or zero. */
function relGain(control, treatment) {
if (control === null || treatment === null || control === 0) return null;
return (treatment - control) / control;
}
/**
* Full measurement over a parsed record set.
*
* Treatment arm = `effort: high` (the only effort at which the loop runs).
* Control arm = every other effort.
*
* - sources: between-arm median gain in `unique_sources`.
* - dimensions: within-run median gain of (dimensions − dimensions_baseline)
* / dimensions_baseline across the treatment arm. It is a
* within-run delta by construction, so it needs no control arm —
* the control arm's is 0, the loop being inert there.
*
* @param {object[]} records
*/
export function measure(records) {
const { eligible, excluded } = partitionEligible(records);
const treatment = eligible.filter((r) => r.effort === 'high');
const control = eligible.filter((r) => r.effort !== 'high');
const srcControl = median(numbers(control, (r) => Number(r.unique_sources)));
const srcTreatment = median(numbers(treatment, (r) => Number(r.unique_sources)));
const sourcesGain = relGain(srcControl, srcTreatment);
const dimDeltas = treatment
.map((r) => ({ d: Number(r.dimensions), b: Number(r.dimensions_baseline) }))
.filter(({ d, b }) => Number.isFinite(d) && Number.isFinite(b) && b > 0)
.map(({ d, b }) => (d - b) / b);
const dimensionsGain = median(dimDeltas);
return {
control: { n: control.length, sources: srcControl },
treatment: { n: treatment.length, sources: srcTreatment },
sources: { control: srcControl, treatment: srcTreatment, gain: sourcesGain },
dimensions: { gain: dimensionsGain, n: dimDeltas.length },
excluded,
verdict: decideVerdict(sourcesGain, dimensionsGain),
};
}
/**
* Pre-registered mapping, verbatim from the brief: "median forbedring >= 30 %
* på (a) eller (b) → adopt. < 15 % → decline." OR on both sides, adopt
* evaluated first — so a strong win on one axis is an adopt even when the
* other axis sits under the decline bar. A stricter AND rule may well be the
* better decision procedure, but changing it here is changing the
* pre-registration after the fact, which is the one thing the constraint
* exists to prevent.
*
* @param {number|null} sourcesGain
* @param {number|null} dimensionsGain
* @returns {'adopt'|'decline'|'inconclusive'|'insufficient-data'}
*/
export function decideVerdict(sourcesGain, dimensionsGain) {
if (sourcesGain === null || sourcesGain === undefined) return 'insufficient-data';
if (dimensionsGain === null || dimensionsGain === undefined) return 'insufficient-data';
if (sourcesGain >= ADOPT_THRESHOLD || dimensionsGain >= ADOPT_THRESHOLD) return 'adopt';
if (sourcesGain < DECLINE_THRESHOLD || dimensionsGain < DECLINE_THRESHOLD) return 'decline';
return 'inconclusive';
}
/**
* SC activation check, BOTH halves.
*
* The SC asks two things of an `effort: high` run: that it discovered at least
* one dimension, AND that the dimension list in the output brief is a TRUE
* SUPERSET of the interview-derived ones. This function used to check only
* `dimensions - dimensions_baseline >= 1`, which is a count delta and says
* nothing about membership — a run that dropped two interview dimensions and
* added three discovered ones passed while violating the second half.
* Supersetness was asserted only by Phase 4.5's prose contract that discovery
* APPENDS; nothing read it.
*
* The record cannot carry the dimension names: names are free prose, and
* the record denies prose by omission. So the run attests
* membership with `dimensions_baseline_preserved`, a low-cardinality boolean set
* in Phase 4.5, and this gate refuses to call activation OK without it. An
* ABSENT attestation is not an attestation — legacy rows fail here rather than
* passing on the old count-only rule.
*
* @param {object[]} records
*/
export function activationCheck(records) {
const high = records.filter((r) => r.effort === 'high');
if (high.length === 0) {
return { ok: false, reason: 'no `effort: high` run found in stats', discovered_dimensions: null };
}
const last = high[high.length - 1];
const d = Number(last.dimensions);
const b = Number(last.dimensions_baseline);
if (!Number.isFinite(d) || !Number.isFinite(b)) {
return { ok: false, reason: 'latest high run lacks dimensions/dimensions_baseline', discovered_dimensions: null };
}
const discovered = d - b;
const preserved = last.dimensions_baseline_preserved;
const base = {
ts: last.ts ?? null,
dimensions: d,
dimensions_baseline: b,
discovered_dimensions: discovered,
dimensions_baseline_preserved: preserved ?? null,
conv_turns: Number(last.conv_turns ?? 0),
empty_turns: Number(last.empty_turns ?? 0),
};
if (typeof preserved !== 'boolean') {
return {
...base,
ok: false,
reason:
'latest high run does not attest `dimensions_baseline_preserved`; the SC needs a true ' +
'superset of the interview dimensions, and a count delta cannot show membership',
};
}
if (preserved === false) {
return {
...base,
ok: false,
reason:
`latest high run discovered ${discovered} dimension(s) but did NOT preserve its interview ` +
'baseline, so the final list is not a superset of it',
};
}
if (discovered < 1) {
return { ...base, ok: false, reason: 'latest high run discovered no dimensions beyond its baseline' };
}
return { ...base, ok: true };
}
// ---- CLI shim ----------------------------------------------------------------
function pct(x) {
return x === null ? 'n/a' : `${(x * 100).toFixed(1)}%`;
}
function defaultStatsPath(env = process.env) {
const dir = env.CLAUDE_PLUGIN_DATA;
return dir ? join(dir, STATS_FILENAME) : null;
}
function parseArgs(argv) {
const o = { stats: null, json: false, activation: false, help: false };
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === '--stats') o.stats = argv[++i];
else if (a === '--json') o.json = true;
else if (a === '--activation-check') o.activation = true;
else if (a === '--help' || a === '-h') o.help = true;
else { process.stderr.write(`Unknown argument: ${a}\n`); process.exit(2); }
}
return o;
}
function mainCli() {
const o = parseArgs(process.argv.slice(2));
if (o.help) {
process.stdout.write(
'Usage: storm-measure.mjs [--stats FILE] [--activation-check] [--json]\n' +
' Default --stats: ${CLAUDE_PLUGIN_DATA}/' + STATS_FILENAME + '\n' +
' Thresholds (pre-registered, docs/storm-measurement.md): ' +
`adopt >= ${ADOPT_THRESHOLD * 100}%, decline < ${DECLINE_THRESHOLD * 100}%\n`,
);
process.exit(0);
}
const statsPath = o.stats || defaultStatsPath();
if (!statsPath) {
process.stderr.write('storm-measure: CLAUDE_PLUGIN_DATA is not set and no --stats FILE was given.\n');
process.exit(2);
}
if (!existsSync(statsPath)) {
process.stderr.write(`storm-measure: stats file not found: ${statsPath}\n`);
process.exit(2);
}
let parsed;
try {
parsed = parseStats(readFileSync(statsPath, 'utf-8'));
} catch (e) {
process.stderr.write(`${e.message}\n`);
process.exit(2);
}
if (o.activation) {
const res = activationCheck(parsed.records);
process.stdout.write(JSON.stringify(res, null, 2) + '\n');
process.exit(res.ok ? 0 : 1);
}
const m = measure(parsed.records);
if (o.json) {
process.stdout.write(JSON.stringify({ statsPath, ...m, malformed: parsed.malformed, legacy: parsed.legacy }, null, 2) + '\n');
process.exit(0);
}
const L = [];
L.push(`STORM adoption gate — ${statsPath}`);
L.push(` control (effort != high): n=${m.control.n} median unique_sources=${m.control.sources ?? 'n/a'}`);
L.push(` treatment (effort = high): n=${m.treatment.n} median unique_sources=${m.treatment.sources ?? 'n/a'}`);
L.push(` excluded (empty_turns > 0): ${m.excluded}`);
if (parsed.legacy) L.push(` legacy rows without \`effort\`: ${parsed.legacy}`);
if (parsed.malformed) L.push(` malformed lines: ${parsed.malformed}`);
L.push('');
L.push(` median gain, unique_sources: ${pct(m.sources.gain)}`);
L.push(` median gain, dimensions over baseline: ${pct(m.dimensions.gain)} (n=${m.dimensions.n})`);
L.push('');
L.push(` thresholds: adopt >= ${pct(ADOPT_THRESHOLD)} on EITHER · decline < ${pct(DECLINE_THRESHOLD)} on EITHER · adopt wins ties`);
L.push(` VERDICT: ${m.verdict}`);
process.stdout.write(L.join('\n') + '\n');
process.exit(0);
}
if (isMain(import.meta.url)) {
mainCli();
}

6
skills/researching-sources/SKILL.md
View file

@ -133,10 +133,8 @@ usable source is an open question, not a rating.
research brief: required frontmatter, `confidence` a number in [0, 1],
the required sections. It does not read sources, dates, versions or
queries.
- `lib/validators/query-privacy-gate.mjs` blocks secret-shaped strings and
warns on local paths in a query, but only in the `/trekresearch` steps
that are off by default (dimension discovery and the bounded follow-up
loop). Ordinary searches do not pass through it.
- `lib/validators/query-privacy-gate.mjs` is not called by any command
since its STORM steps were removed, so no search passes through it.
- Everything else in this skill is **prose only**. No check stops a missing
date, a wrong version, an absence reported as a finding, an injected
instruction that was followed, or a local path in an ordinary search.

35
tests/commands/trekresearch-engine.test.mjs
View file

@ -1,35 +0,0 @@
// tests/commands/trekresearch-engine.test.mjs
// Step 1 (deep-research-engine): pin the contract the `--engine deep-research`
// adapter must hit. The adapted in-context `/deep-research` report, reduced into
// the research-brief schema, must pass research-validator under the strict
// default; and a brief missing a required section must fail. This is the one
// genuinely automatable slice of SC2 (schema, not provenance).
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { validateResearchContent } from '../../lib/validators/research-validator.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const FIXTURE = join(ROOT, 'tests', 'fixtures', 'research-deep-research-adapted.md');
test('deep-research adapter output contract — valid brief passes, missing section fails', () => {
const text = readFileSync(FIXTURE, 'utf-8');
// (a) positive: the adapter's target output passes the validator (default = strict).
const okResult = validateResearchContent(text);
assert.equal(okResult.valid, true, JSON.stringify(okResult.errors));
// (b) negative: stripping a required section makes it fail with RESEARCH_MISSING_SECTION,
// giving the contract teeth (a fixture that always passes proves nothing).
const mutated = text.replace('## Dimensions', '## Removed');
const badResult = validateResearchContent(mutated);
assert.equal(badResult.valid, false);
assert.ok(
badResult.errors.find(e => e.code === 'RESEARCH_MISSING_SECTION'),
'expected RESEARCH_MISSING_SECTION; got ' + JSON.stringify(badResult.errors),
);
});

199
tests/commands/trekresearch.test.mjs
View file

@ -54,7 +54,7 @@ test('trekresearch — low-effort path references --quick equivalent', () => {
'heading-bounded slice must still fail on a genuine removal');
});
// --- Step 7: Phase 5 bounded conversation loop (heading-bounded slices) ---
// --- Phase 5 slice (heading-bounded) ---
// Same bounding discipline as the Composition-rule pin above: slice from the
// phase heading to the NEXT phase heading, never a fixed character window.
@ -70,142 +70,6 @@ function phase5(doc) {
return phaseSlice(doc, '## Phase 5 —', '## Phase 6 —');
}
test('trekresearch — Phase 5 loop is gated on effort == high and names both primitives', () => {
const p5 = phase5(read());
assert.match(p5, /effort == 'high'/, 'Phase 5 loop must be gated on effort == \'high\'');
assert.match(p5, /research-loop-cap\.mjs/, 'Phase 5 must call the loop-cap shim per turn');
assert.match(p5, /query-privacy-gate\.mjs/, 'Phase 5 must route outbound queries through the privacy gate');
assert.match(p5, /\$\{CLAUDE_PLUGIN_ROOT\}/, 'shim invocations must use the ${CLAUDE_PLUGIN_ROOT} path form');
});
// CLAUDE_PLUGIN_DATA and CLAUDE_PLUGIN_ROOT are substituted in this command's
// TEXT but are EMPTY in the Bash tool's process env. Every snippet below runs
// in that env, so each needs a resolution that does not depend on it.
test('trekresearch — the scope-marker snippets resolve a root instead of requiring CLAUDE_PLUGIN_DATA', () => {
const p5 = phase5(read());
const blocks = [...p5.matchAll(/```bash\n([\s\S]*?)```/g)].map((m) => m[1]);
const write = blocks.find((b) => b.includes('trekresearch-loop-scope') && b.includes('printf'));
const remove = blocks.find((b) => b.includes('trekresearch-loop-scope') && b.includes('rm -f'));
assert.ok(write, 'Phase 5 must carry the scope-marker write snippet');
assert.ok(remove, 'Phase 5 must carry the scope-marker removal snippet');
for (const [name, block] of [['write', write], ['remove', remove]]) {
assert.match(
block,
/\$\{CLAUDE_PLUGIN_DATA:-\$HOME\/\.claude\/voyage\}/,
`the ${name} snippet must fall back to the same root research-loop-cap.mjs resolves`,
);
assert.match(
block,
/case .* in\s*\n?\s*\/\*\)/,
`the ${name} snippet must guard on ONE absolute-path test — write and remove must not disagree on what counts as usable`,
);
}
// Unset, ${CLAUDE_CODE_SESSION_ID} composes a marker named `.json`, which no
// hook lookup and no TTL sweep ever matches or cleans up.
assert.match(
write,
/-n "\$\{?CLAUDE_CODE_SESSION_ID/,
'the write snippet must require a non-empty CLAUDE_CODE_SESSION_ID before composing the marker path',
);
});
test('trekresearch — the per-turn gates separate "gate could not run" from "gate says no"', () => {
const p5 = phase5(read());
assert.match(
p5,
/VOYAGE_ROOT/,
'the gate snippet must resolve a plugin root rather than interpolating ${CLAUDE_PLUGIN_ROOT} straight into `node`',
);
assert.match(
p5,
/exit 2|could not run/i,
'an unresolvable gate must be distinguishable from a denial — otherwise every query reads as a privacy violation no rewrite can clear',
);
assert.match(
p5,
/plugins\/cache/,
'the fallback must name the plugin cache location it searches',
);
});
test('trekresearch — Phase 5 declares the loop bound and all three exits', () => {
const p5 = phase5(read());
assert.match(p5, /### Loop bound/, 'Phase 5 must carry a `### Loop bound` sub-heading');
assert.match(
p5,
/\*\*Maximum 3 turns per under-illuminated dimension\.\*\*/,
'the bound must be stated verbatim',
);
// Three exits — converged / cap exhausted / operator stop.
assert.match(p5, /converged/i, 'exit 1 (converged) must be documented');
assert.match(p5, /exhaust/i, 'exit 2 (cap exhausted) must be documented');
assert.match(p5, /operator stop/i, 'exit 3 (operator stop) must be documented');
// Exhaustion must reach the operator — a silent cap is indistinguishable
// from convergence, which is the failure this loop exists to avoid.
assert.match(
p5,
/visibl|visible|print/i,
'cap exhaustion must be written visibly to the operator',
);
});
test('trekresearch — Phase 5 marks empty turns without re-targeting the same dimension', () => {
const p5 = phase5(read());
assert.match(p5, /`empty`/, 'a finding-less or citation-less turn must be marked `empty`');
assert.match(p5, /empty_turns/, 'empty turns must be counted (empty_turns)');
assert.match(
p5,
/does NOT re-target|not re-target/i,
'an empty turn must not re-target the same dimension',
);
});
test('trekresearch — Phase 5 states the no-brief default and the moot precedence matrix', () => {
const p5 = phase5(read());
// (a) no-brief default
assert.match(p5, /effort = 'standard'/, 'no-brief default effort must be stated');
assert.match(
p5,
/--project/,
'the no-brief default must be anchored to the absence of --project/brief.md',
);
// (b) precedence matrix — each entry independently makes the loop moot,
// mirroring the --engine moot gate in Phase 4.
for (const token of ['--quick', '--local', 'external_research_enabled']) {
assert.ok(p5.includes(token), `moot matrix must name ${token}`);
}
assert.match(p5, /moot/i, 'the matrix must use the same moot vocabulary as the engine gate');
// (c) interaction rule — effort: high without model under a cheap profile.
assert.match(
p5,
/effort: high/,
'the interaction rule for a brief carrying effort: high without model must be stated',
);
});
test('trekresearch — Phase 5 restates the honesty rule for loop output', () => {
const p5 = phase5(read());
// Whitespace-tolerant: the pin is on the sentence, not on where the
// paragraph happens to wrap.
assert.match(
p5,
/more\s+turns\s+do\s+not\s+make\s+a\s+finding\s+more\s+credible/i,
'the honesty hard rule must be restated for the loop output',
);
});
test('trekresearch — Phase 5 pins survive only while the prose does (mutation control)', () => {
const text = read();
const mutated = text.replace(/research-loop-cap\.mjs/g, 'removed-cap.mjs');
assert.doesNotMatch(
phase5(mutated),
/research-loop-cap\.mjs/,
'heading-bounded Phase 5 slice must still fail on a genuine removal',
);
});
test('trekresearch — High-effort behavior gives standard effort the wave-2 sentence and keeps the low effort sentence verbatim', () => {
const text = read();
const flatText = text.replace(/\s+/g, ' ');
@ -223,64 +87,6 @@ test('trekresearch — High-effort behavior gives standard effort the wave-2 sen
);
});
// --- Step 8: Phase 4.5 dimension discovery + Independence amendment ---
test('trekresearch — Phase 4.5 exists between Phase 4 and Phase 5 with the effort skip-guard', () => {
const text = read();
const p45 = text.indexOf('## Phase 4.5 —');
assert.ok(p45 >= 0, 'Phase 4.5 heading missing');
const p4 = text.indexOf('## Phase 4 —');
const p5 = text.indexOf('## Phase 5 —');
assert.ok(p4 >= 0 && p5 > p45 && p45 > p4, 'Phase 4.5 must sit between Phase 4 and Phase 5');
const slice = text.slice(p45, p5);
assert.match(
slice,
/\*\*Skip this phase entirely unless `phase_signal_result\.effort == 'high'`/,
'Phase 4.5 must carry the bolded skip-guard in the Phase 3.5 form',
);
assert.match(slice, /query-privacy-gate\.mjs/,
'Phase 4.5 must name the privacy gate as its compensating control');
assert.match(slice, /maxDimensions: 8|maxDimensions` *: *8/,
'Phase 4.5 must augment under the existing maxDimensions ceiling, not raise it');
});
// Phase 4.5 never invokes research-loop-cap.mjs, so the cap's own flag check
// does not reach it. Gating on effort alone means unsetting VOYAGE_STORM_ENABLED
// leaves half the mechanism live and the decline branch unreachable — while
// CLAUDE.md claims both phases go inert. The guard has to name both conditions.
test('trekresearch — Phase 4.5 skip-guard is gated on VOYAGE_STORM_ENABLED as well as effort', () => {
const text = read();
const p45 = text.indexOf('## Phase 4.5 —');
const p5 = text.indexOf('## Phase 5 —');
const slice = text.slice(p45, p5);
const guard = slice.slice(0, slice.indexOf('\n\n', slice.indexOf('**Skip this phase')));
assert.match(guard, /VOYAGE_STORM_ENABLED/,
'the Phase 4.5 skip-guard must name VOYAGE_STORM_ENABLED, not effort alone');
assert.match(guard, /\bAND\b|\*\*and\*\*/,
'the guard must be a conjunction — both conditions, not either');
assert.match(slice, /decline/i,
'Phase 4.5 must say why the flag gates it: the decline branch has to stay reachable');
});
test('trekresearch — Independence hard rule carries an explicit Phase 4.5 amendment', () => {
const text = read();
const rulesIdx = text.indexOf('## Hard rules');
assert.ok(rulesIdx >= 0, 'Hard rules section missing');
const rules = text.slice(rulesIdx);
const indIdx = rules.indexOf('**Independence:**');
assert.ok(indIdx >= 0, 'Independence hard rule missing');
// Bound the rule at the next bullet so the amendment must live inside it.
const nextBullet = rules.indexOf('\n- **', indIdx);
const independence = nextBullet > indIdx ? rules.slice(indIdx, nextBullet) : rules.slice(indIdx);
assert.match(independence, /Amend(ed|ment)/i,
'Independence must be explicitly amended, not silently contradicted');
assert.match(independence, /Phase 4\.5/, 'the amendment must name Phase 4.5 as the crossing');
assert.match(independence, /query-privacy-gate\.mjs/,
'the amendment must name the compensating control');
});
// --- v5.1.1 runtime SC4 + SC7 ---
test('trekresearch — SC4: low-effort fixture → resolver returns {effort: low, model: sonnet}', () => {
@ -331,7 +137,8 @@ function paragraphAt(text, marker) {
test('trekresearch — Phase 5 follow-ups take phase_signal_result.model, else opus', () => {
const text = read();
assert.doesNotMatch(flat(phase5(text)), /model: "opus"/);
const part = paragraphAt(text, 'launch a targeted follow-up agent');
const part = paragraphAt(phase5(text), 'launch a targeted follow-up agent');
assert.match(part, /Maximum 2 follow-ups\./);
assert.match(part, /launch a targeted follow-up agent \(model: phase_signal_result\.model, else opus — a follow-up may be one of Claude Code's built-in agents, which have no Voyage frontmatter to fall back to; see Launch rules\)/);
assert.match(flat(text), /captured as `phase_signal_result` and used at Agent-spawn sites in Phase 4 and Phase 5 to inject the resolved model/);
});

49
tests/fixtures/research-deep-research-adapted.md vendored
View file

@ -1,49 +0,0 @@
---
type: trekresearch-brief
created: 2026-06-30
question: "Should /trekresearch delegate its external phase to the built-in /deep-research workflow?"
confidence: 0.8
dimensions: 2
mcp_servers_used: []
local_agents_used: []
external_agents_used:
- deep-research
---
# Deep-research engine adapter output
> Fixture: a `/deep-research` in-context report reduced into the research-brief
> schema by the `--engine deep-research` adapter (Step 4). Models the target the
> adapter must hit; not real engine output.
## Executive Summary
Delegating the external phase to the built-in `/deep-research` workflow is a
viable opt-in engine that supplies fan-out and cited claim-verification for free.
Confidence is medium-high on the mechanism but lower on availability, because the
workflow exposes no positive "is-enabled" probe. The load-bearing caveat is
provenance: structural validity does not certify that the cited URLs are real, so
a swarm fallback plus a human URL spot-check stay mandatory.
## Dimensions
### Engine mechanism -- Confidence: high
**External findings:**
- `/deep-research` is a built-in dynamic workflow reachable only by prose instruction, with no programmatic API (https://code.claude.com/docs/workflows).
- Its report lands in-context with no on-disk artifact, so the adapter must transform what is already in the turn (https://code.claude.com/docs/commands).
### Fallback ergonomics -- Confidence: high
**External findings:**
- There is no positive availability probe; only `disableWorkflows` / `CLAUDE_CODE_DISABLE_WORKFLOWS` off-switches and a 2.1.154 version floor are documented (https://code.claude.com/docs/skills).
- Disabled-workflow behavior under `claude -p` is undocumented, so the post-hoc presence check must be robust to every failure manifestation (https://github.com/anthropics/claude-code/issues/52272).
## Sources
| # | Source | Type | Quality | Used in |
|---|--------|------|---------|---------|
| 1 | https://code.claude.com/docs/workflows | official | high | Engine mechanism |
| 2 | https://code.claude.com/docs/commands | official | high | Engine mechanism |
| 3 | https://code.claude.com/docs/skills | official | high | Fallback ergonomics |
| 4 | https://github.com/anthropics/claude-code/issues/52272 | community | medium | Fallback ergonomics |

556
tests/hooks/agent-cap.test.mjs
View file

@ -1,556 +0,0 @@
// tests/hooks/agent-cap.test.mjs
// Step 10 — pins hooks/scripts/pre-agent-cap.mjs, the PreToolUse enforcement
// of the /trekresearch Phase 5 loop bound.
//
// The spike (docs/spike-pretooluse-subagent-reach.md, RESULT: FIRES) proved a
// plugin PreToolUse hook observes sub-agent tool calls, so the cap can be
// enforced rather than merely documented. This file pins the two properties
// that matter in opposite directions:
//
// (a) it DENIES (exit 2) once the ledger shows the budget spent, and
// (b) it does NOT over-block — an unrelated session, unparsable stdin, a
// stale marker, or the kill switch all exit 0.
//
// Pattern: tests/hooks/bash-guard.test.mjs (child process via runHook).
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, existsSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
import { tmpdir } from 'node:os';
import { runHookWithEnv } from '../helpers/hook-helper.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const CAP_HOOK = join(ROOT, 'hooks', 'scripts', 'pre-agent-cap.mjs');
const HOOKS_JSON = join(ROOT, 'hooks', 'hooks.json');
const SESSION = 'sess-abc123';
const RUN_ID = 'run-xyz789';
/**
* Build a throwaway CLAUDE_PLUGIN_DATA dir holding a scope marker for
* `sessionId` and `turns` ledger entries for RUN_ID.
*/
function fixture({ turns = 0, sessionId = SESSION, startedAt = new Date(), exhausted = false } = {}) {
const dir = mkdtempSync(join(tmpdir(), 'voyage-cap-'));
mkdirSync(join(dir, 'trekresearch-loop-scope'), { recursive: true });
writeFileSync(
join(dir, 'trekresearch-loop-scope', `${sessionId}.json`),
JSON.stringify({ runId: RUN_ID, startedAt: startedAt.toISOString() }),
);
const lines = Array.from({ length: turns }, (_, i) =>
JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, dimension: `d${i}`, effort: 'high', slot: i + 1 }),
);
// The tombstone research-loop-cap.mjs appends when it denies a turn for
// budget. Its presence is what tells this hook "the gate already said no".
if (exhausted) lines.push(JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, exhausted: true }));
writeFileSync(join(dir, 'trekresearch-loop-ledger.jsonl'), lines.length ? lines.join('\n') + '\n' : '');
return dir;
}
// TREKRESEARCH_MAX_CONV_TURNS=1 => budget = 1 * MAX_TOTAL_DIMENSIONS (8).
const CAPPED_ENV = { VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
const BUDGET = 8;
function searchInput(sessionId = SESSION) {
return {
session_id: sessionId,
hook_event_name: 'PreToolUse',
tool_name: 'WebSearch',
tool_input: { query: 'claude code hooks reference' },
agent_id: 'aa6d19525a4680fe0',
agent_type: 'general-purpose',
};
}
// -----------------------------------------------------------------------
// DENY — budget spent
// -----------------------------------------------------------------------
test('pre-agent-cap DENIES once the budget gate has denied a turn (tombstone present)', async () => {
const dir = fixture({ turns: BUDGET, exhausted: true });
const { code, stderr } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 2);
assert.match(stderr, /loop cap/i, 'stderr must name the cap it enforced');
assert.match(stderr, new RegExp(`${BUDGET}`), 'stderr must state the budget');
});
test('pre-agent-cap DENIES above the budget too (a breached ledger, whatever caused it)', async () => {
const dir = fixture({ turns: BUDGET + 5 });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 2);
});
// -----------------------------------------------------------------------
// ALLOW — under the cap, and ON the cap.
//
// allowTurn appends BEFORE the turn runs, so during the FINAL granted turn the
// ledger already holds `budget` records. Denying at `used >= budget` therefore
// blocked that turn's own tool calls: the primitive granted B turns, the
// harness permitted B-1, and an exhausted run always ended through an exit-2
// denial rather than the graceful "cap exhausted" exit the prose defines. The
// boundary belongs one turn later, and the tombstone above — not the count —
// is what marks a run actually finished.
// -----------------------------------------------------------------------
test('pre-agent-cap ALLOWS a loop turn under the budget', async () => {
const dir = fixture({ turns: BUDGET - 1 });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0);
});
test('pre-agent-cap ALLOWS the FINAL granted turn — its own record is already on the ledger', async () => {
const dir = fixture({ turns: BUDGET });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(
code, 0,
'turn B is granted and in flight; denying it makes the harness permit B-1 turns and forces the wrong exit',
);
});
// -----------------------------------------------------------------------
// DOES NOT OVER-BLOCK — the property that keeps this hook safe to wire
// globally. A broken PreToolUse hook would brick every session on the box.
// -----------------------------------------------------------------------
test('pre-agent-cap ALLOWS an unrelated session even when a loop is exhausted', async () => {
const dir = fixture({ turns: BUDGET });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput('some-other-session'), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0, 'no scope marker for this session_id => out of scope');
});
test('pre-agent-cap ALLOWS when no scope marker directory exists at all', async () => {
const dir = mkdtempSync(join(tmpdir(), 'voyage-cap-empty-'));
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0);
});
test('pre-agent-cap ALLOWS on unparsable stdin', async () => {
const dir = fixture({ turns: BUDGET });
const { code } = await runHookWithEnv(CAP_HOOK, 'not json at all', {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0);
});
test('pre-agent-cap ALLOWS when the input carries no session_id', async () => {
const dir = fixture({ turns: BUDGET });
const input = searchInput();
delete input.session_id;
const { code } = await runHookWithEnv(CAP_HOOK, input, {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0);
});
// -----------------------------------------------------------------------
// TTL / auto-reset — a marker left behind by a crashed run must not deny
// tool calls forever.
// -----------------------------------------------------------------------
test('pre-agent-cap ALLOWS when the scope marker is older than the TTL', async () => {
const dir = fixture({ turns: BUDGET, startedAt: new Date(Date.now() - 48 * 3600 * 1000) });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
VOYAGE_CAP_SCOPE_TTL_MS: '1000',
});
assert.strictEqual(code, 0, 'a stale marker must auto-reset, not deny forever');
});
// The TTL runs from marker.startedAt, not from last activity, and `claude
// --resume` keeps the same session_id — so a run that died leaving its marker
// behind hands the resumed session whatever deny window is left. Two things
// bound that: the window is hours, not the machine's life (below), and it only
// opens at all once the budget gate has actually denied a turn.
test('pre-agent-cap ALLOWS a resumed session whose crashed run never exhausted its budget', async () => {
// Marker still fresh, ledger part-spent, no tombstone: the run died mid-loop.
const dir = fixture({ turns: 5 });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(
code, 0,
'a part-spent run leaves no denial record, so resuming its session must not brick unrelated work',
);
});
test('pre-agent-cap uses a default TTL of hours, not a day — a 3h-old marker auto-resets', async () => {
const dir = fixture({
turns: BUDGET,
exhausted: true,
startedAt: new Date(Date.now() - 3 * 3600 * 1000),
});
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
}); // no VOYAGE_CAP_SCOPE_TTL_MS — this is the built-in default
assert.strictEqual(code, 0, 'no real research run lasts 3h; a marker that old is debris');
});
test('pre-agent-cap names the marker path when it denies, so the operator has a remedy', async () => {
const dir = fixture({ turns: BUDGET, exhausted: true });
const { code, stderr } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 2);
assert.ok(
stderr.includes(join(dir, 'trekresearch-loop-scope', `${SESSION}.json`)),
`stderr must name the marker to delete; got:\n${stderr}`,
);
});
// -----------------------------------------------------------------------
// Fail CLOSED once in scope — the hook's own header says a budget control
// that cannot count must not grant. The unreadable-ledger branch returned 0
// and therefore ALLOWED, which is the opposite. A directory standing where
// the ledger file belongs reproduces it portably (EISDIR).
// -----------------------------------------------------------------------
test('pre-agent-cap DENIES when the ledger cannot be read at all (fail closed)', async () => {
const dir = fixture({ turns: 0 });
const ledgerPath = join(dir, 'trekresearch-loop-ledger.jsonl');
rmSync(ledgerPath, { force: true });
mkdirSync(ledgerPath, { recursive: true });
const { code, stderr } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 2, 'an in-scope run whose ledger cannot be counted must not be granted');
assert.match(stderr, /could not be read|unreadable/i, 'stderr must say counting failed, not that the budget is spent');
});
test('pre-agent-cap ALLOWS an unreadable ledger when the session is OUT of scope', async () => {
const dir = fixture({ turns: 0, sessionId: 'a-different-session' });
const ledgerPath = join(dir, 'trekresearch-loop-ledger.jsonl');
rmSync(ledgerPath, { force: true });
mkdirSync(ledgerPath, { recursive: true });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0, 'fail-closed is scoped to the loop, it must not brick unrelated sessions');
});
// -----------------------------------------------------------------------
// Kill switch + default-off
// -----------------------------------------------------------------------
test('pre-agent-cap kill switch VOYAGE_DISABLE_CAP_HOOK=1 allows an exhausted loop', async () => {
const dir = fixture({ turns: BUDGET });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
VOYAGE_DISABLE_CAP_HOOK: '1',
});
assert.strictEqual(code, 0);
});
test('pre-agent-cap is inert when VOYAGE_STORM_ENABLED is not 1 (default-off)', async () => {
const dir = fixture({ turns: BUDGET });
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
VOYAGE_STORM_ENABLED: '0',
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0);
});
// -----------------------------------------------------------------------
// The measured environment — CLAUDE_PLUGIN_DATA is EMPTY in the Bash tool
// env, so that is the environment every real run happens in. The writer (the
// Phase 5 bash snippet) and the reader (this hook) must land on the SAME
// fallback root, or the hook allows unconditionally while claiming to enforce.
// -----------------------------------------------------------------------
test('pre-agent-cap enforces via the fallback root when CLAUDE_PLUGIN_DATA is absent', async () => {
const home = mkdtempSync(join(tmpdir(), 'voyage-cap-home-'));
const root = join(home, '.claude', 'voyage');
mkdirSync(join(root, 'trekresearch-loop-scope'), { recursive: true });
writeFileSync(
join(root, 'trekresearch-loop-scope', `${SESSION}.json`),
JSON.stringify({ runId: RUN_ID, startedAt: new Date().toISOString() }),
);
writeFileSync(
join(root, 'trekresearch-loop-ledger.jsonl'),
[
...Array.from({ length: BUDGET }, (_, i) =>
JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, dimension: `d${i}`, effort: 'high', slot: i + 1 })),
JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, exhausted: true }),
].join('\n') + '\n',
);
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: '',
HOME: home,
});
assert.strictEqual(code, 2, 'the hook must find marker AND ledger under the fallback root and deny');
});
test('pre-agent-cap allows under budget in the fallback root — the fallback is not a blanket deny', async () => {
const home = mkdtempSync(join(tmpdir(), 'voyage-cap-home-'));
const root = join(home, '.claude', 'voyage');
mkdirSync(join(root, 'trekresearch-loop-scope'), { recursive: true });
writeFileSync(
join(root, 'trekresearch-loop-scope', `${SESSION}.json`),
JSON.stringify({ runId: RUN_ID, startedAt: new Date().toISOString() }),
);
writeFileSync(join(root, 'trekresearch-loop-ledger.jsonl'), '');
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: '',
HOME: home,
});
assert.strictEqual(code, 0);
});
// -----------------------------------------------------------------------
// Append-only counting — the hook reads the ledger, it never writes it.
// Writing per tool call would make the cap count its own enforcement.
// -----------------------------------------------------------------------
test('pre-agent-cap never writes to the ledger', async () => {
const dir = fixture({ turns: 2 });
const ledgerPath = join(dir, 'trekresearch-loop-ledger.jsonl');
const before = readFileSync(ledgerPath, 'utf-8');
await runHookWithEnv(CAP_HOOK, searchInput(), { ...CAPPED_ENV, CLAUDE_PLUGIN_DATA: dir });
assert.strictEqual(readFileSync(ledgerPath, 'utf-8'), before);
});
// -----------------------------------------------------------------------
// Crash-time marker states. These are the states the TTL discussion in the
// hook header anticipates, and none of them had a test: a marker written
// half-way, and a marker whose runId never made it. Both must ALLOW — a
// marker we cannot read cannot tell us which run we are in, and guessing
// would deny tool calls in a session we know nothing about.
// -----------------------------------------------------------------------
test('pre-agent-cap ALLOWS on a partially written (corrupt) scope marker', async () => {
const dir = fixture({ turns: BUDGET, exhausted: true });
// Exactly what an interrupted printf leaves behind: valid prefix, no close.
writeFileSync(join(dir, 'trekresearch-loop-scope', `${SESSION}.json`), '{"runId":"run-xyz789","star');
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0, 'an unparsable marker is not evidence of a loop turn');
});
test('pre-agent-cap ALLOWS a marker that carries no runId', async () => {
const dir = fixture({ turns: BUDGET, exhausted: true });
writeFileSync(
join(dir, 'trekresearch-loop-scope', `${SESSION}.json`),
JSON.stringify({ startedAt: new Date().toISOString() }),
);
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0, 'without a runId there are no ledger lines to count against');
});
test('pre-agent-cap ALLOWS a marker whose runId is not a string', async () => {
const dir = fixture({ turns: BUDGET, exhausted: true });
// Truthy, so it clears the `!marker?.runId` guard and the session counts as
// in scope — but readLedger compares runId with ===, so a number matches no
// record and the run reads as 0 turns spent. Allow is the right answer either
// way, which is why this stays a pin on the OUTCOME and not an argument for a
// type guard: no writer emits a non-string runId, and the two routes are
// indistinguishable from outside.
writeFileSync(
join(dir, 'trekresearch-loop-scope', `${SESSION}.json`),
JSON.stringify({ runId: 5, startedAt: new Date().toISOString() }),
);
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0);
});
test('pre-agent-cap ALLOWS a marker whose runId is the empty string', async () => {
const dir = fixture({ turns: BUDGET, exhausted: true });
writeFileSync(
join(dir, 'trekresearch-loop-scope', `${SESSION}.json`),
JSON.stringify({ runId: '', startedAt: new Date().toISOString() }),
);
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0);
});
// -----------------------------------------------------------------------
// Malformed ledger lines — a truncated final write must not be counted as a
// turn, and must not stop the well-formed lines around it from counting.
// -----------------------------------------------------------------------
test('pre-agent-cap does not count a malformed ledger line as a turn', async () => {
const dir = fixture({ turns: BUDGET - 1 });
const ledgerPath = join(dir, 'trekresearch-loop-ledger.jsonl');
writeFileSync(ledgerPath, readFileSync(ledgerPath, 'utf-8') + '{"runId":"run-xyz789","dimen\n');
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 0, 'a half-written line is not a spent turn');
});
test('pre-agent-cap still finds the tombstone with malformed lines around it', async () => {
const dir = fixture({ turns: BUDGET, exhausted: true });
const ledgerPath = join(dir, 'trekresearch-loop-ledger.jsonl');
writeFileSync(ledgerPath, '{ garbage\n' + readFileSync(ledgerPath, 'utf-8') + 'also garbage\n');
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: dir,
});
assert.strictEqual(code, 2, 'skipping bad lines must not mean skipping the run’s denial record');
});
// -----------------------------------------------------------------------
// The marker snippet is EXECUTED, not asserted about.
//
// Every existing pin on the marker lifecycle is a substring assertion on the
// prose in commands/trekresearch.md. A snippet that emitted invalid JSON, or
// wrote to a path the hook never looks at, would keep the whole suite green
// while the hook silently allowed everything — which is the exact failure S82
// found by hand. So these tests run the real shell blocks out of the command
// file, with CLAUDE_PLUGIN_DATA stripped and HOME sandboxed, and then run the
// real hook against what they produced.
// -----------------------------------------------------------------------
const CMD_FILE = join(ROOT, 'commands', 'trekresearch.md');
/** Pull the ```bash block that contains `needle` out of the command file. */
function bashBlockContaining(needle) {
const text = readFileSync(CMD_FILE, 'utf-8');
const at = text.indexOf(needle);
assert.ok(at > -1, `commands/trekresearch.md no longer contains ${JSON.stringify(needle)}`);
const open = text.lastIndexOf('```bash', at);
assert.ok(open > -1, `no \`\`\`bash fence opens before ${JSON.stringify(needle)}`);
const bodyStart = text.indexOf('\n', open) + 1;
const close = text.indexOf('```', bodyStart);
assert.ok(close > bodyStart, 'unterminated bash fence');
return text.slice(bodyStart, close);
}
function runSnippet(snippet, env) {
return execFileSync('bash', ['-c', snippet], {
encoding: 'utf-8',
env: { PATH: process.env.PATH, ...env },
});
}
test('the marker WRITE snippet lands valid JSON exactly where the hook looks for it', () => {
const home = mkdtempSync(join(tmpdir(), 'voyage-snippet-'));
const sessionId = 'snippet-session-1';
const snippet = bashBlockContaining('Arms the PreToolUse cap').replace(/\{run_id\}/g, 'snippet-run-1');
runSnippet(snippet, { HOME: home, CLAUDE_CODE_SESSION_ID: sessionId });
const markerPath = join(home, '.claude', 'voyage', 'trekresearch-loop-scope', `${sessionId}.json`);
assert.ok(existsSync(markerPath), `snippet wrote no marker at ${markerPath}`);
const marker = JSON.parse(readFileSync(markerPath, 'utf-8')); // throws if the printf emits bad JSON
assert.strictEqual(marker.runId, 'snippet-run-1', 'runId must be the same id passed to --run-id');
assert.ok(Number.isFinite(Date.parse(marker.startedAt)), `startedAt must parse, got ${marker.startedAt}`);
});
test('the marker snippet writes NO file when CLAUDE_CODE_SESSION_ID is empty', () => {
const home = mkdtempSync(join(tmpdir(), 'voyage-snippet-'));
const snippet = bashBlockContaining('Arms the PreToolUse cap').replace(/\{run_id\}/g, 'snippet-run-2');
const out = runSnippet(snippet, { HOME: home });
const scopeDir = join(home, '.claude', 'voyage', 'trekresearch-loop-scope');
assert.ok(!existsSync(join(scopeDir, '.json')), 'an empty session id must not produce a `.json` marker');
assert.match(out, /stays inert/i, 'the snippet must say the harness cap is inert, not fail silently');
});
test('write snippet then real hook: the loop’s own writer arms the enforcement end to end', async () => {
const home = mkdtempSync(join(tmpdir(), 'voyage-snippet-'));
const sessionId = 'snippet-session-3';
const runId = 'snippet-run-3';
runSnippet(
bashBlockContaining('Arms the PreToolUse cap').replace(/\{run_id\}/g, runId),
{ HOME: home, CLAUDE_CODE_SESSION_ID: sessionId },
);
// A spent, tombstoned ledger for that same runId, under the same resolved root.
writeFileSync(
join(home, '.claude', 'voyage', 'trekresearch-loop-ledger.jsonl'),
[
...Array.from({ length: BUDGET }, (_, i) =>
JSON.stringify({ ts: new Date().toISOString(), runId, dimension: `d${i}`, effort: 'high', slot: i + 1 })),
JSON.stringify({ ts: new Date().toISOString(), runId, exhausted: true }),
].join('\n') + '\n',
);
const denied = await runHookWithEnv(CAP_HOOK, searchInput(sessionId), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: '',
HOME: home,
});
assert.strictEqual(denied.code, 2, 'the hook must find the snippet’s marker and enforce against it');
// And the removal snippet must disarm it again — same root, same guard.
runSnippet(
bashBlockContaining('Removal — idempotent').replace(/\{run_id\}/g, runId),
{ HOME: home, CLAUDE_CODE_SESSION_ID: sessionId },
);
assert.ok(
!existsSync(join(home, '.claude', 'voyage', 'trekresearch-loop-scope', `${sessionId}.json`)),
'the removal snippet must delete the marker the write snippet created',
);
const allowed = await runHookWithEnv(CAP_HOOK, searchInput(sessionId), {
...CAPPED_ENV,
CLAUDE_PLUGIN_DATA: '',
HOME: home,
});
assert.strictEqual(allowed.code, 0, 'a removed marker must take the session back out of scope');
});
// -----------------------------------------------------------------------
// Wiring
// -----------------------------------------------------------------------
function invocationOf(h) {
return [h.command || '', ...(h.args || [])].join(' ').trim();
}
test('hooks.json wires pre-agent-cap.mjs on PreToolUse with ${CLAUDE_PLUGIN_ROOT}', () => {
const cfg = JSON.parse(readFileSync(HOOKS_JSON, 'utf8'));
const invocations = (cfg.hooks.PreToolUse || []).flatMap((entry) =>
(entry.hooks || []).map(invocationOf),
);
const capInvocation = invocations.find((cmd) => cmd.includes('pre-agent-cap.mjs'));
assert.ok(capInvocation, `no PreToolUse hook references pre-agent-cap.mjs. Found: ${JSON.stringify(invocations)}`);
assert.match(capInvocation, /\$\{CLAUDE_PLUGIN_ROOT\}/, 'relative paths fail in headless sessions');
assert.match(capInvocation, /^node\s+/);
});
test('hooks.json matcher for pre-agent-cap covers the loop’s outbound surface', () => {
const cfg = JSON.parse(readFileSync(HOOKS_JSON, 'utf8'));
const entry = (cfg.hooks.PreToolUse || []).find((e) =>
(e.hooks || []).some((h) => invocationOf(h).includes('pre-agent-cap.mjs')),
);
assert.ok(entry, 'pre-agent-cap entry missing from PreToolUse');
for (const tool of ['WebSearch', 'WebFetch', 'Task']) {
assert.match(entry.matcher, new RegExp(tool), `matcher must cover ${tool}`);
}
});

26
tests/hooks/storm-matcher.test.mjs
View file

@ -1,26 +0,0 @@
// tests/hooks/storm-matcher.test.mjs
// The STORM cap (pre-agent-cap.mjs) must see subagent spawns. The tool was
// renamed Task -> Agent in Claude Code 2.1.63; the docs state the old name
// still works as an alias «in settings and agent definitions»
// (code.claude.com/docs/en/sub-agents, the rename note) and say nothing about
// hook matchers, which compare against the tool name the hook input carries.
// So the matcher must name `Agent` itself. Resolved with Claude Code's matcher
// rules (hooks § Matcher patterns), as write-guard-coverage.test.mjs does.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { scriptsFor } from './guard-helpers.mjs';
const fires = (tool) => scriptsFor('PreToolUse', tool).some((p) => p.endsWith('pre-agent-cap.mjs'));
test('STORM cap fires on Agent (the tool name since 2.1.63)', () => {
assert.ok(fires('Agent'), 'pre-agent-cap.mjs must fire for tool_name "Agent"');
});
test('STORM cap still fires on WebSearch, WebFetch and the old Task name', () => {
for (const t of ['WebSearch', 'WebFetch', 'Task']) assert.ok(fires(t), t);
});
test('STORM cap does not fire on unrelated tools', () => {
for (const t of ['Bash', 'Edit', 'Write', 'Read', 'AgentOutput', 'ExitPlanMode']) assert.ok(!fires(t), t);
});

297
tests/lib/doc-consistency.test.mjs
View file

@ -980,201 +980,22 @@ test('S18: --min-brief-version is documented at the trekplan + trekresearch boun
}
});
test('deep-research-engine: --engine is documented + consistent across surfaces', () => {
// Cross-doc pin mirroring S18 (:1057). The opt-in external-research engine flag
// must be discoverable wherever /trekresearch flags live: the command itself
// plus the three reference surfaces.
for (const f of ['commands/trekresearch.md', 'docs/command-modes.md', 'CLAUDE.md', 'README.md']) {
assert.ok(
read(f).includes('--engine'),
`${f} must document the --engine flag (deep-research-engine)`,
);
}
// README documents it specifically as an **Engine** mode-table row.
assert.ok(
/\*\*Engine\*\*/.test(read('README.md')),
'README.md must document --engine as an **Engine** mode row',
);
// The command prose must name both engine values and the swarm fallback, so the
// opt-in + graceful-degradation contract is pinned — not merely the flag string.
const research = read('commands/trekresearch.md');
assert.ok(/\bswarm\b/.test(research), 'trekresearch.md must name the swarm engine value');
assert.ok(/\bdeep-research\b/.test(research), 'trekresearch.md must name the deep-research engine value');
assert.ok(
/fall back|falls back/.test(research),
'trekresearch.md must document the swarm fallback (graceful degradation)',
);
});
test('deep-research-engine: the CC 2.1.218 operator-only ceiling is documented on every surface', () => {
// CC 2.1.218 changed /deep-research to "start only when invoked manually"; from
// there the Skill tool refuses a model invocation outright with
// `disable-model-invocation`. The flag stays (additive opt-in, always degrades to
// swarm), but no surface may keep promising a path Claude Code has removed, and
// none may leave `2.1.154+` standing alone as the engine's requirement.
const SURFACES = ['commands/trekresearch.md', 'docs/command-modes.md', 'README.md'];
for (const f of SURFACES) {
const body = read(f);
assert.ok(
body.includes('2.1.218'),
`${f} must document the CC 2.1.218 ceiling for the deep-research engine`,
);
assert.ok(
body.includes('disable-model-invocation'),
`${f} must name disable-model-invocation as the fallback reason from CC 2.1.218`,
);
}
// The one-line reference rows carry the whole truth on their own line — an
// operator reading the flag table never scrolls to the command prose.
const ROWS = [
['docs/command-modes.md', /^.*`--engine \{swarm\\\|deep-research\}`.*$/m],
['README.md', /^\|\s*\*\*Engine\*\*.*$/m],
];
for (const [f, re] of ROWS) {
const m = read(f).match(re);
assert.ok(m, `${f} must still carry the --engine reference row`);
assert.ok(
m[0].includes('2.1.218') && m[0].includes('disable-model-invocation'),
`${f} --engine row must state the real window (2.1.218 + disable-model-invocation)`,
);
}
// Both load-bearing regions of the command itself: the flag bullet and the
// pre-gate that decides whether the delegation is attempted at all.
const research = read('commands/trekresearch.md');
const bullet = research.match(/8\. `--engine <name>`[\s\S]*?Flags can be combined/);
assert.ok(bullet, 'trekresearch.md must still carry the --engine flag bullet');
assert.ok(
bullet[0].includes('2.1.218') && bullet[0].includes('disable-model-invocation'),
'trekresearch.md --engine bullet must state the real window, not just the 2.1.154 floor',
);
const pregate = research.match(/\*\*Coarse pre-gate[\s\S]*?authoritative guard\./);
assert.ok(pregate, 'trekresearch.md must still carry the deep-research pre-gate');
assert.ok(
pregate[0].includes('2.1.218'),
'the pre-gate must carry an upper ceiling at 2.1.218, not only the 2.1.154 floor',
);
});
// ── STORM bounded loop — env-vars documented across the four surfaces ──────
// Same cross-doc shape as the --engine pin above. An operator-facing switch
// documented on one surface is a switch most operators never find; and the
// three below decide cost, so they are the ones worth pinning.
const STORM_SURFACES = ['docs/command-modes.md', 'CLAUDE.md', 'README.md', 'docs/architecture.md'];
const STORM_ENV_VARS = ['VOYAGE_STORM_ENABLED', 'TREKRESEARCH_MAX_CONV_TURNS', 'VOYAGE_DISABLE_CAP_HOOK'];
for (const envVar of STORM_ENV_VARS) {
test(`STORM: ${envVar} is documented on all four reference surfaces`, () => {
for (const f of STORM_SURFACES) {
assert.ok(read(f).includes(envVar), `${f} must document ${envVar} (STORM bounded-loop env-vars)`);
}
});
}
test('STORM: VOYAGE_STORM_ENABLED is documented WITH its default-off contract', () => {
// Naming a switch without its default is how a default-on mechanism ships by
// accident. Default-off is also the decline branch: declining costs nothing.
for (const f of STORM_SURFACES) {
const t = read(f);
const i = t.indexOf('VOYAGE_STORM_ENABLED');
const window = t.slice(Math.max(0, i - 400), i + 400);
assert.match(
window,
/default-off|default off|opt-in/i,
`${f}: VOYAGE_STORM_ENABLED must be documented together with its default-off contract`,
);
}
});
// The flag gates BOTH STORM phases, not just the Phase 5 loop. A surface that
// scopes it to "the loop" tells an operator that unsetting it still leaves
// Phase 4.5 discovery running — which was true until the guard was fixed, and
// is the half-off state the decline branch cannot survive.
test('STORM: VOYAGE_STORM_ENABLED is documented as gating BOTH phases, not the loop alone', () => {
for (const f of STORM_SURFACES) {
const t = read(f);
const i = t.indexOf('VOYAGE_STORM_ENABLED');
const window = t.slice(Math.max(0, i - 400), i + 400);
assert.match(
window,
/4\.5|discovery|dimension discovery|both/i,
`${f}: VOYAGE_STORM_ENABLED must be documented as gating Phase 4.5 too, not only the Phase 5 loop`,
);
}
});
// The variable is empty in the Bash tool env, so "missing CLAUDE_PLUGIN_DATA
// denies" described a loop that could never spend turn 1. The root is resolved
// in code now; a doc that still promises the deny describes a mechanism the
// code does not have.
test('STORM: no surface claims a missing CLAUDE_PLUGIN_DATA denies — the root falls back', () => {
for (const f of ['docs/architecture.md', 'CLAUDE.md', 'README.md', 'docs/command-modes.md']) {
const t = read(f);
assert.doesNotMatch(
t,
/(missing|no|absent) `?CLAUDE_PLUGIN_DATA`?[^.\n]*(denies|fails closed)/i,
`${f}: CLAUDE_PLUGIN_DATA absence no longer denies — it resolves to ~/.claude/voyage`,
);
}
assert.match(
read('docs/architecture.md'),
/~\/\.claude\/voyage/,
'docs/architecture.md must name the fallback data root the cap and the hook share',
);
});
test('STORM: README research-dimension prose stays at the existing 3–8 ceiling', () => {
// Phase 4.5 discovers dimensions UNDER the maxDimensions: 8 ceiling.
// Rewriting this prose upward would raise a ceiling the brief asked us to hold.
test('README research-dimension prose stays at the existing 3–8 ceiling', () => {
// /trekresearch Phase 2 determines 3-8 research dimensions; the README must not raise it.
assert.ok(
read('README.md').includes('3–8 research dimensions'),
'README.md must keep the "3–8 research dimensions" prose — augmentation happens under the existing cap, it does not raise it',
'README.md must keep the "3–8 research dimensions" prose — it matches the Phase 2 ceiling, it does not raise it',
);
});
// The Phase 4.5 amendment to the Independence hard rule crosses that rule
// deliberately: discovered dimensions are mined from Phase-4 output, which holds
// local-agent findings, so a discovered dimension can steer an external query.
// The crossing is defensible — bounded, disclosed, and resolving a tension the
// brief created itself. What was not defensible was naming query-privacy-gate.mjs
// as its compensating control: that gate inspects outbound query CONTENT for
// paths, repo identifiers and secret-shaped strings. It compensates the EGRESS
// risk the crossing creates. It cannot stop a local finding from shaping an
// external agent's question, so the bias risk was left with no named control
// while the text read as though it had one.
test('STORM: the Independence amendment does not name the privacy gate as the BIAS control', () => {
const t = read('commands/trekresearch.md');
const amendment = t.slice(t.indexOf('**Independence:**'), t.indexOf('**Graceful degradation:**'));
assert.ok(amendment.length > 100, 'the Independence hard rule and its amendment must still be present');
assert.ok(
!/compensating\s+control\s+is\s+`query-privacy-gate/.test(amendment),
'query-privacy-gate.mjs compensates egress, not bias — naming it as THE compensating control for the ' +
'Independence crossing claims a control the gate cannot provide',
);
// The bias risk must carry a control that actually bears on bias.
assert.match(
amendment,
/contrarian-researcher/,
'the amendment must name the control that does bear on bias — contrarian-researcher runs unconditionally ' +
'at effort: high, the only effort at which the crossing happens',
);
assert.match(
amendment,
/egress/i,
'the privacy gate should still be named, as the control for the egress risk the same crossing creates',
);
});
test('STORM: no banned Sonnet-swarm phrase introduced on any STORM surface', () => {
test('no banned Sonnet-swarm phrase on the reference surfaces', () => {
const BANNED = [
'Sonnet exploration',
'Sonnet runs the exploration',
'front-loads cheap Sonnet',
'exploration agents stay on Sonnet',
];
for (const f of STORM_SURFACES) {
for (const f of ['docs/command-modes.md', 'CLAUDE.md', 'README.md', 'docs/architecture.md']) {
const t = read(f);
for (const phrase of BANNED) {
assert.ok(!t.includes(phrase), `${f} must not claim "${phrase}" — sub-agents are opus-pinned`);
@ -1182,113 +1003,6 @@ test('STORM: no banned Sonnet-swarm phrase introduced on any STORM surface', ()
}
});
// ── STORM bounded loop — Phase 5 scope-marker wiring (S79) ────────────────
// hooks/scripts/pre-agent-cap.mjs enforces the loop bound ONLY while a scope
// marker exists for the calling session. Nothing wrote that marker, so the
// hook shipped correct-but-latent. These pins bind the two ends of one
// contract: the reader (the hook) and the writer (Phase 5 prose). Both sides
// are derived from the hook source where possible, so drift in EITHER
// direction fails — renaming the directory in the hook breaks the prose pin
// just as rewriting the prose does.
const CAP_HOOK_SRC = read('hooks/scripts/pre-agent-cap.mjs');
const RESEARCH_CMD = read('commands/trekresearch.md');
function phase5Section(text) {
const start = text.indexOf('## Phase 5');
const end = text.indexOf('## Phase 6', start);
assert.ok(start > 0 && end > start, 'commands/trekresearch.md must keep ## Phase 5 … ## Phase 6');
return text.slice(start, end);
}
test('STORM marker: the directory the hook reads is the directory Phase 5 writes', () => {
const m = CAP_HOOK_SRC.match(/const SCOPE_DIRNAME = '([^']+)'/);
assert.ok(m, 'pre-agent-cap.mjs must keep SCOPE_DIRNAME as a single-quoted literal');
const scopeDir = m[1];
assert.ok(
phase5Section(RESEARCH_CMD).includes(scopeDir),
`commands/trekresearch.md Phase 5 must write the scope marker under ${scopeDir}/ — a hook keyed on a directory nobody writes is latent, not enforcing`,
);
assert.ok(
phase5Section(RESEARCH_CMD).includes('CLAUDE_PLUGIN_DATA'),
'Phase 5 must root the marker at CLAUDE_PLUGIN_DATA — the same root the hook resolves',
);
});
test('STORM marker: both payload fields the hook reads are named in Phase 5', () => {
// The hook rejects a marker without runId, and treats an unparsable
// startedAt as stale. A writer that emits neither name produces a marker
// that is silently ignored.
const phase5 = phase5Section(RESEARCH_CMD);
for (const field of ['runId', 'startedAt']) {
assert.ok(
CAP_HOOK_SRC.includes(`marker.${field}`) || CAP_HOOK_SRC.includes(`marker?.${field}`),
`pre-agent-cap.mjs must still read marker.${field}`,
);
assert.ok(
phase5.includes(field),
`commands/trekresearch.md Phase 5 must write the ${field} field the hook reads`,
);
}
});
test('STORM marker: Phase 5 keys the marker filename by the harness session id', () => {
// Verified 2026-08-12: the session_id on a PreToolUse payload equals
// $CLAUDE_CODE_SESSION_ID for the same session. The marker filename is the
// scope key — key it by anything else and the hook never matches.
assert.ok(
phase5Section(RESEARCH_CMD).includes('CLAUDE_CODE_SESSION_ID'),
'Phase 5 must name CLAUDE_CODE_SESSION_ID as the marker filename key (== the hook payload session_id)',
);
});
test('STORM marker: only Phase 5 writes it — Phase 4.5 does not', () => {
// The hook contract says "a marker file only the Phase 5 loop writes".
// Phase 4.5 mines already-retrieved Phase-4 results and spends no loop
// turns; scoping enforcement there widens the window for nothing.
const m = CAP_HOOK_SRC.match(/const SCOPE_DIRNAME = '([^']+)'/);
const scopeDir = m[1];
const start = RESEARCH_CMD.indexOf('## Phase 4.5');
const end = RESEARCH_CMD.indexOf('## Phase 5', start);
assert.ok(start > 0 && end > start, 'commands/trekresearch.md must keep ## Phase 4.5 … ## Phase 5');
assert.ok(
!RESEARCH_CMD.slice(start, end).includes(scopeDir),
'Phase 4.5 must not write the scope marker — only the Phase 5 loop does',
);
});
test('STORM marker: every one of the three loop exits removes the marker', () => {
// A marker left behind after the cap is spent denies WebSearch/WebFetch/Task
// for the REST of the session — Phase 6 synthesis spawns agents. Cleanup on
// the exhausted exit is what keeps enforcement from becoming a session brick.
const phase5 = phase5Section(RESEARCH_CMD);
const exitsStart = phase5.indexOf('### Exits');
assert.ok(exitsStart > 0, 'Phase 5 must keep the ### Exits section');
const exits = phase5.slice(exitsStart, phase5.indexOf('###', exitsStart + 5));
for (const [n, label] of [['1.', 'converged'], ['2.', 'cap exhausted'], ['3.', 'operator stop']]) {
const from = exits.indexOf(`\n${n}`);
assert.ok(from > 0, `Exits must keep numbered entry ${n} (${label})`);
const nextMarker = exits.indexOf(`\n${Number(n[0]) + 1}.`, from);
const entry = exits.slice(from, nextMarker > from ? nextMarker : undefined);
assert.match(
entry,
/remove the (scope )?marker|rm -f/i,
`Exit ${n} (${label}) must remove the scope marker — a marker outliving the loop blocks the rest of the session`,
);
}
});
test('STORM marker: the crash path is documented as TTL auto-reset, not cleanup', () => {
// A crashed session runs no cleanup at all. The honest statement is that the
// hook's TTL covers it; claiming cleanup covers crashes would be false.
const phase5 = phase5Section(RESEARCH_CMD);
assert.match(
phase5,
/TTL/,
'Phase 5 must state that a crashed run is covered by the hook TTL, not by exit cleanup',
);
});
test('S18: HANDOVER-CONTRACTS documents the pre-2.2 zero-framing-enforcement hole', () => {
// The framing defense is producer-elective: a brief declaring ≤ 2.1 sidesteps
// it entirely. Handover 1 (PUBLIC CONTRACT) must disclose this and name the remedy.
@ -1528,7 +1242,6 @@ const GEMINI_LIVE_SURFACES = [
// Historical/measurement records that MUST keep their gemini references.
const GEMINI_HISTORICAL = [
'CHANGELOG.md',
'docs/deep-research-engine-brief.md',
'examples/02-real-cli/REGENERATED.md',
];

1
tests/lib/guards-prose-pins.test.mjs
View file

@ -24,7 +24,6 @@ export const MOD_PRECONDITION = "Voyage's guards are deterministic only when no
const STOP_GUARDS = [
'hooks/scripts/pre-bash-executor.mjs',
'hooks/scripts/pre-write-executor.mjs',
'hooks/scripts/pre-agent-cap.mjs',
'hooks/scripts/pre-plan-gate.mjs',
];

2
tests/lib/living-paths.test.mjs
View file

@ -103,13 +103,11 @@ function repoScan() {
// Named exceptions: "file path" -> reason. Each must still be dead (a stale
// exemption fails below).
const TARGET = 'target-repo example';
const RB = 'removed in R-b';
const EXEMPT = new Map([
['commands/trekplan.md tests/e2e.sh', TARGET],
['docs/HANDOVER-CONTRACTS.md docs/specs/api.md', TARGET],
['templates/trekreview-template.md lib/foo.mjs', TARGET],
['docs/release-runbook.md scripts/release-plugin.mjs', "the catalog repo's script"],
['docs/deep-research-engine-research.md docs/cc-upgrade-2.1.181-decision-matrix.md', RB],
]);
const exemptKey = hit => hit.replace(/:\d+ /, ' ');

3
tests/lib/no-claude-code-launch.test.mjs
View file

@ -51,10 +51,7 @@ const PATTERNS = [
const EXEMPT = new Map([
[SELF, ['self', 17, "the gate's own pattern table and known-positive fixture"]],
['CHANGELOG.md', ['c', 15, 'release history; never rewritten']],
['docs/deep-research-engine-research.md', ['c', 6, 'research record (deep-research engine)']],
['docs/operations.md', ['c', 1, 'Path A decision record: why cache-first was rejected']],
['docs/spike-pretooluse-subagent-reach.md', ['c', 2, 'spike record (PreToolUse reach)']],
['tests/fixtures/research-deep-research-adapted.md', ['c', 1, 'recorded research fixture']],
['docs/proevesett.md', ['operator-instrument', 3, "the operator's own eval run through Anthropic's `claude plugin eval`; Voyage starts nothing - kept by operator decision 2026-10-02"]],
['.gitignore', ['operator-instrument', 1, 'ignores the output of that same operator-run eval']],
['hooks/scripts/post-approval-log.mjs', ['mode-read', 3, "reads the session's permission_mode from hook input so an approval is never logged when no person was asked; starts nothing - operator decision 2026-10-06"]],

4
tests/lib/plugin-root-resolution.test.mjs
View file

@ -48,10 +48,10 @@ function run(blockText, home) {
}
// trekexecute blocks 3 and 4 are Sub-step D4 (step check + regression) and Phase 7's K5/K6 checks (wave 2, order 6).
test('commands carry exactly the six known VOYAGE_ROOT blocks', () => {
test('commands carry exactly the five known VOYAGE_ROOT blocks', () => {
assert.deepEqual(resolutionBlocks().map(b => b.where.split(':')[0]), [
'commands/trekexecute.md', 'commands/trekexecute.md', 'commands/trekexecute.md', 'commands/trekexecute.md',
'commands/trekresearch.md', 'commands/trekreview.md',
'commands/trekreview.md',
]);
});

11
tests/lib/removed-surfaces.test.mjs
View file

@ -47,20 +47,11 @@ export const GROUPS = {
// Groups whose removal stage has not run yet. Each stage of R-b empties its
// own entry; REMOVED_SURFACES_ALL=1 checks every group (the red proof).
const PENDING = new Set(['R2', 'R3']);
const PENDING = new Set(['R3']);
// Named exceptions: "path:line" -> reason. Each must still hit (a stale
// exemption fails below).
const R2_FILE = 'R2 removes this file (the STORM gate reads the old trekresearch stats file); drop the entry with it';
const EXEMPT = new Map([
['docs/storm-measurement.md:80', R2_FILE],
['docs/storm-measurement.md:97', R2_FILE],
['docs/storm-measurement.md:100', R2_FILE],
['docs/storm-measurement.md:109', R2_FILE],
['scripts/storm-measure.mjs:4', R2_FILE],
['scripts/storm-measure.mjs:34', R2_FILE],
['scripts/storm-measure.mjs:47', R2_FILE],
['tests/scripts/storm-measure.test.mjs:8', R2_FILE],
]);
const matches = (term, line) => (typeof term === 'string' ? line.includes(term) : term.test(line));

598
tests/lib/research-loop-cap.test.mjs
View file

@ -1,598 +0,0 @@
// tests/lib/research-loop-cap.test.mjs
// Cover lib/util/research-loop-cap.mjs: default-off, worst-case arithmetic,
// anti-dead-data (different caps → different denial points), statefulness
// (identical args → different answers once the budget is hit), env
// coercion, fail-closed on missing CLAUDE_PLUGIN_DATA, and the CLI shim.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { execFileSync, execFile } from 'node:child_process';
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import {
allowTurn,
isStormEnabled,
resolveMaxConvTurns,
resolveLedgerPath,
resolveDataRoot,
readLedger,
checkDimensionCeiling,
MAX_CONV_TURNS,
MAX_TOTAL_DIMENSIONS,
} from '../../lib/util/research-loop-cap.mjs';
import { childEnv } from '../helpers/child-env.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const SHIM = join(HERE, '..', '..', 'lib', 'util', 'research-loop-cap.mjs');
function withTmpDataDir(fn) {
const dir = mkdtempSync(join(tmpdir(), 'research-loop-cap-'));
try {
return fn(dir);
} finally {
rmSync(dir, { recursive: true, force: true });
}
}
function runShim(args, env) {
try {
const out = execFileSync(process.execPath, [SHIM, ...args], {
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'pipe'],
env: childEnv(env),
});
return { code: 0, out };
} catch (e) {
return { code: e.status ?? 1, out: e.stdout?.toString() ?? '' };
}
}
/**
* Run the shim in a BUILT env rather than an inherited one. Spreading
* process.env means no test can express "CLAUDE_PLUGIN_DATA is absent" — the
* exact condition that holds in every real run — so the shim's behaviour there
* went uncovered while the module was denying turn 1.
*/
function runShimStripped(args, env = {}) {
try {
const out = execFileSync(process.execPath, [SHIM, ...args], {
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'pipe'],
env: { PATH: process.env.PATH, ...env },
});
return { code: 0, out };
} catch (e) {
return { code: e.status ?? 1, out: e.stdout?.toString() ?? '' };
}
}
// ---- (a) default-off --------------------------------------------------------
test('allowTurn — VOYAGE_STORM_ENABLED unset denies with budget 0, regardless of effort', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir };
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
assert.equal(r.ok, false);
assert.equal(r.reason, 'storm_disabled');
assert.equal(r.budget, 0);
});
});
test('allowTurn — VOYAGE_STORM_ENABLED=0 denies same as unset', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '0' };
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
assert.equal(r.ok, false);
assert.equal(r.reason, 'storm_disabled');
});
});
test('allowTurn — enabled but effort !== high denies with budget 0', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'standard' }, { env });
assert.equal(r.ok, false);
assert.equal(r.reason, 'effort_not_high');
assert.equal(r.budget, 0);
});
});
// ---- (f) CLAUDE_PLUGIN_DATA unset => documented fallback root ----------------
//
// CLAUDE_PLUGIN_DATA is EMPTY in the Bash tool's process env (measured in a
// live plugin-enabled session), and the Bash snippet in commands/trekresearch.md
// is the module's only caller. Denying on its absence therefore denied turn 1
// of every real run: the loop could never spend a turn, and the pre-registered
// measurement could not be run at all. The root is resolved in code, not
// demanded of the environment.
test('allowTurn — CLAUDE_PLUGIN_DATA unset falls back to the documented root and grants', () => {
withTmpDataDir((home) => {
const env = { VOYAGE_STORM_ENABLED: '1', HOME: home }; // no CLAUDE_PLUGIN_DATA
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
assert.equal(r.ok, true, 'the loop must be able to spend turn 1 without CLAUDE_PLUGIN_DATA');
assert.equal(r.used, 1);
assert.equal(r.budget, MAX_CONV_TURNS * MAX_TOTAL_DIMENSIONS);
assert.ok(
existsSync(join(home, '.claude', 'voyage', 'trekresearch-loop-ledger.jsonl')),
'the ledger must be written under the fallback root',
);
});
});
test('resolveDataRoot — CLAUDE_PLUGIN_DATA wins; empty or unset falls back to ~/.claude/voyage', () => {
assert.equal(resolveDataRoot({ CLAUDE_PLUGIN_DATA: '/tmp/plugin-data' }), '/tmp/plugin-data');
assert.equal(resolveDataRoot({ CLAUDE_PLUGIN_DATA: '', HOME: '/home/x' }), join('/home/x', '.claude', 'voyage'));
assert.equal(resolveDataRoot({ HOME: '/home/x' }), join('/home/x', '.claude', 'voyage'));
});
// ---- (c) worst-case arithmetic ----------------------------------------------
test('allowTurn — budget is max_conv_turns × max_total_dimensions (default 3×8=24)', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
assert.equal(r.ok, true);
assert.equal(r.budget, 24);
assert.equal(r.used, 1);
});
});
test('allowTurn — grants exactly `budget` turns then denies the next one (default 24)', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
let last;
for (let i = 0; i < 24; i++) {
last = allowTurn({ runId: 'r-exhaust', dimension: `d${i % 8}`, effort: 'high' }, { env });
assert.equal(last.ok, true, `turn ${i + 1} should be granted`);
}
const denied = allowTurn({ runId: 'r-exhaust', dimension: 'd0', effort: 'high' }, { env });
assert.equal(denied.ok, false);
assert.equal(denied.reason, 'budget_exhausted');
assert.equal(denied.used, 24);
assert.equal(denied.budget, 24);
});
});
// ---- (c)/(anti-dead-data) — different caps → observably different denial points
test('allowTurn — TREKRESEARCH_MAX_CONV_TURNS=1 denies after 8 turns (1×8), not 24', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
let last;
for (let i = 0; i < 8; i++) {
last = allowTurn({ runId: 'r-narrow', dimension: `d${i}`, effort: 'high' }, { env });
assert.equal(last.ok, true, `turn ${i + 1} should be granted`);
}
const denied = allowTurn({ runId: 'r-narrow', dimension: 'd8', effort: 'high' }, { env });
assert.equal(denied.ok, false);
assert.equal(denied.budget, 8);
assert.notEqual(denied.budget, 24, 'a narrower cap must produce a different denial point than the default');
});
});
// ---- (d) stateful — identical args give different answers once exhausted ---
test('allowTurn — identical {runId, dimension, effort} args diverge once the budget is hit (proves statefulness)', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
const args = { runId: 'r-identical', dimension: 'same-dim', effort: 'high' };
const results = [];
for (let i = 0; i < 9; i++) results.push(allowTurn(args, { env }));
// First 8 (budget = 1*8) granted, 9th denied — same exact input object each time.
assert.deepEqual(results.slice(0, 8).map(r => r.ok), Array(8).fill(true));
assert.equal(results[8].ok, false);
assert.equal(results[8].reason, 'budget_exhausted');
});
});
// ---- (e) env coercion --------------------------------------------------------
test('resolveMaxConvTurns — NaN string falls back to default', () => {
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: 'abc' }), MAX_CONV_TURNS);
});
test('resolveMaxConvTurns — empty string falls back to default', () => {
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '' }), MAX_CONV_TURNS);
});
test('resolveMaxConvTurns — negative value falls back to default', () => {
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '-5' }), MAX_CONV_TURNS);
});
test('resolveMaxConvTurns — zero falls back to default (never unbounded)', () => {
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '0' }), MAX_CONV_TURNS);
});
test('resolveMaxConvTurns — unset falls back to default', () => {
assert.equal(resolveMaxConvTurns({}), MAX_CONV_TURNS);
});
test('resolveMaxConvTurns — valid positive integer string is honored', () => {
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '2' }), 2);
});
// A fractional value passed the `n <= 0` guard and only THEN floored, so 0.5 and
// 0.9 became 0 and the budget became 0 × 8 = 0 — every turn denied, the loop
// silently dead, while README.md and docs/architecture.md both promise a
// fallback of 3. The guard has to see the floored value, not the raw one.
test('resolveMaxConvTurns — a fractional value below 1 falls back to the default, never 0', () => {
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '0.5' }), MAX_CONV_TURNS);
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '0.9' }), MAX_CONV_TURNS);
});
test('resolveMaxConvTurns — a fractional value above 1 still floors (2.7 → 2)', () => {
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '2.7' }), 2);
});
test('resolveMaxConvTurns — Infinity is not a cap and falls back to the default', () => {
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: 'Infinity' }), MAX_CONV_TURNS);
});
test('allowTurn — a fractional cap below 1 cannot produce a budget of 0', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '0.5' };
const r = allowTurn({ runId: 'r-frac', dimension: 'd1', effort: 'high' }, { env });
assert.equal(r.ok, true, 'a budget of 0 would make the loop silently dead, not bounded');
assert.equal(r.budget, MAX_CONV_TURNS * MAX_TOTAL_DIMENSIONS);
});
});
// ---- pure-core unit coverage --------------------------------------------------
test('isStormEnabled — only the literal string "1" enables', () => {
assert.equal(isStormEnabled({ VOYAGE_STORM_ENABLED: '1' }), true);
assert.equal(isStormEnabled({ VOYAGE_STORM_ENABLED: 'true' }), false);
assert.equal(isStormEnabled({}), false);
});
test('resolveLedgerPath — falls back under ~/.claude/voyage when CLAUDE_PLUGIN_DATA is unset or empty', () => {
const expected = join('/home/x', '.claude', 'voyage', 'trekresearch-loop-ledger.jsonl');
assert.equal(resolveLedgerPath({ HOME: '/home/x' }), expected);
assert.equal(resolveLedgerPath({ CLAUDE_PLUGIN_DATA: '', HOME: '/home/x' }), expected);
});
test('resolveLedgerPath — joins CLAUDE_PLUGIN_DATA with the ledger filename', () => {
const p = resolveLedgerPath({ CLAUDE_PLUGIN_DATA: '/tmp/plugin-data' });
assert.equal(p, join('/tmp/plugin-data', 'trekresearch-loop-ledger.jsonl'));
});
test('allowTurn — missing runId or dimension denies with missing_args', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
const r1 = allowTurn({ dimension: 'd1', effort: 'high' }, { env });
assert.equal(r1.ok, false);
assert.equal(r1.reason, 'missing_args');
const r2 = allowTurn({ runId: 'r1', effort: 'high' }, { env });
assert.equal(r2.ok, false);
assert.equal(r2.reason, 'missing_args');
});
});
// ---- fail-closed on an unreadable ledger ------------------------------------
//
// The module's own header states that a budget control "must never silently
// grant unlimited turns just because the data dir is missing". The
// missing-DIRECTORY case already failed closed; the unreadable-FILE case
// returned 0 from countTurns and therefore re-granted the full budget on every
// call, unbounded — a fail-open in the same module that argues against one.
// A directory standing where the ledger file belongs reproduces it portably
// (EISDIR), with no chmod that a root test runner would ignore.
function withUnreadableLedger(fn) {
return withTmpDataDir((dir) => {
mkdirSync(join(dir, 'trekresearch-loop-ledger.jsonl'), { recursive: true });
return fn(dir);
});
}
test('readLedger — a missing ledger is 0 turns, not an error (turn 1 must be grantable)', () => {
withTmpDataDir((dir) => {
assert.equal(readLedger(join(dir, 'nope.jsonl'), 'r1').granted, 0);
});
});
test('readLedger — an unreadable ledger throws rather than reporting 0 turns spent', () => {
withUnreadableLedger((dir) => {
assert.throws(
() => readLedger(join(dir, 'trekresearch-loop-ledger.jsonl'), 'r1'),
/unreadable/i,
);
});
});
test('readLedger — malformed lines are skipped, well-formed ones for the run still count', () => {
withTmpDataDir((dir) => {
const p = join(dir, 'trekresearch-loop-ledger.jsonl');
writeFileSync(p, [
JSON.stringify({ runId: 'r1', dimension: 'd1' }),
'{ not json',
'',
JSON.stringify({ runId: 'other', dimension: 'd1' }),
JSON.stringify({ runId: 'r1', dimension: 'd2' }),
].join('\n') + '\n');
assert.equal(readLedger(p, 'r1').granted, 2);
});
});
test('allowTurn — an unreadable ledger DENIES the turn instead of granting a fresh budget', () => {
withUnreadableLedger((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
const r = allowTurn({ runId: 'r-unreadable', dimension: 'd1', effort: 'high' }, { env });
assert.equal(r.ok, false, 'a budget control that cannot count must not grant');
assert.match(r.reason, /ledger-read-failed/);
});
});
// ---- the bound holds under concurrency --------------------------------------
//
// countTurns-then-appendFileSync had no atomic claim, while the comment above
// allowTurn asserted "Append-only: never read-modify-write" and named the
// concurrent case (Phase 4.5/5 may spawn several agents in one message) as the
// reason. The decision path WAS read-then-write: N callers that all observe
// used == budget-1 all grant, and the bound is exceeded by N-1.
//
// Two of these tests are deterministic. They do not race anything — they assert
// the invariant the claim introduces: a slot that is already claimed is spent,
// even when the ledger has not caught up yet, which is exactly the state a
// mid-append competitor leaves behind. The third runs real processes.
function seedLedger(dir, runId, n) {
writeFileSync(
join(dir, 'trekresearch-loop-ledger.jsonl'),
Array.from({ length: n }, (_, i) =>
JSON.stringify({ ts: new Date().toISOString(), runId, dimension: `d${i}`, effort: 'high' }),
).join('\n') + (n ? '\n' : ''),
);
}
function seedClaims(dir, runId, slots) {
const claimDir = join(dir, 'trekresearch-loop-claims');
mkdirSync(claimDir, { recursive: true });
for (const s of slots) writeFileSync(join(claimDir, `${runId}-${s}.claim`), '');
}
function runShimAsync(args, env) {
return new Promise((resolve) => {
execFile(
process.execPath,
[SHIM, ...args],
{ encoding: 'utf-8', env: { PATH: process.env.PATH, ...env } },
(err, stdout) => resolve({ code: err ? (err.code ?? 1) : 0, out: stdout ?? '' }),
);
});
}
test('allowTurn — a slot already claimed is spent even when the ledger has not caught up', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
// budget = 1 × 8 = 8. Ledger shows 7 turns; a competitor already claimed
// slot 8 and has not appended yet. Counting the ledger alone says "one slot
// free" and grants a 9th turn overall — the breach this closes.
seedLedger(dir, 'r-race', 7);
seedClaims(dir, 'r-race', [1, 2, 3, 4, 5, 6, 7, 8]);
const r = allowTurn({ runId: 'r-race', dimension: 'd0', effort: 'high' }, { env });
assert.equal(r.ok, false, 'every slot up to the budget is claimed, so there is nothing to grant');
assert.equal(r.reason, 'budget_exhausted');
assert.equal(r.budget, 8);
});
});
test('allowTurn — it takes the first FREE slot and claims it, so a repeat call cannot retake it', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
seedLedger(dir, 'r-slot', 7);
seedClaims(dir, 'r-slot', [1, 2, 3, 4, 5, 6, 7]);
const first = allowTurn({ runId: 'r-slot', dimension: 'd0', effort: 'high' }, { env });
assert.equal(first.ok, true, 'slot 8 is free and must be grantable');
assert.equal(first.used, 8, 'used is the slot number, so it never double-counts a claimed slot');
assert.ok(
existsSync(join(dir, 'trekresearch-loop-claims', 'r-slot-8.claim')),
'the grant must leave the claim behind as the atomic record of the slot',
);
const second = allowTurn({ runId: 'r-slot', dimension: 'd0', effort: 'high' }, { env });
assert.equal(second.ok, false);
assert.equal(second.reason, 'budget_exhausted');
});
});
test('allowTurn — a different runId is unaffected by another run’s claims', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
seedClaims(dir, 'r-other', [1, 2, 3, 4, 5, 6, 7, 8]);
const r = allowTurn({ runId: 'r-mine', dimension: 'd0', effort: 'high' }, { env });
assert.equal(r.ok, true, 'claims are per-run; one run must not exhaust another');
});
});
test('allowTurn — parallel processes at the boundary cannot exceed the budget', async () => {
const dir = mkdtempSync(join(tmpdir(), 'research-loop-cap-par-'));
try {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
seedLedger(dir, 'r-par', 7); // budget 8 → exactly one turn left
const results = await Promise.all(
Array.from({ length: 6 }, (_, i) =>
runShimAsync(['--run-id', 'r-par', '--dimension', `p${i}`, '--effort', 'high'], env),
),
);
const winners = results.filter((r) => r.code === 0).length;
assert.equal(winners, 1, `exactly one of six concurrent callers may take the last slot, got ${winners}`);
// Granted turns, not raw lines: the five denied callers also record
// exhaustion tombstones, and a tombstone is not a turn.
const { granted } = readLedger(join(dir, 'trekresearch-loop-ledger.jsonl'), 'r-par');
assert.equal(granted, 8, `granted turns must never exceed the budget, got ${granted}`);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
// ---- the exhaustion tombstone -----------------------------------------------
//
// allowTurn appends BEFORE the turn runs, so during granted turn N the ledger
// holds N records. The hook denied at `used >= budget`, which blocked every
// tool call of the FINAL granted turn: the primitive granted B turns and the
// harness permitted B-1. Worse, an exhausted run then always terminated through
// an exit-2 tool denial instead of the graceful "cap exhausted" exit at
// commands/trekresearch.md, which is the only exit the prose teaches.
//
// Letting the hook allow at `used == budget` fixes the count but would leave it
// unable to catch the one case it exists for — the loop consults the gate, is
// denied, and issues the tool call anyway. So the denial itself becomes a
// record: a tombstone the hook can see.
test('allowTurn — denying for budget writes an exhaustion tombstone', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
for (let i = 0; i < 8; i++) {
assert.equal(allowTurn({ runId: 'r-tomb', dimension: `d${i}`, effort: 'high' }, { env }).ok, true);
}
const ledgerPath = join(dir, 'trekresearch-loop-ledger.jsonl');
assert.equal(readLedger(ledgerPath, 'r-tomb').exhausted, 0, 'no tombstone before the gate has denied anything');
const denied = allowTurn({ runId: 'r-tomb', dimension: 'd0', effort: 'high' }, { env });
assert.equal(denied.ok, false);
assert.equal(denied.reason, 'budget_exhausted');
const after = readLedger(ledgerPath, 'r-tomb');
assert.equal(after.exhausted, 1, 'the denial must leave a record the harness gate can read');
assert.equal(after.granted, 8, 'a tombstone is not a granted turn and must not count as one');
});
});
test('allowTurn — the tombstone is written once, not once per repeated denial', () => {
withTmpDataDir((dir) => {
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
for (let i = 0; i < 8; i++) allowTurn({ runId: 'r-once', dimension: `d${i}`, effort: 'high' }, { env });
for (let i = 0; i < 5; i++) allowTurn({ runId: 'r-once', dimension: 'd0', effort: 'high' }, { env });
const after = readLedger(join(dir, 'trekresearch-loop-ledger.jsonl'), 'r-once');
assert.equal(after.exhausted, 1, 'a hammered gate must not grow the ledger without bound');
assert.equal(after.granted, 8);
});
});
test('readLedger — a tombstone is reported separately and never as a granted turn', () => {
withTmpDataDir((dir) => {
const p = join(dir, 'trekresearch-loop-ledger.jsonl');
writeFileSync(p, [
JSON.stringify({ runId: 'r1', dimension: 'd1', slot: 1 }),
JSON.stringify({ runId: 'r1', exhausted: true }),
JSON.stringify({ runId: 'other', exhausted: true }),
].join('\n') + '\n');
const l = readLedger(p, 'r1');
assert.equal(l.granted, 1);
assert.equal(l.exhausted, 1);
assert.equal(readLedger(p, 'other').granted, 0, 'another run’s tombstone is not a granted turn either');
});
});
// ---- the discovery ceiling has a reader, not just a sentence ----------------
//
// The bounded-cost NFR asks for explicit ceilings on BOTH axes: max conversation
// turns and max discovered dimensions. The turn axis got MAX_CONV_TURNS, a
// ledger-backed reader and a PreToolUse enforcer. The discovery axis got a
// sentence in Phase 4.5 — "append candidates only while the whole list stays at
// or below maxDimensions: 8" — with no constant of its own, no reader, and no
// test that a run exceeding it is caught. That is the same shape as the
// brief_reviewer_iter_cap failure the operator decision warned about: a cap
// nothing reads.
//
// The ceiling is deliberately the SAME constant that sizes the turn budget. Two
// constants for one maxDimensions value is how the two drift apart.
test('checkDimensionCeiling — a list at the ceiling is accepted', () => {
const r = checkDimensionCeiling(Array.from({ length: MAX_TOTAL_DIMENSIONS }, (_, i) => `d${i}`));
assert.equal(r.ok, true);
assert.equal(r.count, MAX_TOTAL_DIMENSIONS);
assert.equal(r.ceiling, MAX_TOTAL_DIMENSIONS);
});
test('checkDimensionCeiling — one dimension over the ceiling is REJECTED', () => {
const r = checkDimensionCeiling(Array.from({ length: MAX_TOTAL_DIMENSIONS + 1 }, (_, i) => `d${i}`));
assert.equal(r.ok, false, 'a ceiling that accepts ceiling+1 is not a ceiling');
assert.equal(r.reason, 'ceiling_exceeded');
assert.equal(r.count, MAX_TOTAL_DIMENSIONS + 1);
});
test('checkDimensionCeiling — a plain count works as well as a list', () => {
assert.equal(checkDimensionCeiling(8).ok, true);
assert.equal(checkDimensionCeiling(9).ok, false);
assert.equal(checkDimensionCeiling('8').ok, true);
});
test('checkDimensionCeiling — an unreadable count is rejected, never waved through', () => {
for (const bad of ['abc', null, undefined, {}, -1, NaN]) {
const r = checkDimensionCeiling(bad);
assert.equal(r.ok, false, `${JSON.stringify(bad)} must not pass a cost ceiling`);
assert.equal(r.reason, 'unreadable_dimension_count');
}
});
test('checkDimensionCeiling — the ceiling is the same constant that sizes the turn budget', () => {
// Phase 4.5 and the Phase 5 budget must not be able to disagree about 8.
assert.equal(checkDimensionCeiling(0).ceiling, MAX_TOTAL_DIMENSIONS);
});
test('CLI shim — --check-dimensions exits 0 at the ceiling and 1 above it', () => {
const at = runShim(['--check-dimensions', String(MAX_TOTAL_DIMENSIONS)], {});
assert.equal(at.code, 0, `at the ceiling must exit 0; got ${at.out}`);
assert.equal(JSON.parse(at.out.trim()).ok, true);
const over = runShim(['--check-dimensions', String(MAX_TOTAL_DIMENSIONS + 1)], {});
assert.equal(over.code, 1, 'a run over the ceiling must be rejected by exit code, not by prose');
const parsed = JSON.parse(over.out.trim());
assert.equal(parsed.ok, false);
assert.equal(parsed.reason, 'ceiling_exceeded');
});
test('CLI shim — --check-dimensions needs no runId, effort or STORM flag', () => {
// It is a cost ceiling on Phase 4.5, which never calls the budget gate, so it
// must not inherit the budget gate's preconditions.
const r = runShimStripped(['--check-dimensions', '3'], {});
assert.equal(r.code, 0, `should not require --run-id/--effort; got ${r.out}`);
});
// ---- (g) shim contract --------------------------------------------------------
test('CLI shim — grants and exits 0 when enabled + high effort + budget available', () => {
withTmpDataDir((dir) => {
const r = runShim(
['--run-id', 'shim-1', '--dimension', 'd1', '--effort', 'high'],
{ CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' },
);
assert.equal(r.code, 0);
const parsed = JSON.parse(r.out.trim());
assert.equal(parsed.ok, true);
});
});
test('CLI shim — denies and exits 1 when disabled', () => {
const r = runShim(['--run-id', 'shim-2', '--dimension', 'd1', '--effort', 'high'], { VOYAGE_STORM_ENABLED: '0' });
assert.equal(r.code, 1);
const parsed = JSON.parse(r.out.trim());
assert.equal(parsed.ok, false);
assert.equal(parsed.reason, 'storm_disabled');
});
test('CLI shim — grants with CLAUDE_PLUGIN_DATA STRIPPED from the environment', () => {
withTmpDataDir((home) => {
const r = runShimStripped(
['--run-id', 'shim-stripped', '--dimension', 'd1', '--effort', 'high'],
{ VOYAGE_STORM_ENABLED: '1', HOME: home },
);
assert.equal(r.code, 0, `shim must grant without CLAUDE_PLUGIN_DATA; got: ${r.out}`);
const parsed = JSON.parse(r.out.trim());
assert.equal(parsed.ok, true);
assert.ok(existsSync(join(home, '.claude', 'voyage', 'trekresearch-loop-ledger.jsonl')));
});
});
test('CLI shim — missing required args exits 1 with usage reason', () => {
const r = runShim(['--run-id', 'shim-3']);
assert.equal(r.code, 1);
const parsed = JSON.parse(r.out.trim());
assert.equal(parsed.ok, false);
assert.match(parsed.reason, /usage:/);
});

11
tests/lib/skills-research-prose-pins.test.mjs
View file

@ -85,7 +85,7 @@ test('researching-sources: "Enforced by" names the structure-only check and says
assert.match(enforced, /research-validator\.mjs/);
assert.match(enforced, /structure/i);
assert.match(enforced, /query-privacy-gate\.mjs/);
assert.match(enforced, /off by default/i, 'the privacy gate runs only in the default-off steps; say so');
assert.match(enforced, /not called by any command/i, 'no command calls the privacy gate now; say so');
assert.match(enforced, /\*\*prose only\*\*/);
});
@ -187,10 +187,11 @@ test('one confidence scale: the skill table and /trekresearch triangulation defi
assert.deepEqual(cmdRows, skillRows, 'Phase 6 must define the ratings word for word as the skill does');
});
test('one confidence scale: the deep-research adapter and the brief template accept the same four ratings', () => {
const cmd = read('commands/trekresearch.md');
assert.match(cmd, /`### \{Dimension\} -- Confidence: \{high\|medium\|low\|contradictory\}`/);
assert.match(read('templates/research-brief-template.md'), /Confidence: \{high \| medium \| low \| contradictory\}/);
test('one confidence scale: the brief template dimension heading accepts the same four ratings', () => {
assert.match(
read('templates/research-brief-template.md'),
/^### \{Dimension Name\} -- Confidence: \{high \| medium \| low \| contradictory\}$/m,
);
});
// --- the eval cases ---------------------------------------------------------

324
tests/scripts/storm-measure.test.mjs
View file

@ -1,324 +0,0 @@
// tests/scripts/storm-measure.test.mjs
// Step 11 — the STORM adoption gate's deterministic accounting core.
//
// The gate decides ONE thing: does the bounded Phase 5 loop buy enough extra
// source/coverage breadth to be worth flipping VOYAGE_STORM_ENABLED on by
// default. Thresholds are pre-registered in docs/storm-measurement.md BEFORE
// any measurement run, so this file pins the arithmetic that turns a
// trekresearch-stats.jsonl into a verdict — not the verdict itself.
//
// Two properties carry the gate's honesty:
// - runs with empty_turns > 0 are EXCLUDED from the gain and COUNTED, so
// adoption is never decided on a broken denominator, and
// - a stats file with no `effort` field is a loud error, never a silently
// empty group that reads as "no gain".
//
// Pure core, no fixtures on disk.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import {
median,
parseStats,
partitionEligible,
measure,
decideVerdict,
activationCheck,
ADOPT_THRESHOLD,
DECLINE_THRESHOLD,
} from '../../scripts/storm-measure.mjs';
// ---------------------------------------------------------------------------
// helpers — synthetic JSONL, one object per line, exactly as the orchestrator emits
// ---------------------------------------------------------------------------
function run({ effort, unique_sources, dimensions, dimensions_baseline, empty_turns = 0, conv_turns = 0 }) {
return JSON.stringify({
ts: '2026-08-12T00:00:00.000Z',
question: 'q',
mode: 'full',
scope: 'both',
engine: 'swarm',
effort,
unique_sources,
dimensions,
dimensions_baseline,
conv_turns,
empty_turns,
});
}
function jsonl(...lines) {
return lines.join('\n') + '\n';
}
// A control arm at 10 sources / 5 dimensions, and a treatment arm at 13
// sources / 8 dimensions: +30.0% sources, +60.0% dimensions.
const STANDARD = [
run({ effort: 'standard', unique_sources: 10, dimensions: 5, dimensions_baseline: 5 }),
run({ effort: 'standard', unique_sources: 10, dimensions: 5, dimensions_baseline: 5 }),
run({ effort: 'standard', unique_sources: 10, dimensions: 5, dimensions_baseline: 5 }),
];
const HIGH = [
run({ effort: 'high', unique_sources: 13, dimensions: 8, dimensions_baseline: 5, conv_turns: 3 }),
run({ effort: 'high', unique_sources: 13, dimensions: 8, dimensions_baseline: 5, conv_turns: 3 }),
run({ effort: 'high', unique_sources: 13, dimensions: 8, dimensions_baseline: 5, conv_turns: 3 }),
];
// ---------------------------------------------------------------------------
// median
// ---------------------------------------------------------------------------
test('median: odd, even, single', () => {
assert.equal(median([3, 1, 2]), 2);
assert.equal(median([1, 2, 3, 4]), 2.5);
assert.equal(median([7]), 7);
});
test('median: empty list is null, never 0 — 0 would read as a real measurement', () => {
assert.equal(median([]), null);
});
test('median does not mutate its input', () => {
const xs = [3, 1, 2];
median(xs);
assert.deepEqual(xs, [3, 1, 2]);
});
// ---------------------------------------------------------------------------
// parseStats — the loud-error requirement
// ---------------------------------------------------------------------------
test('parseStats: a file with no effort field throws, it does not yield empty groups', () => {
const noEffort = jsonl(
JSON.stringify({ ts: 'x', unique_sources: 10, dimensions: 5, dimensions_baseline: 5 }),
JSON.stringify({ ts: 'y', unique_sources: 11, dimensions: 5, dimensions_baseline: 5 }),
);
assert.throws(() => parseStats(noEffort), /effort/i);
});
test('parseStats: skips blank and malformed lines but keeps the good ones', () => {
const text = jsonl(STANDARD[0], '', 'not json', HIGH[0]);
const { records, malformed } = parseStats(text);
assert.equal(records.length, 2);
assert.equal(malformed, 1);
});
test('parseStats: an empty file throws rather than reporting a zero-gain verdict', () => {
assert.throws(() => parseStats(''), /no records/i);
});
// ---------------------------------------------------------------------------
// exclusion of broken runs
// ---------------------------------------------------------------------------
test('partitionEligible: runs with empty_turns > 0 are excluded and counted', () => {
const { records } = parseStats(jsonl(
...HIGH,
run({ effort: 'high', unique_sources: 99, dimensions: 8, dimensions_baseline: 5, empty_turns: 2 }),
));
const { eligible, excluded } = partitionEligible(records);
assert.equal(eligible.length, 3);
assert.equal(excluded, 1);
});
test('partitionEligible: empty_turns === 0 is eligible; a missing field counts as 0', () => {
const { records } = parseStats(jsonl(
STANDARD[0],
JSON.stringify({ ts: 'z', effort: 'standard', unique_sources: 10, dimensions: 5, dimensions_baseline: 5 }),
));
const { eligible, excluded } = partitionEligible(records);
assert.equal(eligible.length, 2);
assert.equal(excluded, 0);
});
// A malformed empty_turns used to land in the ELIGIBLE arm: Number('many') is
// NaN, NaN fails the isFinite test, and the `else` branch pushed it in. The
// exclusion is one of the two properties carrying this gate's honesty, so a
// garbage value silently re-entering the denominator defeats it — and it does so
// in the direction that flatters adoption, since the run that broke is the run
// whose numbers are least trustworthy.
test('partitionEligible: a non-numeric empty_turns is EXCLUDED, never silently eligible', () => {
const { records } = parseStats(jsonl(
STANDARD[0],
JSON.stringify({
ts: 'm', effort: 'high', unique_sources: 999,
dimensions: 8, dimensions_baseline: 5, empty_turns: 'many',
}),
));
const { eligible, excluded } = partitionEligible(records);
assert.equal(excluded, 1, 'a value that cannot be read as a turn count is not evidence of zero empty turns');
assert.equal(eligible.length, 1);
});
test('partitionEligible: an unparsable empty_turns cannot move the median either', () => {
const m = measure(parseStats(jsonl(
...STANDARD,
...HIGH,
JSON.stringify({
ts: 'm', effort: 'high', unique_sources: 900,
dimensions: 8, dimensions_baseline: 5, empty_turns: {},
}),
)).records);
assert.equal(m.excluded, 1);
assert.equal(m.sources.treatment, 13, 'the 900-source malformed run must not reach the median');
});
// ---------------------------------------------------------------------------
// measure — the known-answer test
// ---------------------------------------------------------------------------
test('measure: median gains on synthetic runs give the known answer', () => {
const m = measure(parseStats(jsonl(...STANDARD, ...HIGH)).records);
assert.equal(m.control.n, 3);
assert.equal(m.treatment.n, 3);
assert.equal(m.sources.control, 10);
assert.equal(m.sources.treatment, 13);
assert.ok(Math.abs(m.sources.gain - 0.30) < 1e-9, `sources gain ${m.sources.gain}`);
// (8 - 5) / 5 = 0.60 within each treatment run.
assert.ok(Math.abs(m.dimensions.gain - 0.60) < 1e-9, `dimensions gain ${m.dimensions.gain}`);
});
test('measure: an excluded run cannot move the median', () => {
const withBroken = jsonl(
...STANDARD,
...HIGH,
run({ effort: 'high', unique_sources: 900, dimensions: 8, dimensions_baseline: 5, empty_turns: 1 }),
);
const m = measure(parseStats(withBroken).records);
assert.equal(m.excluded, 1);
assert.equal(m.sources.treatment, 13, 'the 900-source broken run must not reach the median');
assert.ok(Math.abs(m.sources.gain - 0.30) < 1e-9);
});
test('measure: reports null gain (not 0) when an arm has no eligible runs', () => {
const m = measure(parseStats(jsonl(...HIGH)).records);
assert.equal(m.control.n, 0);
assert.equal(m.sources.gain, null);
assert.equal(m.verdict, 'insufficient-data');
});
// ---------------------------------------------------------------------------
// verdict mapping — both sides of both thresholds
// ---------------------------------------------------------------------------
test('decideVerdict: at and above the adopt threshold', () => {
assert.equal(decideVerdict(ADOPT_THRESHOLD, ADOPT_THRESHOLD), 'adopt');
assert.equal(decideVerdict(0.55, 0.44), 'adopt');
});
test('decideVerdict: both metrics between the bars is inconclusive, not adopt', () => {
assert.equal(decideVerdict(ADOPT_THRESHOLD - 0.0001, 0.2), 'inconclusive');
});
test('decideVerdict: below the decline threshold on both metrics declines', () => {
assert.equal(decideVerdict(0.14, 0.05), 'decline');
assert.equal(decideVerdict(DECLINE_THRESHOLD - 0.0001, 0), 'decline');
});
test('decideVerdict: at the decline threshold is inconclusive, not decline', () => {
assert.equal(decideVerdict(DECLINE_THRESHOLD, DECLINE_THRESHOLD), 'inconclusive');
});
// The brief pre-registers "median forbedring >= 30 % på (a) eller (b) → adopt.
// < 15 % → decline." — OR on both sides, with adopt evaluated first.
test('decideVerdict: adopt needs EITHER metric — one strong metric carries a weak one', () => {
assert.equal(decideVerdict(0.90, 0.10), 'adopt');
assert.equal(decideVerdict(0.10, 0.90), 'adopt');
});
test('decideVerdict: either metric below the decline bar declines', () => {
assert.equal(decideVerdict(0.02, 0.20), 'decline');
assert.equal(decideVerdict(0.20, 0.02), 'decline');
});
test('decideVerdict: adopt outranks decline when one metric clears and the other is under the decline bar', () => {
assert.equal(decideVerdict(0.90, 0.10), 'adopt');
});
test('decideVerdict: a null gain is insufficient data, never a decline', () => {
assert.equal(decideVerdict(null, 0.4), 'insufficient-data');
assert.equal(decideVerdict(0.4, null), 'insufficient-data');
});
// ---------------------------------------------------------------------------
// activationCheck — BOTH halves of the SC, not just the count
//
// The SC requires that an `effort: high` run discovers at least one dimension
// AND that the dimension list in the output brief is a TRUE SUPERSET of the
// interview-derived ones. activationCheck only computed dimensions -
// dimensions_baseline >= 1, so a run that replaced two interview dimensions with
// three discovered ones passed the check while violating the SC's second half.
// Supersetness was asserted only by Phase 4.5's prose contract that discovery
// appends; nothing read it.
//
// The stats record carries counts, not names — names are free prose and
// the record denies prose by omission — so the run attests membership
// with a low-cardinality boolean instead, and the gate refuses to call
// activation OK without it.
// ---------------------------------------------------------------------------
function highRun(over = {}) {
return JSON.stringify({
ts: '2026-08-12T00:00:00.000Z',
effort: 'high',
unique_sources: 13,
dimensions: 8,
dimensions_baseline: 5,
conv_turns: 3,
empty_turns: 0,
dimensions_baseline_preserved: true,
...over,
});
}
test('activationCheck: discovery plus a preserved baseline is activation', () => {
const r = activationCheck(parseStats(jsonl(highRun())).records);
assert.equal(r.ok, true);
assert.equal(r.discovered_dimensions, 3);
assert.equal(r.dimensions_baseline_preserved, true);
});
test('activationCheck: a REPLACED baseline is not activation, however many were discovered', () => {
const r = activationCheck(parseStats(jsonl(
highRun({ dimensions: 8, dimensions_baseline: 5, dimensions_baseline_preserved: false }),
)).records);
assert.equal(r.ok, false, 'a count delta of +3 says nothing about which dimensions survived');
assert.match(r.reason, /superset|baseline/i);
});
test('activationCheck: a run that does not attest baseline membership cannot pass', () => {
const rec = JSON.parse(highRun());
delete rec.dimensions_baseline_preserved;
const r = activationCheck(parseStats(jsonl(JSON.stringify(rec))).records);
assert.equal(r.ok, false, 'an absent attestation is not an attestation');
assert.match(r.reason, /dimensions_baseline_preserved/);
});
test('activationCheck: no discovery is still not activation even with the baseline preserved', () => {
const r = activationCheck(parseStats(jsonl(
highRun({ dimensions: 5, dimensions_baseline: 5 }),
)).records);
assert.equal(r.ok, false);
assert.equal(r.discovered_dimensions, 0);
});
test('activationCheck: no effort:high run at all is reported as such', () => {
const r = activationCheck(parseStats(jsonl(...STANDARD)).records);
assert.equal(r.ok, false);
assert.match(r.reason, /high/);
});
// ---------------------------------------------------------------------------
// pre-registration — the thresholds are the doc's, and the doc is committed first
// ---------------------------------------------------------------------------
test('thresholds are the pre-registered 30% / 15%', () => {
assert.equal(ADOPT_THRESHOLD, 0.30);
assert.equal(DECLINE_THRESHOLD, 0.15);
});
// The human-readable summary is the only form of the rule most readers will
// ever see. It said "on BOTH" on both sides while decideVerdict evaluated OR —
// so the report described a stricter gate than the one that produced the verdict
// printed one line below it.
test('the printed threshold line states the OR rule that decideVerdict actually applies', () => {
const src = readFileSync(new URL('../../scripts/storm-measure.mjs', import.meta.url), 'utf-8');
const line = src.split('\n').find((l) => l.includes('thresholds: adopt'));
assert.ok(line, 'the summary must still print its threshold rule');
assert.doesNotMatch(line, /on BOTH/, 'the rule is OR on both sides — printing BOTH misstates the gate');
assert.match(line, /EITHER/);
});