chore(cleanup): R-b R2 — STORM loop, cap hook and the --engine research choice removed
/trekresearch loses Phase 4.5, the bounded Phase 5 loop (Phase 5 keeps its single pass of at most 2 follow-ups), the --engine flag, the deep-research adapter and its output self-check. hooks.json loses the pre-agent-cap PreToolUse entry (README: 9 hooks = 9 scripts). The STORM env vars leave the docs; query-privacy-gate.mjs stays, called by no command, and says so. Files removed: 13 (4 docs, 1 hook, 1 lib module, 1 script, 5 tests, 1 fixture). Files edited: 17. Tests: 2600 (2598 pass / 0 fail / 2 skipped) before, 2467 (2466 pass / 0 fail / 1 skipped, R3 pending) after. Durable check: group R2 green and no longer pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
21480e8627
commit
32621dcb63
SSH key fingerprint:
SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
30 changed files with 34 additions and 4070 deletions
|
|
@ -12,8 +12,7 @@ Imported from `CLAUDE.md` via pointer.
|
|||
- `lib/review/{verdict-gate,engine-findings,recurrence}.mjs` — the review gate (K7: 0 ALLOW, or WARN with every counting MAJOR answered · 1 BLOCK · 2 never green), Claude Code `/code-review` + `/security-review` findings → reviewer payload, recurring finding classes → operator-approved project-memory lines (review_version 1.2); `lib/review/fill-finding-ids.mjs` computes the coordinator's `@@ID_<n>@@` placeholders over `(file, line, rule_key)`; `lib/validators/brief-review-stamp.mjs` lets `/trekplan` skip a brief review `/trekbrief` already ran on the same text
|
||||
- `lib/validators/{brief,research,plan,progress,session-state}-validator.mjs` — schema validators with CLI shims (`node lib/validators/X.mjs --json <path>`)
|
||||
- `lib/validators/architecture-discovery.mjs` — drift-WARN external-contract discovery for `architecture/overview.md`
|
||||
- `lib/util/research-loop-cap.mjs` — stateful, **default-off** turn budget for the `/trekresearch` bounded conversation loop. `allowTurn()` derives the used-turn count from its own append-only JSONL ledger; it never asks the caller how many turns it has spent, because a cap that does is not a cap. Each grant first claims a turn **slot** with `O_EXCL` under `trekresearch-loop-claims/`, so the bound survives several callers deciding at once — counting the ledger and then appending is read-then-write, and Phase 4.5/5 can spawn several agents in one message. Budget = `TREKRESEARCH_MAX_CONV_TURNS` (default `3`, invalid values fall back to `3`) × `maxDimensions` (8, `MAX_TOTAL_DIMENSIONS`). **Default-off:** grants 0 unless `VOYAGE_STORM_ENABLED=1`, which is also the second condition on Phase 4.5's skip-guard (that phase does not call this module): unset, **both** STORM phases are inert. `resolveDataRoot()` is the single root for everything the loop writes — `CLAUDE_PLUGIN_DATA` when the harness sets it, `~/.claude/voyage` when it does not (it is empty in the Bash tool's process env, which is where the loop actually runs); the cap hook resolves through the same function, so writer and reader cannot disagree. A ledger that cannot be **written** denies the turn, and one that exists but cannot be **read** denies it too — only `ENOENT` counts as zero turns spent, that being the legitimate first-turn state (fail-closed). The exported `readLedger()` is the single counting rule; the cap hook calls it rather than keeping a private copy. `checkDimensionCeiling()` is the reader for the OTHER axis of the bounded-cost NFR — the size of the whole dimension list after Phase 4.5 discovery — against the same `MAX_TOTAL_DIMENSIONS`, so the two axes cannot enforce different numbers for one ceiling; an unreadable count is rejected, not waved through. CLI shim, two modes: `--run-id ID --dimension D --effort E` (budget gate) and `--check-dimensions N` (ceiling, no run id/effort/flag required since Phase 4.5 never calls the budget gate)
|
||||
- `lib/validators/query-privacy-gate.mjs` — gates **every** outbound research query before it leaves the machine; the hard-block tier (secret-shaped strings) is not operator-overridable, so a query that trips it must be reformulated rather than forced through. CLI shim: `node lib/validators/query-privacy-gate.mjs "<query>"`
|
||||
- `lib/validators/query-privacy-gate.mjs` — gates an outbound research query before it leaves the machine (no command calls it now: its callers were the removed STORM steps); the hard-block tier (secret-shaped strings) is not operator-overridable, so a query that trips it must be reformulated rather than forced through. CLI shim: `node lib/validators/query-privacy-gate.mjs "<query>"`
|
||||
|
||||
Wiring points (replaces previous prose-grep instructions):
|
||||
- `/trekbrief` Phase 4g → `brief-validator` (post-write sanity check)
|
||||
|
|
@ -31,8 +30,6 @@ Doc-consistency test at `tests/lib/doc-consistency.test.mjs` pins agent-table co
|
|||
|
||||
`hooks/scripts/session-title.mjs` (UserPromptSubmit, CC v2.1.94+) sets `sessionTitle` to `voyage:<command>:<slug>` for voyage-command invocations. Helps parallel sessions identify themselves in process lists.
|
||||
|
||||
`hooks/scripts/pre-agent-cap.mjs` (PreToolUse on `WebSearch|WebFetch|Task`) enforces the `/trekresearch` Phase 5 loop bound in the harness, so the cap is not merely prose the model is asked to obey. It counts spent turns read-only from the append-only ledger `research-loop-cap.mjs` writes, through that module's own exported `readLedger()`. It denies (exit 2) once the budget **gate has denied a turn** — the primitive records its denials as tombstones, and the tombstone is the boundary rather than the count, because `allowTurn()` appends before the turn runs and so the final granted turn already shows `budget` records. A ledger showing more granted turns than the budget denies too, as a backstop. Being in scope but unable to count the ledger also denies: a budget control that cannot count must not grant. Scope key = `session_id` + a marker file only the loop writes (`<resolveDataRoot()>/trekresearch-loop-scope/<session_id>.json`, the same root the ledger uses); without a marker the hook allows unconditionally, which is what keeps a globally-wired `PreToolUse` hook from over-blocking unrelated sessions. Stale markers auto-reset on a TTL and `VOYAGE_DISABLE_CAP_HOOK=1` is the kill switch. Defence in depth only — `lib/util/research-loop-cap.mjs` must stay correct if the hook stops firing (see `docs/spike-pretooluse-subagent-reach.md`).
|
||||
|
||||
`hooks/scripts/post-compact-flush.mjs` (PostCompact event, v3.4.0) re-injects `.session-state.local.json` after context compaction so multi-session work survives a compaction boundary. Companion to `pre-compact-flush.mjs` (which writes the state file before compaction); together they form the rehydrate cycle that keeps `/trekcontinue` reliable across long-running multi-session work.
|
||||
|
||||
## Architecture
|
||||
|
|
@ -94,7 +91,7 @@ Which native Claude Code primitive each pipeline step runs on today, and the alt
|
|||
| **execute** | Inline step loop; multi-session via `git worktree` + session orders the user runs, wave by wave; deterministic manifest audit | CC `TaskCreate`/`TodoWrite` for progress/resume (insufficient — carries no step status / attempts / SHA / drift → own typed `progress.json` contract) |
|
||||
| **review** | Inline parallel reviewers (no cross-feed) → `review-coordinator` Judge | **Workflow** substrate for Phase 5–6 (bake-off POSITIVE: +4.4 % tokens / +54 % wall-time → shipped opt-in, since removed; wholesale substrate swap declined) |
|
||||
| **continue** | Inline reads `.session-state.local.json` → zero-confirm resume | CC `--resume` (transcript replay, not typed work-state → insufficient) |
|
||||
| **cross-cutting** | 6 hook scripts: `pre-bash` + `pre-write` guards, `pre-agent-cap` loop-bound enforcement, `session-title`, `pre-`/`post-compact` flush | — |
|
||||
| **cross-cutting** | 5 hook scripts: `pre-bash` + `pre-write` guards, `session-title`, `pre-`/`post-compact` flush | — |
|
||||
|
||||
¹ MCP per research agent: `docs-researcher` → Microsoft Learn + Tavily · `community-`/`security-`/`contrarian-researcher` → Tavily (+ WebSearch/WebFetch). Graceful degradation when an MCP server is absent.
|
||||
|
||||
|
|
|
|||
|
|
@ -27,26 +27,9 @@ Always interactive. Phase 3 puts the task in a size class (`ceremony:`, K1; heav
|
|||
| `--gates [true\|false\|closed\|open]` | (v3.4.0) Autonomy flag: bare, `true` or `closed` = on; `false` or `open` = off; any other next token is not its value. See `## Autonomy mode` in `docs/operations.md`. |
|
||||
| `--min-brief-version <ver>` | (S18) Warn — never block — if an attached `--project` brief declares a version below `<ver>` (e.g. `2.2`), i.e. sidesteps framing enforcement |
|
||||
| `--profile <name>` | (v4.1.0) Model profile for the research phase. |
|
||||
| `--engine {swarm\|deep-research}` | (deep-research-engine) Opt-in external-research engine; `deep-research` delegates the external phase to Claude Code's built-in `/deep-research` workflow, which only works on `2.1.154 <= CC < 2.1.218` — from **2.1.218** `/deep-research` is operator-invoked only (the Skill tool refuses with `disable-model-invocation`), so the engine always falls back to `swarm`. Never hard-fails. Default `swarm`. |
|
||||
|
||||
Flags combine: `--project <dir> --local`, `--external --quick`.
|
||||
|
||||
### Bounded conversation loop (Phase 4.5 + Phase 5) — env-vars
|
||||
|
||||
Dimension discovery and the multi-turn follow-up loop are **default-off** and
|
||||
have no flag; they are environment-gated, because turning them on costs turns.
|
||||
They run only at `effort: high` (resolved from the brief's `phase_signals`).
|
||||
|
||||
| Env-var | Default | Behavior |
|
||||
|---------|---------|----------|
|
||||
| `VOYAGE_STORM_ENABLED` | _(unset — default-off)_ | `=1` grants the Phase 5 loop a non-zero turn budget and is the second condition on Phase 4.5's skip-guard. Unset, `research-loop-cap.mjs` grants 0 turns and **both** phases are inert: doing nothing keeps the whole mechanism off. |
|
||||
| `TREKRESEARCH_MAX_CONV_TURNS` | `3` | Max turns per under-illuminated dimension. Budget = this × `maxDimensions` (8, `MAX_TOTAL_DIMENSIONS` in `lib/util/research-loop-cap.mjs`). Empty, non-numeric, zero, negative, `Infinity`, or any fraction that floors below `1` (`0.5`, `0.9`) fall back to `3` — never to unbounded, and never to `0`. A fraction at or above `1` floors (`2.7` → `2`). |
|
||||
| `VOYAGE_DISABLE_CAP_HOOK` | _(unset)_ | `=1` disables `hooks/scripts/pre-agent-cap.mjs`, the `PreToolUse` enforcement of the turn budget. The cap primitive still applies; only the second gate is switched off. |
|
||||
|
||||
Adoption of the loop as a default is gated on a pre-registered measurement —
|
||||
protocol, thresholds, and the exact commands in
|
||||
[`docs/storm-measurement.md`](storm-measurement.md).
|
||||
|
||||
## /trekplan modes
|
||||
|
||||
| Flag | Behavior |
|
||||
|
|
|
|||
|
|
@ -1,62 +0,0 @@
|
|||
---
|
||||
type: trekbrief
|
||||
brief_version: "2.2"
|
||||
task: "Opt-in /deep-research-motor for /trekresearch ekstern-fase"
|
||||
slug: deep-research-engine
|
||||
framing: refine
|
||||
status: ready
|
||||
brief_quality: complete
|
||||
research_topics: 3
|
||||
research_status: complete
|
||||
phase_signals_partial: true
|
||||
---
|
||||
|
||||
# Brief — Opt-in `/deep-research`-motor for `/trekresearch`
|
||||
|
||||
## TL;DR
|
||||
- `--engine {swarm|deep-research}` på `/trekresearch` ekstern-fase; `swarm` default (uendret oppførsel).
|
||||
- `deep-research` delegerer ekstern research til Claude Codes innebygde workflow og adapterer inn i research-brief-skjemaet.
|
||||
- Lokal analyse, triangulering og H2-output (`research/NN-*.md`) er motor-uavhengig.
|
||||
- Feature-detekteres med auto-fallback til `swarm` — aldri hard feil.
|
||||
- Surface-only: `commands/` + `agents/`, ingen nye `lib/`-avhengigheter.
|
||||
|
||||
## Intent
|
||||
`/trekresearch` eier i dag hele den eksterne research-fasen selv (Tavily / MS-Learn / Gemini-sverm). Det er portabelt, men du vedlikeholder fan-out, kryssjekk og syntese selv. Anthropic sin innebygde `/deep-research` vedlikeholder fan-out, adversariell påstandsverifisering, sitatfiltrering og websøk for deg. Lar vi operatøren *velge* `/deep-research` for den eksterne fasen, får brukeren en vedlikeholdsfri "turbo" når den er tilgjengelig — uten at Voyage mister sin egen lokal-analyse, triangulering eller H2-kontrakt.
|
||||
|
||||
## Goal
|
||||
`/trekresearch` får et `--engine {swarm|deep-research}`-valg på den eksterne fasen. `swarm` er default (dagens oppførsel). `deep-research` delegerer den eksterne fasen til den innebygde workflowen og adapterer resultatet inn i research-brief-skjemaet. Lokal analyse, triangulering og H2-output (`research/NN-*.md`) er uendret uansett motor.
|
||||
|
||||
## Non-Goals
|
||||
- Bygge om `/trekresearch` på et eget dynamic-workflow-substrat (vurdert og forkastet, Δ≈0).
|
||||
- Multi-provider (Perplexity / Google) — hører til en senere egen `workflow`-motor, ikke denne.
|
||||
- Endre den lokale svermen, handover-kontraktene eller validator-skjemaene.
|
||||
- Gjøre `/deep-research` til default.
|
||||
|
||||
## Constraints / NFR
|
||||
- `/deep-research` krever Claude Code v2.1.154+ og må være på (Pro: via `/config`). Må feature-detekteres med auto-fallback til `swarm` — aldri hard feil.
|
||||
- Output må passere `research-validator.mjs` (confidence ∈ [0,1], dimensions ≥ 1, påkrevde body-seksjoner).
|
||||
- Ingen nye `lib/`-avhengigheter; overflate-endring i `commands/` + `agents/`.
|
||||
- Doc-consistency: oppdater command-tabellen i `CLAUDE.md` + `README.md`, kjør `npm test`.
|
||||
|
||||
## Success Criteria
|
||||
1. `/trekresearch --external --engine swarm "<q>"` gir identisk oppførsel som før (regresjon grønn).
|
||||
2. `/trekresearch --external --engine deep-research "<q>"` produserer en `research/NN-*.md` der `research-validator.mjs --json` rapporterer `valid: true`.
|
||||
3. Med dynamic workflows avskrudd faller `--engine deep-research` automatisk tilbake til `swarm` og logger valget — ingen exception.
|
||||
4. `npm test` (inkl. doc-consistency) er grønn.
|
||||
|
||||
## Research Plan
|
||||
1. **Programmatisk trigging av `/deep-research`** — Kan en plugin-kommando starte den innebygde workflowen og fange artefakten, eller må `commands/trekresearch.md` instruere Claude til å kjøre den? Scope: external · Konfidens: høy · Kost: lav.
|
||||
`/trekresearch --external "Kan en Claude Code plugin-kommando programmatisk starte /deep-research og hente rapporten?"`
|
||||
2. **Output-adapter** — Hvordan mappe `/deep-research` sin siterte rapport til research-brief-skjemaet (dimensjoner, confidence, sitater)? Scope: both · Konfidens: høy · Kost: lav.
|
||||
`/trekresearch --project <dir> --local "Hva krever research-validator.mjs av research/NN-*.md?"`
|
||||
3. **Feature-deteksjon + fallback** — Hvordan oppdage om workflows er på, og hvor i fasen fallback-grenen bør sitte? Scope: local · Konfidens: middels · Kost: lav.
|
||||
|
||||
> **Research-status (2026-06-30, operatør-beslutning «option A»):** Topic 1 (eneste ekte eksterne ukjente) er undersøkt → `docs/deep-research-engine-research.md` (validator-grønn). Funn: `/deep-research` er en innebygd **dynamic workflow** (ikke skill) → trigging KUN via prosa-instruksjon, output **inline i kontekst** (ingen on-disk-artefakt) → motoren må være instruksjons-basert + in-context transform. **SC3 er korrekt som skrevet** (`/deep-research` ER en dynamic workflow). Topic 2 (validator-skjema: `type/created/question` + `## Executive Summary`/`## Dimensions`, `dimensions ≥ 1`) og topic 3 (fallback-plassering) er lokale kode-spørsmål reklassifisert til `/trekplan`-utforskning. `research_status: complete` reflekterer denne beslutningen.
|
||||
|
||||
## Open Questions / Assumptions
|
||||
- Antar at `/deep-research`-rapporten kan reduseres til ≥ 1 dimensjon med per-påstand-sitater uten å bryte trianguleringen. Verifiseres i topic 2.
|
||||
- Uavklart om delegering skjer via instruksjon (trygt) eller programmatisk API (raskere) — topic 1 avgjør.
|
||||
|
||||
## Prior Attempts
|
||||
- Dvalende synthesis-agent bygget, målt (Δ≈0) og forkastet → samme lærdom gjelder substrat-bytte: mål før du adopterer.
|
||||
- Workflow-substratet er allerede dokumentert som vurdert-og-valgt-bort i `docs/architecture.md` §Primitives per step.
|
||||
|
|
@ -1,213 +0,0 @@
|
|||
---
|
||||
type: trekresearch-brief
|
||||
created: 2026-06-30
|
||||
question: "Can a Claude Code plugin command programmatically trigger the built-in /deep-research workflow and capture its report, or must it instruct Claude to run it?"
|
||||
confidence: 0.85
|
||||
dimensions: 4
|
||||
mcp_servers_used: []
|
||||
local_agents_used: [claude-code-guide]
|
||||
external_agents_used: []
|
||||
slug: deep-research-engine
|
||||
feeds_brief: docs/deep-research-engine-brief.md
|
||||
research_topic: 1
|
||||
---
|
||||
|
||||
# Research — Programmatic trigging of `/deep-research` from a plugin command
|
||||
|
||||
> Targeted single-topic research for the `deep-research-engine` brief (Topic 1 of
|
||||
> the brief's Research Plan). Topics 2 & 3 are local Voyage-code questions folded
|
||||
> into `/trekplan` exploration; only Topic 1 was a genuine external unknown.
|
||||
> Method: `claude-code-guide` agent (Anthropic docs + CHANGELOG, cited) +
|
||||
> direct inspection of this machine (CC 2.1.196 binary, a real local `/deep-research`
|
||||
> run). Not run through the `/trekresearch` swarm — see brief reconcile note.
|
||||
|
||||
## Research Question
|
||||
|
||||
Can a Claude Code **plugin slash-command** (markdown under `commands/`)
|
||||
**programmatically** start the built-in `/deep-research` workflow and **capture its
|
||||
report artifact** for adaptation into the research-brief schema — or must the command
|
||||
instead **instruct** Claude (in prose) to run `/deep-research` and then transform the
|
||||
in-context result?
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Programmatic trigger + file-based capture is **not feasible**: `/deep-research` is a
|
||||
built-in **dynamic workflow** (not a skill), deliberately outside the Skill-tool
|
||||
allowlist, and it returns its report **inline into conversation context with no
|
||||
documented on-disk report artifact**. The **only reliable path is instruction-based
|
||||
delegation** — the command's prose tells Claude to run `/deep-research <q>`, then
|
||||
transforms the in-context report in the same turn. Confidence **high** (Anthropic docs +
|
||||
CHANGELOG + local run), with one residual gap: there is no positive "is it enabled?"
|
||||
probe, so feature-detection must lean on the documented *disable* switches + version
|
||||
floor + a `swarm` default.
|
||||
|
||||
## Dimensions
|
||||
|
||||
### 1. Origin + gating -- Confidence: high
|
||||
|
||||
**External findings:**
|
||||
- `/deep-research` is a **built-in dynamic workflow**, not a command and not a bundled
|
||||
skill. `commands.md` marks the `/deep-research <question>` row as **[Workflow]**;
|
||||
`workflows.md`: "Claude Code includes `/deep-research` as a built-in workflow."
|
||||
[VERIFIED — code.claude.com/docs/en/commands.md, code.claude.com/docs/en/workflows.md]
|
||||
- Gating: available on **all paid plans** (pro/max/team/enterprise) + API/Bedrock/Vertex/
|
||||
Foundry. **On Pro it must be turned on** in the *Dynamic workflows* row of `/config`.
|
||||
Also requires the **WebSearch tool** to be available.
|
||||
[VERIFIED — workflows.md, commands.md bundled-workflows row]
|
||||
- Version floor: dynamic workflows were **introduced in CC 2.1.154**; "Dynamic workflows
|
||||
require Claude Code v2.1.154 or later." The brief's `v2.1.154+` + `(Pro: via /config)`
|
||||
constraints are **both correct**. The exact version that first shipped the *named*
|
||||
`/deep-research` workflow is **[NOT DOCUMENTED]** (only a 2.1.196 bugfix mentions it by
|
||||
name) — treat 2.1.154 as the substrate floor, not a proven introduction point.
|
||||
[VERIFIED — workflows.md + CHANGELOG 2.1.154]
|
||||
|
||||
**Local findings:**
|
||||
- This machine runs **CC 2.1.196** (`claude --version`) — substrate floor satisfied.
|
||||
- The exact skill-description string lives **compiled into the binary**
|
||||
(`/Users/ktg/.local/share/claude/versions/2.1.196`, Mach-O 235 MB); there is **no
|
||||
`SKILL.md`** for it anywhere under `~/.claude` (system-wide `find`/`grep` — only hits
|
||||
are this brief + unrelated harness notes). Confirms "Anthropic-bundled, not user skill."
|
||||
|
||||
### 2. Invocation mechanism (programmatic vs instruction) -- Confidence: high
|
||||
|
||||
**External findings:**
|
||||
- **Not** via the `Skill` tool. The Skill-tool built-in allowlist is closed: only
|
||||
`/init`, `/review`, `/security-review` are reachable; "Other built-in commands such as
|
||||
`/compact` are not." `/deep-research` is a Workflow and is not on that list.
|
||||
[VERIFIED — code.claude.com/docs/en/skills.md]
|
||||
- **Instruction-based delegation is the documented mechanism.** Workflows launch when the
|
||||
user types the command, or **when Claude is asked in natural language** ("use a
|
||||
workflow" / "run a workflow") or via the dedicated opt-in keyword CC ships for
|
||||
multi-agent orchestration (CC 2.1.160; spelled out in
|
||||
`docs/cc-upgrade-2.1.181-decision-matrix.md`, which `verify.sh`'s SC1 excludes
|
||||
precisely because it legitimately cites CC keywords). A plugin command whose
|
||||
markdown instructs Claude to run `/deep-research <q>` is therefore the supported path.
|
||||
[VERIFIED — workflows.md "Have Claude write a workflow"]
|
||||
- **Approval gate caveat:** launching a workflow triggers a per-run approval prompt —
|
||||
*every run* in default/acceptEdits; *first launch only* in auto; **never in `claude -p`
|
||||
/ Agent SDK / bypass-permissions** ("the run starts immediately"). Voyage's headless
|
||||
surface (`claude -p`) thus delegates without an interactive gate; interactive sessions
|
||||
hit a prompt. [VERIFIED — workflows.md "Behavior and limits"]
|
||||
- No documented blanket "commands/skills cannot nest" prohibition beyond the Skill-tool
|
||||
allowlist + workflow runtime limits (no mid-run user input; 16 concurrent agents;
|
||||
1000 agents/run). [VERIFIED — workflows.md; NOT DOCUMENTED for a general nesting ban]
|
||||
|
||||
### 3. Output capture -- Confidence: high
|
||||
|
||||
**External findings:**
|
||||
- "When the run finishes, **the report lands in your session**"; "Claude's context holds
|
||||
only the final answer." The report is **in-context**, not a file.
|
||||
[VERIFIED — workflows.md]
|
||||
- What *is* written to disk is the orchestration **script**, not the report: "Every run
|
||||
writes its script to a file under your session's directory in `~/.claude/projects/`."
|
||||
[VERIFIED — workflows.md "How a workflow runs"]
|
||||
|
||||
**Local findings:**
|
||||
- A real `/deep-research` run on this machine left exactly one file —
|
||||
`~/.claude/projects/<session>/workflows/scripts/deep-research-wf_<id>.js` — and **no
|
||||
`.md` report** beside it. [VERIFIED — local filesystem inspection by claude-code-guide]
|
||||
- **Consequence:** a calling command cannot `grep` a results file off disk (none is
|
||||
documented to exist). It can only consume the report **as it sits in conversation
|
||||
context in the same turn** — Claude reads its own prior output and transforms it into
|
||||
research-brief schema. [INFERENCE — file-based capture not feasible; in-context
|
||||
transform is the only avenue]
|
||||
|
||||
### 4. Feature detection + fallback -- Confidence: medium
|
||||
|
||||
**External findings:**
|
||||
- **No positive enumeration / "is-enabled" API or flag is documented.** `claude --help`
|
||||
exposes `--disable-slash-commands` but no "list skills/workflows" or "is-feature-on"
|
||||
flag. [VERIFIED — local `claude --help`; NOT DOCUMENTED for any positive probe]
|
||||
- The documented signals are the **off-switches**, read defensively: `/config` Dynamic-
|
||||
workflows off; `disableWorkflows: true` / `disableBundledSkills: true` in settings.json;
|
||||
`CLAUDE_CODE_DISABLE_WORKFLOWS=1` / `CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1`.
|
||||
[VERIFIED — workflows.md "Turn workflows off"; CHANGELOG 2.1.x]
|
||||
- **The gap:** the Pro `/config` *on*-state is the very thing you most need to detect, and
|
||||
only the *disable* keys are documented; the persisted key/value for the Pro enable-state
|
||||
is **[NOT DOCUMENTED]**, so it cannot be reliably grepped. A failed/disabled
|
||||
`/deep-research` is also **not documented** to raise a signal a sibling command can
|
||||
catch. [VERIFIED gap]
|
||||
|
||||
**Local findings:**
|
||||
- On this machine both disable keys are unset/absent — workflows are not disabled.
|
||||
|
||||
## External Knowledge
|
||||
|
||||
### Best Practice
|
||||
The "Have Claude write a workflow" + "Behavior and limits" sections of `workflows.md`
|
||||
establish that workflows are operator/Claude-launched, run isolated, and return one
|
||||
in-context report. The Skill-tool allowlist (`skills.md`) is the authoritative statement
|
||||
that only three built-ins are tool-invocable.
|
||||
|
||||
### Known Issues
|
||||
Per-run approval prompts outside `-p`/SDK/bypass mean an interactive `/trekresearch
|
||||
--engine deep-research` will pause for operator approval on each launch — acceptable, but
|
||||
worth documenting in the command UX. The absence of a positive availability probe is the
|
||||
single biggest design constraint (see Dimension 4).
|
||||
|
||||
## Synthesis
|
||||
|
||||
Three cross-cutting insights that only emerge from combining the docs with Voyage's brief:
|
||||
|
||||
1. **The brief's SC3 is correct as written — an earlier review note was wrong.** Because
|
||||
`/deep-research` *is itself* a dynamic workflow, "med dynamic workflows avskrudd faller
|
||||
`--engine deep-research` tilbake til swarm" is the right feature-detection axis. A
|
||||
prior brief-review remark that SC3 "conflated `/deep-research` (skill) with dynamic
|
||||
workflows" was based on a wrong premise (that `/deep-research` was a skill) and is
|
||||
retracted. **No SC3 brief edit is needed.**
|
||||
|
||||
2. **The engine must be instruction-based + in-context, never file-based.** Topic 1's
|
||||
open question ("instruction (safe) vs. programmatic API (faster) — topic 1 decides")
|
||||
resolves decisively to **instruction-based**: there is no programmatic API and no
|
||||
on-disk report. The `deep-research` engine path in `commands/trekresearch.md` must
|
||||
(a) instruct Claude to run `/deep-research <q>`, then (b) transform the in-context
|
||||
report into the `research/NN-*.md` schema in the same turn. This is surface-only
|
||||
(`commands/` prose), matching the brief's "ingen nye `lib/`-avhengigheter."
|
||||
|
||||
3. **SC3's "ingen exception" cannot rest on runtime detection — pin it to a default.**
|
||||
Since no positive availability probe exists, robust fallback = version-floor check
|
||||
(≥ 2.1.154) + disable-key heuristic (`disableWorkflows` / env) + **`--engine` default
|
||||
of `swarm`** (explicit opt-in). The fallback is "graceful degradation by design,"
|
||||
not "catch an exception at runtime." This refines, but does not contradict, SC3.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **Adapter fidelity (brief Topic 2, partly answered locally):** `research-validator.mjs`
|
||||
requires `type: trekresearch-brief` + `created` + `question`; validates `confidence ∈
|
||||
[0,1]` and `dimensions ≥ 1` if present; body must carry `## Executive Summary` +
|
||||
`## Dimensions`. So the `/deep-research` report **can** be reduced to ≥ 1 dimension with
|
||||
per-claim citations and a confidence number — the brief's assumption holds. The
|
||||
remaining open part (how cleanly the in-context report maps to per-dimension
|
||||
local/external splits) is a `/trekplan` exploration concern, not an external unknown.
|
||||
- **Exact intro version of the *named* `/deep-research` workflow** — not documented; the
|
||||
2.1.154 dynamic-workflows floor is the safe pin.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Build the `deep-research` engine as **instruction-based delegation with in-context
|
||||
adaptation**, not a programmatic trigger:
|
||||
1. `--engine deep-research` makes `commands/trekresearch.md`'s external phase instruct
|
||||
Claude to run `/deep-research <q>` and transform the returned in-context report into
|
||||
`research/NN-*.md` (validator-conformant: `## Executive Summary` + `## Dimensions`,
|
||||
`confidence`, `dimensions ≥ 1`).
|
||||
2. Feature-detect by **graceful degradation**: version floor + disable-key heuristic +
|
||||
`--engine` default `swarm`. Do not depend on a positive availability probe (none
|
||||
exists). Log the chosen engine. This satisfies SC3 without a runtime exception.
|
||||
3. Keep `swarm` the default (brief's Non-Goal: "ikke default-bytte"). Document the
|
||||
per-run approval prompt for interactive (non-`-p`) sessions.
|
||||
|
||||
Confidence in the recommendation: **high** for the mechanism (instruction-based +
|
||||
in-context), **medium** for the exact fallback-detection ergonomics (the one documented
|
||||
gap). This is sufficient to green-light `/trekplan` with Topic 1 resolved.
|
||||
|
||||
## Sources
|
||||
|
||||
| # | Source | Type | Quality | Used in |
|
||||
|---|--------|------|---------|---------|
|
||||
| 1 | code.claude.com/docs/en/workflows.md | official | high | Dim 1,2,3,4 + Synthesis |
|
||||
| 2 | code.claude.com/docs/en/commands.md | official | high | Dim 1 (Workflow classification, WebSearch req) |
|
||||
| 3 | code.claude.com/docs/en/skills.md | official | high | Dim 2 (Skill-tool allowlist) |
|
||||
| 4 | github.com/anthropics/claude-code CHANGELOG (2.1.154, 2.1.x, 2.1.196) | official | high | Dim 1,4 (version floor, disable keys) |
|
||||
| 5 | Local: CC 2.1.196 binary inspection (`find`/`grep`, `claude --version`) | codebase | high | Dim 1 (bundled, no SKILL.md) |
|
||||
| 6 | Local: real `/deep-research` run — only `.js` script written, no `.md` report | codebase | high | Dim 3 (no on-disk artifact) |
|
||||
| 7 | Local: `lib/validators/research-validator.mjs` schema | codebase | high | Open Questions (adapter feasibility / Topic 2) |
|
||||
|
|
@ -63,7 +63,7 @@ goes to it under MIT, and to the Anthropic article it is itself derived from.
|
|||
|
||||
| Candidate | Does Voyage already have it | Verdict |
|
||||
|---|---|---|
|
||||
| Hash-chained, tamper-evident approval ledger | No. No hash chain exists anywhere; `lib/util/research-loop-cap.mjs` is append-only but unchained | **Partly.** The idea answers "did we build what we agreed", but a new ledger engine is out of scope; the cheap form is a `prev_hash` field on the event ledger that already exists |
|
||||
| Hash-chained, tamper-evident approval ledger | No. No hash chain exists anywhere | **Partly.** The idea answers "did we build what we agreed", but a new ledger engine is out of scope; the cheap form is a `prev_hash` field on the event ledger that already exists |
|
||||
| Plan as a machine-readable file list with a lock | Yes — `plan_version 1.7` manifest, `plan-validator.mjs --strict`, and the independent Phase 7.5 audit | **Borrow the boundary only:** their named process-file exclusion list, so a state or docs update does not need a manifest entry |
|
||||
| Short intent and spec templates | Partly — the brief already requires `## Intent`, `## Goal` and `## Success Criteria` | **No** to a separate spec artifact (the operator is both orderer and builder). **Yes** to their "Gotchas" section: one fixed place for where two constraints collided and who decided |
|
||||
| A rule → enforcement table ("advisory or enforced, and where") | No such table; the doc-pins are that idea applied narrowly | **Yes, cheap.** It makes visible which of Voyage's own rules are only prose |
|
||||
|
|
|
|||
|
|
@ -1,160 +0,0 @@
|
|||
# Spike: does a plugin `PreToolUse` hook reach sub-agent tool calls?
|
||||
|
||||
**Date:** 2026-08-09
|
||||
**Claude Code version:** 2.1.226
|
||||
**Plugin:** voyage 5.9.1 (installed from `ktg-plugin-marketplace`)
|
||||
**Gates:** Step 10 of `plan.md` (`2026-06-30-trekresearch-storm-upgrade`)
|
||||
|
||||
## Question
|
||||
|
||||
Step 10 wants to enforce the conversation-turn cap in a `PreToolUse` hook. That
|
||||
is only viable if a **plugin** `PreToolUse` hook fires on tool calls made
|
||||
*inside a sub-agent*. If it does not, the cap must be enforced somewhere else
|
||||
and Step 10 becomes a documented downgrade instead.
|
||||
|
||||
The question is genuinely open, not answerable from docs alone: the current
|
||||
[hooks reference](https://code.claude.com/docs/en/hooks) states that sub-agent
|
||||
tool calls fire the same hooks and carry `agent_id` / `agent_type`, while
|
||||
[issue #34692](https://github.com/anthropics/claude-code/issues/34692) reported
|
||||
the exact opposite behaviour. The answer is therefore version-dependent and had
|
||||
to be measured on the version actually in use.
|
||||
|
||||
## Method
|
||||
|
||||
A one-shot probe hook was registered for the `WebSearch` matcher, and a
|
||||
**headless child session** was launched to exercise it. The child was used
|
||||
because hooks are resolved when a session starts — a matcher added mid-session
|
||||
cannot be observed by the session that added it.
|
||||
|
||||
Two deviations from the step as originally written, both forced and both
|
||||
verified not to affect the result:
|
||||
|
||||
1. **The matcher was injected into the installed plugin's `hooks.json`, not the
|
||||
repository's.** The plan assumed the repo working tree *is* the active plugin
|
||||
root. It is not: `~/.claude/plugins/cache/ktg-plugin-marketplace/voyage/5.9.1/`
|
||||
is a plain directory holding its own copy, and that copy is what loads.
|
||||
Editing `hooks/hooks.json` in the repo would have measured nothing. The cache
|
||||
file was backed up, modified, and restored — verified byte-identical to the
|
||||
repo file afterwards.
|
||||
2. **The probe script lives under the session scratchpad, not `${TMPDIR}`.** A
|
||||
pathguard hook refuses writes to `${TMPDIR}`. The load-bearing property was
|
||||
only that the script sit **outside `hooks/scripts/`**, which
|
||||
`tests/lib/doc-consistency.test.mjs:75-84` counts via `readdirSync`; the
|
||||
scratchpad satisfies that just as well. The directory still holds 7 scripts.
|
||||
|
||||
### Probe hook
|
||||
|
||||
Logged every invocation as one JSON line (`tool_name`, `agent_id`,
|
||||
`agent_type`, plus the untouched stdin) and always exited `0`, so it could not
|
||||
alter the child's behaviour.
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
# 1. inject the temporary matcher into the INSTALLED plugin
|
||||
node -e '...push {matcher:"WebSearch", ...voyage-spike-hook.mjs} into hooks.PreToolUse...'
|
||||
|
||||
# 2. exercise it from a fresh child session
|
||||
claude -p "Spawn exactly one sub-agent via the Agent tool (subagent_type: general-purpose).
|
||||
Instruct that sub-agent to perform exactly ONE WebSearch for the query
|
||||
'claude code hooks reference' and report back the first result title.
|
||||
You MUST NOT call WebSearch yourself in the main context - only the
|
||||
sub-agent may call it. When the sub-agent returns, reply with the word DONE." \
|
||||
--allowedTools "Agent,Task,WebSearch" \
|
||||
--max-turns 15
|
||||
|
||||
# 3. restore
|
||||
cp "${TMPDIR}voyage-hooks-backup.json" <cache>/hooks/hooks.json
|
||||
```
|
||||
|
||||
The child returned `DONE`.
|
||||
|
||||
> An earlier attempt additionally passed `--permission-mode bypassPermissions`
|
||||
> and was refused by the auto-mode classifier. The flag was dropped;
|
||||
> `--allowedTools` alone was sufficient.
|
||||
|
||||
## Raw observation
|
||||
|
||||
The log contains **exactly one** record — so the main context did not call
|
||||
`WebSearch` itself, and the single entry is unambiguously the sub-agent's call:
|
||||
|
||||
```json
|
||||
{"at":"2026-08-09T12:07:47.796Z","tool_name":"WebSearch",
|
||||
"agent_id":"aa6d19525a4680fe0","agent_type":"general-purpose",
|
||||
"raw_stdin":"{\"session_id\":\"b126fd6a-...\",\"cwd\":\"/Users/ktg/repos/ktg-plugin-marketplace/voyage\",
|
||||
\"permission_mode\":\"auto\",\"agent_id\":\"aa6d19525a4680fe0\",\"agent_type\":\"general-purpose\",
|
||||
\"effort\":{\"level\":\"xhigh\"},\"hook_event_name\":\"PreToolUse\",\"tool_name\":\"WebSearch\",
|
||||
\"tool_input\":{\"query\":\"claude code hooks reference\"},\"tool_use_id\":\"toolu_01Ka4...\"}"}
|
||||
```
|
||||
|
||||
Both `agent_id` and `agent_type` are populated, matching the documented
|
||||
common-input fields for sub-agent-originated tool events. A main-context call
|
||||
would have carried neither.
|
||||
|
||||
## Consequence for Step 10
|
||||
|
||||
A plugin `PreToolUse` hook **does** observe sub-agent tool calls on CC 2.1.226,
|
||||
and can attribute them via `agent_id` / `agent_type`. Step 10 may therefore take
|
||||
the enforcement branch rather than the documented-downgrade branch.
|
||||
|
||||
Two limits worth carrying forward, neither of which changes the verdict:
|
||||
|
||||
- This measures `WebSearch` on one CC version. The behaviour regressed once
|
||||
before (#34692), so the hook must fail **open**, never assume it is the only
|
||||
gate, and the cap must remain correct if the hook silently stops firing.
|
||||
- The probe only establishes *reach*. Whether a **blocking** (exit 2) decision
|
||||
from inside a sub-agent propagates usefully was not measured — the probe
|
||||
always exited 0 by design.
|
||||
|
||||
RESULT: FIRES
|
||||
|
||||
## Enforcement outcome
|
||||
|
||||
Step 10 took the **enforcement branch**: `hooks/scripts/pre-agent-cap.mjs`
|
||||
(PreToolUse, matcher `WebSearch|WebFetch|Task`), pinned by
|
||||
`tests/hooks/agent-cap.test.mjs`.
|
||||
|
||||
What it does: counts turns spent by a run — read-only, from the append-only
|
||||
ledger that `lib/util/research-loop-cap.mjs` writes — and exits 2 once
|
||||
`turns_used >= max_conv_turns × maxDimensions`. It never appends to the
|
||||
ledger; a cap that recorded its own enforcement would count itself.
|
||||
|
||||
Scope key, the part that makes a globally-wired `PreToolUse` hook safe:
|
||||
`session_id` **+** a marker file only the Phase 5 loop writes, at
|
||||
`${CLAUDE_PLUGIN_DATA}/trekresearch-loop-scope/<session_id>.json`:
|
||||
|
||||
```json
|
||||
{ "runId": "<run id>", "startedAt": "<ISO-8601>" }
|
||||
```
|
||||
|
||||
No marker for the calling session ⇒ out of scope ⇒ allow, unconditionally.
|
||||
An unrelated session is never denied because some other run spent its budget.
|
||||
|
||||
Fail-open and fail-closed are split deliberately:
|
||||
|
||||
| Condition | Outcome | Why |
|
||||
|---|---|---|
|
||||
| No marker / no `session_id` / unparsable stdin | allow | Not evidence of a loop turn |
|
||||
| Marker older than TTL (default 2h, `VOYAGE_CAP_SCOPE_TTL_MS`) | allow + auto-reset | A crashed run must not deny tool calls forever, and `--resume` keeps the same `session_id` |
|
||||
| `VOYAGE_DISABLE_CAP_HOOK=1` | allow | Kill switch |
|
||||
| `VOYAGE_STORM_ENABLED` ≠ `1` | allow | Default-off: no loop runs, nothing to enforce |
|
||||
| In scope, `CLAUDE_PLUGIN_DATA` absent | **deny** | A budget control that cannot count must not grant — same stance as `research-loop-cap.mjs` |
|
||||
| In scope, budget spent | **deny (exit 2)** | The bound |
|
||||
|
||||
Both limits recorded above still hold and are not closed by this step. Reach
|
||||
was measured on one CC version for one tool, and blocking-propagation from
|
||||
inside a sub-agent was never measured — so this hook is **defence in depth**,
|
||||
and `research-loop-cap.mjs` must remain correct on its own if the hook
|
||||
silently stops firing.
|
||||
|
||||
**Follow-up, closed (S79).** Step 10 shipped the hook correct but **latent** —
|
||||
nothing wrote the marker, and it enforces exactly when a marker exists.
|
||||
`commands/trekresearch.md` Phase 5 now writes it (`### Loop scope marker`,
|
||||
keyed by `CLAUDE_CODE_SESSION_ID`, carrying the same `run_id` the ledger is
|
||||
counted under) and removes it on all three exits. Cleanup covers exits only; a
|
||||
crashed session is covered by the TTL above. Verified end-to-end: the snippet
|
||||
as shipped arms the hook, the hook denies at `8/8` turns, and the removal
|
||||
snippet returns it to allow so Phase 6 can still spawn agents. Pinned by
|
||||
`tests/lib/doc-consistency.test.mjs` (STORM marker), which derives the
|
||||
directory name from `SCOPE_DIRNAME` in the hook, so renaming either side
|
||||
fails.
|
||||
|
|
@ -1,136 +0,0 @@
|
|||
# STORM adoption gate — pre-registered measurement protocol
|
||||
|
||||
**Status:** thresholds registered, **no measurement run yet.** This document is
|
||||
committed *before* the first measurement by design: a threshold chosen after
|
||||
seeing the numbers is not a threshold. **Harness:** `scripts/storm-measure.mjs`
|
||||
(pinned by `tests/scripts/storm-measure.test.mjs`). **Decides:** whether the
|
||||
bounded Phase 5 conversation loop (`commands/trekresearch.md` §Loop bound) is
|
||||
worth turning on by default.
|
||||
|
||||
---
|
||||
|
||||
## 1. What this gate measures — and what it does not
|
||||
|
||||
The gate measures **source and coverage breadth**:
|
||||
|
||||
| Metric | Definition | Arm |
|
||||
|---|---|---|
|
||||
| `unique_sources` | distinct external sources cited by the run | between-arm median gain |
|
||||
| dimensions over baseline | `(dimensions − dimensions_baseline) / dimensions_baseline` | within-run median across the treatment arm |
|
||||
|
||||
It does **not** measure outline quality, answer correctness, synthesis
|
||||
usefulness, or operator satisfaction. A breadth win is a *necessary* condition
|
||||
for adoption, never a sufficient one. If the loop widens coverage by 40% and
|
||||
the resulting briefs read worse, the correct action is still decline — the
|
||||
number does not overrule a reading of the artifacts.
|
||||
|
||||
The second metric is a within-run delta by construction (`dimensions_baseline`
|
||||
is the interview dimension count, `dimensions` the post-Phase-4.5 list), so it
|
||||
needs no control arm; the control arm's value is 0 because the loop is inert
|
||||
below `effort: high`.
|
||||
|
||||
## 2. Pre-registered thresholds
|
||||
|
||||
Either metric clearing the bar is enough. The brief pre-registers "median
|
||||
forbedring ≥ 30 % på (a) eller (b) → adopt. < 15 % → decline" — a breadth win
|
||||
on one axis counts, because either axis widening is the effect the loop claims.
|
||||
Adopt is evaluated first, so a run that clears the adopt bar on one metric is
|
||||
an adopt even when the other metric sits under the decline bar.
|
||||
|
||||
| Median gain | Verdict | Action |
|
||||
|---|---|---|
|
||||
| ≥ 30% on **either** metric | **adopt** | Flip the `VOYAGE_STORM_ENABLED` default (see §5) |
|
||||
| < 15% on **either** metric (and no adopt) | **decline** | Leave the mechanism default-off. This is a **no-op**: nothing is rolled back |
|
||||
| both metrics in 15% – 30% | **inconclusive** | Keep default-off, gather more runs, re-measure |
|
||||
| either arm empty | **insufficient-data** | Not a decline — measure more |
|
||||
|
||||
`ADOPT_THRESHOLD = 0.30` and `DECLINE_THRESHOLD = 0.15` are exported constants
|
||||
in `scripts/storm-measure.mjs` and pinned by the test suite. Changing them is a
|
||||
deliberate, reviewable act, not a tuning knob to be nudged toward a result.
|
||||
|
||||
## 3. Excluded runs (the honest denominator)
|
||||
|
||||
Runs with `empty_turns > 0` are **excluded from the gain and reported as a
|
||||
count**. An empty turn is one that spent budget and returned no findings, or
|
||||
findings without citations. A run whose `empty_turns` cannot be **read** as a
|
||||
number is excluded on the same footing: a field that says `"many"` is not
|
||||
evidence of zero empty turns, and treating it as one would let the least
|
||||
trustworthy run back into the denominator in the direction that flatters
|
||||
adoption. Including those runs decides adoption on a broken
|
||||
denominator — the loop looks cheap because its failures are averaged into its
|
||||
successes. The harness prints the excluded count on every invocation; if that
|
||||
count is a large fraction of the treatment arm, the finding is about the loop's
|
||||
reliability, and it should be read before the gain figure is read at all.
|
||||
|
||||
Rows predating the Step 9 measurement fields carry no `effort` and are dropped
|
||||
as `legacy` with a count. A stats file where *no* row carries `effort` is a hard
|
||||
error, not an empty treatment group: a schema gap must never present itself as
|
||||
"no gain".
|
||||
|
||||
## 4. The measurement runs (operator-run, outside this plan)
|
||||
|
||||
The measurement itself is **not** part of the implementation plan that built
|
||||
this harness. It is an operator-run gate between that plan and any adopt commit.
|
||||
|
||||
Protocol:
|
||||
|
||||
- **n ≥ 5 runs per arm.** Fewer, and the median is an anecdote.
|
||||
- **The same question set in both arms.** Two briefs, run as `--project` runs.
|
||||
- The arms differ in exactly one thing: whether the loop is enabled.
|
||||
- Both arms append to the same `trekresearch-stats.jsonl`; `effort` is the
|
||||
grouping key that separates them.
|
||||
|
||||
Each run is a Claude Code session the operator opens (Voyage starts none of
|
||||
them; changed 2026-09-27 before any run — the thresholds above are untouched):
|
||||
|
||||
```text
|
||||
# Control arm (loop inert) — n >= 5, in a session opened with VOYAGE_STORM_ENABLED unset
|
||||
/trekresearch --project .claude/projects/<brief-standard-effort>
|
||||
|
||||
# Treatment arm (loop live, effort: high in the brief's phase_signals) — n >= 5,
|
||||
# in a session opened with VOYAGE_STORM_ENABLED=1 in its environment
|
||||
/trekresearch --project .claude/projects/<brief-high-effort>
|
||||
```
|
||||
|
||||
```bash
|
||||
# The gate
|
||||
node scripts/storm-measure.mjs --stats "${CLAUDE_PLUGIN_DATA}/trekresearch-stats.jsonl"
|
||||
|
||||
# Machine-readable, for a decision record
|
||||
node scripts/storm-measure.mjs --json --stats "${CLAUDE_PLUGIN_DATA}/trekresearch-stats.jsonl"
|
||||
|
||||
# SC activation check — BOTH halves: did the latest high-effort run discover at
|
||||
# least one dimension, AND is its final list a true SUPERSET of the interview
|
||||
# ones? The second half is read from the run's own `dimensions_baseline_preserved`
|
||||
# attestation, because the dimension NAMES that would show it directly are prose
|
||||
# the exporter allowlist denies. A run that does not attest it FAILS — dropping
|
||||
# two interview dimensions and appending three discovered ones is a +1 count
|
||||
# delta and not a superset. Exit 0 = both halves hold.
|
||||
node scripts/storm-measure.mjs --activation-check --stats "${CLAUDE_PLUGIN_DATA}/trekresearch-stats.jsonl"
|
||||
```
|
||||
|
||||
## 5. What "adopt" concretely means
|
||||
|
||||
Adoption is **one constant**: `isStormEnabled()` in
|
||||
`lib/util/research-loop-cap.mjs` currently requires `VOYAGE_STORM_ENABLED === '1'`.
|
||||
Adopt = make the loop's budget non-zero without that opt-in, in a commit that
|
||||
cites the measurement output.
|
||||
|
||||
This asymmetry is deliberate and was designed in before any code was written:
|
||||
|
||||
- **decline costs nothing** — the mechanism ships default-off, so declining is
|
||||
doing nothing. No revert, no removal from a command file two other steps
|
||||
already rewrote.
|
||||
- **adopt costs one constant** — plus the enforcement hook
|
||||
(`hooks/scripts/pre-agent-cap.mjs`) already in place to bound what gets turned
|
||||
on, and the operator-visible cap-exhaustion message already required by
|
||||
Phase 5's exit conditions.
|
||||
|
||||
## 6. Reading the result honestly
|
||||
|
||||
- The gate measures breadth. Say "breadth" in the decision record, not "quality".
|
||||
- Report the excluded count alongside the gain, always. A 35% gain computed
|
||||
after excluding 6 of 10 treatment runs is a finding about instability.
|
||||
- `insufficient-data` is not a decline. Do not resolve it by lowering n.
|
||||
- A verdict computed from a stats file mixing several question sets measures the
|
||||
question sets, not the loop.
|
||||
Loading…
Add table
Add a link
Reference in a new issue