Compare commits
33 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f3874946ad | |||
| d9cba9c6ea | |||
| 3e8af75015 | |||
| 0a569eec55 | |||
| abc5bd8967 | |||
| 0556743bad | |||
| 9a38500a63 | |||
| e1cf545a0c | |||
| 5bb6735c94 | |||
| 9ffeae0e2e | |||
| 2728a43656 | |||
| 79044624db | |||
| 8d39e1d4a5 | |||
| 451969083b | |||
| 0799d6e914 | |||
| 22058459f8 | |||
| cd1d5c8738 | |||
| db3b8f5491 | |||
| 77ccf6ba06 | |||
| dcc71d9577 | |||
| 5c37b95dfb | |||
| 84fbee2313 | |||
| 8b7a849a76 | |||
| 357e17b176 | |||
| 937482067d | |||
| 76818b2459 | |||
| 4ec979747b | |||
| 0e657de023 | |||
| a6bed277d0 | |||
| 581489a513 | |||
| 9d8e043959 | |||
| 60e9e7ae5c | |||
| 926b768543 |
57 changed files with 2544 additions and 189 deletions
|
|
@ -1,12 +1,23 @@
|
|||
{
|
||||
"name": "voyage",
|
||||
"description": "Voyage — brief, research, plan, execute, review, continue. Contract-driven Claude Code pipeline. /trekbrief, /trekplan, and /trekreview each end by building a self-contained operator-annotation HTML (scripts/annotate.mjs, modelled on claude-code-100x): select text or click any element, pick intent (Fiks/Endre/Spørsmål), write comment, copy structured prompt, paste back, Claude revises the .md.",
|
||||
"version": "5.8.0",
|
||||
"version": "5.9.1",
|
||||
"author": {
|
||||
"name": "Kjell Tore Guttormsen"
|
||||
},
|
||||
"homepage": "https://git.fromaitochitta.com/open/ktg-plugin-marketplace/src/branch/main/plugins/voyage",
|
||||
"repository": "https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git",
|
||||
"license": "MIT",
|
||||
"keywords": ["voyage", "trek", "planning", "implementation", "research", "context-engineering", "agents", "adversarial-review", "headless", "execution"]
|
||||
"keywords": [
|
||||
"voyage",
|
||||
"trek",
|
||||
"planning",
|
||||
"implementation",
|
||||
"research",
|
||||
"context-engineering",
|
||||
"agents",
|
||||
"adversarial-review",
|
||||
"headless",
|
||||
"execution"
|
||||
]
|
||||
}
|
||||
|
|
|
|||
50
CHANGELOG.md
50
CHANGELOG.md
|
|
@ -4,6 +4,56 @@ All notable changes to this project will be documented in this file.
|
|||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## v5.9.1 — 2026-07-03 — Fix /trekendsession load-time crash (eager-exec placeholders)
|
||||
|
||||
Patch, no functional additions.
|
||||
|
||||
### Fixed
|
||||
|
||||
- `/trekendsession` was unusable in every invocation: two of its three `` !`...` `` eager-exec blocks (Phase 3 atomic-write, Phase 4 validator call) contained unresolved runtime placeholders (`<project-dir>` etc.). The harness executes eager-exec blocks at command LOAD time, so zsh parsed `<project-dir>` as input redirection and the command aborted before the model saw a single instruction. Both blocks are now plain runtime Bash fences with the `{curly}` placeholder convention (shell-inert), matching `trekplan.md`/`trekresearch.md`. The Phase 1 discovery block (self-contained) keeps its legitimate eager-exec prefix; `trekcontinue.md`'s discovery block was runtime-verified unaffected.
|
||||
- Latent secondary bug in the same blocks: cwd-relative plugin paths (`lib/validators/...`, `./lib/util/atomic-write.mjs`) would have failed with `ERR_MODULE_NOT_FOUND` even after substitution, since the Bash cwd is the user's repo. Both now use absolute `${CLAUDE_PLUGIN_ROOT}` paths per the existing command convention (Node ESM accepts absolute-path import specifiers — verified on Node 18+).
|
||||
|
||||
### Added
|
||||
|
||||
- Regression guard `tests/commands/trekendsession.test.mjs`: scans every `` !` ``-block in `commands/*.md` for unresolved `<angle>`/`{curly}` placeholders (this bug class is silent until first invocation), plus structure tests pinning Phase 3/4 as runtime Bash with `${CLAUDE_PLUGIN_ROOT}` paths and exactly one surviving eager block. Suite baseline 828 → 832 (830 pass / 0 fail / 2 skip).
|
||||
|
||||
## v5.9.0 — 2026-07-02 — Fable model tier + deep-research engine
|
||||
|
||||
Additive, plus one behavior alignment: profile `phase_models` now reach sub-agent spawn sites (previously documented but never wired), and the seven command orchestrators no longer pin `model: opus` — frontmatter omits `model:`, so the orchestrator follows the session model.
|
||||
|
||||
### Fable model tier
|
||||
|
||||
- `fable` (→ Claude Fable 5, Mythos-class, positioned above Opus) is an accepted model value throughout the validation chain: `BASE_ALLOWED_MODELS` widened to `['sonnet', 'opus', 'fable']` in `lib/validators/profile-validator.mjs` — the single source imported by brief-validator and phase-signal-resolver (two-layer gate preserved; accept-fable AND reject-unknown-model covered at both layers). No env gate — haiku's `VOYAGE_ALLOW_HAIKU` opt-in stays as-is.
|
||||
- `/trekbrief` Phase 3.5 tier loop offers a 4th option: `fable → {effort: high, model: fable}`. AskUserQuestion's 4-option maximum is now fully used — a 5th tier requires a loop redesign. The fable tier reuses `effort: high` orchestration semantics; `EFFORT_LEVELS` is unchanged.
|
||||
- New built-in profile `lib/profiles/fable.yaml` (all six phases on `fable`, modeled on premium; registered in `BUILTIN_NAMES` with a `loadProfile('fable')` canary test so a registry regression fails loudly instead of silently resolving premium). Premium stays the default.
|
||||
- Reasoning effort is inherited from the session: Fable 5's default effort is `high`, NOT xhigh, and switching model resets effort — set xhigh at session level (`/effort xhigh`, the `effortLevel` setting, or `CLAUDE_CODE_EFFORT_LEVEL`). Documented canonically in `docs/profiles.md` §Model & effort axes.
|
||||
- `claude-fable-5` added to the cost `PRICE_TABLE` ($10/MTok input, $50/MTok output; cache write 5m $12.50 / 1h $20; cache read $1 — verified 2026-07-02 against the official platform pricing docs). `PRICE_TABLE_VERSION` bumped to `2026-07-02`. Without the entry, every fable run would report `cost_usd: null` in the observability export.
|
||||
- Profile tables + allowlist prose updated across README, `docs/profiles.md`, `docs/operations.md`, `docs/HANDOVER-CONTRACTS.md`, `docs/architecture.md`, `docs/command-modes.md`, templates, and CLAUDE.md; the S15 doc pins now machine-check the fable row cell-for-cell against `fable.yaml`. The `^(opus|sonnet)…` regex claim in two docs was corrected — validation is an exact string match against `BASE_ALLOWED_MODELS`; the regex never existed in code.
|
||||
|
||||
### Behavior alignment: profile `phase_models` now reach sub-agent spawns
|
||||
|
||||
- Pre-existing wiring gap (found in exploration): all four pipeline commands invoked only the brief-only `phase-signal-resolver.mjs`, so the `?? profile.phase_models[<phase>]` half of the documented composition rule never executed — `--profile <x>` never reached sub-agent spawns.
|
||||
- Fixed with a single composed resolver: `resolver.mjs --resolve-phase-model` now returns `{effort, model, source}` (brief > profile > default, with effort passed through atomically) and is the one CLI the four pipeline commands invoke. A doc-consistency pin requires the composed invocation and forbids the brief-only CLI in command Bash blocks.
|
||||
- **Behavior change (contract alignment):** `--profile economy/balanced` now genuinely reaches sub-agent spawn sites for the first time — behavior aligns with what the docs have long claimed. The premium default is unaffected in practice (premium resolves `opus`, which equals the frontmatter fallback).
|
||||
- Command frontmatter: the `model: opus` line is DELETED from all seven commands — omission (not the disputed `inherit` literal) is the spelling both official surfaces document as session-inheritance, guarded by a frontmatter-absence doc pin. Accepted tradeoff: in a sonnet session the orchestrator runs on sonnet; re-add a frontmatter pin for deterministic orchestrator choice. The 24 `agents/*.md` `model: opus` pins are untouched (spawn-time injection wins; frontmatter is the fallback). `/trekcontinue`/`/trekendsession` spawn no exploration swarm and get no spawn-site injection; the continue phase is covered at resolver level and follows the session model.
|
||||
|
||||
### Bundled unreleased work (since v5.8.0)
|
||||
|
||||
- `/trekresearch --engine {swarm|deep-research}` (`581489a..9374820`): opt-in delegation of the external research phase to Claude Code's built-in `/deep-research` workflow, with in-context adapter + self-check, availability fallback to swarm (never hard-fails), and doc-consistency pins across surfaces.
|
||||
- brief-validator CLI no-flag invocation fix (`926b768`).
|
||||
- Deep-research engine research notes + docs (`60e9e7a`, `9d8e043`).
|
||||
|
||||
### Operator + consume-side notes
|
||||
|
||||
- The operator-global CLAUDE.md policy "Opus 4.8 default for all subagents" predates the fable tier; updating it is an operator action outside this repo.
|
||||
- `/plugin update` compares against a stale local marketplace clone and can report "already at the latest version" after this release (Claude Code issues #35752 / #38271, both closed-not-planned). Reliable refresh: remove + re-add the marketplace, or `git pull --ff-only` in the marketplace clone. Cross-version skew consequence: a stale cached v5.8 brief-validator REJECTS fable-bearing briefs with `BRIEF_INVALID_MODEL` — enum widening is safe for new readers of old data, not old readers of new data.
|
||||
- Org `availableModels` with `enforceAvailableModels: true` can make an inheriting orchestrator silently fall back to the first allowed model.
|
||||
|
||||
### Release hygiene
|
||||
|
||||
- Suite (measured with bare `npm test` at release): **828 (826 pass / 0 fail / 2 skipped)** — +17 over the pre-release ground-truth baseline of 811 measured 2026-07-02 (allowlist/gate coverage, fable profile pins, composed-resolver + frontmatter-absence doc pins, PRICE_TABLE case).
|
||||
- Version sync: `plugin.json`, `package.json`, `package-lock.json`, README badge, CHANGELOG top entry all at `5.9.0`, guarded by `doc-consistency.test.mjs`.
|
||||
|
||||
## v5.8.0 — 2026-06-30 — offline gold-scored output eval (SKAL-1·4b)
|
||||
|
||||
Additive — no behavior change, no breaking change. Internal eval infrastructure only (`lib/` + `tests/` + docs); no command, agent, profile, or Handover contract touched.
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ Voyage — a contract-driven Claude Code pipeline: brief, research, plan, execut
|
|||
| Command | Description | Model |
|
||||
|---------|-------------|-------|
|
||||
| `/trekbrief` | Brief — interactive interview produces a task brief with explicit research plan; optionally orchestrates the pipeline | opus |
|
||||
| `/trekresearch` | Research — deep local + external research, produces structured research brief | opus |
|
||||
| `/trekresearch` | Research — deep local + external research, produces structured research brief. Opt-in `--engine {swarm\|deep-research}` delegates the external phase to Claude Code's built-in `/deep-research` workflow (swarm default) | opus |
|
||||
| `/trekplan` | Plan — brief-reviewer, explore, plan, review. Requires `--brief` or `--project`. Auto-discovers `architecture/overview.md` if present | opus |
|
||||
| `/trekexecute` | Execute — disciplined plan/session-spec executor with failure recovery | opus |
|
||||
| `/trekreview` | Review — independent post-hoc review of delivered code against the brief. Produces `review.md` with severity-tagged findings (Handover 6) | opus |
|
||||
|
|
@ -55,13 +55,13 @@ Full flag reference for each command (modes, `--gates`, `--profile`, breaking ch
|
|||
|
||||
> **Inventory (S33 reconcile).** 24 agent files = **21 spawnable** (one, `synthesis-agent`, ships **dormant** — Δ≈0, wired to nothing) **+ 3 orchestrator reference docs** (`planning-/research-/review-orchestrator` document the inline `/trek*` workflow, not spawnable capabilities). All 24 stay `model: opus` (operator pin `40d8742`); the glue/mechanical/retrieval/dormant roles were reconsidered for a sonnet downgrade and **kept opus** — decision record: `docs/voyage-vs-cc-balance-analysis.md` §10.
|
||||
|
||||
> **Model & effort.** `opus` = Opus 4.8 (default reasoning effort `high`); `sonnet` = Sonnet 4.6. Select agents carry native per-spawn `effort:` (retrieval → `medium`, adversarial-reasoning → `high`) — a different axis from brief `phase_signals.effort` (orchestration shape: which agents/passes run). Per-agent table + axes → `docs/profiles.md` §Model & effort axes.
|
||||
> **Model & effort.** `opus` = Opus 4.8 (default reasoning effort `high`); `sonnet` = Sonnet 4.6; `fable` = Fable 5 (Mythos-class, above Opus — reasoning effort inherits from the session; xhigh requires a session-level setting). Select agents carry native per-spawn `effort:` (retrieval → `medium`, adversarial-reasoning → `high`) — a different axis from brief `phase_signals.effort` (orchestration shape: which agents/passes run). Per-agent table + axes → `docs/profiles.md` §Model & effort axes.
|
||||
|
||||
## Reference docs (read on demand)
|
||||
|
||||
- **Architecture, workflows, project-directory contract, state, terminology:** `docs/architecture.md`
|
||||
- **Quality infrastructure (`lib/` validators, parsers, autonomy primitives, hooks):** `docs/architecture.md` §Quality infrastructure
|
||||
- **Autonomy gates (`--gates`), Path A/B/C decision:** `docs/operations.md`
|
||||
- **Profile system (`--profile economy/balanced/premium`), lookup order, custom profiles:** `docs/operations.md`
|
||||
- **Profile system (`--profile economy/balanced/premium/fable`), lookup order, custom profiles:** `docs/operations.md`
|
||||
- **Observability (Stop hook, OTLP/textfile export, SSRF mitigation):** `docs/operations.md`
|
||||
- **Handover contracts (the 7 pipeline handovers):** `docs/HANDOVER-CONTRACTS.md`
|
||||
|
|
|
|||
77
README.md
77
README.md
|
|
@ -1,17 +1,17 @@
|
|||
# trekplan — Brief, Research, Plan, Execute, Review, Continue
|
||||
# voyage
|
||||
|
||||
|
||||
Contract-driven Claude Code pipeline: brief, research, plan, execute, review. Agent swarms, research triangulation, adversarial review, multi-session resumption.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
> **Solo-maintained, fork-and-own.** This plugin is a starting point, not a vendor product. Issues are welcome as signals; pull requests are not accepted. See [GOVERNANCE.md](GOVERNANCE.md) for the full model and what upstream provides.
|
||||
|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development. [Full disclosure →](../../README.md#ai-generated-code-disclosure)*
|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development. Every change is human-directed, reviewed, and validated before commit. Per Anthropic Consumer Terms §4, ownership of outputs is assigned to the user; this plugin is licensed MIT.*
|
||||
|
||||
A [Claude Code](https://docs.anthropic.com/en/docs/claude-code) plugin for deep implementation planning, multi-source research, autonomous execution, independent post-hoc review, and zero-friction multi-session resumption. Six commands, one pipeline:
|
||||
|
||||
> **What's new — v5.8.0: offline gold-scored output eval (SKAL-1·4b).** The review-coordinator self-eval gains a scoring run: `lib/review/gold-scorer.mjs` grades a committed agent-run fixture against the golden corpus at `(file, rule_key)` granularity (precision/recall/f1 + verdict match), and the suite census gains a third category (`goldEval`) so a scoring run is counted apart from behavior coverage and doc-pins. Offline + deterministic — committed reviewer payloads, no live agent spawn (the LLM-in-the-loop tier is the separate 4c). Internal eval infrastructure; no command/agent/Handover change. **v5.7.1:** leaner always-loaded agent listing — `<example>` blocks relocated to agent bodies (~3,180 tok/turn, no behavior change). **v5.7.0:** opt-in per-session token/cost metering (SKAL-2) + eval foundation (SKAL-1·4a). **v5.6.1:** one-line `description:` for the four reference/dormant agents (~700 tok). **v5.5.0:** brief **framing** enforcement (`brief_version 2.2`) + a `/trekreview` reviewer-schema contract. Additive — no breaking changes. **Full version history → [CHANGELOG.md](CHANGELOG.md).**
|
||||
|
||||
| Command | What it does |
|
||||
|---------|-------------|
|
||||
| **`/trekbrief`** | Brief — interactive interview produces a task brief with explicit research plan |
|
||||
|
|
@ -21,6 +21,23 @@ A [Claude Code](https://docs.anthropic.com/en/docs/claude-code) plugin for deep
|
|||
| **`/trekreview`** | Review — independent post-hoc review of delivered code against the brief, severity-tagged findings |
|
||||
| **`/trekcontinue`** | Continue — read `.session-state.local.json` and resume the next session in a multi-session project |
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
|
||||
claude plugin install voyage@ktg-plugin-marketplace
|
||||
```
|
||||
|
||||
Or enable directly in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"voyage@ktg-plugin-marketplace": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`/trekbrief`, `/trekplan`, and `/trekreview` each end by running `scripts/annotate.mjs` against the just-written artifact and printing the resulting `file://<abs path>` link. The operator opens the HTML in a browser, clicks any line of the document, writes their own note in the inline textarea, watches a sidebar of all notes (editable, deletable, persisted in browser `localStorage`), and clicks "Copy Prompt" to get one structured prompt that they paste back into Claude — Claude then revises the `.md` from the notes. **The operator drives every annotation.** See [Reviewing and annotating artifacts](#reviewing-and-annotating-artifacts-v502).
|
||||
|
||||
Every artifact lives in one project directory: `.claude/projects/{YYYY-MM-DD}-{slug}/` contains `brief.md`, `research/NN-*.md`, `plan.md`, `sessions/`, `progress.json`, and `review.md`.
|
||||
|
|
@ -68,9 +85,6 @@ Under the hood, `lib/util/autonomy-gate.mjs` runs a small state machine (`idle
|
|||
## Quick start
|
||||
|
||||
```bash
|
||||
# Install the marketplace, then browse and enable plugins with /plugin
|
||||
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
|
||||
|
||||
# Capture intent (interactive)
|
||||
/trekbrief Add user authentication with JWT tokens
|
||||
# → .claude/projects/2026-04-18-jwt-auth/brief.md
|
||||
|
|
@ -127,7 +141,10 @@ Concrete capabilities, observable in the code — not aspirations.
|
|||
|
||||
**Virksomhet / regulated environment.** Defense-in-depth security across four layers (plugin hooks, prompt-level denylist, pre-execution plan scan, scoped tool access). `disableSkillShellExecution: true` recommendation for fork-ers handling untrusted briefs. No cloud dependency, no GitHub requirement. Validators are plain-Node CLIs — invocable from CI, custom hooks, or external tools, not just from voyage commands.
|
||||
|
||||
**What it doesn't solve:**
|
||||
## Non-goals
|
||||
|
||||
What this pipeline does **not** solve — read this before adopting it:
|
||||
|
||||
- LLM output truthfulness. Validators check shape, not facts. A plan with hallucinated paths passes schema but fails in execute. Plan-critic catches some, not all.
|
||||
- Multi-user concurrency on a single project directory. Two simultaneous executors will clobber `progress.json`.
|
||||
- Cost management. Opus on the orchestrator layer is expensive; documented in [Cost profile](#cost-profile), no automatic model downgrade.
|
||||
|
|
@ -151,7 +168,7 @@ Output: `.claude/projects/{YYYY-MM-DD}-{slug}/brief.md`
|
|||
|------|-------|----------|
|
||||
| **Default** | `/trekbrief <task>` | Dynamic interview until quality gates pass. No question cap. |
|
||||
| **Quick** | `/trekbrief --quick <task>` | Starts compact (optional sections get at most one probe), still escalates on weak required sections or failed review gate. |
|
||||
| **Profile** | `/trekbrief --profile <name> <task>` | (v4.1.0) Pin model profile for the brief phase: `economy` / `balanced` / `premium` / `<custom>`. See [Profile system](#profile-system-v410) below. |
|
||||
| **Profile** | `/trekbrief --profile <name> <task>` | (v4.1.0) Pin model profile for the brief phase: `economy` / `balanced` / `premium` / `fable` / `<custom>`. See [Profile system](#profile-system-v410) below. |
|
||||
|
||||
`/trekbrief` is **always interactive**. There is no foreground/background mode — the interview requires user input.
|
||||
|
||||
|
|
@ -195,6 +212,7 @@ Output:
|
|||
| **External** | `/trekresearch --external <question>` | Only external research agents (skip codebase analysis) |
|
||||
| **Foreground** | `/trekresearch --fg <question>` | No-op alias (foreground is default since v2.4.0) |
|
||||
| **Profile** | `/trekresearch --profile <name> <question>` | (v4.1.0) Pin model profile for the research phase. See [Profile system](#profile-system-v410). |
|
||||
| **Engine** | `/trekresearch --external --engine deep-research <question>` | Delegate the external phase to Claude Code's built-in `/deep-research` workflow; falls back to `swarm` if unavailable. Default `swarm`. |
|
||||
|
||||
Flags combine: `--project <dir> --external`.
|
||||
|
||||
|
|
@ -756,35 +774,16 @@ The `pre-compact-flush.mjs` hook directly fixes the documented P0 in `docs/treke
|
|||
|
||||
**Annotation HTML requires a desktop browser.** `scripts/annotate.mjs` produces a single self-contained `.html` file you open with `file://` in any modern browser (Chrome / Safari / Firefox / Edge — last two versions). No CDN, no server, no npm runtime deps. State persists in `localStorage` so closing and re-opening the tab keeps your work, but it's local to one browser on one machine — not synced anywhere. If you want to annotate without a browser, paste the `.md` into Claude with "comments inline below" and write notes in chat — same end result, just without the visual surface.
|
||||
|
||||
## Installation
|
||||
|
||||
Add the marketplace and browse plugins with `/plugin`:
|
||||
|
||||
```bash
|
||||
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
|
||||
```
|
||||
|
||||
Or enable directly in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"voyage@ktg-plugin-marketplace": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
An optional architect step between research and plan was previously available via a separate plugin; that architect plugin is no longer publicly distributed. The `architecture/overview.md` filesystem slot remains supported by `/trekplan` for any compatible producer.
|
||||
|
||||
## Profile system (v4.1.0)
|
||||
|
||||
Three built-in model profiles plus operator-defined `<custom>.yaml` (drop in `lib/profiles/`). Each profile pins `phase_models` for the six pipeline phases. The active profile is recorded in plan.md frontmatter as `profile: <name>` and emitted to JSONL stats for cost-attribution.
|
||||
Four built-in model profiles plus operator-defined `<custom>.yaml` (drop in `lib/profiles/`). Each profile pins `phase_models` for the six pipeline phases. The active profile is recorded in plan.md frontmatter as `profile: <name>` and emitted to JSONL stats for cost-attribution.
|
||||
|
||||
| Profile | Brief | Research | Plan | Execute | Review | Continue | Use case |
|
||||
|---------|-------|----------|------|---------|--------|----------|----------|
|
||||
| `economy` | sonnet | sonnet | sonnet | sonnet | sonnet | sonnet | ⚠ **Experimental** (uncalibrated Jaccard floor) — lowest cost; high-confidence small-scope tasks (opt-in via `--profile economy`) |
|
||||
| `balanced` | sonnet | sonnet | opus | sonnet | opus | sonnet | Mixed — opus where reasoning depth pays off (opt-in via `--profile balanced`) |
|
||||
| `premium` (default) | opus | opus | opus | opus | opus | opus | Maximum quality — Opus on every phase (default since the 2026-05-13 operator decision) |
|
||||
| `fable` | fable | fable | fable | fable | fable | fable | Max quality — Fable 5 (Mythos-class, above Opus) on every phase (opt-in via `--profile fable`); reasoning effort inherits from the session |
|
||||
|
||||
Lookup order:
|
||||
|
||||
|
|
@ -809,9 +808,9 @@ Default JSONL stats stream (`${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl`) is unchan
|
|||
|
||||
## Cost profile
|
||||
|
||||
The default `premium` profile runs **Opus on every phase** — the orchestrator (one per command), the exploration and review swarms (5–10 sub-agents per command, all `model: opus`-pinned in `agents/*.md`), and the executor (one per plan session). The model is **uniform per phase**: there is no "Opus orchestrates, Sonnet runs the swarms" split — a phase resolves to one model and both the orchestrator and its sub-agents use it. For cheaper runs, opt into `--profile balanced` (Sonnet on brief/research/execute/continue, Opus on plan + review) or `--profile economy` (Sonnet everywhere). Per-command cost is published in `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` if you want exact numbers.
|
||||
The default `premium` profile runs **Opus on every phase** of the pipeline's agent work — the exploration and review swarms (5–10 sub-agents per command; spawn sites inject the composed brief > profile > frontmatter resolution, with `agents/*.md` `model: opus` pins as the fallback) and the executor (one per plan session). The command orchestrator itself is not profile-controlled: as of v5.9, command frontmatter omits `model:`, so the orchestrator follows the session model. For cheaper runs, opt into `--profile balanced` (Sonnet on brief/research/execute/continue, Opus on plan + review) or `--profile economy` (Sonnet everywhere); for maximum quality, `--profile fable` (Fable 5 on every phase). Per-command cost is published in `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` if you want exact numbers.
|
||||
|
||||
The `opus` alias resolves to **Opus 4.8** (default reasoning effort `high`) and `sonnet` to Sonnet 4.6. Note two distinct effort axes that share the word "effort": brief `phase_signals.effort` (low/standard/high) tunes *orchestration shape* — how many agents and passes run — while native `effort:` on selected agents (retrieval at `medium`, adversarial-reasoning at `high`) tunes the *per-spawn reasoning budget*. See [`docs/profiles.md`](docs/profiles.md) § Model & effort axes.
|
||||
The `opus` alias resolves to **Opus 4.8** (default reasoning effort `high`), `sonnet` to Sonnet 4.6, and `fable` to **Fable 5** (Mythos-class, above Opus; default reasoning effort `high` — xhigh requires a session-level setting, see [`docs/profiles.md`](docs/profiles.md)). Note two distinct effort axes that share the word "effort": brief `phase_signals.effort` (low/standard/high) tunes *orchestration shape* — how many agents and passes run — while native `effort:` on selected agents (retrieval at `medium`, adversarial-reasoning at `high`) tunes the *per-spawn reasoning budget*. See [`docs/profiles.md`](docs/profiles.md) § Model & effort axes.
|
||||
|
||||
For per-profile cost estimates, see [`docs/profiles.md`](docs/profiles.md).
|
||||
|
||||
|
|
@ -919,6 +918,18 @@ suppress this, leave the `architecture/` directory absent from your
|
|||
project directory. Discovery is additive — missing file is fine, no
|
||||
error.
|
||||
|
||||
## Changelog
|
||||
|
||||
Full version history → [CHANGELOG.md](CHANGELOG.md).
|
||||
|
||||
Recent, in one line each:
|
||||
|
||||
- **v5.8.0** — offline gold-scored output eval (SKAL-1·4b): `lib/review/gold-scorer.mjs` grades a committed agent-run fixture against the golden corpus at `(file, rule_key)` granularity; suite census gains a `goldEval` category. Offline + deterministic, no live agent spawn.
|
||||
- **v5.7.1** — leaner always-loaded agent listing (`<example>` blocks moved into agent bodies, ~3,180 tok/turn, no behavior change).
|
||||
- **v5.7.0** — opt-in per-session token/cost metering (SKAL-2) + eval foundation (SKAL-1·4a).
|
||||
- **v5.6.1** — one-line `description:` for the four reference/dormant agents (~700 tok).
|
||||
- **v5.5.0** — brief **framing** enforcement (`brief_version 2.2`) + a `/trekreview` reviewer-schema contract. Additive, no breaking changes.
|
||||
|
||||
## Contributing
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
|
|
|||
|
|
@ -7,12 +7,20 @@ tools: ["Read", "Glob", "Grep", "Write", "Edit", "Bash"]
|
|||
---
|
||||
|
||||
<!-- Phase mapping: orchestrator → command
|
||||
Corrected in v5.10: every row below was off by one, and the old last row
|
||||
pointed at a ninth command phase that does not exist — the command ends
|
||||
at Phase 8.
|
||||
Orchestrator Phase 1 = Command Phase 4 (Agent group selection)
|
||||
Orchestrator Phase 2 = Command Phase 5 (Parallel research)
|
||||
Orchestrator Phase 3 = Command Phase 6 (Targeted follow-ups)
|
||||
Orchestrator Phase 4 = Command Phase 7 (Triangulation)
|
||||
Orchestrator Phase 5 = Command Phase 8 (Synthesis + write brief)
|
||||
Orchestrator Phase 6 = Command Phase 9 (Completion)
|
||||
Orchestrator Phase 2 = Command Phase 4 (Parallel research — same
|
||||
command phase; the orchestrator
|
||||
splits selection from launch)
|
||||
(no orchestrator phase)= Command Phase 4.5 (Dimension discovery,
|
||||
high effort only — v5.10)
|
||||
Orchestrator Phase 3 = Command Phase 5 (Targeted follow-ups; bounded
|
||||
conversation loop at high effort)
|
||||
Orchestrator Phase 4 = Command Phase 6 (Triangulation)
|
||||
Orchestrator Phase 5 = Command Phase 7 (Synthesis + write brief)
|
||||
Orchestrator Phase 6 = Command Phase 8 (Present and track / completion)
|
||||
As of v2.4.0, /trekresearch runs these phases inline in main
|
||||
context instead of spawning this agent. Keep this file as the canonical
|
||||
reference for what those phases do. -->
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@
|
|||
name: trekbrief
|
||||
description: Interactive interview that produces a task brief with explicit research plan. Feeds /trekresearch and /trekplan. Optionally orchestrates the full pipeline end-to-end.
|
||||
argument-hint: "[--quick] <task description>"
|
||||
model: opus
|
||||
allowed-tools: Agent, Read, Glob, Grep, Write, Edit, Bash, AskUserQuestion
|
||||
---
|
||||
|
||||
|
|
@ -367,13 +366,14 @@ in the question body so the operator sees why it was picked.
|
|||
### The loop — 4 tier-coupled AskUserQuestion calls
|
||||
|
||||
Loop over `[research, plan, execute, review]` in order. For each phase,
|
||||
issue one `AskUserQuestion` with 3 options:
|
||||
issue one `AskUserQuestion` with 4 options:
|
||||
|
||||
| Option | Maps to phase_signals entry |
|
||||
|--------|----------------------------|
|
||||
| **Low effort** | `{phase: <name>, effort: low, model: sonnet}` |
|
||||
| **Standard (default)** | `{phase: <name>, effort: standard}` *(model omitted — composition falls through to profile)* |
|
||||
| **High effort** | `{phase: <name>, effort: high, model: opus}` |
|
||||
| **Fable (max quality)** | `{phase: <name>, effort: high, model: fable}` |
|
||||
|
||||
The proposed tier per phase (from the default-derivation heuristic) MUST be
|
||||
labelled `(default)` in the option list so the operator can one-click
|
||||
|
|
@ -384,6 +384,15 @@ The mapping table is canonical:
|
|||
- `low → {effort: low, model: sonnet}` (force sonnet for the low-cost path)
|
||||
- `standard → {effort: standard}` (model omitted; composition rule resolves via profile)
|
||||
- `high → {effort: high, model: opus}` (force opus for the high-confidence path)
|
||||
- `fable → {effort: high, model: fable}` (force Fable 5 for the max-quality path)
|
||||
|
||||
The fable tier reuses `effort: high` semantics — full swarm, contrarian +
|
||||
gemini always-on; `EFFORT_LEVELS` is unchanged (Voyage effort is orchestration
|
||||
shape, not model reasoning effort). Model reasoning effort is inherited from
|
||||
the session: Fable 5's default effort is `high`, NOT xhigh. To run xhigh, the
|
||||
operator sets it at session level via `/effort xhigh`, the `effortLevel`
|
||||
setting, or `CLAUDE_CODE_EFFORT_LEVEL` — switching model resets effort to the
|
||||
model default, so it does not follow the model.
|
||||
|
||||
### Force-stop handling
|
||||
|
||||
|
|
@ -891,7 +900,7 @@ Never let stats failures block the workflow.
|
|||
## Profile (v4.1)
|
||||
|
||||
Accepts `--profile <name>` where `<name>` is one of `economy`, `balanced`,
|
||||
`premium`, or a custom profile under `voyage-profiles/`. Default: `premium`.
|
||||
`premium`, `fable`, or a custom profile under `voyage-profiles/`. Default: `premium`.
|
||||
|
||||
Resolution order (per `lib/profiles/resolver.mjs`):
|
||||
1. `--profile` flag (source: `flag`)
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@
|
|||
name: trekcontinue
|
||||
description: Resume the next session in a multi-session trekplan project. Reads .session-state.local.json and immediately begins the next session.
|
||||
argument-hint: "[<project-dir> | --help]"
|
||||
model: opus
|
||||
---
|
||||
|
||||
# Ultracontinue Local v1.0
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@
|
|||
name: trekendsession
|
||||
description: Mark the current session as complete and write session-state pointing at the next session. Helper for informal multi-session flows.
|
||||
argument-hint: "<next-brief-path> <next-label> | --help"
|
||||
model: opus
|
||||
---
|
||||
|
||||
# Voyage End-Session Local v1.0
|
||||
|
|
@ -91,16 +90,16 @@ want an interactive flow, use `/trekcontinue --help` to see the full pipeline.
|
|||
|
||||
## Phase 3 — Atomically write `.session-state.local.json` + sibling NEXT-SESSION-PROMPT.local.md
|
||||
|
||||
Write `<project-dir>/.session-state.local.json` with the schema-v1 object:
|
||||
Write `{project_dir}/.session-state.local.json` with the schema-v1 object:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"project": "<project-dir>",
|
||||
"next_session_brief_path": "<arg 1>",
|
||||
"next_session_label": "<arg 2>",
|
||||
"project": "{project_dir}",
|
||||
"next_session_brief_path": "{arg 1}",
|
||||
"next_session_label": "{arg 2}",
|
||||
"status": "in_progress",
|
||||
"updated_at": "<now, ISO-8601>"
|
||||
"updated_at": "{now, ISO-8601}"
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -115,14 +114,22 @@ Under `node --input-type=module -e "<script>" arg1 arg2 arg3`, Node sets
|
|||
|
||||
This phase ALSO writes a sibling `NEXT-SESSION-PROMPT.local.md` in the
|
||||
project directory with YAML frontmatter (`produced_by: trekendsession`,
|
||||
`produced_at: <ISO-8601>`, `project: <project-dir>`). Both files are written
|
||||
in a single ESM block so the writes succeed or fail together:
|
||||
`produced_at: {ISO-8601}`, `project: {project_dir}`). Both files are written
|
||||
in a single ESM block so the writes succeed or fail together.
|
||||
|
||||
Run the block below via the Bash tool at runtime, substituting the resolved
|
||||
values for the `{curly}` placeholders (Phase 1 gives `{project_dir}`, Phase 2
|
||||
gives `{next_brief_path}` and `{next_label}`). This is NOT an eager-exec
|
||||
block — the values do not exist at command-load time. The import path must
|
||||
stay absolute via `${CLAUDE_PLUGIN_ROOT}` — your Bash cwd is the user's
|
||||
repo, not the plugin root, so a cwd-relative import throws
|
||||
`ERR_MODULE_NOT_FOUND`:
|
||||
|
||||
```bash
|
||||
!`node --input-type=module -e "
|
||||
node --input-type=module -e "
|
||||
import path from 'node:path';
|
||||
import { writeFileSync } from 'node:fs';
|
||||
import { atomicWriteJson } from './lib/util/atomic-write.mjs';
|
||||
import { atomicWriteJson } from '${CLAUDE_PLUGIN_ROOT}/lib/util/atomic-write.mjs';
|
||||
const [, dir, brief, label] = process.argv;
|
||||
const now = new Date().toISOString();
|
||||
const stateObj = { schema_version: 1, project: dir, next_session_brief_path: brief, next_session_label: label, status: 'in_progress', updated_at: now };
|
||||
|
|
@ -133,26 +140,28 @@ const promptBody = '---\\nproduced_by: trekendsession\\nproduced_at: ' + now + '
|
|||
writeFileSync(promptFile, promptBody);
|
||||
console.log(stateFile);
|
||||
console.log(promptFile);
|
||||
" '<project-dir>' '<next-brief-path>' '<next-label>'`
|
||||
" '{project_dir}' '{next_brief_path}' '{next_label}'
|
||||
```
|
||||
|
||||
## Phase 4 — Validate + narrate
|
||||
|
||||
Validate the freshly-written state file:
|
||||
Validate the freshly-written state file via the Bash tool at runtime,
|
||||
substituting the resolved `{project_dir}` (NOT eager-exec — the file does
|
||||
not exist at command-load time):
|
||||
|
||||
```bash
|
||||
!`node lib/validators/session-state-validator.mjs --json <project-dir>/.session-state.local.json`
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/session-state-validator.mjs --json {project_dir}/.session-state.local.json
|
||||
```
|
||||
|
||||
If `valid: true`, print the success block matching `/trekcontinue` Phase 3
|
||||
narration (SC-8 cross-project consistency — same template both sides):
|
||||
|
||||
```
|
||||
Session state written: <project-dir>/.session-state.local.json
|
||||
Session state written: {project_dir}/.session-state.local.json
|
||||
|
||||
Project: <project-dir>
|
||||
Next session: <next-label>
|
||||
Brief: <next-brief-path>
|
||||
Project: {project_dir}
|
||||
Next session: {next_label}
|
||||
Brief: {next_brief_path}
|
||||
|
||||
In a fresh Claude session, run /trekcontinue to resume.
|
||||
```
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@
|
|||
name: trekexecute
|
||||
description: Disciplined plan executor — single-session or multi-session with parallel orchestration, failure recovery, and headless support
|
||||
argument-hint: "[--project <dir>] [--fg | --resume | --dry-run | --validate | --step N | --session N] [plan.md]"
|
||||
model: opus
|
||||
allowed-tools: Read, Write, Edit, Bash, Glob, Grep, AskUserQuestion
|
||||
disallowed-tools: Agent, TeamCreate
|
||||
---
|
||||
|
|
@ -1577,7 +1576,7 @@ Never let stats failures block the workflow.
|
|||
## Profile (v4.1)
|
||||
|
||||
Accepts `--profile <name>` where `<name>` is `economy`, `balanced`, `premium`,
|
||||
or a custom profile under `voyage-profiles/`. Default: `premium`.
|
||||
`fable`, or a custom profile under `voyage-profiles/`. Default: `premium`.
|
||||
|
||||
Resolution order (per `lib/profiles/resolver.mjs`):
|
||||
1. `--profile` flag (source: `flag`)
|
||||
|
|
@ -1609,12 +1608,25 @@ model_for_phase = brief.phase_signals[<phase>]?.model ?? profile.phase_models[
|
|||
```
|
||||
|
||||
The brief signal wins per-phase when present; the profile fills any
|
||||
gaps. Composition is mechanically resolved via
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/phase-signal-resolver.mjs`
|
||||
invoked in Phase 2.4; the resolved JSON is captured as `phase_signal_result`
|
||||
and consumed when picking the orchestration model + parallel-wave
|
||||
strategy. The resolver controls only the orchestrator — sub-agents read
|
||||
`model:` from their own `agents/*.md` frontmatter (still pinned to `opus`).
|
||||
gaps. Both fields are mechanically resolved by the single composed CLI,
|
||||
invoked in Phase 2.4 alongside the sequencing-gate brief-validator call:
|
||||
|
||||
```bash
|
||||
# v5.9 — composed phase-model resolution (brief > profile > default) for the
|
||||
# execute phase. ONE call returns {effort, model, source}; captured as
|
||||
# phase_signal_result. Append --profile {profile} when the operator passed
|
||||
# --profile.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/resolver.mjs --resolve-phase-model --phase execute --brief-path "{dir}/brief.md" [--profile {profile}] --json
|
||||
```
|
||||
|
||||
`phase_signal_result.effort` is consumed when picking the execution
|
||||
strategy (gates auto-escalation, parallel-wave choice — see High-effort
|
||||
behavior below). The resolver does NOT control the orchestrator's own
|
||||
model — that is fixed at invocation time (command frontmatter omits
|
||||
`model:`, so it follows the session model) and cannot be switched mid-turn.
|
||||
`/trekexecute` spawns no sub-agent swarm (Hard Rule 10), so
|
||||
`phase_signal_result.model` has no spawn site here; it is returned for
|
||||
cross-command uniformity and stats.
|
||||
|
||||
For `/trekexecute` specifically: `effort == 'low'` activates `--gates open`
|
||||
+ sequential-only execution (no worktree-isolated parallel waves — runs
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@
|
|||
name: trekplan
|
||||
description: Deep implementation planning from a task brief. Requires --brief or --project. Runs parallel specialized agents, optional external research, and adversarial review.
|
||||
argument-hint: "--brief <path> | --project <dir> [--fg | --quick | --research <brief> | --decompose <plan> | --export headless <plan>]"
|
||||
model: opus
|
||||
allowed-tools: Agent, Read, Glob, Grep, Write, Edit, Bash, AskUserQuestion, TaskCreate, TaskUpdate, TeamCreate, TeamDelete
|
||||
---
|
||||
|
||||
|
|
@ -75,10 +74,11 @@ Parse `$ARGUMENTS` for mode flags. Order of precedence:
|
|||
# older brief that sidesteps framing enforcement.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/brief-validator.mjs --soft --json [--min-version {min_brief_version}] "{dir}/brief.md"
|
||||
|
||||
# v5.1.1 — resolve per-phase brief-signal for plan phase. Result is
|
||||
# captured as phase_signal_result and used at Agent-spawn sites below
|
||||
# to override the orchestrator model when a signal is present.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/phase-signal-resolver.mjs --brief "{dir}/brief.md" --phase plan --json
|
||||
# v5.9 — composed phase-model resolution (brief > profile > default) for
|
||||
# the plan phase. ONE call returns {effort, model, source}; captured as
|
||||
# phase_signal_result and injected at Agent-spawn sites below.
|
||||
# Append --profile {profile} when the operator passed --profile.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/resolver.mjs --resolve-phase-model --phase plan --brief-path "{dir}/brief.md" [--profile {profile}] --json
|
||||
|
||||
# Research briefs (if any) — drift-warn only, none of these block the run
|
||||
[ -d "{dir}/research" ] && \
|
||||
|
|
@ -824,7 +824,7 @@ Never let tracking failures block the main workflow.
|
|||
|
||||
## Profile (v4.1)
|
||||
|
||||
Accepts `--profile <name>` where `<name>` is `economy`, `balanced`, `premium`,
|
||||
Accepts `--profile <name>` where `<name>` is `economy`, `balanced`, `premium`, `fable`,
|
||||
or a custom profile under `voyage-profiles/`. Default: `premium`.
|
||||
|
||||
Resolution order (per `lib/profiles/resolver.mjs`):
|
||||
|
|
@ -859,13 +859,15 @@ model_for_phase = brief.phase_signals[<phase>]?.model ?? profile.phase_models[
|
|||
```
|
||||
|
||||
The brief signal wins per-phase when present; the profile fills any
|
||||
gaps. Composition is mechanically resolved via
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/phase-signal-resolver.mjs`
|
||||
invoked in Phase 1; the resolved JSON is captured as `phase_signal_result`
|
||||
and passed to `Agent` tool calls explicitly. The resolver controls only
|
||||
the orchestrator and the model parameter at Agent-spawn sites — sub-agents
|
||||
otherwise read `model:` from their own `agents/*.md` frontmatter (still
|
||||
pinned to `opus`).
|
||||
gaps. Both fields are mechanically resolved by the single composed CLI
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/resolver.mjs --resolve-phase-model`
|
||||
invoked in Phase 1; the resolved JSON `{effort, model, source}` is captured
|
||||
as `phase_signal_result` and passed to `Agent` tool calls explicitly. The
|
||||
resolver controls the `model` parameter at Agent-spawn sites only — the
|
||||
orchestrator's own model is fixed at invocation time (command frontmatter
|
||||
omits `model:`, so it follows the session model) and cannot be switched
|
||||
mid-turn. Sub-agents fall back to `model:` in their own `agents/*.md`
|
||||
frontmatter when no spawn-site injection happens.
|
||||
|
||||
For `/trekplan` specifically: `effort == 'low'` activates the existing
|
||||
`--quick`-equivalent code-path (skip Phase 5 agent swarm — plan directly
|
||||
|
|
@ -910,10 +912,11 @@ Standard and low effort: do NOT run the additional pass.
|
|||
inadequate, stop and ask the user to run `/trekbrief` again.
|
||||
- **Scope**: Only explore the current working directory and its subdirectories.
|
||||
Never read files outside the repo (no ~/.env, no credentials, no other repos).
|
||||
- **Cost**: Sub-agents use their pinned `model:` frontmatter (currently `opus`).
|
||||
When `phase_signals[<phase>].model` is set, the orchestrator AND Agent-spawn
|
||||
sites use the resolved model (`phase_signal_result.model`) for that phase.
|
||||
Frontmatter is the default; brief signal is the per-phase override.
|
||||
- **Cost**: Model resolution at Agent-spawn sites is a three-layer fallback:
|
||||
brief `phase_signals[<phase>].model` > `profile.phase_models[<phase>]` >
|
||||
agent frontmatter `model:`. The composed resolver returns the first two
|
||||
layers as `phase_signal_result.model`; spawn sites inject it, and agent
|
||||
frontmatter is the fallback when no injection happens.
|
||||
- **Privacy**: Never log, store, or repeat file contents that look like
|
||||
secrets, tokens, or credentials. Never log prompt text.
|
||||
- **No premature execution**: Do not modify any project files until the user
|
||||
|
|
|
|||
|
|
@ -1,8 +1,7 @@
|
|||
---
|
||||
name: trekresearch
|
||||
description: Deep research combining local codebase analysis with external knowledge, producing structured research briefs with triangulation and confidence ratings
|
||||
argument-hint: "[--project <dir>] [--quick | --local | --external | --fg] <research question>"
|
||||
model: opus
|
||||
argument-hint: "[--project <dir>] [--quick | --local | --external | --fg] [--engine swarm|deep-research] <research question>"
|
||||
allowed-tools: Agent, Read, Glob, Grep, Write, Edit, Bash, AskUserQuestion, WebSearch, WebFetch, mcp__tavily__tavily_search, mcp__tavily__tavily_research
|
||||
---
|
||||
|
||||
|
|
@ -55,15 +54,18 @@ Supported flags:
|
|||
Create `{dir}/research/` if it does not already exist.
|
||||
|
||||
When `{dir}/brief.md` exists, ALWAYS run the brief-validator (soft mode)
|
||||
AND the phase-signal-resolver for this command's phase before continuing.
|
||||
The resolver's JSON output is captured as `phase_signal_result` and used
|
||||
at Agent-spawn sites in Phase 4 to inject the brief-resolved model:
|
||||
AND the composed phase-model resolver for this command's phase before
|
||||
continuing. The resolver's JSON output `{effort, model, source}`
|
||||
(brief signal > profile > default) is captured as `phase_signal_result`
|
||||
and used at Agent-spawn sites in Phase 4 to inject the resolved model:
|
||||
|
||||
```bash
|
||||
# When --min-brief-version was passed, append --min-version {min_brief_version}
|
||||
# so an older brief raises BRIEF_VERSION_BELOW_MINIMUM (warn, never block).
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/brief-validator.mjs --soft --json [--min-version {min_brief_version}] "{dir}/brief.md"
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/phase-signal-resolver.mjs --brief "{dir}/brief.md" --phase research --json
|
||||
# v5.9 — composed resolver: ONE call returns {effort, model, source}.
|
||||
# Append --profile {profile} when the operator passed --profile.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/resolver.mjs --resolve-phase-model --phase research --brief-path "{dir}/brief.md" [--profile {profile}] --json
|
||||
```
|
||||
|
||||
6. `--gates` — autonomy control. When present, set `gates_mode = true`. The
|
||||
|
|
@ -80,6 +82,16 @@ Supported flags:
|
|||
enforcement only fires at `≥ 2.2`. Absent → no version check. See
|
||||
`docs/HANDOVER-CONTRACTS.md` §Handover 1 for the pre-2.2 enforcement hole.
|
||||
|
||||
8. `--engine <name>` — opt-in external-research engine. Accepts `--engine <name>`
|
||||
where `<name>` is `swarm` or `deep-research`. **Default: `swarm`** (unchanged
|
||||
behavior). `swarm` runs Voyage's own external-research agent swarm;
|
||||
`deep-research` delegates the external phase to Claude Code's built-in
|
||||
`/deep-research` dynamic workflow and adapts its report into the research-brief
|
||||
schema (requires Claude Code 2.1.154+ and dynamic workflows enabled; falls back
|
||||
to `swarm` and notes the fallback if unavailable — never hard-fails). Orthogonal
|
||||
to `--profile`/`phase_signals`; only affects the external phase. Set
|
||||
**engine = {swarm|deep-research}** (the *requested* engine).
|
||||
|
||||
Flags can be combined:
|
||||
- `--local` — local-only research
|
||||
- `--external --quick` — external-only, lightweight
|
||||
|
|
@ -87,7 +99,7 @@ Flags can be combined:
|
|||
- `--quick` alone implies both local and external (lightweight)
|
||||
|
||||
Defaults: **scope = both**, **execution = foreground** (only mode as of
|
||||
v2.4.0), **project_dir = none**.
|
||||
v2.4.0), **project_dir = none**, **engine = swarm**.
|
||||
|
||||
After stripping flags, the remaining text is the **research question**.
|
||||
|
||||
|
|
@ -108,6 +120,7 @@ Modes:
|
|||
--external Only external research agents (skip codebase analysis)
|
||||
--fg No-op alias (foreground is the only mode as of v2.4.0)
|
||||
--project Write brief into an trekbrief project folder (auto-indexed)
|
||||
--engine Opt-in external-research engine: swarm (default) | deep-research
|
||||
|
||||
Flags can be combined: --local, --external --quick, --project <dir> --external
|
||||
|
||||
|
|
@ -118,6 +131,7 @@ Examples:
|
|||
/trekresearch --external What are the security implications of using Redis for sessions?
|
||||
/trekresearch --fg --local What patterns does this codebase use for database access?
|
||||
/trekresearch --project .claude/projects/2026-04-18-jwt-auth --external What JWT library is best for Node.js?
|
||||
/trekresearch --project <dir> --external --engine deep-research <research question>
|
||||
```
|
||||
|
||||
Do not continue past this step if no question was provided.
|
||||
|
|
@ -126,6 +140,7 @@ Report the detected mode:
|
|||
```
|
||||
Mode: {default | quick}, Scope: {both | local | external}, Execution: foreground
|
||||
Project: {project_dir or "-"}
|
||||
Engine (requested): {swarm | deep-research}
|
||||
Question: {research question}
|
||||
```
|
||||
|
||||
|
|
@ -292,6 +307,63 @@ For each local agent, prompt with the research question, NOT a task description:
|
|||
- convention-scanner: "Discover coding conventions relevant to evaluating {question}.
|
||||
What patterns would a solution need to follow?"
|
||||
|
||||
### Engine selection (scope = both or external)
|
||||
|
||||
`--engine` affects ONLY the external portion of research. The local agents
|
||||
(`### Local agents` above) and Phases 6–7 (triangulation, synthesis, brief
|
||||
writing) are **engine-agnostic** — they run identically regardless of engine.
|
||||
|
||||
`--engine` is **moot** (treated as `swarm`) whenever the external phase does not
|
||||
run at all: `--local`, `--quick`, `effort == 'low'`, or a profile with
|
||||
`external_research_enabled == false` (the `economy`/`balanced` auto-disable — see
|
||||
Profile below). The profile's on/off switch wins. Initialize
|
||||
`effective_engine = {requested engine}`.
|
||||
|
||||
**engine = swarm (default):** run the `### External agents` + `### Bridge agent`
|
||||
blocks below unchanged. This is byte-for-byte the current path, so `--engine swarm`
|
||||
changes nothing (SC1). Keep the native-swarm anchors intact ("in parallel",
|
||||
"single message", `model: "opus"`).
|
||||
|
||||
**engine = deep-research:**
|
||||
|
||||
1. **Coarse pre-gate (best-effort, NOT a trust signal).** `Bash: claude --version`;
|
||||
parse the leading `X.Y.Z` (e.g. from `2.1.196 (Claude Code)`) and compare
|
||||
numerically against `2.1.154` — split each on `.` and compare major, then minor,
|
||||
then patch as integers (do NOT string-compare; lexical comparison mis-orders
|
||||
multi-digit patch numbers). If the version is `< 2.1.154`, OR if
|
||||
`disableWorkflows: true` / `CLAUDE_CODE_DISABLE_WORKFLOWS=1` is set, skip to the
|
||||
fallback (step 4). **If `claude` is not on PATH inside the Bash tool (possible
|
||||
under `claude -p`) or the version cannot be parsed, treat the pre-gate as
|
||||
*indeterminate* and proceed to step 2 — do NOT hard-fail.** There is no positive
|
||||
availability probe (research Dim 4), so a passing pre-gate does not guarantee the
|
||||
workflow runs; the post-hoc check (step 3) is the authoritative guard.
|
||||
|
||||
2. **Run.** Instruct Claude (in prose, this turn) to run
|
||||
`/deep-research <research question>` and request per-claim citations. Note:
|
||||
interactive default/acceptEdits triggers a per-run approval prompt; `claude -p` /
|
||||
SDK / bypass runs immediately.
|
||||
|
||||
3. **Post-hoc presence + provenance check (the real guard).** Verify a real, cited
|
||||
`/deep-research` report actually landed in context — substantive findings with
|
||||
citations, not an empty/denied/errored turn and not bare error text. This check
|
||||
must be **robust to all failure manifestations** (workflow disabled, approval
|
||||
denied, runtime error, empty output), because the disabled-headless behavior is
|
||||
undocumented: no recognizable cited report in context → fall back, regardless of
|
||||
how the failure surfaces.
|
||||
|
||||
4. **On no real report (fallback):** set `effective_engine = swarm`, run the swarm
|
||||
blocks below, and **log the fallback at this decision point** — print
|
||||
`Engine: deep-research → swarm (fallback: <reason>)` and carry the reason into the
|
||||
Phase-8 Present summary and the brief's `## Executive Summary`. **NEVER fabricate
|
||||
or synthesize a substitute report** — a structurally-valid-but-invented brief
|
||||
passes the structure-only validator and silently poisons `/trekplan`; that is the
|
||||
worst outcome of this feature.
|
||||
|
||||
5. **On a real report:** keep `effective_engine = deep-research`, log
|
||||
`Engine: deep-research (active)`, and carry the report into Phase 6 triangulation
|
||||
as the external-findings input (adapted in Phase 7 — see the Deep-research engine
|
||||
adapter below).
|
||||
|
||||
### External agents (scope = both or external)
|
||||
|
||||
Launch the new research-specialized agents:
|
||||
|
|
@ -319,15 +391,128 @@ other agents — the value of Gemini is independence.
|
|||
small = halved, medium/large = default
|
||||
- convention-scanner: medium+ codebases only (50+ files)
|
||||
|
||||
## Phase 4.5 — Dimension discovery
|
||||
|
||||
**Skip this phase entirely unless `phase_signal_result.effort == 'high'`.**
|
||||
|
||||
Phase 4 retrieves more than the interview knew to ask for. This phase mines
|
||||
that surplus: findings that were **retrieved but unintegrated** — material an
|
||||
agent surfaced that no interview dimension claims.
|
||||
|
||||
1. **Mine.** Walk the Phase-4 agent results and collect findings that map to
|
||||
no existing dimension.
|
||||
2. **Rerank.** Order candidates by relevance to the research question **and**
|
||||
dissimilarity to the dimensions already on the list. A candidate that
|
||||
restates an existing dimension is not a discovery.
|
||||
3. **Augment under the existing ceiling.** Append candidates to the dimension
|
||||
list only while the **whole** list (interview + discovered) stays at or
|
||||
below `maxDimensions: 8` (`settings.json:16`). The ceiling is **not**
|
||||
raised here, so the documented 3–8 dimension range stays true and the
|
||||
README prose about it stays untouched. If the interview already produced 8
|
||||
dimensions, this phase discovers nothing and says so.
|
||||
4. **Record the baseline.** Keep the interview-derived count as
|
||||
`dimensions_baseline` so the discovered delta is machine-readable against
|
||||
the final `dimensions` (Phase 8 stats).
|
||||
|
||||
Every outbound query generated from a discovered dimension passes
|
||||
`query-privacy-gate.mjs` before it leaves the machine — see the per-turn
|
||||
protocol in Phase 5. That gate is the compensating control for the
|
||||
Independence crossing this phase makes (see Hard rules → Independence).
|
||||
|
||||
## Phase 5 — Targeted follow-ups
|
||||
|
||||
Review all agent results. Identify knowledge gaps — areas where findings are
|
||||
thin, contradictory, or missing.
|
||||
Review all agent results. Identify knowledge gaps — dimensions where findings
|
||||
are thin, contradictory, or missing (**under-illuminated dimensions**).
|
||||
|
||||
For each significant gap, launch a targeted follow-up agent (model: "opus")
|
||||
with a narrow, specific brief. Maximum 2 follow-ups.
|
||||
**Standard and low effort — unchanged single pass.** For each significant gap,
|
||||
launch a targeted follow-up agent (model: "opus") with a narrow, specific
|
||||
brief. Maximum 2 follow-ups. If no gaps exist, skip: "Initial research
|
||||
sufficient — no follow-ups needed." Then go to Phase 6.
|
||||
|
||||
If no gaps exist, skip: "Initial research sufficient — no follow-ups needed."
|
||||
**The bounded loop below runs ONLY when `phase_signal_result.effort == 'high'`**
|
||||
(resolved in Phase 1; see `### High-effort behavior (v5.1.1)`). At any other
|
||||
effort this whole sub-section is inert — no loop, no cap ledger, no new
|
||||
counters beyond zero.
|
||||
|
||||
### Loop bound
|
||||
|
||||
**Maximum 3 turns per under-illuminated dimension.** The bound is per
|
||||
dimension, not per run: the worst case is 3 turns × the whole dimension list
|
||||
under the `maxDimensions: 8` ceiling (`settings.json:16`), which is what
|
||||
`research-loop-cap.mjs` sizes itself against. The cap counts itself from its
|
||||
own append-only ledger — it never asks this prose how many turns it has used.
|
||||
|
||||
The loop is **default-off**: `research-loop-cap.mjs` grants a budget of 0
|
||||
unless `VOYAGE_STORM_ENABLED=1`. Doing nothing leaves the mechanism off.
|
||||
|
||||
### Per-turn protocol
|
||||
|
||||
Each turn targets exactly one under-illuminated dimension, and runs two gates
|
||||
before it spends anything:
|
||||
|
||||
```bash
|
||||
# 1. Budget gate — per turn, per dimension. Exit 0 = granted, exit 1 = denied.
|
||||
# JSON on stdout: {ok, used, budget, reason?}
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/util/research-loop-cap.mjs \
|
||||
--run-id {run_id} --dimension {dimension} --effort {phase_signal_result.effort}
|
||||
|
||||
# 2. Privacy gate — EVERY outbound query, before it leaves the machine.
|
||||
# Exit 0 = send as-is; exit 1 = rewrite the query and re-gate. Never bypass.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/query-privacy-gate.mjs "{query text}"
|
||||
```
|
||||
|
||||
A denied budget gate is an exit condition, not a retry. A failed privacy gate
|
||||
is a rewrite: the hard-block tier (secret-shaped strings) is never
|
||||
operator-overridable, so a query that trips it must be reformulated, not
|
||||
forced through.
|
||||
|
||||
**Empty turns.** A turn that returns no findings, or findings without
|
||||
citations, is marked `empty`. An empty turn is counted in `empty_turns` and
|
||||
does NOT re-target the same dimension — re-asking the same question of the
|
||||
same silence is how a bounded loop turns into an unbounded one. Move to the
|
||||
next under-illuminated dimension, or exit.
|
||||
|
||||
### Exits (all three, always one of them)
|
||||
|
||||
1. **Converged** — the dimension carries findings with citations and no
|
||||
remaining contradiction. Stop turning on it. This is the normal exit.
|
||||
2. **Cap exhausted** — `research-loop-cap.mjs` denies the turn. Print the
|
||||
exhaustion **visibly** to the operator, never silently:
|
||||
`Loop bound reached for dimension {dimension} after {N} turns — remaining
|
||||
gaps are carried into the brief as open questions.` A silent cap is
|
||||
indistinguishable from convergence, and that confusion is exactly what this
|
||||
phase exists to prevent.
|
||||
3. **Operator stop** — the operator interrupts. Carry whatever has been
|
||||
gathered into Phase 6 and record the remaining gaps as open questions. Do
|
||||
not re-enter the loop after a stop.
|
||||
|
||||
### When the loop does not apply
|
||||
|
||||
**No-brief default.** Without `--project` (or with a project whose `brief.md`
|
||||
is absent), there are no `phase_signals` to resolve, so `effort = 'standard'`,
|
||||
the loop does not run, and all new counters (`conv_turns`, `empty_turns`) are
|
||||
emitted as `0`.
|
||||
|
||||
**Precedence matrix — each entry independently makes the loop moot**, the same
|
||||
way `--engine` is moot when the external phase does not run (see the moot gate
|
||||
in Phase 4):
|
||||
|
||||
| Condition | Effect on the loop |
|
||||
|-----------|--------------------|
|
||||
| `--quick` | Moot — Phase 3.5 skips to Phase 8; the swarm never runs |
|
||||
| `--local` | Moot — no outbound queries to bound |
|
||||
| `external_research_enabled: false` (profile) | Moot — the profile's on/off switch wins |
|
||||
|
||||
**Interaction rule.** A brief that carries `effort: high` **without** a
|
||||
`model`, under a cheap profile (`economy`/`balanced`): the effort signal
|
||||
governs orchestration shape, so the loop is armed, but the profile still
|
||||
supplies the model — and if that profile disables external research, the
|
||||
matrix above wins and the loop is moot regardless of effort.
|
||||
|
||||
**Honesty (hard rule, restated for this loop).** More turns do not make a
|
||||
finding more credible. Turn count is a cost, not evidence: report what the
|
||||
citations support, and let an exhausted cap show up as open questions rather
|
||||
than as confidence.
|
||||
|
||||
## Phase 6 — Triangulation
|
||||
|
||||
|
|
@ -373,6 +558,45 @@ Write the brief to the `brief_destination` computed in Phase 1:
|
|||
|
||||
Create the parent directory if it does not exist.
|
||||
|
||||
### Deep-research engine adapter (engine = deep-research only)
|
||||
|
||||
**Only when `effective_engine == deep-research`.** The swarm path skips this
|
||||
entirely — its findings already flow through Phases 6–7 unchanged (SC1).
|
||||
|
||||
Transform the in-context `/deep-research` report INTO
|
||||
`@${CLAUDE_PLUGIN_ROOT}/templates/research-brief-template.md` — do NOT paste the
|
||||
raw report. Specifically:
|
||||
|
||||
- Reduce the report to ≥ 1 `### {Dimension} -- Confidence: {high|medium|low}`
|
||||
entry, each carrying **External findings** bullets with per-claim source URLs.
|
||||
Local findings still come from the local agents (Phase 4) and are merged in per
|
||||
dimension as usual.
|
||||
- Emit a numeric `confidence ∈ [0,1]` in frontmatter and a 3-sentence
|
||||
`## Executive Summary` (answer, confidence, key caveat).
|
||||
- Populate `## Sources` from the report's citations.
|
||||
- **If the report lacks per-claim URLs, lower the confidence and note the gap in
|
||||
`## Open Questions` — do NOT fabricate URLs.** Provenance you cannot cite is not
|
||||
provenance.
|
||||
- If the report is large, bound the transform to the top dimensions to avoid
|
||||
context truncation.
|
||||
|
||||
### Output self-check (engine = deep-research only)
|
||||
|
||||
**Only when `effective_engine == deep-research`.** After writing to
|
||||
`brief_destination`, run the output validator and repair-or-fall-back. This mirrors
|
||||
the trekplan Phase-8 write→validate→repair self-check; the swarm path does NOT run
|
||||
it, so swarm behavior is unchanged (SC1):
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/research-validator.mjs --json "{brief_destination}"
|
||||
```
|
||||
|
||||
On `valid: false`, repair the brief to satisfy the reported errors and re-run the
|
||||
validator. If it cannot be made valid (e.g. the report was too thin to yield even
|
||||
one dimension), set `effective_engine = swarm`, fall back to the swarm engine for
|
||||
this run (and log the fallback per the Engine selection step), rather than emit an
|
||||
invalid brief.
|
||||
|
||||
## Phase 8 — Present and track
|
||||
|
||||
Present a summary to the user:
|
||||
|
|
@ -384,6 +608,7 @@ Present a summary to the user:
|
|||
**Mode:** {default | quick}, Scope: {both | local | external}
|
||||
**Brief:** {brief_destination}
|
||||
**Project:** {project_dir or "-"}
|
||||
**Engine (effective):** {swarm | deep-research}{, with fallback reason if it fell back}
|
||||
**Confidence:** {overall confidence 0.0-1.0}
|
||||
**Dimensions:** {N} researched
|
||||
**Agents:** {N} local + {N} external + {gemini: used | unavailable | skipped}
|
||||
|
|
@ -418,10 +643,16 @@ Record format (one JSON line):
|
|||
"question": "{research question (first 100 chars)}",
|
||||
"mode": "{default|quick}",
|
||||
"scope": "{both|local|external}",
|
||||
"engine": "{effective engine: swarm|deep-research}",
|
||||
"slug": "{brief slug}",
|
||||
"project_dir": "{project_dir or null}",
|
||||
"brief_path": "{brief_destination}",
|
||||
"dimensions": {N},
|
||||
"dimensions_baseline": {N},
|
||||
"effort": "{low|standard|high}",
|
||||
"conv_turns": {N},
|
||||
"empty_turns": {N},
|
||||
"unique_sources": {N},
|
||||
"agents_local": {N},
|
||||
"agents_external": {N},
|
||||
"gemini_used": {true|false},
|
||||
|
|
@ -431,11 +662,22 @@ Record format (one JSON line):
|
|||
}
|
||||
```
|
||||
|
||||
**The five measurement fields (v5.10).** `effort` is the grouping key — the
|
||||
resolved `phase_signal_result.effort` for the `research` phase, a
|
||||
low-cardinality label (`low|standard|high`), and the only axis on which a
|
||||
high-effort run can be compared against a standard one. The other four are
|
||||
numeric: `unique_sources` (distinct sources cited across the brief),
|
||||
`dimensions_baseline` (the interview-derived dimension count, so the Phase 4.5
|
||||
delta against `dimensions` is machine-readable), `conv_turns` (Phase 5 loop
|
||||
turns actually spent), and `empty_turns` (loop turns that returned no findings
|
||||
or no citations). On a standard run the loop never arms, so
|
||||
`dimensions_baseline == dimensions` and both turn counters are `0`.
|
||||
|
||||
If `${CLAUDE_PLUGIN_DATA}` is not set or not writable, skip tracking silently.
|
||||
|
||||
## Profile (v4.1)
|
||||
|
||||
Accepts `--profile <name>` where `<name>` is `economy`, `balanced`, `premium`,
|
||||
Accepts `--profile <name>` where `<name>` is `economy`, `balanced`, `premium`, `fable`,
|
||||
or a custom profile under `voyage-profiles/`. Default: `premium`.
|
||||
|
||||
Resolution order (per `lib/profiles/resolver.mjs`):
|
||||
|
|
@ -455,8 +697,8 @@ VOYAGE_PROFILE=balanced /trekresearch
|
|||
```
|
||||
|
||||
Stats records emit `profile`, `phase_models`, `parallel_agents`,
|
||||
`external_research_enabled`, and `profile_source` so operators can audit
|
||||
which profile drove which session.
|
||||
`external_research_enabled`, `profile_source`, and `engine` so operators can
|
||||
audit which profile and engine drove which session.
|
||||
|
||||
## Composition rule (v5.1)
|
||||
|
||||
|
|
@ -470,13 +712,15 @@ model_for_phase = brief.phase_signals[<phase>]?.model ?? profile.phase_models[
|
|||
```
|
||||
|
||||
The brief signal wins per-phase when present; the profile fills any
|
||||
gaps. Composition is mechanically resolved via
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/phase-signal-resolver.mjs`
|
||||
invoked in Phase 1; the resolved JSON is captured as `phase_signal_result`
|
||||
and passed to `Agent` tool calls explicitly. The resolver controls only
|
||||
the orchestrator and the model parameter at Agent-spawn sites — sub-agents
|
||||
otherwise read `model:` from their own `agents/*.md` frontmatter (still
|
||||
pinned to `opus`).
|
||||
gaps. Both fields are mechanically resolved by the single composed CLI
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/resolver.mjs --resolve-phase-model`
|
||||
invoked in Phase 1; the resolved JSON `{effort, model, source}` is captured
|
||||
as `phase_signal_result` and passed to `Agent` tool calls explicitly. The
|
||||
resolver controls the `model` parameter at Agent-spawn sites only — the
|
||||
orchestrator's own model is fixed at invocation time (command frontmatter
|
||||
omits `model:`, so it follows the session model) and cannot be switched
|
||||
mid-turn. Sub-agents fall back to `model:` in their own `agents/*.md`
|
||||
frontmatter when no spawn-site injection happens.
|
||||
|
||||
For `/trekresearch` specifically: `effort == 'low'` activates the
|
||||
existing `--quick`-equivalent code-path (inline research, no agent swarm).
|
||||
|
|
@ -508,6 +752,14 @@ significant architectural questions or when triangulation value is
|
|||
high; in high-effort mode it runs unconditionally to provide an
|
||||
independent second opinion.
|
||||
|
||||
High effort additionally arms the Phase 5 bounded follow-up loop (max 3
|
||||
turns per under-illuminated dimension, budgeted by
|
||||
`research-loop-cap.mjs`, every outbound query gated by
|
||||
`query-privacy-gate.mjs`). The loop stays default-off until
|
||||
`VOYAGE_STORM_ENABLED=1`, and the moot matrix in Phase 5 (`--quick`,
|
||||
`--local`, `external_research_enabled: false`) overrides the effort
|
||||
signal whenever the external phase does not run at all.
|
||||
|
||||
Standard effort (or absent): use the existing conditional triggers.
|
||||
Low effort: inline research only, no agent swarm (existing
|
||||
`--quick`-equivalent code-path).
|
||||
|
|
@ -519,12 +771,23 @@ Low effort: inline research only, no agent swarm (existing
|
|||
- **Sources required:** Every claim must cite a source. No unsourced findings.
|
||||
- **Independence:** Do not pre-bias external agents with local findings or vice versa.
|
||||
Triangulate AFTER independent research.
|
||||
**Amended (v5.10) for Phase 4.5:** dimension discovery deliberately crosses this
|
||||
rule. Its candidate dimensions are mined from the Phase-4 result set, which
|
||||
contains output from the five local codebase agents, so a discovered dimension
|
||||
can carry local context into an external query. The crossing is bounded to
|
||||
Phase 4.5 and the Phase 5 loop it feeds, it never applies to the initial
|
||||
external swarm (which stays blind to local findings), and the compensating
|
||||
control is `query-privacy-gate.mjs`: every outbound query is inspected before
|
||||
it leaves the machine, with a hard-block tier for secret-shaped strings that no
|
||||
operator flag can override. Triangulation still happens AFTER independent
|
||||
research.
|
||||
- **Graceful degradation:** If MCP tools are unavailable (Tavily, Gemini, MS Learn),
|
||||
proceed with available tools and note limitations in brief metadata.
|
||||
- **Cost:** Sub-agents use their pinned `model:` frontmatter (currently `opus`).
|
||||
When `phase_signals[<phase>].model` is set, the orchestrator AND Agent-spawn
|
||||
sites use the resolved model (`phase_signal_result.model`) for that phase.
|
||||
Frontmatter is the default; brief signal is the per-phase override.
|
||||
- **Cost:** Model resolution at Agent-spawn sites is a three-layer fallback:
|
||||
brief `phase_signals[<phase>].model` > `profile.phase_models[<phase>]` >
|
||||
agent frontmatter `model:`. The composed resolver returns the first two
|
||||
layers as `phase_signal_result.model`; spawn sites inject it, and agent
|
||||
frontmatter is the fallback when no injection happens.
|
||||
- **Privacy:** Never log secrets, tokens, or credentials.
|
||||
- **Honesty:** If the question is trivially answerable, say so. Don't inflate research.
|
||||
- **Scope of codebase:** Only analyze the current working directory for local research.
|
||||
|
|
|
|||
|
|
@ -5,7 +5,6 @@ description: |
|
|||
review.md with severity-tagged findings (BLOCKER/MAJOR/MINOR/SUGGESTION)
|
||||
per Handover 6 (review → plan).
|
||||
argument-hint: "--project <dir> [--since <ref>] [--quick] [--validate] [--dry-run]"
|
||||
model: opus
|
||||
allowed-tools: Agent, Read, Glob, Grep, Write, Edit, Bash, AskUserQuestion
|
||||
---
|
||||
|
||||
|
|
@ -92,10 +91,12 @@ as the file is parseable:
|
|||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/brief-validator.mjs --soft --json "{brief_path}"
|
||||
|
||||
# v5.1.1 — resolve the review-phase brief signal. The JSON is captured as
|
||||
# v5.9 — composed phase-model resolution (brief > profile > default) for the
|
||||
# review phase. ONE call returns {effort, model, source}; captured as
|
||||
# phase_signal_result and used in Phase 7 at the reviewer-launch site to
|
||||
# inject the brief-resolved model.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/phase-signal-resolver.mjs --brief "{brief_path}" --phase review --json
|
||||
# inject the resolved model. Append --profile {profile} when the operator
|
||||
# passed --profile.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/resolver.mjs --resolve-phase-model --phase review --brief-path "{brief_path}" [--profile {profile}] --json
|
||||
```
|
||||
|
||||
Read the JSON output. If `valid: false` AND any error has code
|
||||
|
|
@ -420,7 +421,7 @@ the contract for that handover (see `docs/HANDOVER-CONTRACTS.md`).
|
|||
|
||||
## Profile (v4.1)
|
||||
|
||||
Accepts `--profile <name>` where `<name>` is `economy`, `balanced`, `premium`,
|
||||
Accepts `--profile <name>` where `<name>` is `economy`, `balanced`, `premium`, `fable`,
|
||||
or a custom profile under `voyage-profiles/`. Default: `premium`.
|
||||
|
||||
Resolution order (per `lib/profiles/resolver.mjs`):
|
||||
|
|
@ -452,13 +453,15 @@ model_for_phase = brief.phase_signals[<phase>]?.model ?? profile.phase_models[
|
|||
```
|
||||
|
||||
The brief signal wins per-phase when present; the profile fills any
|
||||
gaps. Composition is mechanically resolved via
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/phase-signal-resolver.mjs`
|
||||
invoked in Phase 2; the resolved JSON is captured as `phase_signal_result`
|
||||
and passed to `Agent` tool calls explicitly. The resolver controls only
|
||||
the orchestrator and the model parameter at Agent-spawn sites — sub-agents
|
||||
otherwise read `model:` from their own `agents/*.md` frontmatter (still
|
||||
pinned to `opus`).
|
||||
gaps. Both fields are mechanically resolved by the single composed CLI
|
||||
`node ${CLAUDE_PLUGIN_ROOT}/lib/profiles/resolver.mjs --resolve-phase-model`
|
||||
invoked in Phase 2; the resolved JSON `{effort, model, source}` is captured
|
||||
as `phase_signal_result` and passed to `Agent` tool calls explicitly. The
|
||||
resolver controls the `model` parameter at Agent-spawn sites only — the
|
||||
orchestrator's own model is fixed at invocation time (command frontmatter
|
||||
omits `model:`, so it follows the session model) and cannot be switched
|
||||
mid-turn. Sub-agents fall back to `model:` in their own `agents/*.md`
|
||||
frontmatter when no spawn-site injection happens.
|
||||
|
||||
For `/trekreview` specifically: `effort == 'low'` activates the existing
|
||||
`--quick`-equivalent code-path (skip the brief-conformance reviewer; run
|
||||
|
|
@ -515,10 +518,11 @@ Low effort: skip the brief-conformance reviewer entirely (existing
|
|||
`findings:\n - a\n - b`.
|
||||
- **Refuse-with-suggestion above 100 files / 100K tokens.** Never run
|
||||
blind on a giant diff. Use AskUserQuestion to surface the gate.
|
||||
- **Cost.** Sub-agents use their pinned `model:` frontmatter (currently `opus`).
|
||||
When `phase_signals[<phase>].model` is set, the orchestrator AND Agent-spawn
|
||||
sites use the resolved model (`phase_signal_result.model`) for that phase.
|
||||
Frontmatter is the default; brief signal is the per-phase override.
|
||||
- **Cost.** Model resolution at Agent-spawn sites is a three-layer fallback:
|
||||
brief `phase_signals[<phase>].model` > `profile.phase_models[<phase>]` >
|
||||
agent frontmatter `model:`. The composed resolver returns the first two
|
||||
layers as `phase_signal_result.model`; spawn sites inject it, and agent
|
||||
frontmatter is the fallback when no injection happens.
|
||||
- **Privacy.** Never log secrets, tokens, or credentials in review.md.
|
||||
Findings citing files with secret-like content must redact the secret
|
||||
in the `detail` field.
|
||||
|
|
|
|||
|
|
@ -115,9 +115,17 @@ Optional but standard sections: `## Non-Goals`, `## Constraints`, `## Preference
|
|||
- `BRIEF_INVALID_PHASE_SIGNALS` → strict halt; phase_signals must be a list of `{phase, effort?, model?}` entries.
|
||||
- `BRIEF_INVALID_PHASE_SIGNAL_PHASE` → strict halt; phase ∉ `[research, plan, execute, review]`.
|
||||
- `BRIEF_INVALID_EFFORT` → strict halt; effort ∉ `[low, standard, high]`.
|
||||
- `BRIEF_INVALID_MODEL` → strict halt; model ∉ `BASE_ALLOWED_MODELS` (currently `[sonnet, opus]`).
|
||||
- `BRIEF_INVALID_MODEL` → strict halt; model ∉ `BASE_ALLOWED_MODELS` (currently `[sonnet, opus, fable]`).
|
||||
- `BRIEF_SIGNALS_MUTUALLY_EXCLUSIVE` → strict halt; cannot set both `phase_signals` and `phase_signals_partial: true`.
|
||||
|
||||
**Compatibility direction of the v5.9 allowlist widening (`fable`):** enum
|
||||
widening is safe for new readers of old data, not old readers of new data.
|
||||
Existing sonnet/opus briefs stay valid under the v5.9+ validator (non-breaking,
|
||||
no `brief_version` bump — value-space extension, not a schema change). The
|
||||
reverse does NOT hold: a fable-bearing brief REQUIRES a v5.9+ validator — an
|
||||
older cached brief-validator (e.g. a stale v5.8 marketplace clone) rejects it
|
||||
with `BRIEF_INVALID_MODEL`.
|
||||
|
||||
---
|
||||
|
||||
## Handover 2 — research/*.md → plan
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ Doc-consistency test at `tests/lib/doc-consistency.test.mjs` pins agent-table co
|
|||
|
||||
**Brief:** 7-phase workflow: Parse mode → Create project dir → Phase 3 completeness loop (section-driven, no question cap) → Phase 3.5 per-phase effort dialog (v5.1) → Phase 4 draft/review/revise with `brief-reviewer` as stop-gate (max 3 iterations; gate = all dimensions ≥ 4 and research plan = 5) → Finalize (`brief.md` on pass, or `brief_quality: partial` on cap/force-stop) → Manual/auto opt-in → Stats. Always interactive. Auto mode runs research + plan inline in the main context (v2.4.0).
|
||||
|
||||
**Phase 3.5 (v5.1) — adaptive-depth signals:** Between Phase 3 completeness exit and Phase 4 draft, the operator commits an effort level (`low | standard | high`) and an optional `model` (`sonnet | opus`) per downstream phase (`research`, `plan`, `execute`, `review`) via 4 tier-coupled `AskUserQuestion` calls. The choices land in `brief.md` frontmatter as `phase_signals:` (a list of `{phase, effort?, model?}` entries) when committed, or `phase_signals_partial: true` when the operator force-stops. `brief_version: 2.1` activates the **sequencing gate**: validator emits `BRIEF_V51_MISSING_SIGNALS` if a 2.1-versioned brief lacks both fields. Downstream commands surface a friendly hint pointing back to `/trekbrief` — enforcement is validator-only. Composition is documented prose in each downstream command's `## Composition rule (v5.1)` section: `brief.phase_signals[phase] > profile.phase_models[phase]`. The brief signal wins per-phase when present; the profile fills gaps. `effort == low` activates each command's existing `--quick`-equivalent code-path (`/trekexecute` low-effort = `--gates open` + sequential-only). High-effort behavior is deferred to v5.1.1 per brief Non-Goal.
|
||||
**Phase 3.5 (v5.1) — adaptive-depth signals:** Between Phase 3 completeness exit and Phase 4 draft, the operator commits an effort level (`low | standard | high`) and an optional `model` (`sonnet | opus | fable`) per downstream phase (`research`, `plan`, `execute`, `review`) via 4 tier-coupled `AskUserQuestion` calls. The choices land in `brief.md` frontmatter as `phase_signals:` (a list of `{phase, effort?, model?}` entries) when committed, or `phase_signals_partial: true` when the operator force-stops. `brief_version: 2.1` activates the **sequencing gate**: validator emits `BRIEF_V51_MISSING_SIGNALS` if a 2.1-versioned brief lacks both fields. Downstream commands surface a friendly hint pointing back to `/trekbrief` — enforcement is validator-only. Composition is documented prose in each downstream command's `## Composition rule (v5.1)` section: `brief.phase_signals[phase] > profile.phase_models[phase]`. The brief signal wins per-phase when present; the profile fills gaps. `effort == low` activates each command's existing `--quick`-equivalent code-path (`/trekexecute` low-effort = `--gates open` + sequential-only). High-effort behavior is deferred to v5.1.1 per brief Non-Goal.
|
||||
|
||||
**Research:** Foreground workflow (v2.4.0): Parse mode → Interview → Parallel research swarm (5 local + 4 external + 1 bridge, spawned from main context) → Follow-ups → Triangulation → Synthesis + brief → Stats. With `--project`, writes to `{dir}/research/NN-slug.md`.
|
||||
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ Per-command flag tables, imported from `CLAUDE.md` via pointer.
|
|||
| _(default)_ | Dynamic interview until quality gates pass → brief.md with research plan |
|
||||
| `--quick` | Compact start; still escalates if required sections are weak or the brief-review gate fails → brief.md with research plan |
|
||||
| `--gates {true\|false}` | (v3.4.0) Boolean autonomy-gate flag; present → gating on. Policy (`gates_mode`) detailed under `## Autonomy mode` in `docs/operations.md`. |
|
||||
| `--profile <name>` | (v4.1.0) Model profile: `economy` / `balanced` / `premium` / `<custom>`. Sets `phase_models` for the brief phase. See `## Profile system` in `docs/operations.md`. |
|
||||
| `--profile <name>` | (v4.1.0) Model profile: `economy` / `balanced` / `premium` / `fable` / `<custom>`. Sets `phase_models` for the brief phase. See `## Profile system` in `docs/operations.md`. |
|
||||
|
||||
Always interactive. Phase 3 is a section-driven completeness loop (no hard cap on question count); Phase 4 runs a `brief-reviewer` stop-gate with max 3 review iterations. After writing the brief, asks the user to choose manual (print commands) or auto (Claude runs research + plan in foreground).
|
||||
|
||||
|
|
@ -26,6 +26,7 @@ Always interactive. Phase 3 is a section-driven completeness loop (no hard cap o
|
|||
| `--gates {true\|false}` | (v3.4.0) Boolean autonomy-gate flag; present → gating on. Policy (`gates_mode`) detailed under `## Autonomy mode` in `docs/operations.md`. |
|
||||
| `--min-brief-version <ver>` | (S18) Warn — never block — if an attached `--project` brief declares a version below `<ver>` (e.g. `2.2`), i.e. sidesteps framing enforcement |
|
||||
| `--profile <name>` | (v4.1.0) Model profile for the research phase. |
|
||||
| `--engine {swarm\|deep-research}` | (deep-research-engine) Opt-in external-research engine; `deep-research` delegates the external phase to Claude Code's built-in `/deep-research` workflow (CC 2.1.154+), falls back to `swarm`. Default `swarm`. |
|
||||
|
||||
Flags combine: `--project <dir> --local`, `--external --quick`.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,13 +1,25 @@
|
|||
---
|
||||
# Avstem frontmatter mot lib/validators/brief-validator.mjs før /trekplan
|
||||
type: trekbrief
|
||||
brief_version: "2.2"
|
||||
status: draft
|
||||
task: "Opt-in /deep-research-motor for /trekresearch ekstern-fase"
|
||||
slug: deep-research-engine
|
||||
framing: refine
|
||||
status: ready
|
||||
brief_quality: complete
|
||||
research_topics: 3
|
||||
research_status: complete
|
||||
phase_signals_partial: true
|
||||
---
|
||||
|
||||
# Brief — Opt-in `/deep-research`-motor for `/trekresearch`
|
||||
|
||||
## TL;DR
|
||||
- `--engine {swarm|deep-research}` på `/trekresearch` ekstern-fase; `swarm` default (uendret oppførsel).
|
||||
- `deep-research` delegerer ekstern research til Claude Codes innebygde workflow og adapterer inn i research-brief-skjemaet.
|
||||
- Lokal analyse, triangulering og H2-output (`research/NN-*.md`) er motor-uavhengig.
|
||||
- Feature-detekteres med auto-fallback til `swarm` — aldri hard feil.
|
||||
- Surface-only: `commands/` + `agents/`, ingen nye `lib/`-avhengigheter.
|
||||
|
||||
## Intent
|
||||
`/trekresearch` eier i dag hele den eksterne research-fasen selv (Tavily / MS-Learn / Gemini-sverm). Det er portabelt, men du vedlikeholder fan-out, kryssjekk og syntese selv. Anthropic sin innebygde `/deep-research` vedlikeholder fan-out, adversariell påstandsverifisering, sitatfiltrering og websøk for deg. Lar vi operatøren *velge* `/deep-research` for den eksterne fasen, får brukeren en vedlikeholdsfri "turbo" når den er tilgjengelig — uten at Voyage mister sin egen lokal-analyse, triangulering eller H2-kontrakt.
|
||||
|
||||
|
|
@ -39,6 +51,8 @@ research_topics: 3
|
|||
`/trekresearch --project <dir> --local "Hva krever research-validator.mjs av research/NN-*.md?"`
|
||||
3. **Feature-deteksjon + fallback** — Hvordan oppdage om workflows er på, og hvor i fasen fallback-grenen bør sitte? Scope: local · Konfidens: middels · Kost: lav.
|
||||
|
||||
> **Research-status (2026-06-30, operatør-beslutning «option A»):** Topic 1 (eneste ekte eksterne ukjente) er undersøkt → `docs/deep-research-engine-research.md` (validator-grønn). Funn: `/deep-research` er en innebygd **dynamic workflow** (ikke skill) → trigging KUN via prosa-instruksjon, output **inline i kontekst** (ingen on-disk-artefakt) → motoren må være instruksjons-basert + in-context transform. **SC3 er korrekt som skrevet** (`/deep-research` ER en dynamic workflow). Topic 2 (validator-skjema: `type/created/question` + `## Executive Summary`/`## Dimensions`, `dimensions ≥ 1`) og topic 3 (fallback-plassering) er lokale kode-spørsmål reklassifisert til `/trekplan`-utforskning. `research_status: complete` reflekterer denne beslutningen.
|
||||
|
||||
## Open Questions / Assumptions
|
||||
- Antar at `/deep-research`-rapporten kan reduseres til ≥ 1 dimensjon med per-påstand-sitater uten å bryte trianguleringen. Verifiseres i topic 2.
|
||||
- Uavklart om delegering skjer via instruksjon (trygt) eller programmatisk API (raskere) — topic 1 avgjør.
|
||||
|
|
|
|||
210
docs/deep-research-engine-research.md
Normal file
210
docs/deep-research-engine-research.md
Normal file
|
|
@ -0,0 +1,210 @@
|
|||
---
|
||||
type: trekresearch-brief
|
||||
created: 2026-06-30
|
||||
question: "Can a Claude Code plugin command programmatically trigger the built-in /deep-research workflow and capture its report, or must it instruct Claude to run it?"
|
||||
confidence: 0.85
|
||||
dimensions: 4
|
||||
mcp_servers_used: []
|
||||
local_agents_used: [claude-code-guide]
|
||||
external_agents_used: []
|
||||
slug: deep-research-engine
|
||||
feeds_brief: docs/deep-research-engine-brief.md
|
||||
research_topic: 1
|
||||
---
|
||||
|
||||
# Research — Programmatic trigging of `/deep-research` from a plugin command
|
||||
|
||||
> Targeted single-topic research for the `deep-research-engine` brief (Topic 1 of
|
||||
> the brief's Research Plan). Topics 2 & 3 are local Voyage-code questions folded
|
||||
> into `/trekplan` exploration; only Topic 1 was a genuine external unknown.
|
||||
> Method: `claude-code-guide` agent (Anthropic docs + CHANGELOG, cited) +
|
||||
> direct inspection of this machine (CC 2.1.196 binary, a real local `/deep-research`
|
||||
> run). Not run through the `/trekresearch` swarm — see brief reconcile note.
|
||||
|
||||
## Research Question
|
||||
|
||||
Can a Claude Code **plugin slash-command** (markdown under `commands/`)
|
||||
**programmatically** start the built-in `/deep-research` workflow and **capture its
|
||||
report artifact** for adaptation into the research-brief schema — or must the command
|
||||
instead **instruct** Claude (in prose) to run `/deep-research` and then transform the
|
||||
in-context result?
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Programmatic trigger + file-based capture is **not feasible**: `/deep-research` is a
|
||||
built-in **dynamic workflow** (not a skill), deliberately outside the Skill-tool
|
||||
allowlist, and it returns its report **inline into conversation context with no
|
||||
documented on-disk report artifact**. The **only reliable path is instruction-based
|
||||
delegation** — the command's prose tells Claude to run `/deep-research <q>`, then
|
||||
transforms the in-context report in the same turn. Confidence **high** (Anthropic docs +
|
||||
CHANGELOG + local run), with one residual gap: there is no positive "is it enabled?"
|
||||
probe, so feature-detection must lean on the documented *disable* switches + version
|
||||
floor + a `swarm` default.
|
||||
|
||||
## Dimensions
|
||||
|
||||
### 1. Origin + gating -- Confidence: high
|
||||
|
||||
**External findings:**
|
||||
- `/deep-research` is a **built-in dynamic workflow**, not a command and not a bundled
|
||||
skill. `commands.md` marks the `/deep-research <question>` row as **[Workflow]**;
|
||||
`workflows.md`: "Claude Code includes `/deep-research` as a built-in workflow."
|
||||
[VERIFIED — code.claude.com/docs/en/commands.md, code.claude.com/docs/en/workflows.md]
|
||||
- Gating: available on **all paid plans** (pro/max/team/enterprise) + API/Bedrock/Vertex/
|
||||
Foundry. **On Pro it must be turned on** in the *Dynamic workflows* row of `/config`.
|
||||
Also requires the **WebSearch tool** to be available.
|
||||
[VERIFIED — workflows.md, commands.md bundled-workflows row]
|
||||
- Version floor: dynamic workflows were **introduced in CC 2.1.154**; "Dynamic workflows
|
||||
require Claude Code v2.1.154 or later." The brief's `v2.1.154+` + `(Pro: via /config)`
|
||||
constraints are **both correct**. The exact version that first shipped the *named*
|
||||
`/deep-research` workflow is **[NOT DOCUMENTED]** (only a 2.1.196 bugfix mentions it by
|
||||
name) — treat 2.1.154 as the substrate floor, not a proven introduction point.
|
||||
[VERIFIED — workflows.md + CHANGELOG 2.1.154]
|
||||
|
||||
**Local findings:**
|
||||
- This machine runs **CC 2.1.196** (`claude --version`) — substrate floor satisfied.
|
||||
- The exact skill-description string lives **compiled into the binary**
|
||||
(`/Users/ktg/.local/share/claude/versions/2.1.196`, Mach-O 235 MB); there is **no
|
||||
`SKILL.md`** for it anywhere under `~/.claude` (system-wide `find`/`grep` — only hits
|
||||
are this brief + unrelated harness notes). Confirms "Anthropic-bundled, not user skill."
|
||||
|
||||
### 2. Invocation mechanism (programmatic vs instruction) -- Confidence: high
|
||||
|
||||
**External findings:**
|
||||
- **Not** via the `Skill` tool. The Skill-tool built-in allowlist is closed: only
|
||||
`/init`, `/review`, `/security-review` are reachable; "Other built-in commands such as
|
||||
`/compact` are not." `/deep-research` is a Workflow and is not on that list.
|
||||
[VERIFIED — code.claude.com/docs/en/skills.md]
|
||||
- **Instruction-based delegation is the documented mechanism.** Workflows launch when the
|
||||
user types the command, or **when Claude is asked in natural language** ("use a
|
||||
workflow" / "run a workflow") or via the `ultracode` keyword. A plugin command whose
|
||||
markdown instructs Claude to run `/deep-research <q>` is therefore the supported path.
|
||||
[VERIFIED — workflows.md "Have Claude write a workflow"]
|
||||
- **Approval gate caveat:** launching a workflow triggers a per-run approval prompt —
|
||||
*every run* in default/acceptEdits; *first launch only* in auto; **never in `claude -p`
|
||||
/ Agent SDK / bypass-permissions** ("the run starts immediately"). Voyage's headless
|
||||
surface (`claude -p`) thus delegates without an interactive gate; interactive sessions
|
||||
hit a prompt. [VERIFIED — workflows.md "Behavior and limits"]
|
||||
- No documented blanket "commands/skills cannot nest" prohibition beyond the Skill-tool
|
||||
allowlist + workflow runtime limits (no mid-run user input; 16 concurrent agents;
|
||||
1000 agents/run). [VERIFIED — workflows.md; NOT DOCUMENTED for a general nesting ban]
|
||||
|
||||
### 3. Output capture -- Confidence: high
|
||||
|
||||
**External findings:**
|
||||
- "When the run finishes, **the report lands in your session**"; "Claude's context holds
|
||||
only the final answer." The report is **in-context**, not a file.
|
||||
[VERIFIED — workflows.md]
|
||||
- What *is* written to disk is the orchestration **script**, not the report: "Every run
|
||||
writes its script to a file under your session's directory in `~/.claude/projects/`."
|
||||
[VERIFIED — workflows.md "How a workflow runs"]
|
||||
|
||||
**Local findings:**
|
||||
- A real `/deep-research` run on this machine left exactly one file —
|
||||
`~/.claude/projects/<session>/workflows/scripts/deep-research-wf_<id>.js` — and **no
|
||||
`.md` report** beside it. [VERIFIED — local filesystem inspection by claude-code-guide]
|
||||
- **Consequence:** a calling command cannot `grep` a results file off disk (none is
|
||||
documented to exist). It can only consume the report **as it sits in conversation
|
||||
context in the same turn** — Claude reads its own prior output and transforms it into
|
||||
research-brief schema. [INFERENCE — file-based capture not feasible; in-context
|
||||
transform is the only avenue]
|
||||
|
||||
### 4. Feature detection + fallback -- Confidence: medium
|
||||
|
||||
**External findings:**
|
||||
- **No positive enumeration / "is-enabled" API or flag is documented.** `claude --help`
|
||||
exposes `--disable-slash-commands` but no "list skills/workflows" or "is-feature-on"
|
||||
flag. [VERIFIED — local `claude --help`; NOT DOCUMENTED for any positive probe]
|
||||
- The documented signals are the **off-switches**, read defensively: `/config` Dynamic-
|
||||
workflows off; `disableWorkflows: true` / `disableBundledSkills: true` in settings.json;
|
||||
`CLAUDE_CODE_DISABLE_WORKFLOWS=1` / `CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=1`.
|
||||
[VERIFIED — workflows.md "Turn workflows off"; CHANGELOG 2.1.x]
|
||||
- **The gap:** the Pro `/config` *on*-state is the very thing you most need to detect, and
|
||||
only the *disable* keys are documented; the persisted key/value for the Pro enable-state
|
||||
is **[NOT DOCUMENTED]**, so it cannot be reliably grepped. A failed/disabled
|
||||
`/deep-research` is also **not documented** to raise a signal a sibling command can
|
||||
catch. [VERIFIED gap]
|
||||
|
||||
**Local findings:**
|
||||
- On this machine both disable keys are unset/absent — workflows are not disabled.
|
||||
|
||||
## External Knowledge
|
||||
|
||||
### Best Practice
|
||||
The "Have Claude write a workflow" + "Behavior and limits" sections of `workflows.md`
|
||||
establish that workflows are operator/Claude-launched, run isolated, and return one
|
||||
in-context report. The Skill-tool allowlist (`skills.md`) is the authoritative statement
|
||||
that only three built-ins are tool-invocable.
|
||||
|
||||
### Known Issues
|
||||
Per-run approval prompts outside `-p`/SDK/bypass mean an interactive `/trekresearch
|
||||
--engine deep-research` will pause for operator approval on each launch — acceptable, but
|
||||
worth documenting in the command UX. The absence of a positive availability probe is the
|
||||
single biggest design constraint (see Dimension 4).
|
||||
|
||||
## Synthesis
|
||||
|
||||
Three cross-cutting insights that only emerge from combining the docs with Voyage's brief:
|
||||
|
||||
1. **The brief's SC3 is correct as written — an earlier review note was wrong.** Because
|
||||
`/deep-research` *is itself* a dynamic workflow, "med dynamic workflows avskrudd faller
|
||||
`--engine deep-research` tilbake til swarm" is the right feature-detection axis. A
|
||||
prior brief-review remark that SC3 "conflated `/deep-research` (skill) with dynamic
|
||||
workflows" was based on a wrong premise (that `/deep-research` was a skill) and is
|
||||
retracted. **No SC3 brief edit is needed.**
|
||||
|
||||
2. **The engine must be instruction-based + in-context, never file-based.** Topic 1's
|
||||
open question ("instruction (safe) vs. programmatic API (faster) — topic 1 decides")
|
||||
resolves decisively to **instruction-based**: there is no programmatic API and no
|
||||
on-disk report. The `deep-research` engine path in `commands/trekresearch.md` must
|
||||
(a) instruct Claude to run `/deep-research <q>`, then (b) transform the in-context
|
||||
report into the `research/NN-*.md` schema in the same turn. This is surface-only
|
||||
(`commands/` prose), matching the brief's "ingen nye `lib/`-avhengigheter."
|
||||
|
||||
3. **SC3's "ingen exception" cannot rest on runtime detection — pin it to a default.**
|
||||
Since no positive availability probe exists, robust fallback = version-floor check
|
||||
(≥ 2.1.154) + disable-key heuristic (`disableWorkflows` / env) + **`--engine` default
|
||||
of `swarm`** (explicit opt-in). The fallback is "graceful degradation by design,"
|
||||
not "catch an exception at runtime." This refines, but does not contradict, SC3.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **Adapter fidelity (brief Topic 2, partly answered locally):** `research-validator.mjs`
|
||||
requires `type: trekresearch-brief` + `created` + `question`; validates `confidence ∈
|
||||
[0,1]` and `dimensions ≥ 1` if present; body must carry `## Executive Summary` +
|
||||
`## Dimensions`. So the `/deep-research` report **can** be reduced to ≥ 1 dimension with
|
||||
per-claim citations and a confidence number — the brief's assumption holds. The
|
||||
remaining open part (how cleanly the in-context report maps to per-dimension
|
||||
local/external splits) is a `/trekplan` exploration concern, not an external unknown.
|
||||
- **Exact intro version of the *named* `/deep-research` workflow** — not documented; the
|
||||
2.1.154 dynamic-workflows floor is the safe pin.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Build the `deep-research` engine as **instruction-based delegation with in-context
|
||||
adaptation**, not a programmatic trigger:
|
||||
1. `--engine deep-research` makes `commands/trekresearch.md`'s external phase instruct
|
||||
Claude to run `/deep-research <q>` and transform the returned in-context report into
|
||||
`research/NN-*.md` (validator-conformant: `## Executive Summary` + `## Dimensions`,
|
||||
`confidence`, `dimensions ≥ 1`).
|
||||
2. Feature-detect by **graceful degradation**: version floor + disable-key heuristic +
|
||||
`--engine` default `swarm`. Do not depend on a positive availability probe (none
|
||||
exists). Log the chosen engine. This satisfies SC3 without a runtime exception.
|
||||
3. Keep `swarm` the default (brief's Non-Goal: "ikke default-bytte"). Document the
|
||||
per-run approval prompt for interactive (non-`-p`) sessions.
|
||||
|
||||
Confidence in the recommendation: **high** for the mechanism (instruction-based +
|
||||
in-context), **medium** for the exact fallback-detection ergonomics (the one documented
|
||||
gap). This is sufficient to green-light `/trekplan` with Topic 1 resolved.
|
||||
|
||||
## Sources
|
||||
|
||||
| # | Source | Type | Quality | Used in |
|
||||
|---|--------|------|---------|---------|
|
||||
| 1 | code.claude.com/docs/en/workflows.md | official | high | Dim 1,2,3,4 + Synthesis |
|
||||
| 2 | code.claude.com/docs/en/commands.md | official | high | Dim 1 (Workflow classification, WebSearch req) |
|
||||
| 3 | code.claude.com/docs/en/skills.md | official | high | Dim 2 (Skill-tool allowlist) |
|
||||
| 4 | github.com/anthropics/claude-code CHANGELOG (2.1.154, 2.1.x, 2.1.196) | official | high | Dim 1,4 (version floor, disable keys) |
|
||||
| 5 | Local: CC 2.1.196 binary inspection (`find`/`grep`, `claude --version`) | codebase | high | Dim 1 (bundled, no SKILL.md) |
|
||||
| 6 | Local: real `/deep-research` run — only `.js` script written, no `.md` report | codebase | high | Dim 3 (no on-disk artifact) |
|
||||
| 7 | Local: `lib/validators/research-validator.mjs` schema | codebase | high | Open Questions (adapter feasibility / Topic 2) |
|
||||
|
|
@ -28,13 +28,14 @@ A revived Path C (post-v2.2.xxx) would require: (1) re-architecting tool-list to
|
|||
|
||||
## Profile system (`--profile`, v4.1.0)
|
||||
|
||||
Three built-in model profiles plus operator-defined `<custom>.yaml`. Each profile pins `phase_models` for the six pipeline phases (`brief`, `research`, `plan`, `execute`, `review`, `continue`). Profile is recorded in plan.md frontmatter as `profile: <name>` and emitted to `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` for cost-attribution.
|
||||
Four built-in model profiles plus operator-defined `<custom>.yaml`. Each profile pins `phase_models` for the six pipeline phases (`brief`, `research`, `plan`, `execute`, `review`, `continue`). Profile is recorded in plan.md frontmatter as `profile: <name>` and emitted to `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` for cost-attribution.
|
||||
|
||||
| Profile | Brief | Research | Plan | Execute | Review | Continue | Use case |
|
||||
|---------|-------|----------|------|---------|--------|----------|----------|
|
||||
| `economy` | sonnet | sonnet | sonnet | sonnet | sonnet | sonnet | ⚠ **Experimental** (uncalibrated Jaccard floor) — lowest cost; high-confidence small-scope tasks (operator-opt-in via `--profile economy`) |
|
||||
| `balanced` | sonnet | sonnet | opus | sonnet | opus | sonnet | Mixed — opus where reasoning depth pays off (operator-opt-in via `--profile balanced`) |
|
||||
| `premium` (default) | opus | opus | opus | opus | opus | opus | Maximum quality — Opus on every phase. Default since 2026-05-13 operator request; also the hardcoded resolver default returned by `resolveProfile()` in `lib/profiles/resolver.mjs` |
|
||||
| `fable` | fable | fable | fable | fable | fable | fable | Max quality — Fable 5 (Mythos-class, above Opus) on every phase (operator-opt-in via `--profile fable`); reasoning effort inherits from the session — see `docs/profiles.md` §Model & effort axes |
|
||||
|
||||
### Lookup order
|
||||
|
||||
|
|
@ -45,7 +46,7 @@ Three built-in model profiles plus operator-defined `<custom>.yaml`. Each profil
|
|||
|
||||
### Custom profiles
|
||||
|
||||
Create `voyage-profiles/<custom>.yaml` in the repo root (or `~/.claude/voyage-profiles/<custom>.yaml`) to define a **new** tier — the name must not be a built-in. The validator (`lib/validators/profile-validator.mjs`) enforces: every `phase_models[].phase` must be a known phase enum; every `phase_models[].model` must match `^(opus|sonnet)(\b|-).*` or one of the canonical short names. `findProfilePath` (`lib/profiles/resolver.mjs`) resolves **built-in first** (`lib/profiles/<name>.yaml` for `economy`/`balanced`/`premium`), then repo-root `voyage-profiles/`, then `~/.claude/voyage-profiles/`. A custom file named after a built-in therefore **cannot** shadow it (custom profiles must use new names); for the same custom name, repo-root takes precedence over home.
|
||||
Create `voyage-profiles/<custom>.yaml` in the repo root (or `~/.claude/voyage-profiles/<custom>.yaml`) to define a **new** tier — the name must not be a built-in. The validator (`lib/validators/profile-validator.mjs`) enforces: every `phase_models[].phase` must be a known phase enum; every `phase_models[].model` must exactly match an entry in `BASE_ALLOWED_MODELS` (`['sonnet', 'opus', 'fable']`; `haiku` only with `VOYAGE_ALLOW_HAIKU=1`). `findProfilePath` (`lib/profiles/resolver.mjs`) resolves **built-in first** (`lib/profiles/<name>.yaml` for `economy`/`balanced`/`premium`/`fable`), then repo-root `voyage-profiles/`, then `~/.claude/voyage-profiles/`. A custom file named after a built-in therefore **cannot** shadow it (custom profiles must use new names); for the same custom name, repo-root takes precedence over home.
|
||||
|
||||
Drift between plan-frontmatter `profile:` and step-manifest `profile_used:` emits a `MANIFEST_PROFILE_DRIFT` warning from `plan-validator --strict` (Step 20). Plan remains valid; the warning surfaces accidental tier-mismatch.
|
||||
|
||||
|
|
|
|||
|
|
@ -6,14 +6,15 @@ cost estimation (with disclaimer).
|
|||
|
||||
## Built-in profiles
|
||||
|
||||
Three pre-defined tiers ship with v4.1, located at
|
||||
`lib/profiles/{economy,balanced,premium}.yaml`.
|
||||
Four pre-defined tiers ship with the plugin (fable added in v5.9), located at
|
||||
`lib/profiles/{economy,balanced,premium,fable}.yaml`.
|
||||
|
||||
| Profile | Brief | Research | Plan | Execute | Review | Continue | Use case |
|
||||
|---------|-------|----------|------|---------|--------|----------|----------|
|
||||
| `economy` | sonnet | sonnet | sonnet | sonnet | sonnet | sonnet | ⚠ **Experimental** (uncalibrated Jaccard floor) — lowest cost; small-scope tasks where you have high confidence the brief is right |
|
||||
| `balanced` | sonnet | sonnet | opus | sonnet | opus | sonnet | Mixed — opus where reasoning depth pays off (plan synthesis + adversarial review); opt-in via `--profile balanced` |
|
||||
| `premium` (default) | opus | opus | opus | opus | opus | opus | Maximum quality — Opus on every phase + external research on (default since the 2026-05-13 operator decision) |
|
||||
| `fable` | fable | fable | fable | fable | fable | fable | Max quality — Fable 5 (Mythos-class, above Opus) on every phase; opt-in via `--profile fable`; reasoning effort inherits from the session (see Model & effort axes) |
|
||||
|
||||
`premium` is the default tier — set by the 2026-05-13 operator decision and
|
||||
matched by the hardcoded resolver default in `lib/profiles/resolver.mjs`. It
|
||||
|
|
@ -22,7 +23,8 @@ roughly 5× the sub-agent cost of an all-sonnet run, accepted as a deliberate
|
|||
trade-off. Drop to `--profile balanced` (opus only on the two phases where
|
||||
quality matters most — Plan synthesis + Review — and sonnet everywhere else)
|
||||
or `--profile economy` (sonnet everywhere) when cost or latency matters more
|
||||
than depth.
|
||||
than depth. Step up to `--profile fable` (Fable 5 on every phase) when
|
||||
maximum quality is wanted end-to-end and cost is not a constraint.
|
||||
|
||||
`economy` is *strictly experimental* in v4.1, and says so in the profile
|
||||
data itself: `lib/profiles/economy.yaml` carries `experimental: true`. The
|
||||
|
|
@ -36,10 +38,19 @@ back to `balanced`.
|
|||
|
||||
## Model & effort axes
|
||||
|
||||
`opus` and `sonnet` are model **aliases**, not pinned ids. As of Claude Code
|
||||
2.1.154 the `opus` alias resolves to **Opus 4.8**, whose default reasoning
|
||||
effort is **`high`**; `sonnet` resolves to Sonnet 4.6. The profile table above
|
||||
selects *which alias* runs each phase — it does not touch reasoning effort.
|
||||
`opus`, `sonnet`, and `fable` are model **aliases**, not pinned ids. As of
|
||||
Claude Code 2.1.154 the `opus` alias resolves to **Opus 4.8**, whose default
|
||||
reasoning effort is **`high`**; `sonnet` resolves to Sonnet 4.6; `fable`
|
||||
resolves to **Fable 5** (Mythos-class, positioned above Opus), whose default
|
||||
reasoning effort is also `high`. The profile table above selects *which
|
||||
alias* runs each phase — it does not touch reasoning effort.
|
||||
|
||||
**Reasoning effort inherits from the session.** Voyage effort (orchestration
|
||||
shape — which agents/passes run) and model reasoning effort are different
|
||||
axes. Fable 5's default reasoning effort is `high`, NOT xhigh, and switching
|
||||
model resets effort to the model default — xhigh does not follow the model.
|
||||
To run the fable tier at xhigh, set it at session level: `/effort xhigh`, the
|
||||
`effortLevel` setting, or `CLAUDE_CODE_EFFORT_LEVEL`.
|
||||
|
||||
Two different things share the word "effort" in Voyage. They are **orthogonal
|
||||
axes** — same name, different mechanism:
|
||||
|
|
@ -54,8 +65,8 @@ axes** — same name, different mechanism:
|
|||
|
||||
The `phase-signal-resolver.mjs` helper only reads the **orchestration** axis
|
||||
(`phase_signals.effort`, gated against `low/standard/high`) plus the optional
|
||||
per-phase `model` (gated against `['sonnet','opus']`). It never emits native
|
||||
`effort:`.
|
||||
per-phase `model` (gated against `['sonnet','opus','fable']`). It never emits
|
||||
native `effort:`.
|
||||
|
||||
**Native `effort:` on agents.** Voyage sets the reasoning axis statically on
|
||||
selected agents, additively over the Opus-4.8 default:
|
||||
|
|
@ -120,11 +131,13 @@ The validator (`lib/validators/profile-validator.mjs`) enforces:
|
|||
|
||||
- Every `phase_models[].phase` must be a known phase enum:
|
||||
`brief` / `research` / `plan` / `execute` / `review` / `continue`
|
||||
- Every `phase_models[].model` must match `^(opus|sonnet)(\b|-).*` or
|
||||
one of the canonical short names
|
||||
- Every `phase_models[].model` must exactly match an entry in
|
||||
`BASE_ALLOWED_MODELS` (`['sonnet', 'opus', 'fable']` in
|
||||
`lib/validators/profile-validator.mjs`; `haiku` only with
|
||||
`VOYAGE_ALLOW_HAIKU=1`)
|
||||
- All six phases must be present (no partial profiles)
|
||||
|
||||
The three built-in names (`economy`, `balanced`, `premium`) resolve to their
|
||||
The four built-in names (`economy`, `balanced`, `premium`, `fable`) resolve to their
|
||||
bundled yaml first — `findProfilePath()` returns the built-in before consulting
|
||||
`voyage-profiles/`, so a same-named custom file is ignored and cannot shadow a
|
||||
built-in. To customize, give your profile a new name and reference it via
|
||||
|
|
|
|||
109
docs/spike-pretooluse-subagent-reach.md
Normal file
109
docs/spike-pretooluse-subagent-reach.md
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
# Spike: does a plugin `PreToolUse` hook reach sub-agent tool calls?
|
||||
|
||||
**Date:** 2026-08-09
|
||||
**Claude Code version:** 2.1.226
|
||||
**Plugin:** voyage 5.9.1 (installed from `ktg-plugin-marketplace`)
|
||||
**Gates:** Step 10 of `plan.md` (`2026-06-30-trekresearch-storm-upgrade`)
|
||||
|
||||
## Question
|
||||
|
||||
Step 10 wants to enforce the conversation-turn cap in a `PreToolUse` hook. That
|
||||
is only viable if a **plugin** `PreToolUse` hook fires on tool calls made
|
||||
*inside a sub-agent*. If it does not, the cap must be enforced somewhere else
|
||||
and Step 10 becomes a documented downgrade instead.
|
||||
|
||||
The question is genuinely open, not answerable from docs alone: the current
|
||||
[hooks reference](https://code.claude.com/docs/en/hooks) states that sub-agent
|
||||
tool calls fire the same hooks and carry `agent_id` / `agent_type`, while
|
||||
[issue #34692](https://github.com/anthropics/claude-code/issues/34692) reported
|
||||
the exact opposite behaviour. The answer is therefore version-dependent and had
|
||||
to be measured on the version actually in use.
|
||||
|
||||
## Method
|
||||
|
||||
A one-shot probe hook was registered for the `WebSearch` matcher, and a
|
||||
**headless child session** was launched to exercise it. The child was used
|
||||
because hooks are resolved when a session starts — a matcher added mid-session
|
||||
cannot be observed by the session that added it.
|
||||
|
||||
Two deviations from the step as originally written, both forced and both
|
||||
verified not to affect the result:
|
||||
|
||||
1. **The matcher was injected into the installed plugin's `hooks.json`, not the
|
||||
repository's.** The plan assumed the repo working tree *is* the active plugin
|
||||
root. It is not: `~/.claude/plugins/cache/ktg-plugin-marketplace/voyage/5.9.1/`
|
||||
is a plain directory holding its own copy, and that copy is what loads.
|
||||
Editing `hooks/hooks.json` in the repo would have measured nothing. The cache
|
||||
file was backed up, modified, and restored — verified byte-identical to the
|
||||
repo file afterwards.
|
||||
2. **The probe script lives under the session scratchpad, not `${TMPDIR}`.** A
|
||||
pathguard hook refuses writes to `${TMPDIR}`. The load-bearing property was
|
||||
only that the script sit **outside `hooks/scripts/`**, which
|
||||
`tests/lib/doc-consistency.test.mjs:75-84` counts via `readdirSync`; the
|
||||
scratchpad satisfies that just as well. The directory still holds 7 scripts.
|
||||
|
||||
### Probe hook
|
||||
|
||||
Logged every invocation as one JSON line (`tool_name`, `agent_id`,
|
||||
`agent_type`, plus the untouched stdin) and always exited `0`, so it could not
|
||||
alter the child's behaviour.
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
# 1. inject the temporary matcher into the INSTALLED plugin
|
||||
node -e '...push {matcher:"WebSearch", ...voyage-spike-hook.mjs} into hooks.PreToolUse...'
|
||||
|
||||
# 2. exercise it from a fresh child session
|
||||
claude -p "Spawn exactly one sub-agent via the Agent tool (subagent_type: general-purpose).
|
||||
Instruct that sub-agent to perform exactly ONE WebSearch for the query
|
||||
'claude code hooks reference' and report back the first result title.
|
||||
You MUST NOT call WebSearch yourself in the main context - only the
|
||||
sub-agent may call it. When the sub-agent returns, reply with the word DONE." \
|
||||
--allowedTools "Agent,Task,WebSearch" \
|
||||
--max-turns 15
|
||||
|
||||
# 3. restore
|
||||
cp "${TMPDIR}voyage-hooks-backup.json" <cache>/hooks/hooks.json
|
||||
```
|
||||
|
||||
The child returned `DONE`.
|
||||
|
||||
> An earlier attempt additionally passed `--permission-mode bypassPermissions`
|
||||
> and was refused by the auto-mode classifier. The flag was dropped;
|
||||
> `--allowedTools` alone was sufficient.
|
||||
|
||||
## Raw observation
|
||||
|
||||
The log contains **exactly one** record — so the main context did not call
|
||||
`WebSearch` itself, and the single entry is unambiguously the sub-agent's call:
|
||||
|
||||
```json
|
||||
{"at":"2026-08-09T12:07:47.796Z","tool_name":"WebSearch",
|
||||
"agent_id":"aa6d19525a4680fe0","agent_type":"general-purpose",
|
||||
"raw_stdin":"{\"session_id\":\"b126fd6a-...\",\"cwd\":\"/Users/ktg/repos/ktg-plugin-marketplace/voyage\",
|
||||
\"permission_mode\":\"auto\",\"agent_id\":\"aa6d19525a4680fe0\",\"agent_type\":\"general-purpose\",
|
||||
\"effort\":{\"level\":\"xhigh\"},\"hook_event_name\":\"PreToolUse\",\"tool_name\":\"WebSearch\",
|
||||
\"tool_input\":{\"query\":\"claude code hooks reference\"},\"tool_use_id\":\"toolu_01Ka4...\"}"}
|
||||
```
|
||||
|
||||
Both `agent_id` and `agent_type` are populated, matching the documented
|
||||
common-input fields for sub-agent-originated tool events. A main-context call
|
||||
would have carried neither.
|
||||
|
||||
## Consequence for Step 10
|
||||
|
||||
A plugin `PreToolUse` hook **does** observe sub-agent tool calls on CC 2.1.226,
|
||||
and can attribute them via `agent_id` / `agent_type`. Step 10 may therefore take
|
||||
the enforcement branch rather than the documented-downgrade branch.
|
||||
|
||||
Two limits worth carrying forward, neither of which changes the verdict:
|
||||
|
||||
- This measures `WebSearch` on one CC version. The behaviour regressed once
|
||||
before (#34692), so the hook must fail **open**, never assume it is the only
|
||||
gate, and the cap must remain correct if the hook silently stops firing.
|
||||
- The probe only establishes *reach*. Whether a **blocking** (exit 2) decision
|
||||
from inside a sub-agent propagates usefully was not measured — the probe
|
||||
always exited 0 by design.
|
||||
|
||||
RESULT: FIRES
|
||||
|
|
@ -105,7 +105,11 @@ const BLOCK_RULES = [
|
|||
// --- Executor-specific additions ---
|
||||
{
|
||||
name: 'System shutdown/reboot',
|
||||
pattern: /\b(?:shutdown|reboot|halt|poweroff)\b/,
|
||||
// Anchored to command position — start of string/line, or after a
|
||||
// separator (`;`, `|`, `&&`, `||`), with optional `sudo` and an optional
|
||||
// absolute path. An unanchored \b match blocked the bare word anywhere,
|
||||
// including quoted grep patterns, heredoc data, and commit messages.
|
||||
pattern: /(?:^|[\n;|]|&&)\s*(?:sudo\s+(?:-[a-zA-Z]+\s+)*)?(?:[\w./-]*\/)?(?:shutdown|reboot|halt|poweroff)\b/,
|
||||
description: 'System shutdown/reboot commands are blocked during execution.',
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -25,11 +25,23 @@ const TREKBRIEF_ALLOWED = Object.freeze(new Set([
|
|||
]));
|
||||
|
||||
// Source: tests/fixtures/jsonl-schemas.md row 2 (trekresearch)
|
||||
// `engine` is a low-cardinality label (swarm|deep-research) emitted at
|
||||
// commands/trekresearch.md:533 and promised in prose (:570-572).
|
||||
// DENY BY OMISSION: question (free prose), project_dir + brief_path
|
||||
// (filesystem paths) are written into the jsonl but MUST NOT reach the
|
||||
// exporter.
|
||||
// The five v5.10 measurement fields are allowlisted too: `effort` is a
|
||||
// low-cardinality label (low|standard|high) and the grouping key the
|
||||
// measurement gate is computed on; `unique_sources`, `dimensions_baseline`,
|
||||
// `conv_turns` and `empty_turns` are plain counters. None of them carry prose
|
||||
// or paths.
|
||||
const TREKRESEARCH_ALLOWED = Object.freeze(new Set([
|
||||
'ts', 'slug', 'mode', 'scope', 'dimensions', 'agents_local',
|
||||
'ts', 'slug', 'mode', 'scope', 'engine', 'dimensions', 'agents_local',
|
||||
'agents_external', 'gemini_used', 'confidence', 'contradictions',
|
||||
'open_questions', 'profile', 'parallel_agents',
|
||||
'external_research_enabled', 'profile_source',
|
||||
'effort', 'unique_sources', 'dimensions_baseline', 'conv_turns',
|
||||
'empty_turns',
|
||||
]));
|
||||
|
||||
// Source: tests/fixtures/jsonl-schemas.md row 3 (trekplan)
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ const OPTIONAL_KEYS = [
|
|||
const OPTIONAL_BOOLEAN_KEYS = new Set(OPTIONAL_KEYS);
|
||||
|
||||
// Optional string-typed manifest keys (v4.1 Step 3 — additive forward-compat).
|
||||
// `profile_used`: name of the model profile (economy|balanced|premium|<custom>) the
|
||||
// `profile_used`: name of the model profile (economy|balanced|premium|fable|<custom>) the
|
||||
// step was executed under. Absence is fine (v4.0 manifests have no
|
||||
// profile concept); presence MUST be a string.
|
||||
// Unlike OPTIONAL_BOOLEAN_KEYS, absence is NOT defaulted — the field is simply
|
||||
|
|
|
|||
21
lib/profiles/fable.yaml
Normal file
21
lib/profiles/fable.yaml
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
---
|
||||
profile_version: "1.0"
|
||||
name: fable
|
||||
phase_models:
|
||||
- phase: brief
|
||||
model: fable
|
||||
- phase: research
|
||||
model: fable
|
||||
- phase: plan
|
||||
model: fable
|
||||
- phase: execute
|
||||
model: fable
|
||||
- phase: review
|
||||
model: fable
|
||||
- phase: continue
|
||||
model: fable
|
||||
parallel_agents_min: 6
|
||||
parallel_agents_max: 8
|
||||
external_research_enabled: true
|
||||
brief_reviewer_iter_cap: 3
|
||||
---
|
||||
|
|
@ -67,6 +67,11 @@ export function resolvePhaseSignalFromFile(briefPath, phase) {
|
|||
}
|
||||
|
||||
// CLI shim — mirrors lib/validators/brief-validator.mjs:168 pattern.
|
||||
// Footgun guard (v5.9): this shim's `model` output is brief-signal-only — it
|
||||
// never consults the profile layer. For command wiring, the composed resolver
|
||||
// CLI (`resolver.mjs --resolve-phase-model`, brief > profile > default) is the
|
||||
// single resolution source for {effort, model}. Do not re-wire commands/*.md
|
||||
// Bash blocks back to this shim.
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
const args = process.argv.slice(2);
|
||||
const getArg = (name) => {
|
||||
|
|
|
|||
|
|
@ -45,7 +45,7 @@ import { resolvePhaseSignal } from './phase-signal-resolver.mjs';
|
|||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const BUILTIN_PROFILES_DIR = __dirname; // lib/profiles/
|
||||
const BUILTIN_NAMES = new Set(['economy', 'balanced', 'premium']);
|
||||
const BUILTIN_NAMES = new Set(['economy', 'balanced', 'premium', 'fable']);
|
||||
|
||||
/**
|
||||
* Resolve the path to a profile file.
|
||||
|
|
@ -221,7 +221,13 @@ export function validateProfileFile(path, opts = {}) {
|
|||
* @param {string|null} briefPath Absolute or repo-relative path to brief.md, or null
|
||||
* @param {string[]|object} argv Full process.argv array OR parsed flags object
|
||||
* @param {object} [env] Environment-variable record (defaults to process.env)
|
||||
* @returns {{model: string, source: 'brief-signal'|'flag'|'env'|'default'}}
|
||||
* @returns {{effort?: string, model: string, source: 'brief-signal'|'flag'|'env'|'default'}}
|
||||
*
|
||||
* `effort` (v5.9 ADDITIVE) is the brief signal's effort passed through when
|
||||
* present — so commands consume ONE coherent {effort, model, source} result
|
||||
* instead of two split CLI calls. Absent when the brief carries no valid
|
||||
* effort signal for the phase (commands default to 'standard' per the
|
||||
* composition rule).
|
||||
*
|
||||
* Error handling contract:
|
||||
* - Never throws. Any failure (ENOENT on briefPath, malformed YAML, missing
|
||||
|
|
@ -234,7 +240,10 @@ export function validateProfileFile(path, opts = {}) {
|
|||
* directly; commands must inject {resolved model} at Agent-tool spawn sites.
|
||||
*/
|
||||
export function resolvePhaseModel(phase, briefPath, argv, env = process.env) {
|
||||
// Step 1: brief-signal lookup
|
||||
// Step 1: brief-signal lookup. `effort` is captured independently of `model`
|
||||
// so a signal like {effort: high} (no model) still passes effort through
|
||||
// while the model falls to the profile layer.
|
||||
let effort;
|
||||
if (typeof briefPath === 'string' && briefPath.length > 0 && existsSync(briefPath)) {
|
||||
let fm = null;
|
||||
try {
|
||||
|
|
@ -246,8 +255,11 @@ export function resolvePhaseModel(phase, briefPath, argv, env = process.env) {
|
|||
}
|
||||
if (fm) {
|
||||
const signal = resolvePhaseSignal(fm, phase);
|
||||
if (signal && typeof signal.effort === 'string') effort = signal.effort;
|
||||
if (signal && typeof signal.model === 'string' && signal.model.length > 0) {
|
||||
return { model: signal.model, source: 'brief-signal' };
|
||||
return effort !== undefined
|
||||
? { effort, model: signal.model, source: 'brief-signal' }
|
||||
: { model: signal.model, source: 'brief-signal' };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -278,7 +290,9 @@ export function resolvePhaseModel(phase, briefPath, argv, env = process.env) {
|
|||
}
|
||||
}
|
||||
const model = phaseModels[phase] || 'opus';
|
||||
return { model, source: profile_source };
|
||||
return effort !== undefined
|
||||
? { effort, model, source: profile_source }
|
||||
: { model, source: profile_source };
|
||||
}
|
||||
|
||||
// CLI shim — invoked by commands/trek*.md via Bash.
|
||||
|
|
@ -300,7 +314,8 @@ if (import.meta.url === `file://${process.argv[1]}`) {
|
|||
if (args.includes('--json')) {
|
||||
process.stdout.write(JSON.stringify(r) + '\n');
|
||||
} else {
|
||||
process.stdout.write(`model=${r.model} source=${r.source}\n`);
|
||||
const effort = 'effort' in r ? ` effort=${r.effort}` : '';
|
||||
process.stdout.write(`model=${r.model} source=${r.source}${effort}\n`);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ import { join, dirname } from 'node:path';
|
|||
// Per-Mtok USD prices, resolved 2026-06-26 via the claude-api skill reference:
|
||||
// base input/output from the model table; cache rates from the prompt-caching
|
||||
// doc multipliers (cache_read 0.1x, write_5m 1.25x, write_1h 2.0x of input).
|
||||
// claude-fable-5 resolved 2026-07-02 from the official platform pricing docs.
|
||||
export const PRICE_TABLE = Object.freeze({
|
||||
'claude-opus-4-8': Object.freeze({
|
||||
input: 5.0,
|
||||
|
|
@ -35,10 +36,17 @@ export const PRICE_TABLE = Object.freeze({
|
|||
cache_write_5m: 6.25,
|
||||
cache_write_1h: 10.0,
|
||||
}),
|
||||
'claude-fable-5': Object.freeze({
|
||||
input: 10.0,
|
||||
output: 50.0,
|
||||
cache_read: 1.0,
|
||||
cache_write_5m: 12.5,
|
||||
cache_write_1h: 20.0,
|
||||
}),
|
||||
});
|
||||
|
||||
// Date the PRICE_TABLE values were resolved/verified. Bump when prices change.
|
||||
export const PRICE_TABLE_VERSION = '2026-06-26';
|
||||
export const PRICE_TABLE_VERSION = '2026-07-02';
|
||||
|
||||
function num(v) {
|
||||
return typeof v === 'number' && Number.isFinite(v) ? v : 0;
|
||||
|
|
|
|||
144
lib/util/research-loop-cap.mjs
Normal file
144
lib/util/research-loop-cap.mjs
Normal file
|
|
@ -0,0 +1,144 @@
|
|||
// lib/util/research-loop-cap.mjs
|
||||
// Stateful, default-off cost cap for the /trekresearch bounded conversation
|
||||
// loop (Phase 4.5 dimension discovery + Phase 5 loop turns).
|
||||
//
|
||||
// Three properties the plan review required:
|
||||
// (a) Default-off — VOYAGE_STORM_ENABLED must be '1'; otherwise the budget
|
||||
// is 0 regardless of effort. This IS the decline branch: doing nothing
|
||||
// leaves the mechanism off, and adopt is flipping this one constant.
|
||||
// (b) The cap counts itself — allowTurn() derives used-turn count from an
|
||||
// append-only JSONL ledger, never from a caller-supplied number. A cap
|
||||
// that asks the caller how many turns it has used is not a cap.
|
||||
// (c) Correct size bound — worst case is max_conv_turns × max_total_dimensions,
|
||||
// where max_total_dimensions is the WHOLE list (interview + discovered)
|
||||
// under settings.json:16's cap of 8 — not × discovered-only.
|
||||
//
|
||||
// CLAUDE_PLUGIN_DATA absent => DENY (fail-closed). This is the opposite of
|
||||
// lib/stats/event-emit.mjs's fail-open: that module is telemetry (must never
|
||||
// block workflow); this module is a budget control (must never silently
|
||||
// grant unlimited turns just because the data dir is missing).
|
||||
//
|
||||
// CLI shim:
|
||||
// node lib/util/research-loop-cap.mjs --run-id ID --dimension D --effort E
|
||||
// → JSON: { ok, used, budget, reason? } (exit 0 = granted, exit 1 = denied)
|
||||
|
||||
import { existsSync, mkdirSync, appendFileSync, readFileSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
|
||||
export const MAX_CONV_TURNS = 3;
|
||||
export const MAX_TOTAL_DIMENSIONS = 8; // settings.json:16 maxDimensions — whole list, not discovered-only
|
||||
|
||||
const LEDGER_FILENAME = 'trekresearch-loop-ledger.jsonl';
|
||||
|
||||
export function isStormEnabled(env = process.env) {
|
||||
return env.VOYAGE_STORM_ENABLED === '1';
|
||||
}
|
||||
|
||||
/**
|
||||
* Coerce TREKRESEARCH_MAX_CONV_TURNS. NaN, empty, negative, or zero all fall
|
||||
* back to MAX_CONV_TURNS — never to unbounded.
|
||||
*/
|
||||
export function resolveMaxConvTurns(env = process.env) {
|
||||
const raw = env.TREKRESEARCH_MAX_CONV_TURNS;
|
||||
if (raw === undefined || raw === null || raw === '') return MAX_CONV_TURNS;
|
||||
const n = Number(raw);
|
||||
if (!Number.isFinite(n) || n <= 0) return MAX_CONV_TURNS;
|
||||
return Math.floor(n);
|
||||
}
|
||||
|
||||
export function resolveLedgerPath(env = process.env) {
|
||||
const dir = env.CLAUDE_PLUGIN_DATA;
|
||||
if (!dir || typeof dir !== 'string' || dir.length === 0) return null;
|
||||
return join(dir, LEDGER_FILENAME);
|
||||
}
|
||||
|
||||
function countTurns(ledgerPath, runId) {
|
||||
if (!existsSync(ledgerPath)) return 0;
|
||||
let text;
|
||||
try { text = readFileSync(ledgerPath, 'utf-8'); }
|
||||
catch { return 0; }
|
||||
let count = 0;
|
||||
for (const line of text.split('\n')) {
|
||||
if (!line) continue;
|
||||
try {
|
||||
const rec = JSON.parse(line);
|
||||
if (rec.runId === runId) count++;
|
||||
} catch { /* skip malformed lines */ }
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether one more research-loop turn may run. Append-only: never
|
||||
* read-modify-write, because Phase 4.5/5 may spawn multiple agents in a
|
||||
* single message and a read-modify-write counter would lose concurrent
|
||||
* grants.
|
||||
*
|
||||
* @param {{runId: string, dimension: string, effort: string}} args
|
||||
* @param {{env?: object, now?: Date}} [opts]
|
||||
* @returns {{ok: boolean, used: number, budget: number, reason?: string}}
|
||||
*/
|
||||
export function allowTurn({ runId, dimension, effort } = {}, opts = {}) {
|
||||
const env = opts.env || process.env;
|
||||
const now = opts.now || new Date();
|
||||
|
||||
if (!isStormEnabled(env)) {
|
||||
return { ok: false, used: 0, budget: 0, reason: 'storm_disabled' };
|
||||
}
|
||||
if (effort !== 'high') {
|
||||
return { ok: false, used: 0, budget: 0, reason: 'effort_not_high' };
|
||||
}
|
||||
if (!runId || !dimension) {
|
||||
return { ok: false, used: 0, budget: 0, reason: 'missing_args' };
|
||||
}
|
||||
|
||||
const maxConvTurns = resolveMaxConvTurns(env);
|
||||
const budget = maxConvTurns * MAX_TOTAL_DIMENSIONS;
|
||||
|
||||
const ledgerPath = resolveLedgerPath(env);
|
||||
if (!ledgerPath) {
|
||||
return { ok: false, used: 0, budget, reason: 'no_plugin_data_dir' };
|
||||
}
|
||||
|
||||
const used = countTurns(ledgerPath, runId);
|
||||
if (used >= budget) {
|
||||
return { ok: false, used, budget, reason: 'budget_exhausted' };
|
||||
}
|
||||
|
||||
try {
|
||||
const dir = dirname(ledgerPath);
|
||||
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
|
||||
appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, dimension, effort }) + '\n');
|
||||
} catch (e) {
|
||||
return { ok: false, used, budget, reason: `ledger-write-failed: ${e.message}` };
|
||||
}
|
||||
|
||||
return { ok: true, used: used + 1, budget };
|
||||
}
|
||||
|
||||
// ---- CLI shim ----------------------------------------------------------------
|
||||
|
||||
function parseArgs(argv) {
|
||||
const out = {};
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
if (a === '--run-id') out.runId = argv[++i];
|
||||
else if (a === '--dimension') out.dimension = argv[++i];
|
||||
else if (a === '--effort') out.effort = argv[++i];
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
const args = parseArgs(process.argv.slice(2));
|
||||
if (!args.runId || !args.dimension || !args.effort) {
|
||||
process.stdout.write(JSON.stringify({
|
||||
ok: false,
|
||||
reason: 'usage: research-loop-cap.mjs --run-id ID --dimension D --effort standard|high|low',
|
||||
}) + '\n');
|
||||
process.exit(1);
|
||||
}
|
||||
const result = allowTurn(args);
|
||||
process.stdout.write(JSON.stringify(result) + '\n');
|
||||
process.exit(result.ok ? 0 : 1);
|
||||
}
|
||||
|
|
@ -252,7 +252,11 @@ if (import.meta.url === `file://${process.argv[1]}`) {
|
|||
const minIdx = args.indexOf('--min-version');
|
||||
const minBriefVersion = minIdx >= 0 ? args[minIdx + 1] : undefined;
|
||||
// filePath is the first positional, skipping the --min-version value token.
|
||||
const filePath = args.find((a, i) => !a.startsWith('--') && i !== minIdx + 1);
|
||||
// Guard: when --min-version is absent (minIdx === -1) the skip index must be -1,
|
||||
// not 0 — otherwise the no-flag invocation `brief-validator.mjs <brief.md>` drops
|
||||
// the file (which sits at index 0) and bails to Usage.
|
||||
const skipIdx = minIdx >= 0 ? minIdx + 1 : -1;
|
||||
const filePath = args.find((a, i) => !a.startsWith('--') && i !== skipIdx);
|
||||
if (!filePath) {
|
||||
process.stderr.write('Usage: brief-validator.mjs [--soft] [--min-version <x.y>] <brief.md>\n');
|
||||
process.exit(2);
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@
|
|||
// PROFILE_READ_ERROR — file unreadable or parse-error
|
||||
// PROFILE_NOT_FOUND — file does not exist
|
||||
//
|
||||
// Allowed model values: ['sonnet', 'opus']. Haiku is allowed only when
|
||||
// Allowed model values: ['sonnet', 'opus', 'fable']. Haiku is allowed only when
|
||||
// VOYAGE_ALLOW_HAIKU=1 (per global CLAUDE.md modellvalg-prinsipp: Haiku skal
|
||||
// ikke brukes som default; eksplisitt opt-in for spesielle bruksmønstre).
|
||||
|
||||
|
|
@ -42,7 +42,7 @@ export const PROFILE_REQUIRED_PHASES = Object.freeze([
|
|||
'brief', 'research', 'plan', 'execute', 'review', 'continue',
|
||||
]);
|
||||
|
||||
export const BASE_ALLOWED_MODELS = Object.freeze(['sonnet', 'opus']);
|
||||
export const BASE_ALLOWED_MODELS = Object.freeze(['sonnet', 'opus', 'fable']);
|
||||
|
||||
function getAllowedModels(env = process.env) {
|
||||
if (env.VOYAGE_ALLOW_HAIKU === '1') {
|
||||
|
|
|
|||
110
lib/validators/query-privacy-gate.mjs
Normal file
110
lib/validators/query-privacy-gate.mjs
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
// lib/validators/query-privacy-gate.mjs
|
||||
// Inspect an outbound research query before it leaves the machine. Called
|
||||
// only from the new high-effort steps (Phase 4.5 dimension discovery + the
|
||||
// bounded Phase 5 loop turns) — the existing single-pass Phase 5 path is
|
||||
// unchanged (Step 6, plan-v2).
|
||||
//
|
||||
// Two-tier, same shape as lib/exporters/endpoint-validator.mjs's SSRF gate:
|
||||
// - WARN tier — absolute filesystem paths, repo-internal identifiers.
|
||||
// Operator-overridable via `strict: false` / `--soft` (matches
|
||||
// lib/validators/research-validator.mjs's strict/soft convention), and
|
||||
// fully bypassable via the VOYAGE_QUERY_PRIVACY_ALLOW=1 opt-in.
|
||||
// - HARD-BLOCK tier — secret-shaped tokens. NEVER overridable by strict,
|
||||
// --soft, or the opt-in env var — mirrors endpoint-validator.mjs's
|
||||
// HARD_BLOCKED_HOSTS, where an opt-in widens the warn tier but never
|
||||
// unlocks the permanently-blocked one.
|
||||
//
|
||||
// CLI shim:
|
||||
// node lib/validators/query-privacy-gate.mjs [--soft] "<query text>"
|
||||
// → JSON {valid, errors, warnings}; exit 0 valid, 1 invalid.
|
||||
|
||||
import { issue } from '../util/result.mjs';
|
||||
|
||||
// WARN tier — absolute filesystem paths (leaks local directory layout).
|
||||
export const ABSOLUTE_PATH_PATTERNS = Object.freeze([
|
||||
/\/Users\/[^\s"'`]+/,
|
||||
/\/home\/[^\s"'`]+/,
|
||||
/[A-Za-z]:\\[^\s"'`]+/,
|
||||
/\$\{?HOME\}?\/[^\s"'`]+/,
|
||||
]);
|
||||
|
||||
// WARN tier — repo-internal identifiers that don't need to leave the
|
||||
// machine in a generic research query.
|
||||
export const REPO_IDENTIFIER_PATTERNS = Object.freeze([
|
||||
/git\.fromaitochitta\.com[^\s"'`]*/,
|
||||
/\bktg-plugin-marketplace\b/,
|
||||
/\bplugins\/cache\/[^\s"'`]+/,
|
||||
]);
|
||||
|
||||
// HARD-BLOCK tier — secret-shaped strings. Never operator-overridable.
|
||||
export const SECRET_SHAPED_PATTERNS = Object.freeze([
|
||||
/\bsk-[A-Za-z0-9]{20,}\b/, // OpenAI/Anthropic-style API keys
|
||||
/\bAKIA[0-9A-Z]{16}\b/, // AWS access key ID
|
||||
/\bghp_[A-Za-z0-9]{36,}\b/, // GitHub personal access token
|
||||
/\bxox[baprs]-[A-Za-z0-9-]{10,}\b/, // Slack token
|
||||
/-----BEGIN [A-Z ]*PRIVATE KEY-----/, // PEM private key block
|
||||
]);
|
||||
|
||||
function findMatch(patterns, text) {
|
||||
for (const re of patterns) {
|
||||
const m = re.exec(text);
|
||||
if (m) return m[0];
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string} text
|
||||
* @param {{strict?: boolean, env?: object}} [opts]
|
||||
* @returns {{valid: boolean, errors: import('../util/result.mjs').Issue[], warnings: import('../util/result.mjs').Issue[]}}
|
||||
*/
|
||||
export function validateOutboundQuery(text, opts = {}) {
|
||||
const strict = opts.strict !== false;
|
||||
const env = opts.env || process.env;
|
||||
// Bypasses the WARN tier entirely — never affects the hard-block tier below.
|
||||
const allowWarnTier = env.VOYAGE_QUERY_PRIVACY_ALLOW === '1';
|
||||
|
||||
if (typeof text !== 'string' || text.length === 0) {
|
||||
return { valid: false, errors: [issue('PRIVACY_EMPTY_QUERY', 'Outbound query must be a non-empty string')], warnings: [] };
|
||||
}
|
||||
|
||||
const errors = [];
|
||||
const warnings = [];
|
||||
|
||||
// Hard-block tier — checked unconditionally; no opt-in reaches this branch.
|
||||
const secretMatch = findMatch(SECRET_SHAPED_PATTERNS, text);
|
||||
if (secretMatch) {
|
||||
errors.push(issue('PRIVACY_SECRET_SHAPED', `Outbound query contains a secret-shaped token: ${secretMatch}`));
|
||||
}
|
||||
|
||||
if (!allowWarnTier) {
|
||||
const pathMatch = findMatch(ABSOLUTE_PATH_PATTERNS, text);
|
||||
if (pathMatch) {
|
||||
const issueObj = issue('PRIVACY_ABSOLUTE_PATH', `Outbound query contains an absolute filesystem path: ${pathMatch}`);
|
||||
if (strict) errors.push(issueObj); else warnings.push(issueObj);
|
||||
}
|
||||
|
||||
const repoMatch = findMatch(REPO_IDENTIFIER_PATTERNS, text);
|
||||
if (repoMatch) {
|
||||
const issueObj = issue('PRIVACY_REPO_IDENTIFIER', `Outbound query contains a repo-internal identifier: ${repoMatch}`);
|
||||
if (strict) errors.push(issueObj); else warnings.push(issueObj);
|
||||
}
|
||||
}
|
||||
|
||||
return { valid: errors.length === 0, errors, warnings };
|
||||
}
|
||||
|
||||
// ---- CLI shim ----------------------------------------------------------------
|
||||
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
const args = process.argv.slice(2);
|
||||
const strict = !args.includes('--soft');
|
||||
const text = args.find(a => !a.startsWith('--'));
|
||||
if (text === undefined) {
|
||||
process.stderr.write('Usage: query-privacy-gate.mjs [--soft] "<query text>"\n');
|
||||
process.exit(2);
|
||||
}
|
||||
const r = validateOutboundQuery(text, { strict });
|
||||
process.stdout.write(JSON.stringify(r) + '\n');
|
||||
process.exit(r.valid ? 0 : 1);
|
||||
}
|
||||
4
package-lock.json
generated
4
package-lock.json
generated
|
|
@ -1,12 +1,12 @@
|
|||
{
|
||||
"name": "voyage",
|
||||
"version": "5.8.0",
|
||||
"version": "5.9.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "voyage",
|
||||
"version": "5.8.0",
|
||||
"version": "5.9.1",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "voyage",
|
||||
"version": "5.8.0",
|
||||
"version": "5.9.1",
|
||||
"description": "Voyage — brief, research, plan, execute, review, continue. Contract-driven Claude Code pipeline. /trekbrief, /trekplan, and /trekreview each end by building a self-contained operator-annotation HTML (scripts/annotate.mjs, modelled on claude-code-100x): select text or click any heading/paragraph/list-item, pick intent (Fiks/Endre/Spørsmål), write comment, copy structured prompt, paste back, Claude revises the .md.",
|
||||
"type": "module",
|
||||
"engines": {
|
||||
|
|
|
|||
|
|
@ -141,7 +141,7 @@ introduced. This section bridges sessions — it's the "baton" in a relay race.}
|
|||
- **Master plan:** `{plan file path}`
|
||||
- **Steps from plan:** {step N}–{step M}
|
||||
- **Estimated complexity:** {low | medium | high}
|
||||
- **Model recommendation:** {opus | sonnet} — {rationale}
|
||||
- **Model recommendation:** {opus | sonnet | fable} — {rationale}
|
||||
|
||||
## Recovery Metadata
|
||||
|
||||
|
|
|
|||
|
|
@ -17,8 +17,9 @@ source: {interview | manual}
|
|||
# plan polishing a wrong premise after a rejected iteration).
|
||||
framing: {preserve | refine | replace | new-direction}
|
||||
# v5.1 — per-phase effort + model signal (Phase 3.5).
|
||||
# `effort` ∈ {low, standard, high}. Omit `model:` for `standard` so composition
|
||||
# falls through to profile resolver. Force-stop alternative is the commented
|
||||
# `effort` ∈ {low, standard, high}; `model` ∈ {sonnet, opus, fable} (v5.9).
|
||||
# Omit `model:` for `standard` so composition falls through to profile
|
||||
# resolver. Force-stop alternative is the commented
|
||||
# `phase_signals_partial: true` below (mutually exclusive with `phase_signals`).
|
||||
phase_signals:
|
||||
- phase: research
|
||||
|
|
|
|||
|
|
@ -16,6 +16,7 @@ import { dirname, join } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { resolvePhaseSignal } from '../../lib/profiles/phase-signal-resolver.mjs';
|
||||
import { validateBriefContent, PHASE_SIGNAL_PHASES, EFFORT_LEVELS } from '../../lib/validators/brief-validator.mjs';
|
||||
import { BASE_ALLOWED_MODELS } from '../../lib/validators/profile-validator.mjs';
|
||||
import { parseDocument } from '../../lib/util/frontmatter.mjs';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -84,8 +85,8 @@ test('trekbrief — SC1: each of 4 phases has both effort AND model on full-sign
|
|||
assert.ok(EFFORT_LEVELS.includes(r.effort),
|
||||
`phase=${phase}: effort "${r.effort}" not in EFFORT_LEVELS`);
|
||||
if ('model' in r) {
|
||||
assert.ok(['sonnet', 'opus'].includes(r.model),
|
||||
`phase=${phase}: model "${r.model}" not in [sonnet, opus]`);
|
||||
assert.ok(BASE_ALLOWED_MODELS.includes(r.model),
|
||||
`phase=${phase}: model "${r.model}" not in [${BASE_ALLOWED_MODELS.join(', ')}]`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
|
@ -99,6 +100,19 @@ test('trekbrief — SC1: missing phase_signals + brief_version 2.1 triggers BRIE
|
|||
);
|
||||
});
|
||||
|
||||
// --- v5.9 — fable tier option in the Phase 3.5 loop ---
|
||||
|
||||
test('trekbrief — v5.9 Phase 3.5 canonical mapping contains the fable row and offers 4 options', () => {
|
||||
const text = read();
|
||||
const startIdx = text.indexOf('## Phase 3.5');
|
||||
assert.ok(startIdx >= 0, 'Phase 3.5 not found');
|
||||
const section = text.slice(startIdx, text.indexOf('## Phase 4', startIdx));
|
||||
assert.ok(section.includes('fable → {effort: high, model: fable}'),
|
||||
'Phase 3.5 canonical mapping must contain the fable tier row');
|
||||
assert.ok(section.includes('with 4 options'),
|
||||
'Phase 3.5 loop must offer 4 options (AskUserQuestion maxItems: 4)');
|
||||
});
|
||||
|
||||
// --- v5.5 — framing enforcement + TL;DR + memory-alignment prose-pins ---
|
||||
|
||||
test('trekbrief — v5.5 Phase 2.5 framing declaration heading present', () => {
|
||||
|
|
|
|||
127
tests/commands/trekendsession.test.mjs
Normal file
127
tests/commands/trekendsession.test.mjs
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
// tests/commands/trekendsession.test.mjs
|
||||
// Regression tests for /trekendsession (commands/trekendsession.md).
|
||||
//
|
||||
// Bug (2026-07-03): two of the three !`...` eager-exec blocks contained
|
||||
// unresolved placeholders (<project-dir> etc.). The harness executes
|
||||
// eager-exec blocks at command LOAD time, so zsh parsed <project-dir> as
|
||||
// input redirection and the command aborted before the model saw a single
|
||||
// instruction. Eager-exec is only valid for self-contained commands.
|
||||
//
|
||||
// Pattern D (markdown structure) — assertions against command prose.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFileSync, readdirSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = join(HERE, '..', '..');
|
||||
const COMMANDS_DIR = join(ROOT, 'commands');
|
||||
const COMMAND_FILE = join(COMMANDS_DIR, 'trekendsession.md');
|
||||
|
||||
function readCommand() {
|
||||
return readFileSync(COMMAND_FILE, 'utf8');
|
||||
}
|
||||
|
||||
function extractPhase(commandText, phaseHeader) {
|
||||
const startIdx = commandText.indexOf(phaseHeader);
|
||||
if (startIdx === -1) return '';
|
||||
const rest = commandText.slice(startIdx);
|
||||
const nextPhase = rest.search(/\n## (?:Phase |Hard )/);
|
||||
if (nextPhase === -1) return rest;
|
||||
return rest.slice(0, nextPhase);
|
||||
}
|
||||
|
||||
// Extract all eager-exec blocks (!`...`) from a command/skill file,
|
||||
// including multi-line blocks. Returns [{ content, line }].
|
||||
function extractEagerBlocks(text) {
|
||||
const blocks = [];
|
||||
const re = /!`([^`]+)`/g;
|
||||
let m;
|
||||
while ((m = re.exec(text)) !== null) {
|
||||
const line = text.slice(0, m.index).split('\n').length;
|
||||
blocks.push({ content: m[1], line });
|
||||
}
|
||||
return blocks;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------
|
||||
// Marketplace-wide regression guard: eager-exec blocks must be
|
||||
// self-contained. An unresolved placeholder (<angle> or {curly}) in an
|
||||
// eager block is executed verbatim by the shell at load time — <x> is
|
||||
// parsed as input redirection and aborts the whole command load.
|
||||
// ---------------------------------------------------------------
|
||||
|
||||
test('eager-exec guard — no !`-block in commands/ contains an unresolved placeholder', () => {
|
||||
const offenders = [];
|
||||
for (const file of readdirSync(COMMANDS_DIR).filter((f) => f.endsWith('.md'))) {
|
||||
const text = readFileSync(join(COMMANDS_DIR, file), 'utf8');
|
||||
for (const { content, line } of extractEagerBlocks(text)) {
|
||||
// Placeholder conventions: <angle-word> or {curly_word}. Curly must
|
||||
// contain a separator (- or _) so JS destructuring like {join} in a
|
||||
// legitimate self-contained script does not false-positive; angle
|
||||
// placeholders are unambiguous (shell would parse them as redirects).
|
||||
if (/<[a-z][a-z0-9_-]*>/.test(content) || /\{[a-z][a-z0-9]*([_-][a-z0-9]+)+\}/.test(content)) {
|
||||
offenders.push(`${file}:${line}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
assert.deepEqual(
|
||||
offenders,
|
||||
[],
|
||||
`eager-exec !\`-blocks run at command LOAD time and must be self-contained; ` +
|
||||
`placeholder found in: ${offenders.join(', ')}`,
|
||||
);
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------
|
||||
// trekendsession-specific: exactly one eager block (Phase 1 project
|
||||
// discovery — self-contained, legitimate); Phases 3 and 4 are runtime
|
||||
// Bash-tool commands with model-substituted values, never eager.
|
||||
// ---------------------------------------------------------------
|
||||
|
||||
test('trekendsession — exactly one eager-exec block remains (Phase 1 discovery)', () => {
|
||||
const cmd = readCommand();
|
||||
const blocks = extractEagerBlocks(cmd);
|
||||
assert.equal(
|
||||
blocks.length,
|
||||
1,
|
||||
`expected exactly 1 eager-exec block (Phase 1 discovery), got ${blocks.length} at line(s) ${blocks.map((b) => b.line).join(', ')}`,
|
||||
);
|
||||
assert.match(
|
||||
blocks[0].content,
|
||||
/readdirSync\(root\)/,
|
||||
'the surviving eager block must be the self-contained Phase 1 discovery script',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekendsession Phase 3 — atomic-write block is runtime Bash (no eager prefix) with plugin-root import', () => {
|
||||
const phase3 = extractPhase(readCommand(), '## Phase 3 ');
|
||||
assert.doesNotMatch(phase3, /!`/, 'Phase 3 must not use eager-exec — values exist only at runtime');
|
||||
assert.match(
|
||||
phase3,
|
||||
/\$\{CLAUDE_PLUGIN_ROOT\}\/lib\/util\/atomic-write\.mjs/,
|
||||
'Phase 3 import must use the absolute ${CLAUDE_PLUGIN_ROOT} path — cwd is the user repo, not the plugin root',
|
||||
);
|
||||
assert.doesNotMatch(
|
||||
phase3,
|
||||
/['"]\.\/lib\/util\/atomic-write\.mjs['"]/,
|
||||
'Phase 3 must not import atomic-write.mjs via a cwd-relative path',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekendsession Phase 4 — validator call is runtime Bash (no eager prefix) with plugin-root path', () => {
|
||||
const phase4 = extractPhase(readCommand(), '## Phase 4 ');
|
||||
assert.doesNotMatch(phase4, /!`/, 'Phase 4 must not use eager-exec — the state-file path exists only at runtime');
|
||||
assert.match(
|
||||
phase4,
|
||||
/\$\{CLAUDE_PLUGIN_ROOT\}\/lib\/validators\/session-state-validator\.mjs/,
|
||||
'Phase 4 validator path must use the absolute ${CLAUDE_PLUGIN_ROOT} convention',
|
||||
);
|
||||
assert.doesNotMatch(
|
||||
phase4,
|
||||
/<[a-z][a-z0-9_-]*>/,
|
||||
'Phase 4 must not use <angle> placeholders in commands — zsh parses <x> as input redirection',
|
||||
);
|
||||
});
|
||||
35
tests/commands/trekresearch-engine.test.mjs
Normal file
35
tests/commands/trekresearch-engine.test.mjs
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
// tests/commands/trekresearch-engine.test.mjs
|
||||
// Step 1 (deep-research-engine): pin the contract the `--engine deep-research`
|
||||
// adapter must hit. The adapted in-context `/deep-research` report, reduced into
|
||||
// the research-brief schema, must pass research-validator under the strict
|
||||
// default; and a brief missing a required section must fail. This is the one
|
||||
// genuinely automatable slice of SC2 (schema, not provenance).
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { validateResearchContent } from '../../lib/validators/research-validator.mjs';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = join(HERE, '..', '..');
|
||||
const FIXTURE = join(ROOT, 'tests', 'fixtures', 'research-deep-research-adapted.md');
|
||||
|
||||
test('deep-research adapter output contract — valid brief passes, missing section fails', () => {
|
||||
const text = readFileSync(FIXTURE, 'utf-8');
|
||||
|
||||
// (a) positive: the adapter's target output passes the validator (default = strict).
|
||||
const okResult = validateResearchContent(text);
|
||||
assert.equal(okResult.valid, true, JSON.stringify(okResult.errors));
|
||||
|
||||
// (b) negative: stripping a required section makes it fail with RESEARCH_MISSING_SECTION,
|
||||
// giving the contract teeth (a fixture that always passes proves nothing).
|
||||
const mutated = text.replace('## Dimensions', '## Removed');
|
||||
const badResult = validateResearchContent(mutated);
|
||||
assert.equal(badResult.valid, false);
|
||||
assert.ok(
|
||||
badResult.errors.find(e => e.code === 'RESEARCH_MISSING_SECTION'),
|
||||
'expected RESEARCH_MISSING_SECTION; got ' + JSON.stringify(badResult.errors),
|
||||
);
|
||||
});
|
||||
|
|
@ -34,10 +34,206 @@ test('trekresearch — sequencing-gate surface mentions BRIEF_V51_MISSING_SIGNAL
|
|||
|
||||
test('trekresearch — low-effort path references --quick equivalent', () => {
|
||||
const text = read();
|
||||
const compIdx = text.indexOf('## Composition rule (v5.1)');
|
||||
assert.ok(compIdx >= 0, 'Composition rule (v5.1) section missing');
|
||||
const section = text.slice(compIdx, compIdx + 2000);
|
||||
assert.match(section, /--quick/, 'Low-effort path must mention --quick equivalent');
|
||||
// Bound the Composition rule section by the next `###` heading rather than a
|
||||
// magic 2000-character window: a fixed count silently drops the match as soon
|
||||
// as prose is inserted above it, turning a real pin into a no-op.
|
||||
const sectionOf = (doc) => {
|
||||
const compIdx = doc.indexOf('## Composition rule (v5.1)');
|
||||
assert.ok(compIdx >= 0, 'Composition rule (v5.1) section missing');
|
||||
const nextHeading = doc.indexOf('\n### ', compIdx);
|
||||
return nextHeading > compIdx ? doc.slice(compIdx, nextHeading) : doc.slice(compIdx);
|
||||
};
|
||||
|
||||
// (a) positive: the low-effort path is documented inside the bounded section.
|
||||
assert.match(sectionOf(text), /--quick/, 'Low-effort path must mention --quick equivalent');
|
||||
|
||||
// (b) negative: an actual removal must still be caught — a bound that can
|
||||
// never fail proves nothing.
|
||||
const mutated = text.replace(/--quick/g, '--removed');
|
||||
assert.doesNotMatch(sectionOf(mutated), /--quick/,
|
||||
'heading-bounded slice must still fail on a genuine removal');
|
||||
});
|
||||
|
||||
// --- Step 7: Phase 5 bounded conversation loop (heading-bounded slices) ---
|
||||
|
||||
// Same bounding discipline as the Composition-rule pin above: slice from the
|
||||
// phase heading to the NEXT phase heading, never a fixed character window.
|
||||
function phaseSlice(doc, startHeading, endHeading) {
|
||||
const start = doc.indexOf(startHeading);
|
||||
assert.ok(start >= 0, `${startHeading} missing`);
|
||||
const end = doc.indexOf(endHeading, start);
|
||||
assert.ok(end > start, `${endHeading} missing — could not bound ${startHeading}`);
|
||||
return doc.slice(start, end);
|
||||
}
|
||||
|
||||
function phase5(doc) {
|
||||
return phaseSlice(doc, '## Phase 5 —', '## Phase 6 —');
|
||||
}
|
||||
|
||||
test('trekresearch — Phase 5 loop is gated on effort == high and names both primitives', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /effort == 'high'/, 'Phase 5 loop must be gated on effort == \'high\'');
|
||||
assert.match(p5, /research-loop-cap\.mjs/, 'Phase 5 must call the loop-cap shim per turn');
|
||||
assert.match(p5, /query-privacy-gate\.mjs/, 'Phase 5 must route outbound queries through the privacy gate');
|
||||
assert.match(p5, /\$\{CLAUDE_PLUGIN_ROOT\}/, 'shim invocations must use the ${CLAUDE_PLUGIN_ROOT} path form');
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 declares the loop bound and all three exits', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /### Loop bound/, 'Phase 5 must carry a `### Loop bound` sub-heading');
|
||||
assert.match(
|
||||
p5,
|
||||
/\*\*Maximum 3 turns per under-illuminated dimension\.\*\*/,
|
||||
'the bound must be stated verbatim',
|
||||
);
|
||||
// Three exits — converged / cap exhausted / operator stop.
|
||||
assert.match(p5, /converged/i, 'exit 1 (converged) must be documented');
|
||||
assert.match(p5, /exhaust/i, 'exit 2 (cap exhausted) must be documented');
|
||||
assert.match(p5, /operator stop/i, 'exit 3 (operator stop) must be documented');
|
||||
// Exhaustion must reach the operator — a silent cap is indistinguishable
|
||||
// from convergence, which is the failure this loop exists to avoid.
|
||||
assert.match(
|
||||
p5,
|
||||
/visibl|visible|print/i,
|
||||
'cap exhaustion must be written visibly to the operator',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 marks empty turns without re-targeting the same dimension', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /`empty`/, 'a finding-less or citation-less turn must be marked `empty`');
|
||||
assert.match(p5, /empty_turns/, 'empty turns must be counted (empty_turns)');
|
||||
assert.match(
|
||||
p5,
|
||||
/does NOT re-target|not re-target/i,
|
||||
'an empty turn must not re-target the same dimension',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 states the no-brief default and the moot precedence matrix', () => {
|
||||
const p5 = phase5(read());
|
||||
// (a) no-brief default
|
||||
assert.match(p5, /effort = 'standard'/, 'no-brief default effort must be stated');
|
||||
assert.match(
|
||||
p5,
|
||||
/--project/,
|
||||
'the no-brief default must be anchored to the absence of --project/brief.md',
|
||||
);
|
||||
// (b) precedence matrix — each entry independently makes the loop moot,
|
||||
// mirroring the --engine moot gate in Phase 4.
|
||||
for (const token of ['--quick', '--local', 'external_research_enabled']) {
|
||||
assert.ok(p5.includes(token), `moot matrix must name ${token}`);
|
||||
}
|
||||
assert.match(p5, /moot/i, 'the matrix must use the same moot vocabulary as the engine gate');
|
||||
// (c) interaction rule — effort: high without model under a cheap profile.
|
||||
assert.match(
|
||||
p5,
|
||||
/effort: high/,
|
||||
'the interaction rule for a brief carrying effort: high without model must be stated',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 restates the honesty rule for loop output', () => {
|
||||
const p5 = phase5(read());
|
||||
// Whitespace-tolerant: the pin is on the sentence, not on where the
|
||||
// paragraph happens to wrap.
|
||||
assert.match(
|
||||
p5,
|
||||
/more\s+turns\s+do\s+not\s+make\s+a\s+finding\s+more\s+credible/i,
|
||||
'the honesty hard rule must be restated for the loop output',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 pins survive only while the prose does (mutation control)', () => {
|
||||
const text = read();
|
||||
const mutated = text.replace(/research-loop-cap\.mjs/g, 'removed-cap.mjs');
|
||||
assert.doesNotMatch(
|
||||
phase5(mutated),
|
||||
/research-loop-cap\.mjs/,
|
||||
'heading-bounded Phase 5 slice must still fail on a genuine removal',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — High-effort behavior keeps the standard/low effort sentences verbatim', () => {
|
||||
const text = read();
|
||||
assert.ok(
|
||||
text.includes('Standard effort (or absent): use the existing conditional triggers.'),
|
||||
'the standard-effort sentence must survive the Phase 5 rewrite verbatim',
|
||||
);
|
||||
assert.ok(
|
||||
text.includes('Low effort: inline research only, no agent swarm'),
|
||||
'the low-effort sentence must survive the Phase 5 rewrite verbatim',
|
||||
);
|
||||
});
|
||||
|
||||
// --- Step 8: Phase 4.5 dimension discovery + Independence amendment ---
|
||||
|
||||
const ORCHESTRATOR_FILE = join(ROOT, 'agents', 'research-orchestrator.md');
|
||||
function readOrchestrator() { return readFileSync(ORCHESTRATOR_FILE, 'utf8'); }
|
||||
|
||||
test('trekresearch — Phase 4.5 exists between Phase 4 and Phase 5 with the effort skip-guard', () => {
|
||||
const text = read();
|
||||
const p45 = text.indexOf('## Phase 4.5 —');
|
||||
assert.ok(p45 >= 0, 'Phase 4.5 heading missing');
|
||||
const p4 = text.indexOf('## Phase 4 —');
|
||||
const p5 = text.indexOf('## Phase 5 —');
|
||||
assert.ok(p4 >= 0 && p5 > p45 && p45 > p4, 'Phase 4.5 must sit between Phase 4 and Phase 5');
|
||||
|
||||
const slice = text.slice(p45, p5);
|
||||
assert.match(
|
||||
slice,
|
||||
/\*\*Skip this phase entirely unless `phase_signal_result\.effort == 'high'`\.\*\*/,
|
||||
'Phase 4.5 must carry the bolded skip-guard in the Phase 3.5 form',
|
||||
);
|
||||
assert.match(slice, /query-privacy-gate\.mjs/,
|
||||
'Phase 4.5 must name the privacy gate as its compensating control');
|
||||
assert.match(slice, /maxDimensions: 8|maxDimensions` *: *8/,
|
||||
'Phase 4.5 must augment under the existing maxDimensions ceiling, not raise it');
|
||||
});
|
||||
|
||||
test('trekresearch — Independence hard rule carries an explicit Phase 4.5 amendment', () => {
|
||||
const text = read();
|
||||
const rulesIdx = text.indexOf('## Hard rules');
|
||||
assert.ok(rulesIdx >= 0, 'Hard rules section missing');
|
||||
const rules = text.slice(rulesIdx);
|
||||
const indIdx = rules.indexOf('**Independence:**');
|
||||
assert.ok(indIdx >= 0, 'Independence hard rule missing');
|
||||
// Bound the rule at the next bullet so the amendment must live inside it.
|
||||
const nextBullet = rules.indexOf('\n- **', indIdx);
|
||||
const independence = nextBullet > indIdx ? rules.slice(indIdx, nextBullet) : rules.slice(indIdx);
|
||||
assert.match(independence, /Amend(ed|ment)/i,
|
||||
'Independence must be explicitly amended, not silently contradicted');
|
||||
assert.match(independence, /Phase 4\.5/, 'the amendment must name Phase 4.5 as the crossing');
|
||||
assert.match(independence, /query-privacy-gate\.mjs/,
|
||||
'the amendment must name the compensating control');
|
||||
});
|
||||
|
||||
test('trekresearch — orchestrator phase map is correct, has no Phase 9, and carries Phase 4.5', () => {
|
||||
const doc = readOrchestrator();
|
||||
const start = doc.indexOf('<!-- Phase mapping');
|
||||
assert.ok(start >= 0, 'phase mapping comment missing');
|
||||
const end = doc.indexOf('-->', start);
|
||||
assert.ok(end > start, 'phase mapping comment not terminated');
|
||||
const map = doc.slice(start, end);
|
||||
|
||||
assert.doesNotMatch(map, /Command Phase 9/,
|
||||
'the command ends at Phase 8 — a Command Phase 9 row is a fiction');
|
||||
|
||||
// Six orchestrator rows, each pointing at the phase the command actually has.
|
||||
const expected = [
|
||||
[1, '4'],
|
||||
[2, '4'],
|
||||
[3, '5'],
|
||||
[4, '6'],
|
||||
[5, '7'],
|
||||
[6, '8'],
|
||||
];
|
||||
for (const [orch, cmd] of expected) {
|
||||
const re = new RegExp(`Orchestrator Phase ${orch}\\s+= Command Phase ${cmd.replace('.', '\\.')}\\b`);
|
||||
assert.match(map, re, `map row for Orchestrator Phase ${orch} must point at Command Phase ${cmd}`);
|
||||
}
|
||||
|
||||
assert.match(map, /Command Phase 4\.5/, 'the map must carry the new Phase 4.5 row');
|
||||
});
|
||||
|
||||
// --- v5.1.1 runtime SC4 + SC7 ---
|
||||
|
|
|
|||
45
tests/fixtures/brief-effort-fable.md
vendored
Normal file
45
tests/fixtures/brief-effort-fable.md
vendored
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
type: trekbrief
|
||||
brief_version: "2.1"
|
||||
created: 2026-07-02
|
||||
task: "Fixture: high-effort all phases on fable (v5.9 allowlist test)"
|
||||
slug: brief-effort-fable
|
||||
project_dir: .claude/projects/2026-07-02-brief-effort-fable/
|
||||
research_topics: 0
|
||||
research_status: complete
|
||||
auto_research: false
|
||||
interview_turns: 4
|
||||
source: fixture
|
||||
phase_signals:
|
||||
- phase: research
|
||||
effort: high
|
||||
model: fable
|
||||
- phase: plan
|
||||
effort: high
|
||||
model: fable
|
||||
- phase: execute
|
||||
effort: high
|
||||
model: fable
|
||||
- phase: review
|
||||
effort: high
|
||||
model: fable
|
||||
---
|
||||
|
||||
# Task: High-effort fable fixture
|
||||
|
||||
## Intent
|
||||
|
||||
Test fixture for the v5.9 fable model tier — all 4 phases at the
|
||||
high effort tier with explicit fable model overrides. Mirrors
|
||||
brief-effort-high.md with `model: opus` replaced by `model: fable`.
|
||||
|
||||
## Goal
|
||||
|
||||
Resolver returns `{effort: 'high', model: 'fable'}` for each of the 4
|
||||
PHASE_SIGNAL_PHASES.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Validator passes with no BRIEF_INVALID_MODEL.
|
||||
- resolvePhaseSignal(fm, phase).effort === 'high' for all 4 phases.
|
||||
- resolvePhaseSignal(fm, phase).model === 'fable' for all 4 phases.
|
||||
22
tests/fixtures/expected.prom
vendored
22
tests/fixtures/expected.prom
vendored
|
|
@ -33,19 +33,31 @@ voyage_trekplan_deep_dives{_schema_id="trekplan",slug="add-auth",mode="default",
|
|||
voyage_trekplan_research_briefs_used{_schema_id="trekplan",slug="add-auth",mode="default",profile="premium",profile_source="flag"} 3
|
||||
# HELP voyage_trekresearch_agents_external voyage stats — trekresearch_agents_external
|
||||
# TYPE voyage_trekresearch_agents_external gauge
|
||||
voyage_trekresearch_agents_external{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 3
|
||||
voyage_trekresearch_agents_external{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 3
|
||||
# HELP voyage_trekresearch_agents_local voyage stats — trekresearch_agents_local
|
||||
# TYPE voyage_trekresearch_agents_local gauge
|
||||
voyage_trekresearch_agents_local{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 5
|
||||
voyage_trekresearch_agents_local{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 5
|
||||
# HELP voyage_trekresearch_contradictions voyage stats — trekresearch_contradictions
|
||||
# TYPE voyage_trekresearch_contradictions gauge
|
||||
voyage_trekresearch_contradictions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 1
|
||||
voyage_trekresearch_contradictions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 1
|
||||
# HELP voyage_trekresearch_conv_turns voyage stats — trekresearch_conv_turns
|
||||
# TYPE voyage_trekresearch_conv_turns gauge
|
||||
voyage_trekresearch_conv_turns{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 5
|
||||
# HELP voyage_trekresearch_dimensions voyage stats — trekresearch_dimensions
|
||||
# TYPE voyage_trekresearch_dimensions gauge
|
||||
voyage_trekresearch_dimensions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 4
|
||||
voyage_trekresearch_dimensions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 4
|
||||
# HELP voyage_trekresearch_dimensions_baseline voyage stats — trekresearch_dimensions_baseline
|
||||
# TYPE voyage_trekresearch_dimensions_baseline gauge
|
||||
voyage_trekresearch_dimensions_baseline{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 3
|
||||
# HELP voyage_trekresearch_empty_turns voyage stats — trekresearch_empty_turns
|
||||
# TYPE voyage_trekresearch_empty_turns gauge
|
||||
voyage_trekresearch_empty_turns{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 1
|
||||
# HELP voyage_trekresearch_open_questions voyage stats — trekresearch_open_questions
|
||||
# TYPE voyage_trekresearch_open_questions gauge
|
||||
voyage_trekresearch_open_questions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 2
|
||||
voyage_trekresearch_open_questions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 2
|
||||
# HELP voyage_trekresearch_unique_sources voyage stats — trekresearch_unique_sources
|
||||
# TYPE voyage_trekresearch_unique_sources gauge
|
||||
voyage_trekresearch_unique_sources{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 17
|
||||
# HELP voyage_trekreview_duration_ms voyage stats — trekreview_duration_ms
|
||||
# TYPE voyage_trekreview_duration_ms histogram
|
||||
voyage_trekreview_duration_ms{_schema_id="trekreview",slug="add-auth",verdict="ALLOW",mode="default",profile="balanced",profile_source="flag"} 4521
|
||||
|
|
|
|||
2
tests/fixtures/jsonl-schemas.md
vendored
2
tests/fixtures/jsonl-schemas.md
vendored
|
|
@ -20,7 +20,7 @@
|
|||
| schema_id | fields | writer_path | line_ref | v4.1 additive | PII |
|
||||
|-----------|--------|-------------|----------|---------------|-----|
|
||||
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, project_dir | commands/trekbrief.md (orchestrator-emit Phase 7) | trekbrief.md:657-672 | profile, phase_models, profile_source | none |
|
||||
| trekresearch-stats | ts, question, mode, scope, slug, project_dir, brief_path, dimensions, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | commands/trekresearch.md (orchestrator-emit Stats tracking) | trekresearch.md:388-410 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
|
||||
| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | commands/trekresearch.md (orchestrator-emit Stats tracking) | trekresearch.md:634-676 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
|
||||
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | commands/trekplan.md (orchestrator-emit Phase 12) | trekplan.md:805-826 | profile, phase_models, parallel_agents, profile_source | none |
|
||||
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | commands/trekexecute.md (orchestrator-emit Phase 9) | trekexecute.md:1479-1494 | profile, phase_models, profile_source | none |
|
||||
| trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:64-86 | payload.profile, payload.phase_models, payload.profile_source | none |
|
||||
|
|
|
|||
49
tests/fixtures/research-deep-research-adapted.md
vendored
Normal file
49
tests/fixtures/research-deep-research-adapted.md
vendored
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
---
|
||||
type: trekresearch-brief
|
||||
created: 2026-06-30
|
||||
question: "Should /trekresearch delegate its external phase to the built-in /deep-research workflow?"
|
||||
confidence: 0.8
|
||||
dimensions: 2
|
||||
mcp_servers_used: []
|
||||
local_agents_used: []
|
||||
external_agents_used:
|
||||
- deep-research
|
||||
---
|
||||
|
||||
# Deep-research engine adapter output
|
||||
|
||||
> Fixture: a `/deep-research` in-context report reduced into the research-brief
|
||||
> schema by the `--engine deep-research` adapter (Step 4). Models the target the
|
||||
> adapter must hit; not real engine output.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Delegating the external phase to the built-in `/deep-research` workflow is a
|
||||
viable opt-in engine that supplies fan-out and cited claim-verification for free.
|
||||
Confidence is medium-high on the mechanism but lower on availability, because the
|
||||
workflow exposes no positive "is-enabled" probe. The load-bearing caveat is
|
||||
provenance: structural validity does not certify that the cited URLs are real, so
|
||||
a swarm fallback plus a human URL spot-check stay mandatory.
|
||||
|
||||
## Dimensions
|
||||
|
||||
### Engine mechanism -- Confidence: high
|
||||
|
||||
**External findings:**
|
||||
- `/deep-research` is a built-in dynamic workflow reachable only by prose instruction, with no programmatic API (https://code.claude.com/docs/workflows).
|
||||
- Its report lands in-context with no on-disk artifact, so the adapter must transform what is already in the turn (https://code.claude.com/docs/commands).
|
||||
|
||||
### Fallback ergonomics -- Confidence: high
|
||||
|
||||
**External findings:**
|
||||
- There is no positive availability probe; only `disableWorkflows` / `CLAUDE_CODE_DISABLE_WORKFLOWS` off-switches and a 2.1.154 version floor are documented (https://code.claude.com/docs/skills).
|
||||
- Disabled-workflow behavior under `claude -p` is undocumented, so the post-hoc presence check must be robust to every failure manifestation (https://github.com/anthropics/claude-code/issues/52272).
|
||||
|
||||
## Sources
|
||||
|
||||
| # | Source | Type | Quality | Used in |
|
||||
|---|--------|------|---------|---------|
|
||||
| 1 | https://code.claude.com/docs/workflows | official | high | Engine mechanism |
|
||||
| 2 | https://code.claude.com/docs/commands | official | high | Engine mechanism |
|
||||
| 3 | https://code.claude.com/docs/skills | official | high | Fallback ergonomics |
|
||||
| 4 | https://github.com/anthropics/claude-code/issues/52272 | community | medium | Fallback ergonomics |
|
||||
2
tests/fixtures/stats-sample.jsonl
vendored
2
tests/fixtures/stats-sample.jsonl
vendored
|
|
@ -2,4 +2,4 @@
|
|||
{"_schema_id":"trekexecute","ts":"2026-05-09T08:30:00.000Z","plan":"trekplan-add-auth.md","plan_type":"plan","mode":"execute","result":"completed","steps_total":12,"steps_passed":12,"steps_failed":0,"steps_skipped":0,"profile":"premium","profile_source":"inheritance"}
|
||||
{"_schema_id":"trekreview","ts":"2026-05-09T09:00:00.000Z","slug":"add-auth","verdict":"ALLOW","reviewed_files_count":18,"mode":"default","duration_ms":4521,"profile":"balanced","profile_source":"flag"}
|
||||
{"_schema_id":"trekbrief","ts":"2026-05-09T07:00:00.000Z","slug":"add-auth","mode":"default","interview_turns":7,"review_iterations":2,"research_topics":3,"profile":"economy","profile_source":"env"}
|
||||
{"_schema_id":"trekresearch","ts":"2026-05-09T07:30:00.000Z","slug":"add-auth","mode":"default","scope":"both","dimensions":4,"agents_local":5,"agents_external":3,"contradictions":1,"open_questions":2,"profile":"premium","profile_source":"default"}
|
||||
{"_schema_id":"trekresearch","ts":"2026-05-09T07:30:00.000Z","slug":"add-auth","mode":"default","scope":"both","dimensions":4,"dimensions_baseline":3,"effort":"high","conv_turns":5,"empty_turns":1,"unique_sources":17,"agents_local":5,"agents_external":3,"contradictions":1,"open_questions":2,"profile":"premium","profile_source":"default"}
|
||||
|
|
|
|||
|
|
@ -129,6 +129,40 @@ test('pre-bash-executor BLOCKS system shutdown command', async () => {
|
|||
assert.strictEqual(code, 2);
|
||||
});
|
||||
|
||||
test('pre-bash-executor BLOCKS a privileged halt at command position', async () => {
|
||||
const { code } = await runHook(PRE_BASH, bashInput('sudo shutdown -h now'));
|
||||
assert.strictEqual(code, 2);
|
||||
});
|
||||
|
||||
test('pre-bash-executor BLOCKS a destructive keyword after a separator', async () => {
|
||||
const { code } = await runHook(PRE_BASH, bashInput('echo done && poweroff'));
|
||||
assert.strictEqual(code, 2);
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// ALLOW — the same keywords as DATA, not at command position.
|
||||
// The rule matched the bare word anywhere in the string, so a quoted grep
|
||||
// pattern, ordinary prose, or a commit message that merely named the rule
|
||||
// was blocked. Anchoring to command position is what separates the two.
|
||||
// -----------------------------------------------------------------------
|
||||
test('pre-bash-executor ALLOWS the keyword inside a quoted grep pattern', async () => {
|
||||
const { code } = await runHook(PRE_BASH, bashInput("grep 'halt' f.mjs"));
|
||||
assert.strictEqual(code, 0);
|
||||
});
|
||||
|
||||
test('pre-bash-executor ALLOWS the keyword inside echoed prose', async () => {
|
||||
const { code } = await runHook(PRE_BASH, bashInput('echo "we should halt here"'));
|
||||
assert.strictEqual(code, 0);
|
||||
});
|
||||
|
||||
test('pre-bash-executor ALLOWS a commit message that names the rule', async () => {
|
||||
const { code } = await runHook(
|
||||
PRE_BASH,
|
||||
bashInput('git commit -m "fix(hooks): anchor shutdown rule to command position"'),
|
||||
);
|
||||
assert.strictEqual(code, 0);
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// BLOCK — cron persistence
|
||||
// -----------------------------------------------------------------------
|
||||
|
|
|
|||
|
|
@ -31,6 +31,25 @@ test('SC #12: stats-sample.jsonl → expected.prom snapshot byte-for-byte match'
|
|||
` node scripts/gen-expected-prom.mjs > tests/fixtures/expected.prom`);
|
||||
});
|
||||
|
||||
test('Step 9: the four numeric STORM fields are metric families and effort is a label', () => {
|
||||
const expected = readFileSync(join(FIXTURES, 'expected.prom'), 'utf-8');
|
||||
for (const field of ['unique_sources', 'dimensions_baseline', 'conv_turns', 'empty_turns']) {
|
||||
assert.match(
|
||||
expected,
|
||||
new RegExp(`^# TYPE voyage_trekresearch_${field} `, 'm'),
|
||||
`${field} must appear as its own metric family — a numeric that never becomes a metric cannot be measured`,
|
||||
);
|
||||
}
|
||||
// effort is a low-cardinality string: it must ride along as a LABEL, never
|
||||
// as a metric family (a label is what makes high-vs-standard groupable).
|
||||
assert.match(expected, /effort="[a-z]+"/, 'effort must be emitted as a label');
|
||||
assert.doesNotMatch(
|
||||
expected,
|
||||
/^# TYPE voyage_trekresearch_effort /m,
|
||||
'effort must not become a metric family',
|
||||
);
|
||||
});
|
||||
|
||||
test('empty-input handling: [] returns empty string (no headers)', () => {
|
||||
assert.equal(transformToPrometheus([]), '');
|
||||
assert.equal(transformToPrometheus(null), '');
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ import {
|
|||
POST_BASH_STATS_ALLOWED,
|
||||
EVENT_EMIT_PAYLOAD_ALLOWED,
|
||||
TOKEN_USAGE_ALLOWED,
|
||||
TREKRESEARCH_ALLOWED,
|
||||
} from '../../lib/exporters/field-allowlist.mjs';
|
||||
|
||||
// ---- path-validator: CWE-22 mitigation -------------------------------------
|
||||
|
|
@ -278,6 +279,43 @@ test('field-allowlist: token-usage INCLUDES numeric/label fields, EXCLUDES sessi
|
|||
assert.equal('cwd' in out, false, 'cwd MUST be stripped (CWE-212)');
|
||||
});
|
||||
|
||||
// ---- trekresearch allowlist: the `engine` field ----------------------------
|
||||
|
||||
test('field-allowlist: trekresearch INCLUDES engine, EXCLUDES question/project_dir/brief_path (two-sided)', () => {
|
||||
const record = {
|
||||
ts: '2026-08-09T12:00:00.000Z',
|
||||
question: 'which retrieval strategy survives contradiction?',
|
||||
mode: 'default',
|
||||
scope: 'both',
|
||||
engine: 'deep-research',
|
||||
slug: 'storm-upgrade',
|
||||
project_dir: '/Users/ktg/secret/project',
|
||||
brief_path: '/Users/ktg/secret/project/brief.md',
|
||||
dimensions: 4,
|
||||
agents_local: 7,
|
||||
agents_external: 4,
|
||||
gemini_used: false,
|
||||
confidence: 0.82,
|
||||
contradictions: 1,
|
||||
open_questions: 3,
|
||||
};
|
||||
const out = applyFieldAllowlist(record, 'trekresearch');
|
||||
// INCLUDED — low-cardinality label, emitted (trekresearch.md:533) and
|
||||
// promised in prose (:570-572); it was silently dropped before this pin.
|
||||
assert.equal('engine' in out, true, 'engine MUST be allowlisted — it is emitted and documented');
|
||||
assert.equal(out.engine, 'deep-research');
|
||||
assert.equal(out._schema_id, 'trekresearch');
|
||||
// EXCLUDED (CWE-212 boundary)
|
||||
assert.equal('question' in out, false, 'question MUST be stripped (prose, CWE-212)');
|
||||
assert.equal('project_dir' in out, false, 'project_dir MUST be stripped (path, CWE-212)');
|
||||
assert.equal('brief_path' in out, false, 'brief_path MUST be stripped (path, CWE-212)');
|
||||
});
|
||||
|
||||
test('field-allowlist: TREKRESEARCH_ALLOWED is frozen (drift-pin)', () => {
|
||||
assert.equal(Object.isFrozen(TREKRESEARCH_ALLOWED), true,
|
||||
'TREKRESEARCH_ALLOWED must be frozen — runtime mutation prevention');
|
||||
});
|
||||
|
||||
test('field-allowlist: null/undefined record handled safely', () => {
|
||||
assert.deepEqual(applyFieldAllowlist(null, 'trekplan'), {});
|
||||
assert.deepEqual(applyFieldAllowlist(undefined, 'trekplan'), {});
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ import { fileURLToPath } from 'node:url';
|
|||
import { parseDocument } from '../../lib/util/frontmatter.mjs';
|
||||
import { resolveProfile, loadProfile } from '../../lib/profiles/resolver.mjs';
|
||||
import { STATES } from '../../lib/util/autonomy-gate.mjs';
|
||||
import { TREKRESEARCH_ALLOWED } from '../../lib/exporters/field-allowlist.mjs';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = join(HERE, '..', '..');
|
||||
|
|
@ -918,7 +919,7 @@ test('S15: default-profile name is consistent across resolver + all profile docs
|
|||
assert.equal(profile_source, 'default', 'resolveProfile({}, {}) must report source=default');
|
||||
assert.equal(def, 'premium', 'resolver hardcoded default is premium (operator decision 2026-05-13, commit 40d8742)');
|
||||
|
||||
const OTHERS = ['economy', 'balanced', 'premium'].filter((p) => p !== def);
|
||||
const OTHERS = ['economy', 'balanced', 'premium', 'fable'].filter((p) => p !== def);
|
||||
for (const doc of PROFILE_DOCS) {
|
||||
const body = read(doc);
|
||||
assert.ok(
|
||||
|
|
@ -941,7 +942,7 @@ test('S15: default-profile name is consistent across resolver + all profile docs
|
|||
test('S15: profile tables encode each built-in yaml phase_models exactly', () => {
|
||||
// Column order in every profile table: Profile | Brief | Research | Plan | Execute | Review | Continue | Use case
|
||||
const PHASES = ['brief', 'research', 'plan', 'execute', 'review', 'continue'];
|
||||
for (const name of ['economy', 'balanced', 'premium']) {
|
||||
for (const name of ['economy', 'balanced', 'premium', 'fable']) {
|
||||
const pm = loadProfile(name).phase_models; // {brief:'opus', ...}
|
||||
const expected = PHASES.map((ph) => pm[ph]);
|
||||
for (const doc of PROFILE_DOCS) {
|
||||
|
|
@ -961,6 +962,31 @@ test('S15: profile tables encode each built-in yaml phase_models exactly', () =>
|
|||
}
|
||||
});
|
||||
|
||||
// STRUCTURAL pin: the exporter allowlist and the authoring fixture must agree.
|
||||
// `engine` was emitted, documented in prose, and still dropped at the export
|
||||
// boundary because nothing tied the two together. Derives one side from the
|
||||
// frozen Set, so it survives rewording of the fixture row.
|
||||
test('S74: every TREKRESEARCH_ALLOWED name is declared in the jsonl-schemas fixture row', () => {
|
||||
const row = read('tests/fixtures/jsonl-schemas.md')
|
||||
.split('\n')
|
||||
.find((l) => l.startsWith('| trekresearch-stats '));
|
||||
assert.ok(row, 'jsonl-schemas.md is missing the `trekresearch-stats` row');
|
||||
// Columns: '' | schema_id | fields | writer_path | line_ref | v4.1 additive | PII | ''
|
||||
const cells = row.split('|').map((c) => c.trim());
|
||||
// Both columns are required: profile/profile_source/parallel_agents live in
|
||||
// the `v4.1 additive` column only, so checking `fields` alone fails at once.
|
||||
const declared = new Set(
|
||||
[cells[2], cells[5]].flatMap((c) => c.split(',').map((f) => f.trim())).filter(Boolean),
|
||||
);
|
||||
for (const name of TREKRESEARCH_ALLOWED) {
|
||||
assert.ok(
|
||||
declared.has(name),
|
||||
`\`${name}\` is allowlisted in field-allowlist.mjs but absent from the fixture row's `
|
||||
+ '`fields` + `v4.1 additive` columns — fix the SOURCE, not this pin',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
// --- S34 (V30) — economy is self-declared experimental until the cross-tier
|
||||
// Jaccard floor (0.55) is empirically calibrated (Step-17 calibration deferred
|
||||
// to v4.2). The status must be visible in BOTH the profile data
|
||||
|
|
@ -1065,6 +1091,32 @@ test('S18: --min-brief-version is documented at the trekplan + trekresearch boun
|
|||
}
|
||||
});
|
||||
|
||||
test('deep-research-engine: --engine is documented + consistent across surfaces', () => {
|
||||
// Cross-doc pin mirroring S18 (:1057). The opt-in external-research engine flag
|
||||
// must be discoverable wherever /trekresearch flags live: the command itself
|
||||
// plus the three reference surfaces.
|
||||
for (const f of ['commands/trekresearch.md', 'docs/command-modes.md', 'CLAUDE.md', 'README.md']) {
|
||||
assert.ok(
|
||||
read(f).includes('--engine'),
|
||||
`${f} must document the --engine flag (deep-research-engine)`,
|
||||
);
|
||||
}
|
||||
// README documents it specifically as an **Engine** mode-table row.
|
||||
assert.ok(
|
||||
/\*\*Engine\*\*/.test(read('README.md')),
|
||||
'README.md must document --engine as an **Engine** mode row',
|
||||
);
|
||||
// The command prose must name both engine values and the swarm fallback, so the
|
||||
// opt-in + graceful-degradation contract is pinned — not merely the flag string.
|
||||
const research = read('commands/trekresearch.md');
|
||||
assert.ok(/\bswarm\b/.test(research), 'trekresearch.md must name the swarm engine value');
|
||||
assert.ok(/\bdeep-research\b/.test(research), 'trekresearch.md must name the deep-research engine value');
|
||||
assert.ok(
|
||||
/fall back|falls back/.test(research),
|
||||
'trekresearch.md must document the swarm fallback (graceful degradation)',
|
||||
);
|
||||
});
|
||||
|
||||
test('S18: HANDOVER-CONTRACTS documents the pre-2.2 zero-framing-enforcement hole', () => {
|
||||
// The framing defense is producer-elective: a brief declaring ≤ 2.1 sidesteps
|
||||
// it entirely. Handover 1 (PUBLIC CONTRACT) must disclose this and name the remedy.
|
||||
|
|
@ -1266,3 +1318,45 @@ test('S38: forward-guard — no product-facing doc asserts main-context relief u
|
|||
offenders.join('\n'),
|
||||
);
|
||||
});
|
||||
|
||||
// ── v5.9 (fable-tier step 6) — composed-resolver wiring pin ────────────────
|
||||
// The four pipeline commands must resolve {effort, model} via the composed
|
||||
// CLI (`resolver.mjs --resolve-phase-model`, brief > profile > default). A
|
||||
// direct `phase-signal-resolver.mjs --brief` invocation in a command Bash
|
||||
// block is the brief-only CLI: it silently drops the profile layer (the
|
||||
// AP2-1 footgun — `--profile <x>` would never reach sub-agent spawns again).
|
||||
// If this pin fails, re-wire the command to the composed CLI — do not relax
|
||||
// the pin.
|
||||
|
||||
for (const cmd of ['trekresearch', 'trekplan', 'trekreview', 'trekexecute']) {
|
||||
test(`v5.9: commands/${cmd}.md invokes the composed resolver, not the brief-only CLI`, () => {
|
||||
const text = read(`commands/${cmd}.md`);
|
||||
assert.ok(
|
||||
text.includes('--resolve-phase-model'),
|
||||
`commands/${cmd}.md must invoke resolver.mjs --resolve-phase-model (composed brief > profile > default)`,
|
||||
);
|
||||
assert.ok(
|
||||
!/phase-signal-resolver\.mjs --brief/.test(text),
|
||||
`commands/${cmd}.md must not invoke the brief-only phase-signal-resolver CLI — the profile layer would be silently dropped`,
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
// ── v5.9 (fable-tier step 7) — orchestrator model-pin ABSENCE pin ───────────
|
||||
// Command frontmatter deliberately omits `model:` — omission is the only
|
||||
// session-inheritance spelling documented by BOTH official surfaces (AP2-2:
|
||||
// the `inherit` literal is disputed between skills.md and the plugin-dev
|
||||
// command-frontmatter reference). Re-adding a pin locks the orchestrator to a
|
||||
// fixed model in every session; if deterministic pinning is ever wanted again,
|
||||
// do it consciously and update this pin's rationale.
|
||||
|
||||
test('v5.9: no commands/*.md frontmatter carries a model: key (session inheritance by omission)', () => {
|
||||
const offenders = [];
|
||||
for (const f of listMd('commands')) {
|
||||
const doc = parseDocument(read(`commands/${f}`));
|
||||
const fm = doc.parsed && doc.parsed.frontmatter;
|
||||
if (fm && 'model' in fm) offenders.push(f);
|
||||
}
|
||||
assert.deepEqual(offenders, [],
|
||||
`command frontmatter must omit model: (orchestrator follows the session model); offenders: ${offenders.join(', ')}`);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ test('resolvePhaseSignal — defensive: null/non-object input returns null', ()
|
|||
|
||||
test('resolvePhaseSignal — drops model not in BASE_ALLOWED_MODELS (defense-in-depth gate)', () => {
|
||||
// MAJOR fix (S4): line that copies `model` must gate against the same
|
||||
// allowlist brief-validator uses (BASE_ALLOWED_MODELS = ['sonnet','opus']),
|
||||
// allowlist brief-validator uses (BASE_ALLOWED_MODELS = ['sonnet','opus','fable']),
|
||||
// mirroring how effort is gated against EFFORT_LEVELS. A brief that slipped
|
||||
// validation (hand-edited, validation skipped) must not hand a junk model
|
||||
// string to a command that then spawns an agent with `model: <junk>`.
|
||||
|
|
@ -70,15 +70,17 @@ test('resolvePhaseSignal — drops model not in BASE_ALLOWED_MODELS (defense-in-
|
|||
assert.ok(!('model' in review), 'model key absent for haiku');
|
||||
});
|
||||
|
||||
test('resolvePhaseSignal — keeps valid models (sonnet, opus) after gating', () => {
|
||||
test('resolvePhaseSignal — keeps valid models (sonnet, opus, fable) after gating', () => {
|
||||
const fm = {
|
||||
phase_signals: [
|
||||
{ phase: 'research', effort: 'low', model: 'sonnet' },
|
||||
{ phase: 'execute', effort: 'high', model: 'opus' },
|
||||
{ phase: 'review', effort: 'high', model: 'fable' },
|
||||
],
|
||||
};
|
||||
assert.equal(resolvePhaseSignal(fm, 'research').model, 'sonnet');
|
||||
assert.equal(resolvePhaseSignal(fm, 'execute').model, 'opus');
|
||||
assert.equal(resolvePhaseSignal(fm, 'review').model, 'fable');
|
||||
});
|
||||
|
||||
test('resolvePhaseSignalFromFile + CLI shim — writes JSON to stdout, exit 0', () => {
|
||||
|
|
|
|||
|
|
@ -50,6 +50,20 @@ test('SC #5: loadProfile("premium") returns all-opus', () => {
|
|||
}
|
||||
});
|
||||
|
||||
test('SC #5: loadProfile("fable") returns all-fable (BUILTIN_NAMES canary)', () => {
|
||||
// Throws PROFILE_NOT_FOUND if 'fable' regresses out of BUILTIN_NAMES —
|
||||
// the silent-fallback-to-premium failure mode this pin exists to catch.
|
||||
const p = loadProfile('fable');
|
||||
assert.equal(p.name, 'fable');
|
||||
for (const phase of ['brief', 'research', 'plan', 'execute', 'review', 'continue']) {
|
||||
assert.equal(p.phase_models[phase], 'fable', `fable ${phase} should be fable`);
|
||||
}
|
||||
assert.equal(p.parallel_agents_min, 6);
|
||||
assert.equal(p.parallel_agents_max, 8);
|
||||
assert.equal(p.external_research_enabled, true);
|
||||
assert.equal(p.brief_reviewer_iter_cap, 3);
|
||||
});
|
||||
|
||||
test('SC #5: loadProfile throws PROFILE_NOT_FOUND for unknown profile', () => {
|
||||
try {
|
||||
loadProfile('does-not-exist-xyz');
|
||||
|
|
|
|||
|
|
@ -60,3 +60,30 @@ test('resolvePhaseModel — Case 6 (defensive): null briefPath falls through to
|
|||
assert.equal(r.model, 'opus', 'premium.plan default = opus');
|
||||
assert.equal(r.source, 'default');
|
||||
});
|
||||
|
||||
// v5.9 — fable profile composition (brief SC 4) + effort passthrough coherence
|
||||
|
||||
test('resolvePhaseModel — --profile fable: all six phases resolve model fable (no brief signal)', () => {
|
||||
for (const phase of ['brief', 'research', 'plan', 'execute', 'review', 'continue']) {
|
||||
const r = resolvePhaseModel(phase, null, ['--profile', 'fable'], {});
|
||||
assert.equal(r.model, 'fable', `fable.${phase} should be fable; got ${JSON.stringify(r)}`);
|
||||
assert.equal(r.source, 'flag');
|
||||
}
|
||||
});
|
||||
|
||||
test('resolvePhaseModel — brief signal (opus) beats --profile fable (composition precedence pin)', () => {
|
||||
// brief-effort-high.md pins execute to model: opus. Brief must beat the fable profile.
|
||||
const r = resolvePhaseModel('execute', FIXTURE('brief-effort-high.md'), ['--profile', 'fable'], {});
|
||||
assert.equal(r.model, 'opus', `brief signal should beat fable profile; got ${JSON.stringify(r)}`);
|
||||
assert.equal(r.source, 'brief-signal');
|
||||
});
|
||||
|
||||
test('resolvePhaseModel — composed output carries {effort, model} atomically (v5.9 passthrough)', () => {
|
||||
// brief-effort-high.md: execute → {effort: high, model: opus}. The composed
|
||||
// resolver must return BOTH fields from one call — the split-CLI design this
|
||||
// passthrough replaced could drift effort and model apart.
|
||||
const r = resolvePhaseModel('execute', FIXTURE('brief-effort-high.md'), [], {});
|
||||
assert.equal(r.effort, 'high', `effort must pass through; got ${JSON.stringify(r)}`);
|
||||
assert.equal(r.model, 'opus');
|
||||
assert.equal(r.source, 'brief-signal');
|
||||
});
|
||||
|
|
|
|||
|
|
@ -94,6 +94,81 @@ test('SC #11(b): commands/trekplan.md prose mentions phase_models + parallel_age
|
|||
'trekplan.md prose must mention parallel_agents (additive stats field)');
|
||||
});
|
||||
|
||||
// --- Step 9: the five STORM measurement fields -----------------------------
|
||||
// Four numerics + one low-cardinality label. Without `effort` there is no axis
|
||||
// to group high-vs-standard runs on, and the measurement gate cannot be
|
||||
// computed at all.
|
||||
|
||||
const STORM_FIELDS = [
|
||||
'effort',
|
||||
'unique_sources',
|
||||
'dimensions_baseline',
|
||||
'conv_turns',
|
||||
'empty_turns',
|
||||
];
|
||||
|
||||
test('Step 9: a standard-run trekresearch record parses and survives applyFieldAllowlist', async () => {
|
||||
const { applyFieldAllowlist } = await import('../../lib/exporters/field-allowlist.mjs');
|
||||
// A standard run: loop never armed, so no discovered dimensions and no turns.
|
||||
const raw = JSON.parse(JSON.stringify({
|
||||
_schema_id: 'trekresearch',
|
||||
ts: '2026-08-09T12:00:00.000Z',
|
||||
slug: 'add-auth',
|
||||
mode: 'default',
|
||||
scope: 'both',
|
||||
engine: 'swarm',
|
||||
question: 'free prose that must never reach the exporter',
|
||||
project_dir: '/Users/somebody/repos/x',
|
||||
brief_path: '/Users/somebody/repos/x/brief.md',
|
||||
dimensions: 4,
|
||||
dimensions_baseline: 4,
|
||||
conv_turns: 0,
|
||||
empty_turns: 0,
|
||||
unique_sources: 11,
|
||||
effort: 'standard',
|
||||
agents_local: 5,
|
||||
agents_external: 4,
|
||||
gemini_used: false,
|
||||
confidence: 0.8,
|
||||
contradictions: 1,
|
||||
open_questions: 2,
|
||||
profile: 'premium',
|
||||
profile_source: 'default',
|
||||
}));
|
||||
|
||||
assert.equal(raw.dimensions_baseline, raw.dimensions,
|
||||
'a standard run discovers no dimensions — baseline must equal the final count');
|
||||
|
||||
const out = applyFieldAllowlist(raw, 'trekresearch');
|
||||
for (const field of STORM_FIELDS) {
|
||||
assert.ok(field in out, `${field} must survive the trekresearch allowlist`);
|
||||
}
|
||||
assert.equal(out.conv_turns, 0);
|
||||
assert.equal(out.empty_turns, 0);
|
||||
assert.equal(out.effort, 'standard');
|
||||
// Deny-by-omission must still hold for the PII-ish fields.
|
||||
for (const denied of ['question', 'project_dir', 'brief_path']) {
|
||||
assert.equal(denied in out, false, `${denied} must NOT reach the exporter`);
|
||||
}
|
||||
});
|
||||
|
||||
test('Step 9: commands/trekresearch.md prose names all five measurement fields', () => {
|
||||
const content = readFileSync(join(REPO_ROOT, 'commands', 'trekresearch.md'), 'utf-8');
|
||||
for (const field of STORM_FIELDS) {
|
||||
assert.ok(content.includes(field),
|
||||
`trekresearch.md prose must name ${field} — an emitted-but-undocumented field is unauditable`);
|
||||
}
|
||||
});
|
||||
|
||||
test('Step 9: tests/fixtures/jsonl-schemas.md trekresearch row lists the five fields', () => {
|
||||
const doc = readFileSync(join(REPO_ROOT, 'tests', 'fixtures', 'jsonl-schemas.md'), 'utf-8');
|
||||
const row = doc.split('\n').find(l => l.startsWith('| trekresearch-stats '));
|
||||
assert.ok(row, 'jsonl-schemas.md is missing the trekresearch-stats row');
|
||||
for (const field of STORM_FIELDS) {
|
||||
assert.ok(row.includes(field), `authoring reference must list ${field}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('SC #11(b): commands/trekresearch.md prose mentions external_research_enabled', () => {
|
||||
const content = readFileSync(join(REPO_ROOT, 'commands', 'trekresearch.md'), 'utf-8');
|
||||
assert.match(content, /external_research_enabled/,
|
||||
|
|
|
|||
233
tests/lib/research-loop-cap.test.mjs
Normal file
233
tests/lib/research-loop-cap.test.mjs
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
// tests/lib/research-loop-cap.test.mjs
|
||||
// Cover lib/util/research-loop-cap.mjs: default-off, worst-case arithmetic,
|
||||
// anti-dead-data (different caps → different denial points), statefulness
|
||||
// (identical args → different answers once the budget is hit), env
|
||||
// coercion, fail-closed on missing CLAUDE_PLUGIN_DATA, and the CLI shim.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdtempSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
allowTurn,
|
||||
isStormEnabled,
|
||||
resolveMaxConvTurns,
|
||||
resolveLedgerPath,
|
||||
MAX_CONV_TURNS,
|
||||
MAX_TOTAL_DIMENSIONS,
|
||||
} from '../../lib/util/research-loop-cap.mjs';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const SHIM = join(HERE, '..', '..', 'lib', 'util', 'research-loop-cap.mjs');
|
||||
|
||||
function withTmpDataDir(fn) {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'research-loop-cap-'));
|
||||
try {
|
||||
return fn(dir);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function runShim(args, env) {
|
||||
try {
|
||||
const out = execFileSync(process.execPath, [SHIM, ...args], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
env: { ...process.env, ...env },
|
||||
});
|
||||
return { code: 0, out };
|
||||
} catch (e) {
|
||||
return { code: e.status ?? 1, out: e.stdout?.toString() ?? '' };
|
||||
}
|
||||
}
|
||||
|
||||
// ---- (a) default-off --------------------------------------------------------
|
||||
|
||||
test('allowTurn — VOYAGE_STORM_ENABLED unset denies with budget 0, regardless of effort', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir };
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'storm_disabled');
|
||||
assert.equal(r.budget, 0);
|
||||
});
|
||||
});
|
||||
|
||||
test('allowTurn — VOYAGE_STORM_ENABLED=0 denies same as unset', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '0' };
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'storm_disabled');
|
||||
});
|
||||
});
|
||||
|
||||
test('allowTurn — enabled but effort !== high denies with budget 0', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'standard' }, { env });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'effort_not_high');
|
||||
assert.equal(r.budget, 0);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (f) CLAUDE_PLUGIN_DATA unset => fail-closed deny -----------------------
|
||||
|
||||
test('allowTurn — CLAUDE_PLUGIN_DATA unset denies even when enabled + high effort', () => {
|
||||
const env = { VOYAGE_STORM_ENABLED: '1' }; // no CLAUDE_PLUGIN_DATA
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'no_plugin_data_dir');
|
||||
assert.equal(r.budget, MAX_CONV_TURNS * MAX_TOTAL_DIMENSIONS);
|
||||
});
|
||||
|
||||
// ---- (c) worst-case arithmetic ----------------------------------------------
|
||||
|
||||
test('allowTurn — budget is max_conv_turns × max_total_dimensions (default 3×8=24)', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r.ok, true);
|
||||
assert.equal(r.budget, 24);
|
||||
assert.equal(r.used, 1);
|
||||
});
|
||||
});
|
||||
|
||||
test('allowTurn — grants exactly `budget` turns then denies the next one (default 24)', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
|
||||
let last;
|
||||
for (let i = 0; i < 24; i++) {
|
||||
last = allowTurn({ runId: 'r-exhaust', dimension: `d${i % 8}`, effort: 'high' }, { env });
|
||||
assert.equal(last.ok, true, `turn ${i + 1} should be granted`);
|
||||
}
|
||||
const denied = allowTurn({ runId: 'r-exhaust', dimension: 'd0', effort: 'high' }, { env });
|
||||
assert.equal(denied.ok, false);
|
||||
assert.equal(denied.reason, 'budget_exhausted');
|
||||
assert.equal(denied.used, 24);
|
||||
assert.equal(denied.budget, 24);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (c)/(anti-dead-data) — different caps → observably different denial points
|
||||
|
||||
test('allowTurn — TREKRESEARCH_MAX_CONV_TURNS=1 denies after 8 turns (1×8), not 24', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
|
||||
let last;
|
||||
for (let i = 0; i < 8; i++) {
|
||||
last = allowTurn({ runId: 'r-narrow', dimension: `d${i}`, effort: 'high' }, { env });
|
||||
assert.equal(last.ok, true, `turn ${i + 1} should be granted`);
|
||||
}
|
||||
const denied = allowTurn({ runId: 'r-narrow', dimension: 'd8', effort: 'high' }, { env });
|
||||
assert.equal(denied.ok, false);
|
||||
assert.equal(denied.budget, 8);
|
||||
assert.notEqual(denied.budget, 24, 'a narrower cap must produce a different denial point than the default');
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (d) stateful — identical args give different answers once exhausted ---
|
||||
|
||||
test('allowTurn — identical {runId, dimension, effort} args diverge once the budget is hit (proves statefulness)', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
|
||||
const args = { runId: 'r-identical', dimension: 'same-dim', effort: 'high' };
|
||||
const results = [];
|
||||
for (let i = 0; i < 9; i++) results.push(allowTurn(args, { env }));
|
||||
// First 8 (budget = 1*8) granted, 9th denied — same exact input object each time.
|
||||
assert.deepEqual(results.slice(0, 8).map(r => r.ok), Array(8).fill(true));
|
||||
assert.equal(results[8].ok, false);
|
||||
assert.equal(results[8].reason, 'budget_exhausted');
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (e) env coercion --------------------------------------------------------
|
||||
|
||||
test('resolveMaxConvTurns — NaN string falls back to default', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: 'abc' }), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — empty string falls back to default', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '' }), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — negative value falls back to default', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '-5' }), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — zero falls back to default (never unbounded)', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '0' }), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — unset falls back to default', () => {
|
||||
assert.equal(resolveMaxConvTurns({}), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — valid positive integer string is honored', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '2' }), 2);
|
||||
});
|
||||
|
||||
// ---- pure-core unit coverage --------------------------------------------------
|
||||
|
||||
test('isStormEnabled — only the literal string "1" enables', () => {
|
||||
assert.equal(isStormEnabled({ VOYAGE_STORM_ENABLED: '1' }), true);
|
||||
assert.equal(isStormEnabled({ VOYAGE_STORM_ENABLED: 'true' }), false);
|
||||
assert.equal(isStormEnabled({}), false);
|
||||
});
|
||||
|
||||
test('resolveLedgerPath — null when CLAUDE_PLUGIN_DATA unset or empty', () => {
|
||||
assert.equal(resolveLedgerPath({}), null);
|
||||
assert.equal(resolveLedgerPath({ CLAUDE_PLUGIN_DATA: '' }), null);
|
||||
});
|
||||
|
||||
test('resolveLedgerPath — joins CLAUDE_PLUGIN_DATA with the ledger filename', () => {
|
||||
const p = resolveLedgerPath({ CLAUDE_PLUGIN_DATA: '/tmp/plugin-data' });
|
||||
assert.equal(p, join('/tmp/plugin-data', 'trekresearch-loop-ledger.jsonl'));
|
||||
});
|
||||
|
||||
test('allowTurn — missing runId or dimension denies with missing_args', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
|
||||
const r1 = allowTurn({ dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r1.ok, false);
|
||||
assert.equal(r1.reason, 'missing_args');
|
||||
const r2 = allowTurn({ runId: 'r1', effort: 'high' }, { env });
|
||||
assert.equal(r2.ok, false);
|
||||
assert.equal(r2.reason, 'missing_args');
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (g) shim contract --------------------------------------------------------
|
||||
|
||||
test('CLI shim — grants and exits 0 when enabled + high effort + budget available', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const r = runShim(
|
||||
['--run-id', 'shim-1', '--dimension', 'd1', '--effort', 'high'],
|
||||
{ CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' },
|
||||
);
|
||||
assert.equal(r.code, 0);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.ok, true);
|
||||
});
|
||||
});
|
||||
|
||||
test('CLI shim — denies and exits 1 when disabled', () => {
|
||||
const r = runShim(['--run-id', 'shim-2', '--dimension', 'd1', '--effort', 'high'], { VOYAGE_STORM_ENABLED: '0' });
|
||||
assert.equal(r.code, 1);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.ok, false);
|
||||
assert.equal(parsed.reason, 'storm_disabled');
|
||||
});
|
||||
|
||||
test('CLI shim — missing required args exits 1 with usage reason', () => {
|
||||
const r = runShim(['--run-id', 'shim-3']);
|
||||
assert.equal(r.code, 1);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.ok, false);
|
||||
assert.match(parsed.reason, /usage:/);
|
||||
});
|
||||
|
|
@ -75,6 +75,15 @@ test('deriveCost — hand-computed value for a known model (cache-aware)', () =>
|
|||
assert.equal(is_estimate, false);
|
||||
});
|
||||
|
||||
test('deriveCost — hand-computed value for claude-fable-5 (v5.9 fable tier)', () => {
|
||||
const totals = { tokens_input: 3000, tokens_output: 500, tokens_cache_creation: 1000, tokens_cache_read: 6000 };
|
||||
const { cost_usd, is_estimate } = deriveCost(totals, 'claude-fable-5');
|
||||
// (3000×10 + 500×50 + 1000×12.5 + 6000×1) / 1e6
|
||||
// = (30000 + 25000 + 12500 + 6000) / 1e6 = 73500 / 1e6 = 0.0735
|
||||
assert.equal(cost_usd, 0.0735);
|
||||
assert.equal(is_estimate, false);
|
||||
});
|
||||
|
||||
test('deriveCost — refuse-to-estimate for an unknown model', () => {
|
||||
const totals = { tokens_input: 3000, tokens_output: 500, tokens_cache_creation: 1000, tokens_cache_read: 6000 };
|
||||
const out = deriveCost(totals, 'claude-unknown-9');
|
||||
|
|
|
|||
|
|
@ -1,5 +1,9 @@
|
|||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { validateBriefContent } from '../../lib/validators/brief-validator.mjs';
|
||||
|
||||
const GOOD_BRIEF = `---
|
||||
|
|
@ -251,6 +255,25 @@ test('validateBrief — v5.1.1: UNQUOTED brief_version 2.1 WITH phase_signals is
|
|||
assert.ok(!r.errors.find(e => e.code === 'BRIEF_V51_MISSING_SIGNALS'));
|
||||
});
|
||||
|
||||
// --- v5.9 — fable model tier (BASE_ALLOWED_MODELS widened to three values) ---
|
||||
|
||||
test('validateBrief — v5.9: fable phase_signals fixture accepted (no BRIEF_INVALID_MODEL)', () => {
|
||||
const t = readFileSync(new URL('../fixtures/brief-effort-fable.md', import.meta.url), 'utf-8');
|
||||
const r = validateBriefContent(t, { strict: true });
|
||||
assert.equal(r.valid, true, JSON.stringify(r.errors));
|
||||
assert.ok(!r.errors.find(e => e.code === 'BRIEF_INVALID_MODEL'));
|
||||
});
|
||||
|
||||
test('validateBrief — v5.9: unknown model gpt5 in phase_signals rejected with BRIEF_INVALID_MODEL', () => {
|
||||
const t = GOOD_BRIEF
|
||||
.replace('brief_version: "2.0"', 'brief_version: "2.1"')
|
||||
.replace('source: interview\n', `source: interview\n${SIGNALS_BLOCK.replace('model: opus', 'model: gpt5')}`);
|
||||
const r = validateBriefContent(t, { strict: true });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === 'BRIEF_INVALID_MODEL'),
|
||||
`expected BRIEF_INVALID_MODEL for gpt5, got: ${JSON.stringify(r.errors)}`);
|
||||
});
|
||||
|
||||
// --- v5.5 — framing enforcement + obligatory TL;DR (gated at brief_version ≥ 2.2) ---
|
||||
// Operator decision (S6, option A1): framing + TL;DR are hard BLOCKERs for briefs
|
||||
// declaring brief_version ≥ 2.2; existing 2.0/2.1 briefs stay valid (forward-compat,
|
||||
|
|
@ -393,3 +416,38 @@ test('validateBrief — S18 min-version: trekreview brief is exempt (no framing
|
|||
const r = validateBriefContent(REVIEW_AS_BRIEF, { minBriefVersion: '2.2' });
|
||||
assert.ok(!r.warnings.find(w => w.code === 'BRIEF_VERSION_BELOW_MINIMUM'));
|
||||
});
|
||||
|
||||
// S56 — CLI arg-parsing regression. The no-flag invocation `brief-validator.mjs <brief.md>`
|
||||
// used to bail to Usage/exit 2 because the --min-version skip index was 0 when the flag
|
||||
// was absent, dropping the file positional (which sits at argv index 0).
|
||||
test('CLI — no-flag invocation reaches validation, does not bail to Usage (S56 regression)', () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'brief-cli-'));
|
||||
try {
|
||||
const file = join(dir, 'brief.md');
|
||||
writeFileSync(file, GOOD_BRIEF);
|
||||
// execFileSync throws on non-zero exit; pre-fix this bailed to Usage (exit 2).
|
||||
const out = execFileSync(process.execPath, [
|
||||
'lib/validators/brief-validator.mjs',
|
||||
file,
|
||||
], { encoding: 'utf-8' });
|
||||
assert.match(out, /PASS/);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('CLI — --min-version still locates the file positional after the value token (S56)', () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'brief-cli-'));
|
||||
try {
|
||||
const file = join(dir, 'brief.md');
|
||||
writeFileSync(file, GOOD_BRIEF);
|
||||
const out = execFileSync(process.execPath, [
|
||||
'lib/validators/brief-validator.mjs',
|
||||
'--min-version', '2.0',
|
||||
file,
|
||||
], { encoding: 'utf-8' });
|
||||
assert.match(out, /PASS/);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
// tests/validators/profile-validator.test.mjs
|
||||
// SC #1, #2, #3: profile-validator validates lib/profiles/{economy,balanced,premium}.yaml
|
||||
// SC #1, #2, #3: profile-validator validates lib/profiles/{economy,balanced,premium,fable}.yaml
|
||||
// (innebygde profiler) plus rejects invalid models and invalid enum types.
|
||||
|
||||
import { test } from 'node:test';
|
||||
|
|
@ -12,14 +12,15 @@ import {
|
|||
validateProfileContent,
|
||||
PROFILE_REQUIRED_FIELDS,
|
||||
PROFILE_REQUIRED_PHASES,
|
||||
BASE_ALLOWED_MODELS,
|
||||
} from '../../lib/validators/profile-validator.mjs';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const REPO_ROOT = join(__dirname, '..', '..');
|
||||
|
||||
// SC #1: alle 3 innebygde profiler grønne
|
||||
// SC #1: alle 4 innebygde profiler grønne
|
||||
|
||||
for (const profileName of ['economy', 'balanced', 'premium']) {
|
||||
for (const profileName of ['economy', 'balanced', 'premium', 'fable']) {
|
||||
test(`SC #1: lib/profiles/${profileName}.yaml validates clean`, () => {
|
||||
const r = validateProfile(join(REPO_ROOT, 'lib', 'profiles', `${profileName}.yaml`));
|
||||
assert.equal(r.valid, true,
|
||||
|
|
@ -93,6 +94,77 @@ brief_reviewer_iter_cap: 1
|
|||
`expected valid with VOYAGE_ALLOW_HAIKU=1, got: ${JSON.stringify(allowed.errors)}`);
|
||||
});
|
||||
|
||||
// Fable tier (v5.9): fable accepted under DEFAULT env (no opt-in flag),
|
||||
// unknown models still rejected — the allowlist gate must demonstrably fire.
|
||||
|
||||
test('fable accepted in phase_models under default env (no env flag)', () => {
|
||||
const fableProfile = `---
|
||||
profile_version: "1.0"
|
||||
name: fable-inline
|
||||
phase_models:
|
||||
- phase: brief
|
||||
model: fable
|
||||
- phase: research
|
||||
model: fable
|
||||
- phase: plan
|
||||
model: fable
|
||||
- phase: execute
|
||||
model: fable
|
||||
- phase: review
|
||||
model: fable
|
||||
- phase: continue
|
||||
model: fable
|
||||
parallel_agents_min: 6
|
||||
parallel_agents_max: 8
|
||||
external_research_enabled: true
|
||||
brief_reviewer_iter_cap: 3
|
||||
---
|
||||
`;
|
||||
const r = validateProfileContent(fableProfile, { env: { /* default: no flags */ } });
|
||||
assert.equal(r.valid, true,
|
||||
`expected fable accepted under default env, got: ${JSON.stringify(r.errors)}`);
|
||||
assert.equal(r.errors.length, 0);
|
||||
});
|
||||
|
||||
test('unknown model gpt5 rejected with PROFILE_INVALID_MODEL under default env', () => {
|
||||
const gpt5Profile = `---
|
||||
profile_version: "1.0"
|
||||
name: gpt5-inline
|
||||
phase_models:
|
||||
- phase: brief
|
||||
model: gpt5
|
||||
- phase: research
|
||||
model: sonnet
|
||||
- phase: plan
|
||||
model: opus
|
||||
- phase: execute
|
||||
model: sonnet
|
||||
- phase: review
|
||||
model: opus
|
||||
- phase: continue
|
||||
model: sonnet
|
||||
parallel_agents_min: 2
|
||||
parallel_agents_max: 4
|
||||
external_research_enabled: false
|
||||
brief_reviewer_iter_cap: 1
|
||||
---
|
||||
`;
|
||||
const r = validateProfileContent(gpt5Profile, { env: { /* default: no flags */ } });
|
||||
assert.equal(r.valid, false);
|
||||
const found = r.errors.find(e => e.code === 'PROFILE_INVALID_MODEL' && /gpt5/.test(e.message));
|
||||
assert.ok(found, `expected PROFILE_INVALID_MODEL for gpt5, got: ${JSON.stringify(r.errors)}`);
|
||||
});
|
||||
|
||||
// BASE_ALLOWED_MODELS allowlist drift-pin (mirrors the PROFILE_REQUIRED_FIELDS pin)
|
||||
|
||||
test('BASE_ALLOWED_MODELS export drift-pin', () => {
|
||||
assert.deepEqual(
|
||||
[...BASE_ALLOWED_MODELS],
|
||||
['sonnet', 'opus', 'fable'],
|
||||
'BASE_ALLOWED_MODELS contract drift — pin contract',
|
||||
);
|
||||
});
|
||||
|
||||
// Required fields presence
|
||||
|
||||
test('PROFILE_MISSING_FIELD when name absent', () => {
|
||||
|
|
|
|||
143
tests/validators/query-privacy-gate.test.mjs
Normal file
143
tests/validators/query-privacy-gate.test.mjs
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
// tests/validators/query-privacy-gate.test.mjs
|
||||
// Cover lib/validators/query-privacy-gate.mjs: two-sided code table
|
||||
// (absolute path / repo-internal identifier / secret-shaped token), a
|
||||
// benign query passing untouched, the opt-in env var reaching only the
|
||||
// warn tier (never the hard-block tier), strict/soft severity, and the
|
||||
// CLI shim.
|
||||
//
|
||||
// Secret-shaped fixtures are built via string concatenation/repeat, never
|
||||
// as literal tokens — the repo's own secrets pre-edit hook (correctly)
|
||||
// treats a literal AKIA/sk-/ghp_ string as a real credential.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
validateOutboundQuery,
|
||||
ABSOLUTE_PATH_PATTERNS,
|
||||
REPO_IDENTIFIER_PATTERNS,
|
||||
SECRET_SHAPED_PATTERNS,
|
||||
} from '../../lib/validators/query-privacy-gate.mjs';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const SHIM = join(HERE, '..', '..', 'lib', 'validators', 'query-privacy-gate.mjs');
|
||||
|
||||
const FAKE_OPENAI_KEY = 'sk-' + 'a'.repeat(24);
|
||||
const FAKE_AWS_KEY = 'AKIA' + 'Q'.repeat(16);
|
||||
const FAKE_GITHUB_PAT = 'ghp_' + 'b'.repeat(36);
|
||||
|
||||
function runShim(args) {
|
||||
try {
|
||||
const out = execFileSync(process.execPath, [SHIM, ...args], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
});
|
||||
return { code: 0, out };
|
||||
} catch (e) {
|
||||
return { code: e.status ?? 1, out: e.stdout?.toString() ?? '' };
|
||||
}
|
||||
}
|
||||
|
||||
// ---- two-sided code table ----------------------------------------------------
|
||||
|
||||
const TABLE = [
|
||||
{ label: 'absolute path (/Users/...)', text: 'find every caller of foo in /Users/ktg/repos/voyage/lib/util/foo.mjs', code: 'PRIVACY_ABSOLUTE_PATH' },
|
||||
{ label: 'absolute path (/home/...)', text: 'trace /home/alice/projects/app/src/index.js for imports', code: 'PRIVACY_ABSOLUTE_PATH' },
|
||||
{ label: 'repo-internal identifier (forgejo host)', text: 'what changed recently on git.fromaitochitta.com/open/voyage', code: 'PRIVACY_REPO_IDENTIFIER' },
|
||||
{ label: 'repo-internal identifier (repo name)', text: 'search issues for ktg-plugin-marketplace regressions', code: 'PRIVACY_REPO_IDENTIFIER' },
|
||||
{ label: 'secret-shaped (OpenAI/Anthropic-style key)', text: `auth failing with key ${FAKE_OPENAI_KEY}`, code: 'PRIVACY_SECRET_SHAPED' },
|
||||
{ label: 'secret-shaped (AWS access key)', text: `rotate ${FAKE_AWS_KEY} now`, code: 'PRIVACY_SECRET_SHAPED' },
|
||||
{ label: 'secret-shaped (GitHub PAT)', text: `token leaked: ${FAKE_GITHUB_PAT}`, code: 'PRIVACY_SECRET_SHAPED' },
|
||||
];
|
||||
|
||||
for (const { label, text, code } of TABLE) {
|
||||
test(`validateOutboundQuery — ${label} → ${code} (strict, error)`, () => {
|
||||
const r = validateOutboundQuery(text, { strict: true, env: {} });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === code), JSON.stringify(r.errors));
|
||||
});
|
||||
}
|
||||
|
||||
test('validateOutboundQuery — benign generic query passes untouched', () => {
|
||||
const r = validateOutboundQuery('What are the tradeoffs between optimistic and pessimistic locking?', { env: {} });
|
||||
assert.equal(r.valid, true);
|
||||
assert.deepEqual(r.errors, []);
|
||||
assert.deepEqual(r.warnings, []);
|
||||
});
|
||||
|
||||
// ---- strict vs soft (warn tier only) -----------------------------------------
|
||||
|
||||
test('validateOutboundQuery — soft mode downgrades warn-tier findings to warnings, stays valid', () => {
|
||||
const r = validateOutboundQuery('inspect /Users/ktg/repos/voyage', { strict: false, env: {} });
|
||||
assert.equal(r.valid, true);
|
||||
assert.equal(r.errors.length, 0);
|
||||
assert.ok(r.warnings.find(w => w.code === 'PRIVACY_ABSOLUTE_PATH'));
|
||||
});
|
||||
|
||||
test('validateOutboundQuery — soft mode does NOT downgrade the hard-block tier', () => {
|
||||
const r = validateOutboundQuery(`leaked ${FAKE_OPENAI_KEY}`, { strict: false, env: {} });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === 'PRIVACY_SECRET_SHAPED'));
|
||||
});
|
||||
|
||||
// ---- opt-in env var reaches only the warn tier -------------------------------
|
||||
|
||||
test('validateOutboundQuery — VOYAGE_QUERY_PRIVACY_ALLOW=1 bypasses the warn tier entirely', () => {
|
||||
const r = validateOutboundQuery('inspect /Users/ktg/repos/voyage', { env: { VOYAGE_QUERY_PRIVACY_ALLOW: '1' } });
|
||||
assert.equal(r.valid, true);
|
||||
assert.equal(r.errors.length, 0);
|
||||
assert.equal(r.warnings.length, 0);
|
||||
});
|
||||
|
||||
test('validateOutboundQuery — VOYAGE_QUERY_PRIVACY_ALLOW=1 does NOT open the hard-block tier', () => {
|
||||
const r = validateOutboundQuery(`leaked ${FAKE_OPENAI_KEY}`, { env: { VOYAGE_QUERY_PRIVACY_ALLOW: '1' } });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === 'PRIVACY_SECRET_SHAPED'), 'opt-in must never unlock the hard-block tier');
|
||||
});
|
||||
|
||||
test('validateOutboundQuery — VOYAGE_QUERY_PRIVACY_ALLOW=1 combined with a secret still denies', () => {
|
||||
const r = validateOutboundQuery(`/Users/ktg/x leaked ${FAKE_OPENAI_KEY}`, { env: { VOYAGE_QUERY_PRIVACY_ALLOW: '1' } });
|
||||
assert.equal(r.valid, false);
|
||||
assert.equal(r.errors.length, 1);
|
||||
assert.equal(r.errors[0].code, 'PRIVACY_SECRET_SHAPED');
|
||||
});
|
||||
|
||||
// ---- empty input --------------------------------------------------------------
|
||||
|
||||
test('validateOutboundQuery — empty string is invalid', () => {
|
||||
const r = validateOutboundQuery('', { env: {} });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === 'PRIVACY_EMPTY_QUERY'));
|
||||
});
|
||||
|
||||
// ---- pattern set is frozen ----------------------------------------------------
|
||||
|
||||
test('pattern sets are Object.frozen', () => {
|
||||
assert.equal(Object.isFrozen(ABSOLUTE_PATH_PATTERNS), true);
|
||||
assert.equal(Object.isFrozen(REPO_IDENTIFIER_PATTERNS), true);
|
||||
assert.equal(Object.isFrozen(SECRET_SHAPED_PATTERNS), true);
|
||||
});
|
||||
|
||||
// ---- CLI shim -----------------------------------------------------------------
|
||||
|
||||
test('CLI shim — benign query exits 0 with valid:true', () => {
|
||||
const r = runShim(['harmless generic question about caching strategies']);
|
||||
assert.equal(r.code, 0);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.valid, true);
|
||||
});
|
||||
|
||||
test('CLI shim — secret-shaped query exits 1 even with --soft', () => {
|
||||
const r = runShim(['--soft', `leaked ${FAKE_OPENAI_KEY}`]);
|
||||
assert.equal(r.code, 1);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.valid, false);
|
||||
assert.ok(parsed.errors.find(e => e.code === 'PRIVACY_SECRET_SHAPED'));
|
||||
});
|
||||
|
||||
test('CLI shim — missing query argument exits 2 (usage error)', () => {
|
||||
const r = runShim([]);
|
||||
assert.equal(r.code, 2);
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue