Compare commits
13 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f3874946ad | |||
| d9cba9c6ea | |||
| 3e8af75015 | |||
| 0a569eec55 | |||
| abc5bd8967 | |||
| 0556743bad | |||
| 9a38500a63 | |||
| e1cf545a0c | |||
| 5bb6735c94 | |||
| 9ffeae0e2e | |||
| 2728a43656 | |||
| 79044624db | |||
| 8d39e1d4a5 |
25 changed files with 1553 additions and 73 deletions
|
|
@ -1,12 +1,23 @@
|
|||
{
|
||||
"name": "voyage",
|
||||
"description": "Voyage — brief, research, plan, execute, review, continue. Contract-driven Claude Code pipeline. /trekbrief, /trekplan, and /trekreview each end by building a self-contained operator-annotation HTML (scripts/annotate.mjs, modelled on claude-code-100x): select text or click any element, pick intent (Fiks/Endre/Spørsmål), write comment, copy structured prompt, paste back, Claude revises the .md.",
|
||||
"version": "5.9.0",
|
||||
"version": "5.9.1",
|
||||
"author": {
|
||||
"name": "Kjell Tore Guttormsen"
|
||||
},
|
||||
"homepage": "https://git.fromaitochitta.com/open/ktg-plugin-marketplace/src/branch/main/plugins/voyage",
|
||||
"repository": "https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git",
|
||||
"license": "MIT",
|
||||
"keywords": ["voyage", "trek", "planning", "implementation", "research", "context-engineering", "agents", "adversarial-review", "headless", "execution"]
|
||||
"keywords": [
|
||||
"voyage",
|
||||
"trek",
|
||||
"planning",
|
||||
"implementation",
|
||||
"research",
|
||||
"context-engineering",
|
||||
"agents",
|
||||
"adversarial-review",
|
||||
"headless",
|
||||
"execution"
|
||||
]
|
||||
}
|
||||
|
|
|
|||
13
CHANGELOG.md
13
CHANGELOG.md
|
|
@ -4,6 +4,19 @@ All notable changes to this project will be documented in this file.
|
|||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## v5.9.1 — 2026-07-03 — Fix /trekendsession load-time crash (eager-exec placeholders)
|
||||
|
||||
Patch, no functional additions.
|
||||
|
||||
### Fixed
|
||||
|
||||
- `/trekendsession` was unusable in every invocation: two of its three `` !`...` `` eager-exec blocks (Phase 3 atomic-write, Phase 4 validator call) contained unresolved runtime placeholders (`<project-dir>` etc.). The harness executes eager-exec blocks at command LOAD time, so zsh parsed `<project-dir>` as input redirection and the command aborted before the model saw a single instruction. Both blocks are now plain runtime Bash fences with the `{curly}` placeholder convention (shell-inert), matching `trekplan.md`/`trekresearch.md`. The Phase 1 discovery block (self-contained) keeps its legitimate eager-exec prefix; `trekcontinue.md`'s discovery block was runtime-verified unaffected.
|
||||
- Latent secondary bug in the same blocks: cwd-relative plugin paths (`lib/validators/...`, `./lib/util/atomic-write.mjs`) would have failed with `ERR_MODULE_NOT_FOUND` even after substitution, since the Bash cwd is the user's repo. Both now use absolute `${CLAUDE_PLUGIN_ROOT}` paths per the existing command convention (Node ESM accepts absolute-path import specifiers — verified on Node 18+).
|
||||
|
||||
### Added
|
||||
|
||||
- Regression guard `tests/commands/trekendsession.test.mjs`: scans every `` !` ``-block in `commands/*.md` for unresolved `<angle>`/`{curly}` placeholders (this bug class is silent until first invocation), plus structure tests pinning Phase 3/4 as runtime Bash with `${CLAUDE_PLUGIN_ROOT}` paths and exactly one surviving eager block. Suite baseline 828 → 832 (830 pass / 0 fail / 2 skip).
|
||||
|
||||
## v5.9.0 — 2026-07-02 — Fable model tier + deep-research engine
|
||||
|
||||
Additive, plus one behavior alignment: profile `phase_models` now reach sub-agent spawn sites (previously documented but never wired), and the seven command orchestrators no longer pin `model: opus` — frontmatter omits `model:`, so the orchestrator follows the session model.
|
||||
|
|
|
|||
67
README.md
67
README.md
|
|
@ -1,17 +1,17 @@
|
|||
# trekplan — Brief, Research, Plan, Execute, Review, Continue
|
||||
# voyage
|
||||
|
||||

|
||||
Contract-driven Claude Code pipeline: brief, research, plan, execute, review. Agent swarms, research triangulation, adversarial review, multi-session resumption.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
> **Solo-maintained, fork-and-own.** This plugin is a starting point, not a vendor product. Issues are welcome as signals; pull requests are not accepted. See [GOVERNANCE.md](GOVERNANCE.md) for the full model and what upstream provides.
|
||||
|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development. [Full disclosure →](../../README.md#ai-generated-code-disclosure)*
|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development. Every change is human-directed, reviewed, and validated before commit. Per Anthropic Consumer Terms §4, ownership of outputs is assigned to the user; this plugin is licensed MIT.*
|
||||
|
||||
A [Claude Code](https://docs.anthropic.com/en/docs/claude-code) plugin for deep implementation planning, multi-source research, autonomous execution, independent post-hoc review, and zero-friction multi-session resumption. Six commands, one pipeline:
|
||||
|
||||
> **What's new — v5.8.0: offline gold-scored output eval (SKAL-1·4b).** The review-coordinator self-eval gains a scoring run: `lib/review/gold-scorer.mjs` grades a committed agent-run fixture against the golden corpus at `(file, rule_key)` granularity (precision/recall/f1 + verdict match), and the suite census gains a third category (`goldEval`) so a scoring run is counted apart from behavior coverage and doc-pins. Offline + deterministic — committed reviewer payloads, no live agent spawn (the LLM-in-the-loop tier is the separate 4c). Internal eval infrastructure; no command/agent/Handover change. **v5.7.1:** leaner always-loaded agent listing — `<example>` blocks relocated to agent bodies (~3,180 tok/turn, no behavior change). **v5.7.0:** opt-in per-session token/cost metering (SKAL-2) + eval foundation (SKAL-1·4a). **v5.6.1:** one-line `description:` for the four reference/dormant agents (~700 tok). **v5.5.0:** brief **framing** enforcement (`brief_version 2.2`) + a `/trekreview` reviewer-schema contract. Additive — no breaking changes. **Full version history → [CHANGELOG.md](CHANGELOG.md).**
|
||||
|
||||
| Command | What it does |
|
||||
|---------|-------------|
|
||||
| **`/trekbrief`** | Brief — interactive interview produces a task brief with explicit research plan |
|
||||
|
|
@ -21,6 +21,23 @@ A [Claude Code](https://docs.anthropic.com/en/docs/claude-code) plugin for deep
|
|||
| **`/trekreview`** | Review — independent post-hoc review of delivered code against the brief, severity-tagged findings |
|
||||
| **`/trekcontinue`** | Continue — read `.session-state.local.json` and resume the next session in a multi-session project |
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
|
||||
claude plugin install voyage@ktg-plugin-marketplace
|
||||
```
|
||||
|
||||
Or enable directly in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"voyage@ktg-plugin-marketplace": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`/trekbrief`, `/trekplan`, and `/trekreview` each end by running `scripts/annotate.mjs` against the just-written artifact and printing the resulting `file://<abs path>` link. The operator opens the HTML in a browser, clicks any line of the document, writes their own note in the inline textarea, watches a sidebar of all notes (editable, deletable, persisted in browser `localStorage`), and clicks "Copy Prompt" to get one structured prompt that they paste back into Claude — Claude then revises the `.md` from the notes. **The operator drives every annotation.** See [Reviewing and annotating artifacts](#reviewing-and-annotating-artifacts-v502).
|
||||
|
||||
Every artifact lives in one project directory: `.claude/projects/{YYYY-MM-DD}-{slug}/` contains `brief.md`, `research/NN-*.md`, `plan.md`, `sessions/`, `progress.json`, and `review.md`.
|
||||
|
|
@ -68,9 +85,6 @@ Under the hood, `lib/util/autonomy-gate.mjs` runs a small state machine (`idle
|
|||
## Quick start
|
||||
|
||||
```bash
|
||||
# Install the marketplace, then browse and enable plugins with /plugin
|
||||
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
|
||||
|
||||
# Capture intent (interactive)
|
||||
/trekbrief Add user authentication with JWT tokens
|
||||
# → .claude/projects/2026-04-18-jwt-auth/brief.md
|
||||
|
|
@ -127,7 +141,10 @@ Concrete capabilities, observable in the code — not aspirations.
|
|||
|
||||
**Virksomhet / regulated environment.** Defense-in-depth security across four layers (plugin hooks, prompt-level denylist, pre-execution plan scan, scoped tool access). `disableSkillShellExecution: true` recommendation for fork-ers handling untrusted briefs. No cloud dependency, no GitHub requirement. Validators are plain-Node CLIs — invocable from CI, custom hooks, or external tools, not just from voyage commands.
|
||||
|
||||
**What it doesn't solve:**
|
||||
## Non-goals
|
||||
|
||||
What this pipeline does **not** solve — read this before adopting it:
|
||||
|
||||
- LLM output truthfulness. Validators check shape, not facts. A plan with hallucinated paths passes schema but fails in execute. Plan-critic catches some, not all.
|
||||
- Multi-user concurrency on a single project directory. Two simultaneous executors will clobber `progress.json`.
|
||||
- Cost management. Opus on the orchestrator layer is expensive; documented in [Cost profile](#cost-profile), no automatic model downgrade.
|
||||
|
|
@ -757,26 +774,6 @@ The `pre-compact-flush.mjs` hook directly fixes the documented P0 in `docs/treke
|
|||
|
||||
**Annotation HTML requires a desktop browser.** `scripts/annotate.mjs` produces a single self-contained `.html` file you open with `file://` in any modern browser (Chrome / Safari / Firefox / Edge — last two versions). No CDN, no server, no npm runtime deps. State persists in `localStorage` so closing and re-opening the tab keeps your work, but it's local to one browser on one machine — not synced anywhere. If you want to annotate without a browser, paste the `.md` into Claude with "comments inline below" and write notes in chat — same end result, just without the visual surface.
|
||||
|
||||
## Installation
|
||||
|
||||
Add the marketplace and browse plugins with `/plugin`:
|
||||
|
||||
```bash
|
||||
claude plugin marketplace add https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git
|
||||
```
|
||||
|
||||
Or enable directly in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"voyage@ktg-plugin-marketplace": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
An optional architect step between research and plan was previously available via a separate plugin; that architect plugin is no longer publicly distributed. The `architecture/overview.md` filesystem slot remains supported by `/trekplan` for any compatible producer.
|
||||
|
||||
## Profile system (v4.1.0)
|
||||
|
||||
Four built-in model profiles plus operator-defined `<custom>.yaml` (drop in `lib/profiles/`). Each profile pins `phase_models` for the six pipeline phases. The active profile is recorded in plan.md frontmatter as `profile: <name>` and emitted to JSONL stats for cost-attribution.
|
||||
|
|
@ -921,6 +918,18 @@ suppress this, leave the `architecture/` directory absent from your
|
|||
project directory. Discovery is additive — missing file is fine, no
|
||||
error.
|
||||
|
||||
## Changelog
|
||||
|
||||
Full version history → [CHANGELOG.md](CHANGELOG.md).
|
||||
|
||||
Recent, in one line each:
|
||||
|
||||
- **v5.8.0** — offline gold-scored output eval (SKAL-1·4b): `lib/review/gold-scorer.mjs` grades a committed agent-run fixture against the golden corpus at `(file, rule_key)` granularity; suite census gains a `goldEval` category. Offline + deterministic, no live agent spawn.
|
||||
- **v5.7.1** — leaner always-loaded agent listing (`<example>` blocks moved into agent bodies, ~3,180 tok/turn, no behavior change).
|
||||
- **v5.7.0** — opt-in per-session token/cost metering (SKAL-2) + eval foundation (SKAL-1·4a).
|
||||
- **v5.6.1** — one-line `description:` for the four reference/dormant agents (~700 tok).
|
||||
- **v5.5.0** — brief **framing** enforcement (`brief_version 2.2`) + a `/trekreview` reviewer-schema contract. Additive, no breaking changes.
|
||||
|
||||
## Contributing
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
|
|
|||
|
|
@ -7,12 +7,20 @@ tools: ["Read", "Glob", "Grep", "Write", "Edit", "Bash"]
|
|||
---
|
||||
|
||||
<!-- Phase mapping: orchestrator → command
|
||||
Corrected in v5.10: every row below was off by one, and the old last row
|
||||
pointed at a ninth command phase that does not exist — the command ends
|
||||
at Phase 8.
|
||||
Orchestrator Phase 1 = Command Phase 4 (Agent group selection)
|
||||
Orchestrator Phase 2 = Command Phase 5 (Parallel research)
|
||||
Orchestrator Phase 3 = Command Phase 6 (Targeted follow-ups)
|
||||
Orchestrator Phase 4 = Command Phase 7 (Triangulation)
|
||||
Orchestrator Phase 5 = Command Phase 8 (Synthesis + write brief)
|
||||
Orchestrator Phase 6 = Command Phase 9 (Completion)
|
||||
Orchestrator Phase 2 = Command Phase 4 (Parallel research — same
|
||||
command phase; the orchestrator
|
||||
splits selection from launch)
|
||||
(no orchestrator phase)= Command Phase 4.5 (Dimension discovery,
|
||||
high effort only — v5.10)
|
||||
Orchestrator Phase 3 = Command Phase 5 (Targeted follow-ups; bounded
|
||||
conversation loop at high effort)
|
||||
Orchestrator Phase 4 = Command Phase 6 (Triangulation)
|
||||
Orchestrator Phase 5 = Command Phase 7 (Synthesis + write brief)
|
||||
Orchestrator Phase 6 = Command Phase 8 (Present and track / completion)
|
||||
As of v2.4.0, /trekresearch runs these phases inline in main
|
||||
context instead of spawning this agent. Keep this file as the canonical
|
||||
reference for what those phases do. -->
|
||||
|
|
|
|||
|
|
@ -90,16 +90,16 @@ want an interactive flow, use `/trekcontinue --help` to see the full pipeline.
|
|||
|
||||
## Phase 3 — Atomically write `.session-state.local.json` + sibling NEXT-SESSION-PROMPT.local.md
|
||||
|
||||
Write `<project-dir>/.session-state.local.json` with the schema-v1 object:
|
||||
Write `{project_dir}/.session-state.local.json` with the schema-v1 object:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"project": "<project-dir>",
|
||||
"next_session_brief_path": "<arg 1>",
|
||||
"next_session_label": "<arg 2>",
|
||||
"project": "{project_dir}",
|
||||
"next_session_brief_path": "{arg 1}",
|
||||
"next_session_label": "{arg 2}",
|
||||
"status": "in_progress",
|
||||
"updated_at": "<now, ISO-8601>"
|
||||
"updated_at": "{now, ISO-8601}"
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -114,14 +114,22 @@ Under `node --input-type=module -e "<script>" arg1 arg2 arg3`, Node sets
|
|||
|
||||
This phase ALSO writes a sibling `NEXT-SESSION-PROMPT.local.md` in the
|
||||
project directory with YAML frontmatter (`produced_by: trekendsession`,
|
||||
`produced_at: <ISO-8601>`, `project: <project-dir>`). Both files are written
|
||||
in a single ESM block so the writes succeed or fail together:
|
||||
`produced_at: {ISO-8601}`, `project: {project_dir}`). Both files are written
|
||||
in a single ESM block so the writes succeed or fail together.
|
||||
|
||||
Run the block below via the Bash tool at runtime, substituting the resolved
|
||||
values for the `{curly}` placeholders (Phase 1 gives `{project_dir}`, Phase 2
|
||||
gives `{next_brief_path}` and `{next_label}`). This is NOT an eager-exec
|
||||
block — the values do not exist at command-load time. The import path must
|
||||
stay absolute via `${CLAUDE_PLUGIN_ROOT}` — your Bash cwd is the user's
|
||||
repo, not the plugin root, so a cwd-relative import throws
|
||||
`ERR_MODULE_NOT_FOUND`:
|
||||
|
||||
```bash
|
||||
!`node --input-type=module -e "
|
||||
node --input-type=module -e "
|
||||
import path from 'node:path';
|
||||
import { writeFileSync } from 'node:fs';
|
||||
import { atomicWriteJson } from './lib/util/atomic-write.mjs';
|
||||
import { atomicWriteJson } from '${CLAUDE_PLUGIN_ROOT}/lib/util/atomic-write.mjs';
|
||||
const [, dir, brief, label] = process.argv;
|
||||
const now = new Date().toISOString();
|
||||
const stateObj = { schema_version: 1, project: dir, next_session_brief_path: brief, next_session_label: label, status: 'in_progress', updated_at: now };
|
||||
|
|
@ -132,26 +140,28 @@ const promptBody = '---\\nproduced_by: trekendsession\\nproduced_at: ' + now + '
|
|||
writeFileSync(promptFile, promptBody);
|
||||
console.log(stateFile);
|
||||
console.log(promptFile);
|
||||
" '<project-dir>' '<next-brief-path>' '<next-label>'`
|
||||
" '{project_dir}' '{next_brief_path}' '{next_label}'
|
||||
```
|
||||
|
||||
## Phase 4 — Validate + narrate
|
||||
|
||||
Validate the freshly-written state file:
|
||||
Validate the freshly-written state file via the Bash tool at runtime,
|
||||
substituting the resolved `{project_dir}` (NOT eager-exec — the file does
|
||||
not exist at command-load time):
|
||||
|
||||
```bash
|
||||
!`node lib/validators/session-state-validator.mjs --json <project-dir>/.session-state.local.json`
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/session-state-validator.mjs --json {project_dir}/.session-state.local.json
|
||||
```
|
||||
|
||||
If `valid: true`, print the success block matching `/trekcontinue` Phase 3
|
||||
narration (SC-8 cross-project consistency — same template both sides):
|
||||
|
||||
```
|
||||
Session state written: <project-dir>/.session-state.local.json
|
||||
Session state written: {project_dir}/.session-state.local.json
|
||||
|
||||
Project: <project-dir>
|
||||
Next session: <next-label>
|
||||
Brief: <next-brief-path>
|
||||
Project: {project_dir}
|
||||
Next session: {next_label}
|
||||
Brief: {next_brief_path}
|
||||
|
||||
In a fresh Claude session, run /trekcontinue to resume.
|
||||
```
|
||||
|
|
|
|||
|
|
@ -391,15 +391,128 @@ other agents — the value of Gemini is independence.
|
|||
small = halved, medium/large = default
|
||||
- convention-scanner: medium+ codebases only (50+ files)
|
||||
|
||||
## Phase 4.5 — Dimension discovery
|
||||
|
||||
**Skip this phase entirely unless `phase_signal_result.effort == 'high'`.**
|
||||
|
||||
Phase 4 retrieves more than the interview knew to ask for. This phase mines
|
||||
that surplus: findings that were **retrieved but unintegrated** — material an
|
||||
agent surfaced that no interview dimension claims.
|
||||
|
||||
1. **Mine.** Walk the Phase-4 agent results and collect findings that map to
|
||||
no existing dimension.
|
||||
2. **Rerank.** Order candidates by relevance to the research question **and**
|
||||
dissimilarity to the dimensions already on the list. A candidate that
|
||||
restates an existing dimension is not a discovery.
|
||||
3. **Augment under the existing ceiling.** Append candidates to the dimension
|
||||
list only while the **whole** list (interview + discovered) stays at or
|
||||
below `maxDimensions: 8` (`settings.json:16`). The ceiling is **not**
|
||||
raised here, so the documented 3–8 dimension range stays true and the
|
||||
README prose about it stays untouched. If the interview already produced 8
|
||||
dimensions, this phase discovers nothing and says so.
|
||||
4. **Record the baseline.** Keep the interview-derived count as
|
||||
`dimensions_baseline` so the discovered delta is machine-readable against
|
||||
the final `dimensions` (Phase 8 stats).
|
||||
|
||||
Every outbound query generated from a discovered dimension passes
|
||||
`query-privacy-gate.mjs` before it leaves the machine — see the per-turn
|
||||
protocol in Phase 5. That gate is the compensating control for the
|
||||
Independence crossing this phase makes (see Hard rules → Independence).
|
||||
|
||||
## Phase 5 — Targeted follow-ups
|
||||
|
||||
Review all agent results. Identify knowledge gaps — areas where findings are
|
||||
thin, contradictory, or missing.
|
||||
Review all agent results. Identify knowledge gaps — dimensions where findings
|
||||
are thin, contradictory, or missing (**under-illuminated dimensions**).
|
||||
|
||||
For each significant gap, launch a targeted follow-up agent (model: "opus")
|
||||
with a narrow, specific brief. Maximum 2 follow-ups.
|
||||
**Standard and low effort — unchanged single pass.** For each significant gap,
|
||||
launch a targeted follow-up agent (model: "opus") with a narrow, specific
|
||||
brief. Maximum 2 follow-ups. If no gaps exist, skip: "Initial research
|
||||
sufficient — no follow-ups needed." Then go to Phase 6.
|
||||
|
||||
If no gaps exist, skip: "Initial research sufficient — no follow-ups needed."
|
||||
**The bounded loop below runs ONLY when `phase_signal_result.effort == 'high'`**
|
||||
(resolved in Phase 1; see `### High-effort behavior (v5.1.1)`). At any other
|
||||
effort this whole sub-section is inert — no loop, no cap ledger, no new
|
||||
counters beyond zero.
|
||||
|
||||
### Loop bound
|
||||
|
||||
**Maximum 3 turns per under-illuminated dimension.** The bound is per
|
||||
dimension, not per run: the worst case is 3 turns × the whole dimension list
|
||||
under the `maxDimensions: 8` ceiling (`settings.json:16`), which is what
|
||||
`research-loop-cap.mjs` sizes itself against. The cap counts itself from its
|
||||
own append-only ledger — it never asks this prose how many turns it has used.
|
||||
|
||||
The loop is **default-off**: `research-loop-cap.mjs` grants a budget of 0
|
||||
unless `VOYAGE_STORM_ENABLED=1`. Doing nothing leaves the mechanism off.
|
||||
|
||||
### Per-turn protocol
|
||||
|
||||
Each turn targets exactly one under-illuminated dimension, and runs two gates
|
||||
before it spends anything:
|
||||
|
||||
```bash
|
||||
# 1. Budget gate — per turn, per dimension. Exit 0 = granted, exit 1 = denied.
|
||||
# JSON on stdout: {ok, used, budget, reason?}
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/util/research-loop-cap.mjs \
|
||||
--run-id {run_id} --dimension {dimension} --effort {phase_signal_result.effort}
|
||||
|
||||
# 2. Privacy gate — EVERY outbound query, before it leaves the machine.
|
||||
# Exit 0 = send as-is; exit 1 = rewrite the query and re-gate. Never bypass.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/query-privacy-gate.mjs "{query text}"
|
||||
```
|
||||
|
||||
A denied budget gate is an exit condition, not a retry. A failed privacy gate
|
||||
is a rewrite: the hard-block tier (secret-shaped strings) is never
|
||||
operator-overridable, so a query that trips it must be reformulated, not
|
||||
forced through.
|
||||
|
||||
**Empty turns.** A turn that returns no findings, or findings without
|
||||
citations, is marked `empty`. An empty turn is counted in `empty_turns` and
|
||||
does NOT re-target the same dimension — re-asking the same question of the
|
||||
same silence is how a bounded loop turns into an unbounded one. Move to the
|
||||
next under-illuminated dimension, or exit.
|
||||
|
||||
### Exits (all three, always one of them)
|
||||
|
||||
1. **Converged** — the dimension carries findings with citations and no
|
||||
remaining contradiction. Stop turning on it. This is the normal exit.
|
||||
2. **Cap exhausted** — `research-loop-cap.mjs` denies the turn. Print the
|
||||
exhaustion **visibly** to the operator, never silently:
|
||||
`Loop bound reached for dimension {dimension} after {N} turns — remaining
|
||||
gaps are carried into the brief as open questions.` A silent cap is
|
||||
indistinguishable from convergence, and that confusion is exactly what this
|
||||
phase exists to prevent.
|
||||
3. **Operator stop** — the operator interrupts. Carry whatever has been
|
||||
gathered into Phase 6 and record the remaining gaps as open questions. Do
|
||||
not re-enter the loop after a stop.
|
||||
|
||||
### When the loop does not apply
|
||||
|
||||
**No-brief default.** Without `--project` (or with a project whose `brief.md`
|
||||
is absent), there are no `phase_signals` to resolve, so `effort = 'standard'`,
|
||||
the loop does not run, and all new counters (`conv_turns`, `empty_turns`) are
|
||||
emitted as `0`.
|
||||
|
||||
**Precedence matrix — each entry independently makes the loop moot**, the same
|
||||
way `--engine` is moot when the external phase does not run (see the moot gate
|
||||
in Phase 4):
|
||||
|
||||
| Condition | Effect on the loop |
|
||||
|-----------|--------------------|
|
||||
| `--quick` | Moot — Phase 3.5 skips to Phase 8; the swarm never runs |
|
||||
| `--local` | Moot — no outbound queries to bound |
|
||||
| `external_research_enabled: false` (profile) | Moot — the profile's on/off switch wins |
|
||||
|
||||
**Interaction rule.** A brief that carries `effort: high` **without** a
|
||||
`model`, under a cheap profile (`economy`/`balanced`): the effort signal
|
||||
governs orchestration shape, so the loop is armed, but the profile still
|
||||
supplies the model — and if that profile disables external research, the
|
||||
matrix above wins and the loop is moot regardless of effort.
|
||||
|
||||
**Honesty (hard rule, restated for this loop).** More turns do not make a
|
||||
finding more credible. Turn count is a cost, not evidence: report what the
|
||||
citations support, and let an exhausted cap show up as open questions rather
|
||||
than as confidence.
|
||||
|
||||
## Phase 6 — Triangulation
|
||||
|
||||
|
|
@ -535,6 +648,11 @@ Record format (one JSON line):
|
|||
"project_dir": "{project_dir or null}",
|
||||
"brief_path": "{brief_destination}",
|
||||
"dimensions": {N},
|
||||
"dimensions_baseline": {N},
|
||||
"effort": "{low|standard|high}",
|
||||
"conv_turns": {N},
|
||||
"empty_turns": {N},
|
||||
"unique_sources": {N},
|
||||
"agents_local": {N},
|
||||
"agents_external": {N},
|
||||
"gemini_used": {true|false},
|
||||
|
|
@ -544,6 +662,17 @@ Record format (one JSON line):
|
|||
}
|
||||
```
|
||||
|
||||
**The five measurement fields (v5.10).** `effort` is the grouping key — the
|
||||
resolved `phase_signal_result.effort` for the `research` phase, a
|
||||
low-cardinality label (`low|standard|high`), and the only axis on which a
|
||||
high-effort run can be compared against a standard one. The other four are
|
||||
numeric: `unique_sources` (distinct sources cited across the brief),
|
||||
`dimensions_baseline` (the interview-derived dimension count, so the Phase 4.5
|
||||
delta against `dimensions` is machine-readable), `conv_turns` (Phase 5 loop
|
||||
turns actually spent), and `empty_turns` (loop turns that returned no findings
|
||||
or no citations). On a standard run the loop never arms, so
|
||||
`dimensions_baseline == dimensions` and both turn counters are `0`.
|
||||
|
||||
If `${CLAUDE_PLUGIN_DATA}` is not set or not writable, skip tracking silently.
|
||||
|
||||
## Profile (v4.1)
|
||||
|
|
@ -623,6 +752,14 @@ significant architectural questions or when triangulation value is
|
|||
high; in high-effort mode it runs unconditionally to provide an
|
||||
independent second opinion.
|
||||
|
||||
High effort additionally arms the Phase 5 bounded follow-up loop (max 3
|
||||
turns per under-illuminated dimension, budgeted by
|
||||
`research-loop-cap.mjs`, every outbound query gated by
|
||||
`query-privacy-gate.mjs`). The loop stays default-off until
|
||||
`VOYAGE_STORM_ENABLED=1`, and the moot matrix in Phase 5 (`--quick`,
|
||||
`--local`, `external_research_enabled: false`) overrides the effort
|
||||
signal whenever the external phase does not run at all.
|
||||
|
||||
Standard effort (or absent): use the existing conditional triggers.
|
||||
Low effort: inline research only, no agent swarm (existing
|
||||
`--quick`-equivalent code-path).
|
||||
|
|
@ -634,6 +771,16 @@ Low effort: inline research only, no agent swarm (existing
|
|||
- **Sources required:** Every claim must cite a source. No unsourced findings.
|
||||
- **Independence:** Do not pre-bias external agents with local findings or vice versa.
|
||||
Triangulate AFTER independent research.
|
||||
**Amended (v5.10) for Phase 4.5:** dimension discovery deliberately crosses this
|
||||
rule. Its candidate dimensions are mined from the Phase-4 result set, which
|
||||
contains output from the five local codebase agents, so a discovered dimension
|
||||
can carry local context into an external query. The crossing is bounded to
|
||||
Phase 4.5 and the Phase 5 loop it feeds, it never applies to the initial
|
||||
external swarm (which stays blind to local findings), and the compensating
|
||||
control is `query-privacy-gate.mjs`: every outbound query is inspected before
|
||||
it leaves the machine, with a hard-block tier for secret-shaped strings that no
|
||||
operator flag can override. Triangulation still happens AFTER independent
|
||||
research.
|
||||
- **Graceful degradation:** If MCP tools are unavailable (Tavily, Gemini, MS Learn),
|
||||
proceed with available tools and note limitations in brief metadata.
|
||||
- **Cost:** Model resolution at Agent-spawn sites is a three-layer fallback:
|
||||
|
|
|
|||
109
docs/spike-pretooluse-subagent-reach.md
Normal file
109
docs/spike-pretooluse-subagent-reach.md
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
# Spike: does a plugin `PreToolUse` hook reach sub-agent tool calls?
|
||||
|
||||
**Date:** 2026-08-09
|
||||
**Claude Code version:** 2.1.226
|
||||
**Plugin:** voyage 5.9.1 (installed from `ktg-plugin-marketplace`)
|
||||
**Gates:** Step 10 of `plan.md` (`2026-06-30-trekresearch-storm-upgrade`)
|
||||
|
||||
## Question
|
||||
|
||||
Step 10 wants to enforce the conversation-turn cap in a `PreToolUse` hook. That
|
||||
is only viable if a **plugin** `PreToolUse` hook fires on tool calls made
|
||||
*inside a sub-agent*. If it does not, the cap must be enforced somewhere else
|
||||
and Step 10 becomes a documented downgrade instead.
|
||||
|
||||
The question is genuinely open, not answerable from docs alone: the current
|
||||
[hooks reference](https://code.claude.com/docs/en/hooks) states that sub-agent
|
||||
tool calls fire the same hooks and carry `agent_id` / `agent_type`, while
|
||||
[issue #34692](https://github.com/anthropics/claude-code/issues/34692) reported
|
||||
the exact opposite behaviour. The answer is therefore version-dependent and had
|
||||
to be measured on the version actually in use.
|
||||
|
||||
## Method
|
||||
|
||||
A one-shot probe hook was registered for the `WebSearch` matcher, and a
|
||||
**headless child session** was launched to exercise it. The child was used
|
||||
because hooks are resolved when a session starts — a matcher added mid-session
|
||||
cannot be observed by the session that added it.
|
||||
|
||||
Two deviations from the step as originally written, both forced and both
|
||||
verified not to affect the result:
|
||||
|
||||
1. **The matcher was injected into the installed plugin's `hooks.json`, not the
|
||||
repository's.** The plan assumed the repo working tree *is* the active plugin
|
||||
root. It is not: `~/.claude/plugins/cache/ktg-plugin-marketplace/voyage/5.9.1/`
|
||||
is a plain directory holding its own copy, and that copy is what loads.
|
||||
Editing `hooks/hooks.json` in the repo would have measured nothing. The cache
|
||||
file was backed up, modified, and restored — verified byte-identical to the
|
||||
repo file afterwards.
|
||||
2. **The probe script lives under the session scratchpad, not `${TMPDIR}`.** A
|
||||
pathguard hook refuses writes to `${TMPDIR}`. The load-bearing property was
|
||||
only that the script sit **outside `hooks/scripts/`**, which
|
||||
`tests/lib/doc-consistency.test.mjs:75-84` counts via `readdirSync`; the
|
||||
scratchpad satisfies that just as well. The directory still holds 7 scripts.
|
||||
|
||||
### Probe hook
|
||||
|
||||
Logged every invocation as one JSON line (`tool_name`, `agent_id`,
|
||||
`agent_type`, plus the untouched stdin) and always exited `0`, so it could not
|
||||
alter the child's behaviour.
|
||||
|
||||
### Commands
|
||||
|
||||
```bash
|
||||
# 1. inject the temporary matcher into the INSTALLED plugin
|
||||
node -e '...push {matcher:"WebSearch", ...voyage-spike-hook.mjs} into hooks.PreToolUse...'
|
||||
|
||||
# 2. exercise it from a fresh child session
|
||||
claude -p "Spawn exactly one sub-agent via the Agent tool (subagent_type: general-purpose).
|
||||
Instruct that sub-agent to perform exactly ONE WebSearch for the query
|
||||
'claude code hooks reference' and report back the first result title.
|
||||
You MUST NOT call WebSearch yourself in the main context - only the
|
||||
sub-agent may call it. When the sub-agent returns, reply with the word DONE." \
|
||||
--allowedTools "Agent,Task,WebSearch" \
|
||||
--max-turns 15
|
||||
|
||||
# 3. restore
|
||||
cp "${TMPDIR}voyage-hooks-backup.json" <cache>/hooks/hooks.json
|
||||
```
|
||||
|
||||
The child returned `DONE`.
|
||||
|
||||
> An earlier attempt additionally passed `--permission-mode bypassPermissions`
|
||||
> and was refused by the auto-mode classifier. The flag was dropped;
|
||||
> `--allowedTools` alone was sufficient.
|
||||
|
||||
## Raw observation
|
||||
|
||||
The log contains **exactly one** record — so the main context did not call
|
||||
`WebSearch` itself, and the single entry is unambiguously the sub-agent's call:
|
||||
|
||||
```json
|
||||
{"at":"2026-08-09T12:07:47.796Z","tool_name":"WebSearch",
|
||||
"agent_id":"aa6d19525a4680fe0","agent_type":"general-purpose",
|
||||
"raw_stdin":"{\"session_id\":\"b126fd6a-...\",\"cwd\":\"/Users/ktg/repos/ktg-plugin-marketplace/voyage\",
|
||||
\"permission_mode\":\"auto\",\"agent_id\":\"aa6d19525a4680fe0\",\"agent_type\":\"general-purpose\",
|
||||
\"effort\":{\"level\":\"xhigh\"},\"hook_event_name\":\"PreToolUse\",\"tool_name\":\"WebSearch\",
|
||||
\"tool_input\":{\"query\":\"claude code hooks reference\"},\"tool_use_id\":\"toolu_01Ka4...\"}"}
|
||||
```
|
||||
|
||||
Both `agent_id` and `agent_type` are populated, matching the documented
|
||||
common-input fields for sub-agent-originated tool events. A main-context call
|
||||
would have carried neither.
|
||||
|
||||
## Consequence for Step 10
|
||||
|
||||
A plugin `PreToolUse` hook **does** observe sub-agent tool calls on CC 2.1.226,
|
||||
and can attribute them via `agent_id` / `agent_type`. Step 10 may therefore take
|
||||
the enforcement branch rather than the documented-downgrade branch.
|
||||
|
||||
Two limits worth carrying forward, neither of which changes the verdict:
|
||||
|
||||
- This measures `WebSearch` on one CC version. The behaviour regressed once
|
||||
before (#34692), so the hook must fail **open**, never assume it is the only
|
||||
gate, and the cap must remain correct if the hook silently stops firing.
|
||||
- The probe only establishes *reach*. Whether a **blocking** (exit 2) decision
|
||||
from inside a sub-agent propagates usefully was not measured — the probe
|
||||
always exited 0 by design.
|
||||
|
||||
RESULT: FIRES
|
||||
|
|
@ -105,7 +105,11 @@ const BLOCK_RULES = [
|
|||
// --- Executor-specific additions ---
|
||||
{
|
||||
name: 'System shutdown/reboot',
|
||||
pattern: /\b(?:shutdown|reboot|halt|poweroff)\b/,
|
||||
// Anchored to command position — start of string/line, or after a
|
||||
// separator (`;`, `|`, `&&`, `||`), with optional `sudo` and an optional
|
||||
// absolute path. An unanchored \b match blocked the bare word anywhere,
|
||||
// including quoted grep patterns, heredoc data, and commit messages.
|
||||
pattern: /(?:^|[\n;|]|&&)\s*(?:sudo\s+(?:-[a-zA-Z]+\s+)*)?(?:[\w./-]*\/)?(?:shutdown|reboot|halt|poweroff)\b/,
|
||||
description: 'System shutdown/reboot commands are blocked during execution.',
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -25,11 +25,23 @@ const TREKBRIEF_ALLOWED = Object.freeze(new Set([
|
|||
]));
|
||||
|
||||
// Source: tests/fixtures/jsonl-schemas.md row 2 (trekresearch)
|
||||
// `engine` is a low-cardinality label (swarm|deep-research) emitted at
|
||||
// commands/trekresearch.md:533 and promised in prose (:570-572).
|
||||
// DENY BY OMISSION: question (free prose), project_dir + brief_path
|
||||
// (filesystem paths) are written into the jsonl but MUST NOT reach the
|
||||
// exporter.
|
||||
// The five v5.10 measurement fields are allowlisted too: `effort` is a
|
||||
// low-cardinality label (low|standard|high) and the grouping key the
|
||||
// measurement gate is computed on; `unique_sources`, `dimensions_baseline`,
|
||||
// `conv_turns` and `empty_turns` are plain counters. None of them carry prose
|
||||
// or paths.
|
||||
const TREKRESEARCH_ALLOWED = Object.freeze(new Set([
|
||||
'ts', 'slug', 'mode', 'scope', 'dimensions', 'agents_local',
|
||||
'ts', 'slug', 'mode', 'scope', 'engine', 'dimensions', 'agents_local',
|
||||
'agents_external', 'gemini_used', 'confidence', 'contradictions',
|
||||
'open_questions', 'profile', 'parallel_agents',
|
||||
'external_research_enabled', 'profile_source',
|
||||
'effort', 'unique_sources', 'dimensions_baseline', 'conv_turns',
|
||||
'empty_turns',
|
||||
]));
|
||||
|
||||
// Source: tests/fixtures/jsonl-schemas.md row 3 (trekplan)
|
||||
|
|
|
|||
144
lib/util/research-loop-cap.mjs
Normal file
144
lib/util/research-loop-cap.mjs
Normal file
|
|
@ -0,0 +1,144 @@
|
|||
// lib/util/research-loop-cap.mjs
|
||||
// Stateful, default-off cost cap for the /trekresearch bounded conversation
|
||||
// loop (Phase 4.5 dimension discovery + Phase 5 loop turns).
|
||||
//
|
||||
// Three properties the plan review required:
|
||||
// (a) Default-off — VOYAGE_STORM_ENABLED must be '1'; otherwise the budget
|
||||
// is 0 regardless of effort. This IS the decline branch: doing nothing
|
||||
// leaves the mechanism off, and adopt is flipping this one constant.
|
||||
// (b) The cap counts itself — allowTurn() derives used-turn count from an
|
||||
// append-only JSONL ledger, never from a caller-supplied number. A cap
|
||||
// that asks the caller how many turns it has used is not a cap.
|
||||
// (c) Correct size bound — worst case is max_conv_turns × max_total_dimensions,
|
||||
// where max_total_dimensions is the WHOLE list (interview + discovered)
|
||||
// under settings.json:16's cap of 8 — not × discovered-only.
|
||||
//
|
||||
// CLAUDE_PLUGIN_DATA absent => DENY (fail-closed). This is the opposite of
|
||||
// lib/stats/event-emit.mjs's fail-open: that module is telemetry (must never
|
||||
// block workflow); this module is a budget control (must never silently
|
||||
// grant unlimited turns just because the data dir is missing).
|
||||
//
|
||||
// CLI shim:
|
||||
// node lib/util/research-loop-cap.mjs --run-id ID --dimension D --effort E
|
||||
// → JSON: { ok, used, budget, reason? } (exit 0 = granted, exit 1 = denied)
|
||||
|
||||
import { existsSync, mkdirSync, appendFileSync, readFileSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
|
||||
export const MAX_CONV_TURNS = 3;
|
||||
export const MAX_TOTAL_DIMENSIONS = 8; // settings.json:16 maxDimensions — whole list, not discovered-only
|
||||
|
||||
const LEDGER_FILENAME = 'trekresearch-loop-ledger.jsonl';
|
||||
|
||||
export function isStormEnabled(env = process.env) {
|
||||
return env.VOYAGE_STORM_ENABLED === '1';
|
||||
}
|
||||
|
||||
/**
|
||||
* Coerce TREKRESEARCH_MAX_CONV_TURNS. NaN, empty, negative, or zero all fall
|
||||
* back to MAX_CONV_TURNS — never to unbounded.
|
||||
*/
|
||||
export function resolveMaxConvTurns(env = process.env) {
|
||||
const raw = env.TREKRESEARCH_MAX_CONV_TURNS;
|
||||
if (raw === undefined || raw === null || raw === '') return MAX_CONV_TURNS;
|
||||
const n = Number(raw);
|
||||
if (!Number.isFinite(n) || n <= 0) return MAX_CONV_TURNS;
|
||||
return Math.floor(n);
|
||||
}
|
||||
|
||||
export function resolveLedgerPath(env = process.env) {
|
||||
const dir = env.CLAUDE_PLUGIN_DATA;
|
||||
if (!dir || typeof dir !== 'string' || dir.length === 0) return null;
|
||||
return join(dir, LEDGER_FILENAME);
|
||||
}
|
||||
|
||||
function countTurns(ledgerPath, runId) {
|
||||
if (!existsSync(ledgerPath)) return 0;
|
||||
let text;
|
||||
try { text = readFileSync(ledgerPath, 'utf-8'); }
|
||||
catch { return 0; }
|
||||
let count = 0;
|
||||
for (const line of text.split('\n')) {
|
||||
if (!line) continue;
|
||||
try {
|
||||
const rec = JSON.parse(line);
|
||||
if (rec.runId === runId) count++;
|
||||
} catch { /* skip malformed lines */ }
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether one more research-loop turn may run. Append-only: never
|
||||
* read-modify-write, because Phase 4.5/5 may spawn multiple agents in a
|
||||
* single message and a read-modify-write counter would lose concurrent
|
||||
* grants.
|
||||
*
|
||||
* @param {{runId: string, dimension: string, effort: string}} args
|
||||
* @param {{env?: object, now?: Date}} [opts]
|
||||
* @returns {{ok: boolean, used: number, budget: number, reason?: string}}
|
||||
*/
|
||||
export function allowTurn({ runId, dimension, effort } = {}, opts = {}) {
|
||||
const env = opts.env || process.env;
|
||||
const now = opts.now || new Date();
|
||||
|
||||
if (!isStormEnabled(env)) {
|
||||
return { ok: false, used: 0, budget: 0, reason: 'storm_disabled' };
|
||||
}
|
||||
if (effort !== 'high') {
|
||||
return { ok: false, used: 0, budget: 0, reason: 'effort_not_high' };
|
||||
}
|
||||
if (!runId || !dimension) {
|
||||
return { ok: false, used: 0, budget: 0, reason: 'missing_args' };
|
||||
}
|
||||
|
||||
const maxConvTurns = resolveMaxConvTurns(env);
|
||||
const budget = maxConvTurns * MAX_TOTAL_DIMENSIONS;
|
||||
|
||||
const ledgerPath = resolveLedgerPath(env);
|
||||
if (!ledgerPath) {
|
||||
return { ok: false, used: 0, budget, reason: 'no_plugin_data_dir' };
|
||||
}
|
||||
|
||||
const used = countTurns(ledgerPath, runId);
|
||||
if (used >= budget) {
|
||||
return { ok: false, used, budget, reason: 'budget_exhausted' };
|
||||
}
|
||||
|
||||
try {
|
||||
const dir = dirname(ledgerPath);
|
||||
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
|
||||
appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, dimension, effort }) + '\n');
|
||||
} catch (e) {
|
||||
return { ok: false, used, budget, reason: `ledger-write-failed: ${e.message}` };
|
||||
}
|
||||
|
||||
return { ok: true, used: used + 1, budget };
|
||||
}
|
||||
|
||||
// ---- CLI shim ----------------------------------------------------------------
|
||||
|
||||
function parseArgs(argv) {
|
||||
const out = {};
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
if (a === '--run-id') out.runId = argv[++i];
|
||||
else if (a === '--dimension') out.dimension = argv[++i];
|
||||
else if (a === '--effort') out.effort = argv[++i];
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
const args = parseArgs(process.argv.slice(2));
|
||||
if (!args.runId || !args.dimension || !args.effort) {
|
||||
process.stdout.write(JSON.stringify({
|
||||
ok: false,
|
||||
reason: 'usage: research-loop-cap.mjs --run-id ID --dimension D --effort standard|high|low',
|
||||
}) + '\n');
|
||||
process.exit(1);
|
||||
}
|
||||
const result = allowTurn(args);
|
||||
process.stdout.write(JSON.stringify(result) + '\n');
|
||||
process.exit(result.ok ? 0 : 1);
|
||||
}
|
||||
110
lib/validators/query-privacy-gate.mjs
Normal file
110
lib/validators/query-privacy-gate.mjs
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
// lib/validators/query-privacy-gate.mjs
|
||||
// Inspect an outbound research query before it leaves the machine. Called
|
||||
// only from the new high-effort steps (Phase 4.5 dimension discovery + the
|
||||
// bounded Phase 5 loop turns) — the existing single-pass Phase 5 path is
|
||||
// unchanged (Step 6, plan-v2).
|
||||
//
|
||||
// Two-tier, same shape as lib/exporters/endpoint-validator.mjs's SSRF gate:
|
||||
// - WARN tier — absolute filesystem paths, repo-internal identifiers.
|
||||
// Operator-overridable via `strict: false` / `--soft` (matches
|
||||
// lib/validators/research-validator.mjs's strict/soft convention), and
|
||||
// fully bypassable via the VOYAGE_QUERY_PRIVACY_ALLOW=1 opt-in.
|
||||
// - HARD-BLOCK tier — secret-shaped tokens. NEVER overridable by strict,
|
||||
// --soft, or the opt-in env var — mirrors endpoint-validator.mjs's
|
||||
// HARD_BLOCKED_HOSTS, where an opt-in widens the warn tier but never
|
||||
// unlocks the permanently-blocked one.
|
||||
//
|
||||
// CLI shim:
|
||||
// node lib/validators/query-privacy-gate.mjs [--soft] "<query text>"
|
||||
// → JSON {valid, errors, warnings}; exit 0 valid, 1 invalid.
|
||||
|
||||
import { issue } from '../util/result.mjs';
|
||||
|
||||
// WARN tier — absolute filesystem paths (leaks local directory layout).
|
||||
export const ABSOLUTE_PATH_PATTERNS = Object.freeze([
|
||||
/\/Users\/[^\s"'`]+/,
|
||||
/\/home\/[^\s"'`]+/,
|
||||
/[A-Za-z]:\\[^\s"'`]+/,
|
||||
/\$\{?HOME\}?\/[^\s"'`]+/,
|
||||
]);
|
||||
|
||||
// WARN tier — repo-internal identifiers that don't need to leave the
|
||||
// machine in a generic research query.
|
||||
export const REPO_IDENTIFIER_PATTERNS = Object.freeze([
|
||||
/git\.fromaitochitta\.com[^\s"'`]*/,
|
||||
/\bktg-plugin-marketplace\b/,
|
||||
/\bplugins\/cache\/[^\s"'`]+/,
|
||||
]);
|
||||
|
||||
// HARD-BLOCK tier — secret-shaped strings. Never operator-overridable.
|
||||
export const SECRET_SHAPED_PATTERNS = Object.freeze([
|
||||
/\bsk-[A-Za-z0-9]{20,}\b/, // OpenAI/Anthropic-style API keys
|
||||
/\bAKIA[0-9A-Z]{16}\b/, // AWS access key ID
|
||||
/\bghp_[A-Za-z0-9]{36,}\b/, // GitHub personal access token
|
||||
/\bxox[baprs]-[A-Za-z0-9-]{10,}\b/, // Slack token
|
||||
/-----BEGIN [A-Z ]*PRIVATE KEY-----/, // PEM private key block
|
||||
]);
|
||||
|
||||
function findMatch(patterns, text) {
|
||||
for (const re of patterns) {
|
||||
const m = re.exec(text);
|
||||
if (m) return m[0];
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string} text
|
||||
* @param {{strict?: boolean, env?: object}} [opts]
|
||||
* @returns {{valid: boolean, errors: import('../util/result.mjs').Issue[], warnings: import('../util/result.mjs').Issue[]}}
|
||||
*/
|
||||
export function validateOutboundQuery(text, opts = {}) {
|
||||
const strict = opts.strict !== false;
|
||||
const env = opts.env || process.env;
|
||||
// Bypasses the WARN tier entirely — never affects the hard-block tier below.
|
||||
const allowWarnTier = env.VOYAGE_QUERY_PRIVACY_ALLOW === '1';
|
||||
|
||||
if (typeof text !== 'string' || text.length === 0) {
|
||||
return { valid: false, errors: [issue('PRIVACY_EMPTY_QUERY', 'Outbound query must be a non-empty string')], warnings: [] };
|
||||
}
|
||||
|
||||
const errors = [];
|
||||
const warnings = [];
|
||||
|
||||
// Hard-block tier — checked unconditionally; no opt-in reaches this branch.
|
||||
const secretMatch = findMatch(SECRET_SHAPED_PATTERNS, text);
|
||||
if (secretMatch) {
|
||||
errors.push(issue('PRIVACY_SECRET_SHAPED', `Outbound query contains a secret-shaped token: ${secretMatch}`));
|
||||
}
|
||||
|
||||
if (!allowWarnTier) {
|
||||
const pathMatch = findMatch(ABSOLUTE_PATH_PATTERNS, text);
|
||||
if (pathMatch) {
|
||||
const issueObj = issue('PRIVACY_ABSOLUTE_PATH', `Outbound query contains an absolute filesystem path: ${pathMatch}`);
|
||||
if (strict) errors.push(issueObj); else warnings.push(issueObj);
|
||||
}
|
||||
|
||||
const repoMatch = findMatch(REPO_IDENTIFIER_PATTERNS, text);
|
||||
if (repoMatch) {
|
||||
const issueObj = issue('PRIVACY_REPO_IDENTIFIER', `Outbound query contains a repo-internal identifier: ${repoMatch}`);
|
||||
if (strict) errors.push(issueObj); else warnings.push(issueObj);
|
||||
}
|
||||
}
|
||||
|
||||
return { valid: errors.length === 0, errors, warnings };
|
||||
}
|
||||
|
||||
// ---- CLI shim ----------------------------------------------------------------
|
||||
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
const args = process.argv.slice(2);
|
||||
const strict = !args.includes('--soft');
|
||||
const text = args.find(a => !a.startsWith('--'));
|
||||
if (text === undefined) {
|
||||
process.stderr.write('Usage: query-privacy-gate.mjs [--soft] "<query text>"\n');
|
||||
process.exit(2);
|
||||
}
|
||||
const r = validateOutboundQuery(text, { strict });
|
||||
process.stdout.write(JSON.stringify(r) + '\n');
|
||||
process.exit(r.valid ? 0 : 1);
|
||||
}
|
||||
4
package-lock.json
generated
4
package-lock.json
generated
|
|
@ -1,12 +1,12 @@
|
|||
{
|
||||
"name": "voyage",
|
||||
"version": "5.9.0",
|
||||
"version": "5.9.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "voyage",
|
||||
"version": "5.9.0",
|
||||
"version": "5.9.1",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "voyage",
|
||||
"version": "5.9.0",
|
||||
"version": "5.9.1",
|
||||
"description": "Voyage — brief, research, plan, execute, review, continue. Contract-driven Claude Code pipeline. /trekbrief, /trekplan, and /trekreview each end by building a self-contained operator-annotation HTML (scripts/annotate.mjs, modelled on claude-code-100x): select text or click any heading/paragraph/list-item, pick intent (Fiks/Endre/Spørsmål), write comment, copy structured prompt, paste back, Claude revises the .md.",
|
||||
"type": "module",
|
||||
"engines": {
|
||||
|
|
|
|||
127
tests/commands/trekendsession.test.mjs
Normal file
127
tests/commands/trekendsession.test.mjs
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
// tests/commands/trekendsession.test.mjs
|
||||
// Regression tests for /trekendsession (commands/trekendsession.md).
|
||||
//
|
||||
// Bug (2026-07-03): two of the three !`...` eager-exec blocks contained
|
||||
// unresolved placeholders (<project-dir> etc.). The harness executes
|
||||
// eager-exec blocks at command LOAD time, so zsh parsed <project-dir> as
|
||||
// input redirection and the command aborted before the model saw a single
|
||||
// instruction. Eager-exec is only valid for self-contained commands.
|
||||
//
|
||||
// Pattern D (markdown structure) — assertions against command prose.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFileSync, readdirSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = join(HERE, '..', '..');
|
||||
const COMMANDS_DIR = join(ROOT, 'commands');
|
||||
const COMMAND_FILE = join(COMMANDS_DIR, 'trekendsession.md');
|
||||
|
||||
function readCommand() {
|
||||
return readFileSync(COMMAND_FILE, 'utf8');
|
||||
}
|
||||
|
||||
function extractPhase(commandText, phaseHeader) {
|
||||
const startIdx = commandText.indexOf(phaseHeader);
|
||||
if (startIdx === -1) return '';
|
||||
const rest = commandText.slice(startIdx);
|
||||
const nextPhase = rest.search(/\n## (?:Phase |Hard )/);
|
||||
if (nextPhase === -1) return rest;
|
||||
return rest.slice(0, nextPhase);
|
||||
}
|
||||
|
||||
// Extract all eager-exec blocks (!`...`) from a command/skill file,
|
||||
// including multi-line blocks. Returns [{ content, line }].
|
||||
function extractEagerBlocks(text) {
|
||||
const blocks = [];
|
||||
const re = /!`([^`]+)`/g;
|
||||
let m;
|
||||
while ((m = re.exec(text)) !== null) {
|
||||
const line = text.slice(0, m.index).split('\n').length;
|
||||
blocks.push({ content: m[1], line });
|
||||
}
|
||||
return blocks;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------
|
||||
// Marketplace-wide regression guard: eager-exec blocks must be
|
||||
// self-contained. An unresolved placeholder (<angle> or {curly}) in an
|
||||
// eager block is executed verbatim by the shell at load time — <x> is
|
||||
// parsed as input redirection and aborts the whole command load.
|
||||
// ---------------------------------------------------------------
|
||||
|
||||
test('eager-exec guard — no !`-block in commands/ contains an unresolved placeholder', () => {
|
||||
const offenders = [];
|
||||
for (const file of readdirSync(COMMANDS_DIR).filter((f) => f.endsWith('.md'))) {
|
||||
const text = readFileSync(join(COMMANDS_DIR, file), 'utf8');
|
||||
for (const { content, line } of extractEagerBlocks(text)) {
|
||||
// Placeholder conventions: <angle-word> or {curly_word}. Curly must
|
||||
// contain a separator (- or _) so JS destructuring like {join} in a
|
||||
// legitimate self-contained script does not false-positive; angle
|
||||
// placeholders are unambiguous (shell would parse them as redirects).
|
||||
if (/<[a-z][a-z0-9_-]*>/.test(content) || /\{[a-z][a-z0-9]*([_-][a-z0-9]+)+\}/.test(content)) {
|
||||
offenders.push(`${file}:${line}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
assert.deepEqual(
|
||||
offenders,
|
||||
[],
|
||||
`eager-exec !\`-blocks run at command LOAD time and must be self-contained; ` +
|
||||
`placeholder found in: ${offenders.join(', ')}`,
|
||||
);
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------
|
||||
// trekendsession-specific: exactly one eager block (Phase 1 project
|
||||
// discovery — self-contained, legitimate); Phases 3 and 4 are runtime
|
||||
// Bash-tool commands with model-substituted values, never eager.
|
||||
// ---------------------------------------------------------------
|
||||
|
||||
test('trekendsession — exactly one eager-exec block remains (Phase 1 discovery)', () => {
|
||||
const cmd = readCommand();
|
||||
const blocks = extractEagerBlocks(cmd);
|
||||
assert.equal(
|
||||
blocks.length,
|
||||
1,
|
||||
`expected exactly 1 eager-exec block (Phase 1 discovery), got ${blocks.length} at line(s) ${blocks.map((b) => b.line).join(', ')}`,
|
||||
);
|
||||
assert.match(
|
||||
blocks[0].content,
|
||||
/readdirSync\(root\)/,
|
||||
'the surviving eager block must be the self-contained Phase 1 discovery script',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekendsession Phase 3 — atomic-write block is runtime Bash (no eager prefix) with plugin-root import', () => {
|
||||
const phase3 = extractPhase(readCommand(), '## Phase 3 ');
|
||||
assert.doesNotMatch(phase3, /!`/, 'Phase 3 must not use eager-exec — values exist only at runtime');
|
||||
assert.match(
|
||||
phase3,
|
||||
/\$\{CLAUDE_PLUGIN_ROOT\}\/lib\/util\/atomic-write\.mjs/,
|
||||
'Phase 3 import must use the absolute ${CLAUDE_PLUGIN_ROOT} path — cwd is the user repo, not the plugin root',
|
||||
);
|
||||
assert.doesNotMatch(
|
||||
phase3,
|
||||
/['"]\.\/lib\/util\/atomic-write\.mjs['"]/,
|
||||
'Phase 3 must not import atomic-write.mjs via a cwd-relative path',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekendsession Phase 4 — validator call is runtime Bash (no eager prefix) with plugin-root path', () => {
|
||||
const phase4 = extractPhase(readCommand(), '## Phase 4 ');
|
||||
assert.doesNotMatch(phase4, /!`/, 'Phase 4 must not use eager-exec — the state-file path exists only at runtime');
|
||||
assert.match(
|
||||
phase4,
|
||||
/\$\{CLAUDE_PLUGIN_ROOT\}\/lib\/validators\/session-state-validator\.mjs/,
|
||||
'Phase 4 validator path must use the absolute ${CLAUDE_PLUGIN_ROOT} convention',
|
||||
);
|
||||
assert.doesNotMatch(
|
||||
phase4,
|
||||
/<[a-z][a-z0-9_-]*>/,
|
||||
'Phase 4 must not use <angle> placeholders in commands — zsh parses <x> as input redirection',
|
||||
);
|
||||
});
|
||||
|
|
@ -34,10 +34,206 @@ test('trekresearch — sequencing-gate surface mentions BRIEF_V51_MISSING_SIGNAL
|
|||
|
||||
test('trekresearch — low-effort path references --quick equivalent', () => {
|
||||
const text = read();
|
||||
const compIdx = text.indexOf('## Composition rule (v5.1)');
|
||||
assert.ok(compIdx >= 0, 'Composition rule (v5.1) section missing');
|
||||
const section = text.slice(compIdx, compIdx + 2000);
|
||||
assert.match(section, /--quick/, 'Low-effort path must mention --quick equivalent');
|
||||
// Bound the Composition rule section by the next `###` heading rather than a
|
||||
// magic 2000-character window: a fixed count silently drops the match as soon
|
||||
// as prose is inserted above it, turning a real pin into a no-op.
|
||||
const sectionOf = (doc) => {
|
||||
const compIdx = doc.indexOf('## Composition rule (v5.1)');
|
||||
assert.ok(compIdx >= 0, 'Composition rule (v5.1) section missing');
|
||||
const nextHeading = doc.indexOf('\n### ', compIdx);
|
||||
return nextHeading > compIdx ? doc.slice(compIdx, nextHeading) : doc.slice(compIdx);
|
||||
};
|
||||
|
||||
// (a) positive: the low-effort path is documented inside the bounded section.
|
||||
assert.match(sectionOf(text), /--quick/, 'Low-effort path must mention --quick equivalent');
|
||||
|
||||
// (b) negative: an actual removal must still be caught — a bound that can
|
||||
// never fail proves nothing.
|
||||
const mutated = text.replace(/--quick/g, '--removed');
|
||||
assert.doesNotMatch(sectionOf(mutated), /--quick/,
|
||||
'heading-bounded slice must still fail on a genuine removal');
|
||||
});
|
||||
|
||||
// --- Step 7: Phase 5 bounded conversation loop (heading-bounded slices) ---
|
||||
|
||||
// Same bounding discipline as the Composition-rule pin above: slice from the
|
||||
// phase heading to the NEXT phase heading, never a fixed character window.
|
||||
function phaseSlice(doc, startHeading, endHeading) {
|
||||
const start = doc.indexOf(startHeading);
|
||||
assert.ok(start >= 0, `${startHeading} missing`);
|
||||
const end = doc.indexOf(endHeading, start);
|
||||
assert.ok(end > start, `${endHeading} missing — could not bound ${startHeading}`);
|
||||
return doc.slice(start, end);
|
||||
}
|
||||
|
||||
function phase5(doc) {
|
||||
return phaseSlice(doc, '## Phase 5 —', '## Phase 6 —');
|
||||
}
|
||||
|
||||
test('trekresearch — Phase 5 loop is gated on effort == high and names both primitives', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /effort == 'high'/, 'Phase 5 loop must be gated on effort == \'high\'');
|
||||
assert.match(p5, /research-loop-cap\.mjs/, 'Phase 5 must call the loop-cap shim per turn');
|
||||
assert.match(p5, /query-privacy-gate\.mjs/, 'Phase 5 must route outbound queries through the privacy gate');
|
||||
assert.match(p5, /\$\{CLAUDE_PLUGIN_ROOT\}/, 'shim invocations must use the ${CLAUDE_PLUGIN_ROOT} path form');
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 declares the loop bound and all three exits', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /### Loop bound/, 'Phase 5 must carry a `### Loop bound` sub-heading');
|
||||
assert.match(
|
||||
p5,
|
||||
/\*\*Maximum 3 turns per under-illuminated dimension\.\*\*/,
|
||||
'the bound must be stated verbatim',
|
||||
);
|
||||
// Three exits — converged / cap exhausted / operator stop.
|
||||
assert.match(p5, /converged/i, 'exit 1 (converged) must be documented');
|
||||
assert.match(p5, /exhaust/i, 'exit 2 (cap exhausted) must be documented');
|
||||
assert.match(p5, /operator stop/i, 'exit 3 (operator stop) must be documented');
|
||||
// Exhaustion must reach the operator — a silent cap is indistinguishable
|
||||
// from convergence, which is the failure this loop exists to avoid.
|
||||
assert.match(
|
||||
p5,
|
||||
/visibl|visible|print/i,
|
||||
'cap exhaustion must be written visibly to the operator',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 marks empty turns without re-targeting the same dimension', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /`empty`/, 'a finding-less or citation-less turn must be marked `empty`');
|
||||
assert.match(p5, /empty_turns/, 'empty turns must be counted (empty_turns)');
|
||||
assert.match(
|
||||
p5,
|
||||
/does NOT re-target|not re-target/i,
|
||||
'an empty turn must not re-target the same dimension',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 states the no-brief default and the moot precedence matrix', () => {
|
||||
const p5 = phase5(read());
|
||||
// (a) no-brief default
|
||||
assert.match(p5, /effort = 'standard'/, 'no-brief default effort must be stated');
|
||||
assert.match(
|
||||
p5,
|
||||
/--project/,
|
||||
'the no-brief default must be anchored to the absence of --project/brief.md',
|
||||
);
|
||||
// (b) precedence matrix — each entry independently makes the loop moot,
|
||||
// mirroring the --engine moot gate in Phase 4.
|
||||
for (const token of ['--quick', '--local', 'external_research_enabled']) {
|
||||
assert.ok(p5.includes(token), `moot matrix must name ${token}`);
|
||||
}
|
||||
assert.match(p5, /moot/i, 'the matrix must use the same moot vocabulary as the engine gate');
|
||||
// (c) interaction rule — effort: high without model under a cheap profile.
|
||||
assert.match(
|
||||
p5,
|
||||
/effort: high/,
|
||||
'the interaction rule for a brief carrying effort: high without model must be stated',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 restates the honesty rule for loop output', () => {
|
||||
const p5 = phase5(read());
|
||||
// Whitespace-tolerant: the pin is on the sentence, not on where the
|
||||
// paragraph happens to wrap.
|
||||
assert.match(
|
||||
p5,
|
||||
/more\s+turns\s+do\s+not\s+make\s+a\s+finding\s+more\s+credible/i,
|
||||
'the honesty hard rule must be restated for the loop output',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 pins survive only while the prose does (mutation control)', () => {
|
||||
const text = read();
|
||||
const mutated = text.replace(/research-loop-cap\.mjs/g, 'removed-cap.mjs');
|
||||
assert.doesNotMatch(
|
||||
phase5(mutated),
|
||||
/research-loop-cap\.mjs/,
|
||||
'heading-bounded Phase 5 slice must still fail on a genuine removal',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — High-effort behavior keeps the standard/low effort sentences verbatim', () => {
|
||||
const text = read();
|
||||
assert.ok(
|
||||
text.includes('Standard effort (or absent): use the existing conditional triggers.'),
|
||||
'the standard-effort sentence must survive the Phase 5 rewrite verbatim',
|
||||
);
|
||||
assert.ok(
|
||||
text.includes('Low effort: inline research only, no agent swarm'),
|
||||
'the low-effort sentence must survive the Phase 5 rewrite verbatim',
|
||||
);
|
||||
});
|
||||
|
||||
// --- Step 8: Phase 4.5 dimension discovery + Independence amendment ---
|
||||
|
||||
const ORCHESTRATOR_FILE = join(ROOT, 'agents', 'research-orchestrator.md');
|
||||
function readOrchestrator() { return readFileSync(ORCHESTRATOR_FILE, 'utf8'); }
|
||||
|
||||
test('trekresearch — Phase 4.5 exists between Phase 4 and Phase 5 with the effort skip-guard', () => {
|
||||
const text = read();
|
||||
const p45 = text.indexOf('## Phase 4.5 —');
|
||||
assert.ok(p45 >= 0, 'Phase 4.5 heading missing');
|
||||
const p4 = text.indexOf('## Phase 4 —');
|
||||
const p5 = text.indexOf('## Phase 5 —');
|
||||
assert.ok(p4 >= 0 && p5 > p45 && p45 > p4, 'Phase 4.5 must sit between Phase 4 and Phase 5');
|
||||
|
||||
const slice = text.slice(p45, p5);
|
||||
assert.match(
|
||||
slice,
|
||||
/\*\*Skip this phase entirely unless `phase_signal_result\.effort == 'high'`\.\*\*/,
|
||||
'Phase 4.5 must carry the bolded skip-guard in the Phase 3.5 form',
|
||||
);
|
||||
assert.match(slice, /query-privacy-gate\.mjs/,
|
||||
'Phase 4.5 must name the privacy gate as its compensating control');
|
||||
assert.match(slice, /maxDimensions: 8|maxDimensions` *: *8/,
|
||||
'Phase 4.5 must augment under the existing maxDimensions ceiling, not raise it');
|
||||
});
|
||||
|
||||
test('trekresearch — Independence hard rule carries an explicit Phase 4.5 amendment', () => {
|
||||
const text = read();
|
||||
const rulesIdx = text.indexOf('## Hard rules');
|
||||
assert.ok(rulesIdx >= 0, 'Hard rules section missing');
|
||||
const rules = text.slice(rulesIdx);
|
||||
const indIdx = rules.indexOf('**Independence:**');
|
||||
assert.ok(indIdx >= 0, 'Independence hard rule missing');
|
||||
// Bound the rule at the next bullet so the amendment must live inside it.
|
||||
const nextBullet = rules.indexOf('\n- **', indIdx);
|
||||
const independence = nextBullet > indIdx ? rules.slice(indIdx, nextBullet) : rules.slice(indIdx);
|
||||
assert.match(independence, /Amend(ed|ment)/i,
|
||||
'Independence must be explicitly amended, not silently contradicted');
|
||||
assert.match(independence, /Phase 4\.5/, 'the amendment must name Phase 4.5 as the crossing');
|
||||
assert.match(independence, /query-privacy-gate\.mjs/,
|
||||
'the amendment must name the compensating control');
|
||||
});
|
||||
|
||||
test('trekresearch — orchestrator phase map is correct, has no Phase 9, and carries Phase 4.5', () => {
|
||||
const doc = readOrchestrator();
|
||||
const start = doc.indexOf('<!-- Phase mapping');
|
||||
assert.ok(start >= 0, 'phase mapping comment missing');
|
||||
const end = doc.indexOf('-->', start);
|
||||
assert.ok(end > start, 'phase mapping comment not terminated');
|
||||
const map = doc.slice(start, end);
|
||||
|
||||
assert.doesNotMatch(map, /Command Phase 9/,
|
||||
'the command ends at Phase 8 — a Command Phase 9 row is a fiction');
|
||||
|
||||
// Six orchestrator rows, each pointing at the phase the command actually has.
|
||||
const expected = [
|
||||
[1, '4'],
|
||||
[2, '4'],
|
||||
[3, '5'],
|
||||
[4, '6'],
|
||||
[5, '7'],
|
||||
[6, '8'],
|
||||
];
|
||||
for (const [orch, cmd] of expected) {
|
||||
const re = new RegExp(`Orchestrator Phase ${orch}\\s+= Command Phase ${cmd.replace('.', '\\.')}\\b`);
|
||||
assert.match(map, re, `map row for Orchestrator Phase ${orch} must point at Command Phase ${cmd}`);
|
||||
}
|
||||
|
||||
assert.match(map, /Command Phase 4\.5/, 'the map must carry the new Phase 4.5 row');
|
||||
});
|
||||
|
||||
// --- v5.1.1 runtime SC4 + SC7 ---
|
||||
|
|
|
|||
22
tests/fixtures/expected.prom
vendored
22
tests/fixtures/expected.prom
vendored
|
|
@ -33,19 +33,31 @@ voyage_trekplan_deep_dives{_schema_id="trekplan",slug="add-auth",mode="default",
|
|||
voyage_trekplan_research_briefs_used{_schema_id="trekplan",slug="add-auth",mode="default",profile="premium",profile_source="flag"} 3
|
||||
# HELP voyage_trekresearch_agents_external voyage stats — trekresearch_agents_external
|
||||
# TYPE voyage_trekresearch_agents_external gauge
|
||||
voyage_trekresearch_agents_external{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 3
|
||||
voyage_trekresearch_agents_external{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 3
|
||||
# HELP voyage_trekresearch_agents_local voyage stats — trekresearch_agents_local
|
||||
# TYPE voyage_trekresearch_agents_local gauge
|
||||
voyage_trekresearch_agents_local{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 5
|
||||
voyage_trekresearch_agents_local{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 5
|
||||
# HELP voyage_trekresearch_contradictions voyage stats — trekresearch_contradictions
|
||||
# TYPE voyage_trekresearch_contradictions gauge
|
||||
voyage_trekresearch_contradictions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 1
|
||||
voyage_trekresearch_contradictions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 1
|
||||
# HELP voyage_trekresearch_conv_turns voyage stats — trekresearch_conv_turns
|
||||
# TYPE voyage_trekresearch_conv_turns gauge
|
||||
voyage_trekresearch_conv_turns{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 5
|
||||
# HELP voyage_trekresearch_dimensions voyage stats — trekresearch_dimensions
|
||||
# TYPE voyage_trekresearch_dimensions gauge
|
||||
voyage_trekresearch_dimensions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 4
|
||||
voyage_trekresearch_dimensions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 4
|
||||
# HELP voyage_trekresearch_dimensions_baseline voyage stats — trekresearch_dimensions_baseline
|
||||
# TYPE voyage_trekresearch_dimensions_baseline gauge
|
||||
voyage_trekresearch_dimensions_baseline{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 3
|
||||
# HELP voyage_trekresearch_empty_turns voyage stats — trekresearch_empty_turns
|
||||
# TYPE voyage_trekresearch_empty_turns gauge
|
||||
voyage_trekresearch_empty_turns{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 1
|
||||
# HELP voyage_trekresearch_open_questions voyage stats — trekresearch_open_questions
|
||||
# TYPE voyage_trekresearch_open_questions gauge
|
||||
voyage_trekresearch_open_questions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",profile="premium",profile_source="default"} 2
|
||||
voyage_trekresearch_open_questions{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 2
|
||||
# HELP voyage_trekresearch_unique_sources voyage stats — trekresearch_unique_sources
|
||||
# TYPE voyage_trekresearch_unique_sources gauge
|
||||
voyage_trekresearch_unique_sources{_schema_id="trekresearch",slug="add-auth",mode="default",scope="both",effort="high",profile="premium",profile_source="default"} 17
|
||||
# HELP voyage_trekreview_duration_ms voyage stats — trekreview_duration_ms
|
||||
# TYPE voyage_trekreview_duration_ms histogram
|
||||
voyage_trekreview_duration_ms{_schema_id="trekreview",slug="add-auth",verdict="ALLOW",mode="default",profile="balanced",profile_source="flag"} 4521
|
||||
|
|
|
|||
2
tests/fixtures/jsonl-schemas.md
vendored
2
tests/fixtures/jsonl-schemas.md
vendored
|
|
@ -20,7 +20,7 @@
|
|||
| schema_id | fields | writer_path | line_ref | v4.1 additive | PII |
|
||||
|-----------|--------|-------------|----------|---------------|-----|
|
||||
| trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, project_dir | commands/trekbrief.md (orchestrator-emit Phase 7) | trekbrief.md:657-672 | profile, phase_models, profile_source | none |
|
||||
| trekresearch-stats | ts, question, mode, scope, slug, project_dir, brief_path, dimensions, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | commands/trekresearch.md (orchestrator-emit Stats tracking) | trekresearch.md:388-410 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
|
||||
| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | commands/trekresearch.md (orchestrator-emit Stats tracking) | trekresearch.md:634-676 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
|
||||
| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | commands/trekplan.md (orchestrator-emit Phase 12) | trekplan.md:805-826 | profile, phase_models, parallel_agents, profile_source | none |
|
||||
| trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | commands/trekexecute.md (orchestrator-emit Phase 9) | trekexecute.md:1479-1494 | profile, phase_models, profile_source | none |
|
||||
| trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:64-86 | payload.profile, payload.phase_models, payload.profile_source | none |
|
||||
|
|
|
|||
2
tests/fixtures/stats-sample.jsonl
vendored
2
tests/fixtures/stats-sample.jsonl
vendored
|
|
@ -2,4 +2,4 @@
|
|||
{"_schema_id":"trekexecute","ts":"2026-05-09T08:30:00.000Z","plan":"trekplan-add-auth.md","plan_type":"plan","mode":"execute","result":"completed","steps_total":12,"steps_passed":12,"steps_failed":0,"steps_skipped":0,"profile":"premium","profile_source":"inheritance"}
|
||||
{"_schema_id":"trekreview","ts":"2026-05-09T09:00:00.000Z","slug":"add-auth","verdict":"ALLOW","reviewed_files_count":18,"mode":"default","duration_ms":4521,"profile":"balanced","profile_source":"flag"}
|
||||
{"_schema_id":"trekbrief","ts":"2026-05-09T07:00:00.000Z","slug":"add-auth","mode":"default","interview_turns":7,"review_iterations":2,"research_topics":3,"profile":"economy","profile_source":"env"}
|
||||
{"_schema_id":"trekresearch","ts":"2026-05-09T07:30:00.000Z","slug":"add-auth","mode":"default","scope":"both","dimensions":4,"agents_local":5,"agents_external":3,"contradictions":1,"open_questions":2,"profile":"premium","profile_source":"default"}
|
||||
{"_schema_id":"trekresearch","ts":"2026-05-09T07:30:00.000Z","slug":"add-auth","mode":"default","scope":"both","dimensions":4,"dimensions_baseline":3,"effort":"high","conv_turns":5,"empty_turns":1,"unique_sources":17,"agents_local":5,"agents_external":3,"contradictions":1,"open_questions":2,"profile":"premium","profile_source":"default"}
|
||||
|
|
|
|||
|
|
@ -129,6 +129,40 @@ test('pre-bash-executor BLOCKS system shutdown command', async () => {
|
|||
assert.strictEqual(code, 2);
|
||||
});
|
||||
|
||||
test('pre-bash-executor BLOCKS a privileged halt at command position', async () => {
|
||||
const { code } = await runHook(PRE_BASH, bashInput('sudo shutdown -h now'));
|
||||
assert.strictEqual(code, 2);
|
||||
});
|
||||
|
||||
test('pre-bash-executor BLOCKS a destructive keyword after a separator', async () => {
|
||||
const { code } = await runHook(PRE_BASH, bashInput('echo done && poweroff'));
|
||||
assert.strictEqual(code, 2);
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// ALLOW — the same keywords as DATA, not at command position.
|
||||
// The rule matched the bare word anywhere in the string, so a quoted grep
|
||||
// pattern, ordinary prose, or a commit message that merely named the rule
|
||||
// was blocked. Anchoring to command position is what separates the two.
|
||||
// -----------------------------------------------------------------------
|
||||
test('pre-bash-executor ALLOWS the keyword inside a quoted grep pattern', async () => {
|
||||
const { code } = await runHook(PRE_BASH, bashInput("grep 'halt' f.mjs"));
|
||||
assert.strictEqual(code, 0);
|
||||
});
|
||||
|
||||
test('pre-bash-executor ALLOWS the keyword inside echoed prose', async () => {
|
||||
const { code } = await runHook(PRE_BASH, bashInput('echo "we should halt here"'));
|
||||
assert.strictEqual(code, 0);
|
||||
});
|
||||
|
||||
test('pre-bash-executor ALLOWS a commit message that names the rule', async () => {
|
||||
const { code } = await runHook(
|
||||
PRE_BASH,
|
||||
bashInput('git commit -m "fix(hooks): anchor shutdown rule to command position"'),
|
||||
);
|
||||
assert.strictEqual(code, 0);
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// BLOCK — cron persistence
|
||||
// -----------------------------------------------------------------------
|
||||
|
|
|
|||
|
|
@ -31,6 +31,25 @@ test('SC #12: stats-sample.jsonl → expected.prom snapshot byte-for-byte match'
|
|||
` node scripts/gen-expected-prom.mjs > tests/fixtures/expected.prom`);
|
||||
});
|
||||
|
||||
test('Step 9: the four numeric STORM fields are metric families and effort is a label', () => {
|
||||
const expected = readFileSync(join(FIXTURES, 'expected.prom'), 'utf-8');
|
||||
for (const field of ['unique_sources', 'dimensions_baseline', 'conv_turns', 'empty_turns']) {
|
||||
assert.match(
|
||||
expected,
|
||||
new RegExp(`^# TYPE voyage_trekresearch_${field} `, 'm'),
|
||||
`${field} must appear as its own metric family — a numeric that never becomes a metric cannot be measured`,
|
||||
);
|
||||
}
|
||||
// effort is a low-cardinality string: it must ride along as a LABEL, never
|
||||
// as a metric family (a label is what makes high-vs-standard groupable).
|
||||
assert.match(expected, /effort="[a-z]+"/, 'effort must be emitted as a label');
|
||||
assert.doesNotMatch(
|
||||
expected,
|
||||
/^# TYPE voyage_trekresearch_effort /m,
|
||||
'effort must not become a metric family',
|
||||
);
|
||||
});
|
||||
|
||||
test('empty-input handling: [] returns empty string (no headers)', () => {
|
||||
assert.equal(transformToPrometheus([]), '');
|
||||
assert.equal(transformToPrometheus(null), '');
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ import {
|
|||
POST_BASH_STATS_ALLOWED,
|
||||
EVENT_EMIT_PAYLOAD_ALLOWED,
|
||||
TOKEN_USAGE_ALLOWED,
|
||||
TREKRESEARCH_ALLOWED,
|
||||
} from '../../lib/exporters/field-allowlist.mjs';
|
||||
|
||||
// ---- path-validator: CWE-22 mitigation -------------------------------------
|
||||
|
|
@ -278,6 +279,43 @@ test('field-allowlist: token-usage INCLUDES numeric/label fields, EXCLUDES sessi
|
|||
assert.equal('cwd' in out, false, 'cwd MUST be stripped (CWE-212)');
|
||||
});
|
||||
|
||||
// ---- trekresearch allowlist: the `engine` field ----------------------------
|
||||
|
||||
test('field-allowlist: trekresearch INCLUDES engine, EXCLUDES question/project_dir/brief_path (two-sided)', () => {
|
||||
const record = {
|
||||
ts: '2026-08-09T12:00:00.000Z',
|
||||
question: 'which retrieval strategy survives contradiction?',
|
||||
mode: 'default',
|
||||
scope: 'both',
|
||||
engine: 'deep-research',
|
||||
slug: 'storm-upgrade',
|
||||
project_dir: '/Users/ktg/secret/project',
|
||||
brief_path: '/Users/ktg/secret/project/brief.md',
|
||||
dimensions: 4,
|
||||
agents_local: 7,
|
||||
agents_external: 4,
|
||||
gemini_used: false,
|
||||
confidence: 0.82,
|
||||
contradictions: 1,
|
||||
open_questions: 3,
|
||||
};
|
||||
const out = applyFieldAllowlist(record, 'trekresearch');
|
||||
// INCLUDED — low-cardinality label, emitted (trekresearch.md:533) and
|
||||
// promised in prose (:570-572); it was silently dropped before this pin.
|
||||
assert.equal('engine' in out, true, 'engine MUST be allowlisted — it is emitted and documented');
|
||||
assert.equal(out.engine, 'deep-research');
|
||||
assert.equal(out._schema_id, 'trekresearch');
|
||||
// EXCLUDED (CWE-212 boundary)
|
||||
assert.equal('question' in out, false, 'question MUST be stripped (prose, CWE-212)');
|
||||
assert.equal('project_dir' in out, false, 'project_dir MUST be stripped (path, CWE-212)');
|
||||
assert.equal('brief_path' in out, false, 'brief_path MUST be stripped (path, CWE-212)');
|
||||
});
|
||||
|
||||
test('field-allowlist: TREKRESEARCH_ALLOWED is frozen (drift-pin)', () => {
|
||||
assert.equal(Object.isFrozen(TREKRESEARCH_ALLOWED), true,
|
||||
'TREKRESEARCH_ALLOWED must be frozen — runtime mutation prevention');
|
||||
});
|
||||
|
||||
test('field-allowlist: null/undefined record handled safely', () => {
|
||||
assert.deepEqual(applyFieldAllowlist(null, 'trekplan'), {});
|
||||
assert.deepEqual(applyFieldAllowlist(undefined, 'trekplan'), {});
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ import { fileURLToPath } from 'node:url';
|
|||
import { parseDocument } from '../../lib/util/frontmatter.mjs';
|
||||
import { resolveProfile, loadProfile } from '../../lib/profiles/resolver.mjs';
|
||||
import { STATES } from '../../lib/util/autonomy-gate.mjs';
|
||||
import { TREKRESEARCH_ALLOWED } from '../../lib/exporters/field-allowlist.mjs';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = join(HERE, '..', '..');
|
||||
|
|
@ -961,6 +962,31 @@ test('S15: profile tables encode each built-in yaml phase_models exactly', () =>
|
|||
}
|
||||
});
|
||||
|
||||
// STRUCTURAL pin: the exporter allowlist and the authoring fixture must agree.
|
||||
// `engine` was emitted, documented in prose, and still dropped at the export
|
||||
// boundary because nothing tied the two together. Derives one side from the
|
||||
// frozen Set, so it survives rewording of the fixture row.
|
||||
test('S74: every TREKRESEARCH_ALLOWED name is declared in the jsonl-schemas fixture row', () => {
|
||||
const row = read('tests/fixtures/jsonl-schemas.md')
|
||||
.split('\n')
|
||||
.find((l) => l.startsWith('| trekresearch-stats '));
|
||||
assert.ok(row, 'jsonl-schemas.md is missing the `trekresearch-stats` row');
|
||||
// Columns: '' | schema_id | fields | writer_path | line_ref | v4.1 additive | PII | ''
|
||||
const cells = row.split('|').map((c) => c.trim());
|
||||
// Both columns are required: profile/profile_source/parallel_agents live in
|
||||
// the `v4.1 additive` column only, so checking `fields` alone fails at once.
|
||||
const declared = new Set(
|
||||
[cells[2], cells[5]].flatMap((c) => c.split(',').map((f) => f.trim())).filter(Boolean),
|
||||
);
|
||||
for (const name of TREKRESEARCH_ALLOWED) {
|
||||
assert.ok(
|
||||
declared.has(name),
|
||||
`\`${name}\` is allowlisted in field-allowlist.mjs but absent from the fixture row's `
|
||||
+ '`fields` + `v4.1 additive` columns — fix the SOURCE, not this pin',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
// --- S34 (V30) — economy is self-declared experimental until the cross-tier
|
||||
// Jaccard floor (0.55) is empirically calibrated (Step-17 calibration deferred
|
||||
// to v4.2). The status must be visible in BOTH the profile data
|
||||
|
|
|
|||
|
|
@ -94,6 +94,81 @@ test('SC #11(b): commands/trekplan.md prose mentions phase_models + parallel_age
|
|||
'trekplan.md prose must mention parallel_agents (additive stats field)');
|
||||
});
|
||||
|
||||
// --- Step 9: the five STORM measurement fields -----------------------------
|
||||
// Four numerics + one low-cardinality label. Without `effort` there is no axis
|
||||
// to group high-vs-standard runs on, and the measurement gate cannot be
|
||||
// computed at all.
|
||||
|
||||
const STORM_FIELDS = [
|
||||
'effort',
|
||||
'unique_sources',
|
||||
'dimensions_baseline',
|
||||
'conv_turns',
|
||||
'empty_turns',
|
||||
];
|
||||
|
||||
test('Step 9: a standard-run trekresearch record parses and survives applyFieldAllowlist', async () => {
|
||||
const { applyFieldAllowlist } = await import('../../lib/exporters/field-allowlist.mjs');
|
||||
// A standard run: loop never armed, so no discovered dimensions and no turns.
|
||||
const raw = JSON.parse(JSON.stringify({
|
||||
_schema_id: 'trekresearch',
|
||||
ts: '2026-08-09T12:00:00.000Z',
|
||||
slug: 'add-auth',
|
||||
mode: 'default',
|
||||
scope: 'both',
|
||||
engine: 'swarm',
|
||||
question: 'free prose that must never reach the exporter',
|
||||
project_dir: '/Users/somebody/repos/x',
|
||||
brief_path: '/Users/somebody/repos/x/brief.md',
|
||||
dimensions: 4,
|
||||
dimensions_baseline: 4,
|
||||
conv_turns: 0,
|
||||
empty_turns: 0,
|
||||
unique_sources: 11,
|
||||
effort: 'standard',
|
||||
agents_local: 5,
|
||||
agents_external: 4,
|
||||
gemini_used: false,
|
||||
confidence: 0.8,
|
||||
contradictions: 1,
|
||||
open_questions: 2,
|
||||
profile: 'premium',
|
||||
profile_source: 'default',
|
||||
}));
|
||||
|
||||
assert.equal(raw.dimensions_baseline, raw.dimensions,
|
||||
'a standard run discovers no dimensions — baseline must equal the final count');
|
||||
|
||||
const out = applyFieldAllowlist(raw, 'trekresearch');
|
||||
for (const field of STORM_FIELDS) {
|
||||
assert.ok(field in out, `${field} must survive the trekresearch allowlist`);
|
||||
}
|
||||
assert.equal(out.conv_turns, 0);
|
||||
assert.equal(out.empty_turns, 0);
|
||||
assert.equal(out.effort, 'standard');
|
||||
// Deny-by-omission must still hold for the PII-ish fields.
|
||||
for (const denied of ['question', 'project_dir', 'brief_path']) {
|
||||
assert.equal(denied in out, false, `${denied} must NOT reach the exporter`);
|
||||
}
|
||||
});
|
||||
|
||||
test('Step 9: commands/trekresearch.md prose names all five measurement fields', () => {
|
||||
const content = readFileSync(join(REPO_ROOT, 'commands', 'trekresearch.md'), 'utf-8');
|
||||
for (const field of STORM_FIELDS) {
|
||||
assert.ok(content.includes(field),
|
||||
`trekresearch.md prose must name ${field} — an emitted-but-undocumented field is unauditable`);
|
||||
}
|
||||
});
|
||||
|
||||
test('Step 9: tests/fixtures/jsonl-schemas.md trekresearch row lists the five fields', () => {
|
||||
const doc = readFileSync(join(REPO_ROOT, 'tests', 'fixtures', 'jsonl-schemas.md'), 'utf-8');
|
||||
const row = doc.split('\n').find(l => l.startsWith('| trekresearch-stats '));
|
||||
assert.ok(row, 'jsonl-schemas.md is missing the trekresearch-stats row');
|
||||
for (const field of STORM_FIELDS) {
|
||||
assert.ok(row.includes(field), `authoring reference must list ${field}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('SC #11(b): commands/trekresearch.md prose mentions external_research_enabled', () => {
|
||||
const content = readFileSync(join(REPO_ROOT, 'commands', 'trekresearch.md'), 'utf-8');
|
||||
assert.match(content, /external_research_enabled/,
|
||||
|
|
|
|||
233
tests/lib/research-loop-cap.test.mjs
Normal file
233
tests/lib/research-loop-cap.test.mjs
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
// tests/lib/research-loop-cap.test.mjs
|
||||
// Cover lib/util/research-loop-cap.mjs: default-off, worst-case arithmetic,
|
||||
// anti-dead-data (different caps → different denial points), statefulness
|
||||
// (identical args → different answers once the budget is hit), env
|
||||
// coercion, fail-closed on missing CLAUDE_PLUGIN_DATA, and the CLI shim.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdtempSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
allowTurn,
|
||||
isStormEnabled,
|
||||
resolveMaxConvTurns,
|
||||
resolveLedgerPath,
|
||||
MAX_CONV_TURNS,
|
||||
MAX_TOTAL_DIMENSIONS,
|
||||
} from '../../lib/util/research-loop-cap.mjs';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const SHIM = join(HERE, '..', '..', 'lib', 'util', 'research-loop-cap.mjs');
|
||||
|
||||
function withTmpDataDir(fn) {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'research-loop-cap-'));
|
||||
try {
|
||||
return fn(dir);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function runShim(args, env) {
|
||||
try {
|
||||
const out = execFileSync(process.execPath, [SHIM, ...args], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
env: { ...process.env, ...env },
|
||||
});
|
||||
return { code: 0, out };
|
||||
} catch (e) {
|
||||
return { code: e.status ?? 1, out: e.stdout?.toString() ?? '' };
|
||||
}
|
||||
}
|
||||
|
||||
// ---- (a) default-off --------------------------------------------------------
|
||||
|
||||
test('allowTurn — VOYAGE_STORM_ENABLED unset denies with budget 0, regardless of effort', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir };
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'storm_disabled');
|
||||
assert.equal(r.budget, 0);
|
||||
});
|
||||
});
|
||||
|
||||
test('allowTurn — VOYAGE_STORM_ENABLED=0 denies same as unset', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '0' };
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'storm_disabled');
|
||||
});
|
||||
});
|
||||
|
||||
test('allowTurn — enabled but effort !== high denies with budget 0', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'standard' }, { env });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'effort_not_high');
|
||||
assert.equal(r.budget, 0);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (f) CLAUDE_PLUGIN_DATA unset => fail-closed deny -----------------------
|
||||
|
||||
test('allowTurn — CLAUDE_PLUGIN_DATA unset denies even when enabled + high effort', () => {
|
||||
const env = { VOYAGE_STORM_ENABLED: '1' }; // no CLAUDE_PLUGIN_DATA
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'no_plugin_data_dir');
|
||||
assert.equal(r.budget, MAX_CONV_TURNS * MAX_TOTAL_DIMENSIONS);
|
||||
});
|
||||
|
||||
// ---- (c) worst-case arithmetic ----------------------------------------------
|
||||
|
||||
test('allowTurn — budget is max_conv_turns × max_total_dimensions (default 3×8=24)', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
|
||||
const r = allowTurn({ runId: 'r1', dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r.ok, true);
|
||||
assert.equal(r.budget, 24);
|
||||
assert.equal(r.used, 1);
|
||||
});
|
||||
});
|
||||
|
||||
test('allowTurn — grants exactly `budget` turns then denies the next one (default 24)', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
|
||||
let last;
|
||||
for (let i = 0; i < 24; i++) {
|
||||
last = allowTurn({ runId: 'r-exhaust', dimension: `d${i % 8}`, effort: 'high' }, { env });
|
||||
assert.equal(last.ok, true, `turn ${i + 1} should be granted`);
|
||||
}
|
||||
const denied = allowTurn({ runId: 'r-exhaust', dimension: 'd0', effort: 'high' }, { env });
|
||||
assert.equal(denied.ok, false);
|
||||
assert.equal(denied.reason, 'budget_exhausted');
|
||||
assert.equal(denied.used, 24);
|
||||
assert.equal(denied.budget, 24);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (c)/(anti-dead-data) — different caps → observably different denial points
|
||||
|
||||
test('allowTurn — TREKRESEARCH_MAX_CONV_TURNS=1 denies after 8 turns (1×8), not 24', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
|
||||
let last;
|
||||
for (let i = 0; i < 8; i++) {
|
||||
last = allowTurn({ runId: 'r-narrow', dimension: `d${i}`, effort: 'high' }, { env });
|
||||
assert.equal(last.ok, true, `turn ${i + 1} should be granted`);
|
||||
}
|
||||
const denied = allowTurn({ runId: 'r-narrow', dimension: 'd8', effort: 'high' }, { env });
|
||||
assert.equal(denied.ok, false);
|
||||
assert.equal(denied.budget, 8);
|
||||
assert.notEqual(denied.budget, 24, 'a narrower cap must produce a different denial point than the default');
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (d) stateful — identical args give different answers once exhausted ---
|
||||
|
||||
test('allowTurn — identical {runId, dimension, effort} args diverge once the budget is hit (proves statefulness)', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
|
||||
const args = { runId: 'r-identical', dimension: 'same-dim', effort: 'high' };
|
||||
const results = [];
|
||||
for (let i = 0; i < 9; i++) results.push(allowTurn(args, { env }));
|
||||
// First 8 (budget = 1*8) granted, 9th denied — same exact input object each time.
|
||||
assert.deepEqual(results.slice(0, 8).map(r => r.ok), Array(8).fill(true));
|
||||
assert.equal(results[8].ok, false);
|
||||
assert.equal(results[8].reason, 'budget_exhausted');
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (e) env coercion --------------------------------------------------------
|
||||
|
||||
test('resolveMaxConvTurns — NaN string falls back to default', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: 'abc' }), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — empty string falls back to default', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '' }), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — negative value falls back to default', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '-5' }), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — zero falls back to default (never unbounded)', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '0' }), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — unset falls back to default', () => {
|
||||
assert.equal(resolveMaxConvTurns({}), MAX_CONV_TURNS);
|
||||
});
|
||||
|
||||
test('resolveMaxConvTurns — valid positive integer string is honored', () => {
|
||||
assert.equal(resolveMaxConvTurns({ TREKRESEARCH_MAX_CONV_TURNS: '2' }), 2);
|
||||
});
|
||||
|
||||
// ---- pure-core unit coverage --------------------------------------------------
|
||||
|
||||
test('isStormEnabled — only the literal string "1" enables', () => {
|
||||
assert.equal(isStormEnabled({ VOYAGE_STORM_ENABLED: '1' }), true);
|
||||
assert.equal(isStormEnabled({ VOYAGE_STORM_ENABLED: 'true' }), false);
|
||||
assert.equal(isStormEnabled({}), false);
|
||||
});
|
||||
|
||||
test('resolveLedgerPath — null when CLAUDE_PLUGIN_DATA unset or empty', () => {
|
||||
assert.equal(resolveLedgerPath({}), null);
|
||||
assert.equal(resolveLedgerPath({ CLAUDE_PLUGIN_DATA: '' }), null);
|
||||
});
|
||||
|
||||
test('resolveLedgerPath — joins CLAUDE_PLUGIN_DATA with the ledger filename', () => {
|
||||
const p = resolveLedgerPath({ CLAUDE_PLUGIN_DATA: '/tmp/plugin-data' });
|
||||
assert.equal(p, join('/tmp/plugin-data', 'trekresearch-loop-ledger.jsonl'));
|
||||
});
|
||||
|
||||
test('allowTurn — missing runId or dimension denies with missing_args', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' };
|
||||
const r1 = allowTurn({ dimension: 'd1', effort: 'high' }, { env });
|
||||
assert.equal(r1.ok, false);
|
||||
assert.equal(r1.reason, 'missing_args');
|
||||
const r2 = allowTurn({ runId: 'r1', effort: 'high' }, { env });
|
||||
assert.equal(r2.ok, false);
|
||||
assert.equal(r2.reason, 'missing_args');
|
||||
});
|
||||
});
|
||||
|
||||
// ---- (g) shim contract --------------------------------------------------------
|
||||
|
||||
test('CLI shim — grants and exits 0 when enabled + high effort + budget available', () => {
|
||||
withTmpDataDir((dir) => {
|
||||
const r = runShim(
|
||||
['--run-id', 'shim-1', '--dimension', 'd1', '--effort', 'high'],
|
||||
{ CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1' },
|
||||
);
|
||||
assert.equal(r.code, 0);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.ok, true);
|
||||
});
|
||||
});
|
||||
|
||||
test('CLI shim — denies and exits 1 when disabled', () => {
|
||||
const r = runShim(['--run-id', 'shim-2', '--dimension', 'd1', '--effort', 'high'], { VOYAGE_STORM_ENABLED: '0' });
|
||||
assert.equal(r.code, 1);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.ok, false);
|
||||
assert.equal(parsed.reason, 'storm_disabled');
|
||||
});
|
||||
|
||||
test('CLI shim — missing required args exits 1 with usage reason', () => {
|
||||
const r = runShim(['--run-id', 'shim-3']);
|
||||
assert.equal(r.code, 1);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.ok, false);
|
||||
assert.match(parsed.reason, /usage:/);
|
||||
});
|
||||
143
tests/validators/query-privacy-gate.test.mjs
Normal file
143
tests/validators/query-privacy-gate.test.mjs
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
// tests/validators/query-privacy-gate.test.mjs
|
||||
// Cover lib/validators/query-privacy-gate.mjs: two-sided code table
|
||||
// (absolute path / repo-internal identifier / secret-shaped token), a
|
||||
// benign query passing untouched, the opt-in env var reaching only the
|
||||
// warn tier (never the hard-block tier), strict/soft severity, and the
|
||||
// CLI shim.
|
||||
//
|
||||
// Secret-shaped fixtures are built via string concatenation/repeat, never
|
||||
// as literal tokens — the repo's own secrets pre-edit hook (correctly)
|
||||
// treats a literal AKIA/sk-/ghp_ string as a real credential.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
validateOutboundQuery,
|
||||
ABSOLUTE_PATH_PATTERNS,
|
||||
REPO_IDENTIFIER_PATTERNS,
|
||||
SECRET_SHAPED_PATTERNS,
|
||||
} from '../../lib/validators/query-privacy-gate.mjs';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const SHIM = join(HERE, '..', '..', 'lib', 'validators', 'query-privacy-gate.mjs');
|
||||
|
||||
const FAKE_OPENAI_KEY = 'sk-' + 'a'.repeat(24);
|
||||
const FAKE_AWS_KEY = 'AKIA' + 'Q'.repeat(16);
|
||||
const FAKE_GITHUB_PAT = 'ghp_' + 'b'.repeat(36);
|
||||
|
||||
function runShim(args) {
|
||||
try {
|
||||
const out = execFileSync(process.execPath, [SHIM, ...args], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
});
|
||||
return { code: 0, out };
|
||||
} catch (e) {
|
||||
return { code: e.status ?? 1, out: e.stdout?.toString() ?? '' };
|
||||
}
|
||||
}
|
||||
|
||||
// ---- two-sided code table ----------------------------------------------------
|
||||
|
||||
const TABLE = [
|
||||
{ label: 'absolute path (/Users/...)', text: 'find every caller of foo in /Users/ktg/repos/voyage/lib/util/foo.mjs', code: 'PRIVACY_ABSOLUTE_PATH' },
|
||||
{ label: 'absolute path (/home/...)', text: 'trace /home/alice/projects/app/src/index.js for imports', code: 'PRIVACY_ABSOLUTE_PATH' },
|
||||
{ label: 'repo-internal identifier (forgejo host)', text: 'what changed recently on git.fromaitochitta.com/open/voyage', code: 'PRIVACY_REPO_IDENTIFIER' },
|
||||
{ label: 'repo-internal identifier (repo name)', text: 'search issues for ktg-plugin-marketplace regressions', code: 'PRIVACY_REPO_IDENTIFIER' },
|
||||
{ label: 'secret-shaped (OpenAI/Anthropic-style key)', text: `auth failing with key ${FAKE_OPENAI_KEY}`, code: 'PRIVACY_SECRET_SHAPED' },
|
||||
{ label: 'secret-shaped (AWS access key)', text: `rotate ${FAKE_AWS_KEY} now`, code: 'PRIVACY_SECRET_SHAPED' },
|
||||
{ label: 'secret-shaped (GitHub PAT)', text: `token leaked: ${FAKE_GITHUB_PAT}`, code: 'PRIVACY_SECRET_SHAPED' },
|
||||
];
|
||||
|
||||
for (const { label, text, code } of TABLE) {
|
||||
test(`validateOutboundQuery — ${label} → ${code} (strict, error)`, () => {
|
||||
const r = validateOutboundQuery(text, { strict: true, env: {} });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === code), JSON.stringify(r.errors));
|
||||
});
|
||||
}
|
||||
|
||||
test('validateOutboundQuery — benign generic query passes untouched', () => {
|
||||
const r = validateOutboundQuery('What are the tradeoffs between optimistic and pessimistic locking?', { env: {} });
|
||||
assert.equal(r.valid, true);
|
||||
assert.deepEqual(r.errors, []);
|
||||
assert.deepEqual(r.warnings, []);
|
||||
});
|
||||
|
||||
// ---- strict vs soft (warn tier only) -----------------------------------------
|
||||
|
||||
test('validateOutboundQuery — soft mode downgrades warn-tier findings to warnings, stays valid', () => {
|
||||
const r = validateOutboundQuery('inspect /Users/ktg/repos/voyage', { strict: false, env: {} });
|
||||
assert.equal(r.valid, true);
|
||||
assert.equal(r.errors.length, 0);
|
||||
assert.ok(r.warnings.find(w => w.code === 'PRIVACY_ABSOLUTE_PATH'));
|
||||
});
|
||||
|
||||
test('validateOutboundQuery — soft mode does NOT downgrade the hard-block tier', () => {
|
||||
const r = validateOutboundQuery(`leaked ${FAKE_OPENAI_KEY}`, { strict: false, env: {} });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === 'PRIVACY_SECRET_SHAPED'));
|
||||
});
|
||||
|
||||
// ---- opt-in env var reaches only the warn tier -------------------------------
|
||||
|
||||
test('validateOutboundQuery — VOYAGE_QUERY_PRIVACY_ALLOW=1 bypasses the warn tier entirely', () => {
|
||||
const r = validateOutboundQuery('inspect /Users/ktg/repos/voyage', { env: { VOYAGE_QUERY_PRIVACY_ALLOW: '1' } });
|
||||
assert.equal(r.valid, true);
|
||||
assert.equal(r.errors.length, 0);
|
||||
assert.equal(r.warnings.length, 0);
|
||||
});
|
||||
|
||||
test('validateOutboundQuery — VOYAGE_QUERY_PRIVACY_ALLOW=1 does NOT open the hard-block tier', () => {
|
||||
const r = validateOutboundQuery(`leaked ${FAKE_OPENAI_KEY}`, { env: { VOYAGE_QUERY_PRIVACY_ALLOW: '1' } });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === 'PRIVACY_SECRET_SHAPED'), 'opt-in must never unlock the hard-block tier');
|
||||
});
|
||||
|
||||
test('validateOutboundQuery — VOYAGE_QUERY_PRIVACY_ALLOW=1 combined with a secret still denies', () => {
|
||||
const r = validateOutboundQuery(`/Users/ktg/x leaked ${FAKE_OPENAI_KEY}`, { env: { VOYAGE_QUERY_PRIVACY_ALLOW: '1' } });
|
||||
assert.equal(r.valid, false);
|
||||
assert.equal(r.errors.length, 1);
|
||||
assert.equal(r.errors[0].code, 'PRIVACY_SECRET_SHAPED');
|
||||
});
|
||||
|
||||
// ---- empty input --------------------------------------------------------------
|
||||
|
||||
test('validateOutboundQuery — empty string is invalid', () => {
|
||||
const r = validateOutboundQuery('', { env: {} });
|
||||
assert.equal(r.valid, false);
|
||||
assert.ok(r.errors.find(e => e.code === 'PRIVACY_EMPTY_QUERY'));
|
||||
});
|
||||
|
||||
// ---- pattern set is frozen ----------------------------------------------------
|
||||
|
||||
test('pattern sets are Object.frozen', () => {
|
||||
assert.equal(Object.isFrozen(ABSOLUTE_PATH_PATTERNS), true);
|
||||
assert.equal(Object.isFrozen(REPO_IDENTIFIER_PATTERNS), true);
|
||||
assert.equal(Object.isFrozen(SECRET_SHAPED_PATTERNS), true);
|
||||
});
|
||||
|
||||
// ---- CLI shim -----------------------------------------------------------------
|
||||
|
||||
test('CLI shim — benign query exits 0 with valid:true', () => {
|
||||
const r = runShim(['harmless generic question about caching strategies']);
|
||||
assert.equal(r.code, 0);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.valid, true);
|
||||
});
|
||||
|
||||
test('CLI shim — secret-shaped query exits 1 even with --soft', () => {
|
||||
const r = runShim(['--soft', `leaked ${FAKE_OPENAI_KEY}`]);
|
||||
assert.equal(r.code, 1);
|
||||
const parsed = JSON.parse(r.out.trim());
|
||||
assert.equal(parsed.valid, false);
|
||||
assert.ok(parsed.errors.find(e => e.code === 'PRIVACY_SECRET_SHAPED'));
|
||||
});
|
||||
|
||||
test('CLI shim — missing query argument exits 2 (usage error)', () => {
|
||||
const r = runShim([]);
|
||||
assert.equal(r.code, 2);
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue