Compare commits
No commits in common. "main" and "v5.10.0" have entirely different histories.
28 changed files with 332 additions and 1596 deletions
|
|
@ -1,11 +1,11 @@
|
|||
{
|
||||
"name": "voyage",
|
||||
"description": "Voyage — brief, research, plan, execute, review, continue. Contract-driven Claude Code pipeline. /trekbrief, /trekplan, and /trekreview each end by building a self-contained operator-annotation HTML (scripts/annotate.mjs, modelled on claude-code-100x): select text or click any element, pick intent (Fiks/Endre/Spørsmål), write comment, copy structured prompt, paste back, Claude revises the .md.",
|
||||
"version": "5.10.1",
|
||||
"version": "5.10.0",
|
||||
"author": {
|
||||
"name": "Kjell Tore Guttormsen"
|
||||
},
|
||||
"homepage": "https://git.fromaitochitta.com/open/voyage",
|
||||
"homepage": "https://git.fromaitochitta.com/open/ktg-plugin-marketplace/src/branch/main/plugins/voyage",
|
||||
"repository": "https://git.fromaitochitta.com/open/ktg-plugin-marketplace.git",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
|
|
|
|||
3
.gitignore
vendored
3
.gitignore
vendored
|
|
@ -48,6 +48,3 @@ NEXT-SESSION-PROMPT*.local.md
|
|||
# broad `git add -A docs`, which needed a history rewrite to undo. Ignoring the
|
||||
# type is cheaper than remembering not to stage it.
|
||||
docs/*.pdf
|
||||
# Page renders of those same documents (SLDC-AI-1_1.png etc). Narrow on purpose:
|
||||
# legitimate plugin assets may still be .png under docs/.
|
||||
docs/SLDC-AI*.png
|
||||
|
|
|
|||
105
CHANGELOG.md
105
CHANGELOG.md
|
|
@ -4,111 +4,6 @@ All notable changes to this project will be documented in this file.
|
|||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## v5.10.1 — 2026-09-03 — gemini-bridge dropped; fail-closed review verdicts
|
||||
|
||||
### Removed
|
||||
|
||||
- **`gemini-bridge` is dropped from the pipeline (operator directive).** The
|
||||
agent file is deleted and every live surface that could still steer a run
|
||||
toward it is cleared: `commands/trekresearch.md` (the Bridge-agent launch
|
||||
block, the `--local` help text, the high-effort always-on pair, the stats
|
||||
record, the graceful-degradation list), `commands/trekplan.md`,
|
||||
`commands/trekbrief.md`, `agents/research-orchestrator.md` (mode table,
|
||||
agent table, prompting block, summary line), `README.md` (feature prose, mode
|
||||
table, agent prose, the mermaid EXTERNAL node, the `* = conditional` legend,
|
||||
the optional-MCP requirement, the `--local` section), `CLAUDE.md`,
|
||||
`settings.json` (the whole `trekresearch.geminiBridge` block),
|
||||
`templates/research-brief-template.md` (the *Gemini Second Opinion* section
|
||||
and the sources-table `gemini` type), `templates/trekbrief-template.md`,
|
||||
`docs/architecture.md`, `docs/command-modes.md`.
|
||||
- **`/trekplan` high-effort Adversarial Pass 2 degrades explicitly.** It now
|
||||
emits a `## Adversarial Pass 2 (v5.1.1 high-effort)` section whose status
|
||||
reads **"unavailable, skipped"**, rather than failing or vanishing: a
|
||||
high-effort plan carrying no Pass 2 marker is indistinguishable from one whose
|
||||
Pass 2 crashed. `/trekresearch` high effort forces only `contrarian-researcher`
|
||||
always-on, which remains the complete adversarial coverage.
|
||||
- **Not removed:** `gemini_used` survives as a vestigial `/trekresearch` stats
|
||||
field pinned to `false`. Dropping it would break the observability export
|
||||
schema for existing consumers, and no consumer asked for that.
|
||||
- Agent inventory: **24 → 23 files** (20 spawnable + 3 orchestrator reference
|
||||
docs; `synthesis-agent` still dormant). Counts reconciled in `README.md` and
|
||||
`CLAUDE.md`; the `<example>`-block floor in
|
||||
`tests/lib/agent-frontmatter.test.mjs` moves 34 → 32 because an agent
|
||||
legitimately left the inventory — not because examples went missing from a
|
||||
surviving one.
|
||||
- Pinned by 5 new tests in `tests/lib/doc-consistency.test.mjs`, including a
|
||||
**known-positive control** asserting the historical records *still do* carry
|
||||
gemini references — so the empty result on live surfaces is a measurement and
|
||||
not a broken query. CHANGELOG entries and measurement/decision docs keep their
|
||||
references deliberately: they record what a past version did.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`/trekreview` verdicts are fail-closed.** A finding removed by Pass 2 or
|
||||
Pass 3, and a reviewer whose payload was thrown away or never arrived, were
|
||||
arithmetically identical to a finding that never existed — all three pushed
|
||||
the verdict toward `ALLOW`. Measured before the fix: a BLOCKER with a
|
||||
101-character title → `ALLOW`; a payload carrying one ad-hoc `rule_key` was
|
||||
skipped whole at ingest, taking a valid BLOCKER sibling with it → `ALLOW`.
|
||||
`lib/review/coordinator-contract.mjs` now splits every removal two ways —
|
||||
**dropped** when the test refuted the finding as a claim about this codebase,
|
||||
**unverified** otherwise — and a non-empty `unverified` bucket, or a reviewer
|
||||
in `expectedReviewers` that did not report, forbids `ALLOW` (verdict `WARN`,
|
||||
with `allow_blocked_by` naming why). The rule never *raises* a verdict: the
|
||||
severity catalogue and the BLOCKER/MAJOR thresholds are untouched. New
|
||||
exports: `classifySuppression`, `REFUTING_REASONS`, `UNVERIFIED_REASONS`;
|
||||
`runContract` gains `unverified`, `missing_reviewers`,
|
||||
`unattributable_payloads` and `allow_blocked_by` (`suppressed` stays the
|
||||
union, so existing consumers keep their meaning). A payload that fails schema
|
||||
without carrying a `reviewer` name is counted as unattributable rather than
|
||||
reported as a reviewer called "unnamed reviewer" — naming one would invent an
|
||||
agent nobody launched and double-count with `expectedReviewers`.
|
||||
Mirrored in `agents/review-coordinator.md` (Pass 2/3 fate columns, the new
|
||||
§*Suppression is two-valued*, Pass 4 threshold table) and
|
||||
`commands/trekreview.md` (Phase 5 reviewer accounting → STOP; Phase 6).
|
||||
Driven test-first: 11 new tests in `tests/lib/coordinator-contract.test.mjs`,
|
||||
incl. a known-positive control proving `ALLOW` is still reachable.
|
||||
|
||||
### Docs
|
||||
|
||||
- **T1 §6 PoC status corrected — it was already run and declined.**
|
||||
`docs/T1-cc26-delegated-orchestration.md` §8 item 3 still claimed both the §6
|
||||
synthesis-agent PoC and the §5 full bake-off were "designed but unbuilt". That
|
||||
was written in S7 and falsified the same afternoon by S12, which ran the §6
|
||||
PoC and recorded **Δ main-context (faithful flow) = 0.0%, NEGATIVE**
|
||||
(`docs/T1-synthesis-poc-results.md`; reproduce with
|
||||
`node scripts/synthesis-measure.mjs`). The stale wording caused the settled
|
||||
PoC to be re-ordered as new work on 2026-09-02, so it is struck rather than
|
||||
deleted and §6 gained an explicit **RUN AND DECLINED** status block. The
|
||||
finding is structural, not stochastic: `/trekplan` Phase 5 spawns the
|
||||
exploration swarm **foreground** (`commands/trekplan.md:158,338-341`), so the
|
||||
6–10 outputs are already resident in main before Phase 7 begins — delegating
|
||||
only the Phase-7 digest returns on top of them and evicts nothing. A saving
|
||||
needs the Phase-5 delivery moved off-main, which is §5.
|
||||
- **§5's environment premise updated (measured 2026-09-03, CC 2.1.259).** S12
|
||||
recorded the live bake-off as blocked for want of an `ANTHROPIC_API_KEY`;
|
||||
`claude -p --output-format stream-json` in fact runs on subscription auth with
|
||||
no API key and now emits a `subagent_stats` block (spawned / max_depth /
|
||||
refused) that did not exist in June — a ready-made classifier-interference
|
||||
counter for §5's feasibility guard. S12's other blocker (the installed plugin
|
||||
is the cache copy, so a repo-local agent is invisible to `claude -p`) stands
|
||||
unmeasured. This changes nothing about the §6 verdict; it lowers the cost of §5.
|
||||
|
||||
- **`/trekresearch --engine deep-research`: document the real version window.**
|
||||
Claude Code **2.1.218** changed `/deep-research` to start only when the operator
|
||||
invokes it; from there the Skill tool refuses a model invocation outright with
|
||||
`disable-model-invocation` (measured in a real `--engine deep-research` run on
|
||||
2026-09-01 — the SC3 fallback to `swarm` held). The engine was built against
|
||||
2.1.196 and the prose still promised a path Claude Code has removed
|
||||
("requires Claude Code 2.1.154+"). `commands/trekresearch.md` (flag bullet +
|
||||
pre-gate + fallback reason tokens), `docs/command-modes.md` and `README.md` now
|
||||
state the closed window `2.1.154 <= CC < 2.1.218` and name
|
||||
`disable-model-invocation` as the expected fallback reason on any current CC.
|
||||
The pre-gate gained an upper ceiling (still a numeric comparison, not a string
|
||||
one). **The flag is kept** as an additive opt-in that never hard-fails; no
|
||||
`lib/` change, no adapter-contract change, no default change, no version bump.
|
||||
Pinned by `tests/lib/doc-consistency.test.mjs`.
|
||||
|
||||
## v5.10.0 — 2026-08-18 — STORM bounded research loop (default-off) + the Agent-tool `name` spawn defect
|
||||
|
||||
Additive. Every new research mechanism ships **inert**: `VOYAGE_STORM_ENABLED` is
|
||||
|
|
|
|||
|
|
@ -65,8 +65,9 @@ Full flag reference for each command (modes, `--gates`, `--profile`, breaking ch
|
|||
| community-researcher | opus | Community experience: issues, blogs, discussions |
|
||||
| security-researcher | opus | CVEs, audit history, supply chain risks |
|
||||
| contrarian-researcher | opus | Counter-evidence, overlooked alternatives |
|
||||
| gemini-bridge | opus | Gemini Deep Research second opinion (conditional) |
|
||||
|
||||
> **Inventory (S33 reconcile, updated v5.10.1).** 23 agent files = **20 spawnable** (one, `synthesis-agent`, ships **dormant** — Δ≈0, wired to nothing) **+ 3 orchestrator reference docs** (`planning-/research-/review-orchestrator` document the inline `/trek*` workflow, not spawnable capabilities). All 23 stay `model: opus` (operator pin `40d8742`); the glue/mechanical/retrieval/dormant roles were reconsidered for a sonnet downgrade and **kept opus** — decision record: `docs/voyage-vs-cc-balance-analysis.md` §10.
|
||||
> **Inventory (S33 reconcile).** 24 agent files = **21 spawnable** (one, `synthesis-agent`, ships **dormant** — Δ≈0, wired to nothing) **+ 3 orchestrator reference docs** (`planning-/research-/review-orchestrator` document the inline `/trek*` workflow, not spawnable capabilities). All 24 stay `model: opus` (operator pin `40d8742`); the glue/mechanical/retrieval/dormant roles were reconsidered for a sonnet downgrade and **kept opus** — decision record: `docs/voyage-vs-cc-balance-analysis.md` §10.
|
||||
|
||||
> **Model & effort.** `opus` = Opus 4.8 (default reasoning effort `high`); `sonnet` = Sonnet 4.6; `fable` = Fable 5 (Mythos-class, above Opus — reasoning effort inherits from the session; xhigh requires a session-level setting). Select agents carry native per-spawn `effort:` (retrieval → `medium`, adversarial-reasoning → `high`) — a different axis from brief `phase_signals.effort` (orchestration shape: which agents/passes run). Per-agent table + axes → `docs/profiles.md` §Model & effort axes.
|
||||
|
||||
|
|
|
|||
25
README.md
25
README.md
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Contract-driven Claude Code pipeline: brief, research, plan, execute, review. Agent swarms, research triangulation, adversarial review, multi-session resumption.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -195,7 +195,7 @@ If you say "stop" or "enough" during Phase 4, the current review findings are su
|
|||
|
||||
Deep, multi-phase research that combines local codebase analysis with external knowledge. Uses specialized agent swarms to investigate multiple dimensions in parallel, then triangulates findings.
|
||||
|
||||
A parallel swarm of up to 5 local + 4 external agents investigates 3–8 research dimensions. Findings are triangulated (local vs. external, confidence per dimension, contradictions flagged) and synthesized into a structured research brief.
|
||||
A parallel swarm of up to 5 local + 4 external agents investigates 3–8 research dimensions, with optional Gemini Deep Research as an independent second opinion. Findings are triangulated (local vs. external, confidence per dimension, contradictions flagged) and synthesized into a structured research brief.
|
||||
|
||||
Output:
|
||||
- With `--project <dir>`: `{dir}/research/{NN}-{slug}.md` (auto-incremented index)
|
||||
|
|
@ -205,14 +205,14 @@ Output:
|
|||
|
||||
| Mode | Usage | Behavior |
|
||||
|------|-------|----------|
|
||||
| **Default** | `/trekresearch <question>` | Interview + research swarm (local + external), foreground |
|
||||
| **Default** | `/trekresearch <question>` | Interview + research swarm (local + external + Gemini), foreground |
|
||||
| **Project** | `/trekresearch --project <dir> <question>` | Write brief into `{dir}/research/NN-slug.md` |
|
||||
| **Quick** | `/trekresearch --quick <question>` | Interview (short) + inline research, no agent swarm |
|
||||
| **Local** | `/trekresearch --local <question>` | Only codebase analysis agents (skip external research) |
|
||||
| **Local** | `/trekresearch --local <question>` | Only codebase analysis agents (skip external + Gemini) |
|
||||
| **External** | `/trekresearch --external <question>` | Only external research agents (skip codebase analysis) |
|
||||
| **Foreground** | `/trekresearch --fg <question>` | No-op alias (foreground is default since v2.4.0) |
|
||||
| **Profile** | `/trekresearch --profile <name> <question>` | (v4.1.0) Pin model profile for the research phase. See [Profile system](#profile-system-v410). |
|
||||
| **Engine** | `/trekresearch --external --engine deep-research <question>` | Delegate the external phase to Claude Code's built-in `/deep-research` workflow. Only works on `2.1.154 <= CC < 2.1.218`; from **2.1.218** `/deep-research` is operator-invoked only (Skill tool: `disable-model-invocation`), so it always falls back to `swarm`. Never hard-fails. Default `swarm`. |
|
||||
| **Engine** | `/trekresearch --external --engine deep-research <question>` | Delegate the external phase to Claude Code's built-in `/deep-research` workflow; falls back to `swarm` if unavailable. Default `swarm`. |
|
||||
|
||||
Flags combine: `--project <dir> --external`.
|
||||
|
||||
|
|
@ -234,7 +234,7 @@ how many turns it has used. Whether the loop becomes the default is decided by a
|
|||
pre-registered measurement, not by preference: see
|
||||
[`docs/storm-measurement.md`](docs/storm-measurement.md).
|
||||
|
||||
Research uses up to 5 local agents (architecture-mapper, dependency-tracer, task-finder, git-historian, convention-scanner) and 4 external agents (docs-researcher, community-researcher, security-researcher, contrarian-researcher). Per-agent details in [`agents/`](agents/).
|
||||
Research uses up to 5 local agents (architecture-mapper, dependency-tracer, task-finder, git-historian, convention-scanner) and 4 external agents (docs-researcher, community-researcher, security-researcher, contrarian-researcher) plus the optional Gemini bridge for an independent second opinion. Per-agent details in [`agents/`](agents/).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -657,7 +657,7 @@ flowchart TB
|
|||
end
|
||||
subgraph RES["/trekresearch · Phase 4 — parallel"]
|
||||
RL["LOCAL: architecture-mapper · dependency-tracer<br/>task-finder · git-historian · convention-scanner*"]
|
||||
RE["EXTERNAL: docs-researcher · community-researcher<br/>security-researcher* · contrarian-researcher*"]
|
||||
RE["EXTERNAL: docs-researcher · community-researcher<br/>security-researcher* · contrarian-researcher* · gemini-bridge*"]
|
||||
end
|
||||
subgraph PL["/trekplan"]
|
||||
PLG["Phase 4b · brief-reviewer (gate)"]
|
||||
|
|
@ -676,7 +676,7 @@ flowchart TB
|
|||
BR --> RES --> PL --> EX --> RV
|
||||
```
|
||||
|
||||
`* = conditional`: convention-scanner / test-strategist on medium+ codebases (50+ files); research-scout for unknown external tech; security-/contrarian-researcher when a leading recommendation forms (or always at `effort=high`); brief-conformance-reviewer skipped under `--quick`.
|
||||
`* = conditional`: convention-scanner / test-strategist on medium+ codebases (50+ files); research-scout for unknown external tech; security-/contrarian-researcher + gemini-bridge when a leading recommendation forms (or always at `effort=high`); brief-conformance-reviewer skipped under `--quick`.
|
||||
|
||||
> **Which Claude Code primitive each phase uses — and the alternatives considered (Workflow substrate, delegated orchestrator, dormant synthesis-agent)** → see [docs/architecture.md §Primitives per step](docs/architecture.md#primitives-per-step-decision-matrix).
|
||||
|
||||
|
|
@ -837,6 +837,7 @@ For per-profile cost estimates, see [`docs/profiles.md`](docs/profiles.md).
|
|||
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) (CLI, desktop app, or web app)
|
||||
- Claude subscription with Opus access (Max plan recommended)
|
||||
- Optional: [Tavily MCP server](https://github.com/tavily-ai/tavily-mcp) for enhanced external research
|
||||
- Optional: a Gemini Deep Research MCP server exposing `gemini_deep_research`, `gemini_get_research_status`, `gemini_get_research_result`, `gemini_research_followup` for an independent second opinion (voyage calls these tools; any server implementing them works)
|
||||
|
||||
## Architecture
|
||||
|
||||
|
|
@ -844,8 +845,8 @@ Top-level layout:
|
|||
|
||||
```
|
||||
trekplan/
|
||||
├── agents/ 23 specialized agents (all `model: opus`-pinned; per-phase model set by `--profile`)
|
||||
│ └ 20 spawnable (1 dormant: synthesis-agent, Δ≈0) + 3 orchestrator reference docs (not spawned)
|
||||
├── agents/ 24 specialized agents (all `model: opus`-pinned; per-phase model set by `--profile`)
|
||||
│ └ 21 spawnable (1 dormant: synthesis-agent, Δ≈0) + 3 orchestrator reference docs (not spawned)
|
||||
├── commands/ 6 slash commands (trekbrief, trekresearch, trekplan, trekexecute, trekreview, trekcontinue) + trekendsession helper
|
||||
├── templates/ Frontmatter templates for brief, research, plan, session, launch
|
||||
├── hooks/ 8 hooks (pre-bash, pre-write, pre-agent-cap, session-title, post-bash-stats, pre-compact-flush, post-compact-flush, otel-export)
|
||||
|
|
@ -904,8 +905,8 @@ just run `--profile economy`, to put the whole pipeline on Sonnet.
|
|||
|
||||
### Disable external research
|
||||
|
||||
`/trekresearch --local` skips Tavily and Microsoft Learn. To make
|
||||
`--local` the default, edit the front of
|
||||
`/trekresearch --local` skips Tavily, Microsoft Learn, and the
|
||||
Gemini bridge. To make `--local` the default, edit the front of
|
||||
`commands/trekresearch.md` Phase 1 and flip the default branch
|
||||
of the `--local` argument check. Or just always pass `--local` and
|
||||
document it in your team's CLAUDE.md.
|
||||
|
|
|
|||
11
SECURITY.md
11
SECURITY.md
|
|
@ -2,14 +2,13 @@
|
|||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Report privately to <security@fromaitochitta.com> — do not open a public
|
||||
issue. There is no SLA — this is a solo-maintained plugin — but
|
||||
acknowledged reports are usually triaged within 7 days.
|
||||
Open a **private** issue on Forgejo:
|
||||
|
||||
If you already have an account on the Forgejo instance, you may instead
|
||||
open a **private** issue on the repo, tagged `security`:
|
||||
> https://git.fromaitochitta.com/open/ktg-plugin-marketplace
|
||||
|
||||
> https://git.fromaitochitta.com/open/voyage
|
||||
Tag it `security` and mark it private. Do not file public issues for
|
||||
unpatched vulnerabilities. There is no SLA — this is a solo-maintained
|
||||
plugin — but acknowledged reports are usually triaged within 7 days.
|
||||
|
||||
## Supported versions
|
||||
|
||||
|
|
|
|||
151
agents/gemini-bridge.md
Normal file
151
agents/gemini-bridge.md
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
---
|
||||
name: gemini-bridge
|
||||
description: |
|
||||
Use this agent when an independent second opinion from Gemini Deep Research is
|
||||
needed on a technology choice, architectural question, or complex research topic.
|
||||
Provides triangulation value by running a completely independent research path
|
||||
that can confirm or challenge findings from other agents.
|
||||
model: opus
|
||||
color: magenta
|
||||
tools: ["mcp__gemini-mcp__gemini_deep_research", "mcp__gemini-mcp__gemini_get_research_status", "mcp__gemini-mcp__gemini_get_research_result", "mcp__gemini-mcp__gemini_research_followup"]
|
||||
---
|
||||
|
||||
You are a bridge to Google Gemini Deep Research. Your role is to obtain an independent,
|
||||
thorough research result that provides triangulation value — a completely independent
|
||||
research path that can confirm or challenge findings from other agents.
|
||||
|
||||
The value of this agent is INDEPENDENCE. Do not pre-bias Gemini with conclusions from
|
||||
other agents. Submit the research question cleanly so Gemini's findings stand on their
|
||||
own merits.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Check availability
|
||||
|
||||
Attempt to call gemini_deep_research. If the tool is not available (MCP server not
|
||||
connected), return IMMEDIATELY with:
|
||||
|
||||
```
|
||||
## Gemini Bridge Result
|
||||
**Status:** Unavailable
|
||||
**Reason:** Gemini MCP server not connected. Proceeding without second opinion.
|
||||
```
|
||||
|
||||
Do NOT error, block, or retry. Unavailability is an expected operational state.
|
||||
|
||||
### 2. Formulate query
|
||||
|
||||
Take the research question and reformulate it for Gemini to maximize result quality:
|
||||
|
||||
- Add context about what dimensions to cover (trade-offs, maturity, ecosystem, operational
|
||||
concerns, known failure modes, community consensus)
|
||||
- Use format_instructions to request structured output with clear sections, source citations,
|
||||
and explicit confidence levels per claim
|
||||
- Set parameters:
|
||||
- `research_mode`: "custom"
|
||||
- `source_tier`: 2
|
||||
- `research_window_days`: 90
|
||||
|
||||
Example format_instructions to include:
|
||||
> "Structure your response with: Executive Summary, Key Findings (bullet points),
|
||||
> Trade-offs, Known Issues and Gotchas, Community Consensus, and Sources. For each
|
||||
> major claim, indicate your confidence level (high/medium/low) and cite the source."
|
||||
|
||||
### 3. Submit research
|
||||
|
||||
Call `gemini_deep_research` with the reformulated query and parameters.
|
||||
|
||||
### 4. Poll for completion
|
||||
|
||||
Call `gemini_get_research_status` repeatedly until the research completes:
|
||||
|
||||
- Call the status tool, then call it again after it returns — repeat until done
|
||||
- Do not use bash or sleep commands — use repeated tool calls to simulate waiting
|
||||
- Continue polling until status is `"completed"` or `"failed"`
|
||||
- If `"failed"`: report the failure reason and return gracefully — do not retry
|
||||
- Timeout: if still running after 40 polls (~20 minutes of equivalent wait), report
|
||||
timeout and return whatever partial result is available
|
||||
|
||||
### 5. Retrieve result
|
||||
|
||||
Call `gemini_get_research_result` with `include_citations: true`.
|
||||
|
||||
### 6. Optional follow-up
|
||||
|
||||
If the result has clear gaps on specific dimensions that are directly relevant to the
|
||||
research question, call `gemini_research_followup` with a targeted follow-up question.
|
||||
|
||||
Rules for follow-up:
|
||||
- Maximum 1 follow-up call
|
||||
- Only if there is a genuine gap — do not follow up out of habit
|
||||
- Make the follow-up question narrow and specific, not a re-statement of the original
|
||||
|
||||
### 7. Format output
|
||||
|
||||
Structure the final result as:
|
||||
|
||||
```
|
||||
## Gemini Bridge Result
|
||||
**Status:** Completed
|
||||
**Research duration:** {time taken}
|
||||
**Sources cited:** {count}
|
||||
|
||||
### Key Findings
|
||||
- {finding 1}
|
||||
- {finding 2}
|
||||
- {finding 3}
|
||||
|
||||
### Trade-offs and Known Issues
|
||||
- {trade-off or issue 1}
|
||||
- {trade-off or issue 2}
|
||||
|
||||
### Sources
|
||||
| # | Source | Relevance |
|
||||
|---|--------|-----------|
|
||||
| 1 | {URL} | {one-line relevance} |
|
||||
|
||||
### Areas for Triangulation
|
||||
*Claims that should be cross-checked against local codebase analysis
|
||||
and other external agents:*
|
||||
- {claim 1 — check against local architecture}
|
||||
- {claim 2 — verify with community experience}
|
||||
- {claim 3 — validate against codebase constraints}
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
- **Never block the research pipeline.** If Gemini is slow or unavailable, return what
|
||||
you have with a clear status note.
|
||||
- **Do not interpret or editorialize.** Report Gemini's findings as-is, formatted for
|
||||
integration. Your job is formatting and delivery, not analysis.
|
||||
- **Flag "Areas for Triangulation"** — claims that the research-orchestrator or other
|
||||
agents should cross-check against local codebase analysis, team experience, or other
|
||||
external sources.
|
||||
- **Independence is the point.** Do not include findings from other agents in your query
|
||||
to Gemini. The value of a second opinion is that it is uninfluenced by the first.
|
||||
- **Cite everything.** Every major claim in the output must trace to a source in the
|
||||
Sources table. Remove claims that Gemini did not support with a source.
|
||||
- **Graceful degradation at every step.** Unavailable tool, failed research, timeout —
|
||||
all are handled with a clear status message and immediate return. Never leave the
|
||||
pipeline hanging.
|
||||
|
||||
## When to use — examples
|
||||
|
||||
<example>
|
||||
Context: trekresearch launches gemini-bridge for an independent second opinion on a technology choice
|
||||
user: "/trekplan Should we use Kafka or NATS for our event streaming layer?"
|
||||
assistant: "Launching gemini-bridge for an independent second opinion on Kafka vs NATS."
|
||||
<commentary>
|
||||
Technology choice with significant architectural implications triggers gemini-bridge
|
||||
to provide an independent research path alongside local exploration agents.
|
||||
</commentary>
|
||||
</example>
|
||||
|
||||
<example>
|
||||
Context: user wants deep research via Gemini on a complex architectural question
|
||||
user: "Get me a Gemini deep research on event sourcing patterns for distributed systems"
|
||||
assistant: "I'll use the gemini-bridge agent to run a deep research on event sourcing patterns."
|
||||
<commentary>
|
||||
Direct request for Gemini research on a complex architectural question triggers the agent.
|
||||
</commentary>
|
||||
</example>
|
||||
|
|
@ -61,16 +61,13 @@ Execute these phases in order. Do not skip phases.
|
|||
|
||||
Based on the mode, determine which agent groups to launch:
|
||||
|
||||
| Mode | Local agents | External agents |
|
||||
|------|-------------|-----------------|
|
||||
| `default` | Yes | Yes |
|
||||
| `local` | Yes | No |
|
||||
| `external` | No | Yes |
|
||||
| Mode | Local agents | External agents | Gemini bridge |
|
||||
|------|-------------|-----------------|---------------|
|
||||
| `default` | Yes | Yes | Yes (if enabled in settings) |
|
||||
| `local` | Yes | No | No |
|
||||
| `external` | No | Yes | Yes (if enabled) |
|
||||
| `quick` | N/A — handled inline by the command, not the orchestrator |
|
||||
|
||||
> **No bridge agent (v5.10.1).** The third-party second-opinion bridge was dropped
|
||||
> by operator directive; there is no bridge column to select.
|
||||
|
||||
**Local agents** (reuse existing plugin agents with research-focused prompts):
|
||||
|
||||
| Agent | Purpose in research context |
|
||||
|
|
@ -90,6 +87,12 @@ Based on the mode, determine which agent groups to launch:
|
|||
| `security-researcher` | CVEs, audit history, supply chain risks |
|
||||
| `contrarian-researcher` | Counter-evidence, overlooked alternatives, reasons to reconsider |
|
||||
|
||||
**Bridge agent:**
|
||||
|
||||
| Agent | Purpose |
|
||||
|-------|---------|
|
||||
| `gemini-bridge` | Independent second opinion via Gemini Deep Research |
|
||||
|
||||
### Phase 2 — Parallel research
|
||||
|
||||
Launch ALL selected agents **in parallel** using the Agent tool — one message,
|
||||
|
|
@ -124,6 +127,11 @@ Examples:
|
|||
Pass the research question, specific dimensions to investigate, and any context from
|
||||
the interview about what the user already knows or cares about.
|
||||
|
||||
**Prompting gemini-bridge:**
|
||||
|
||||
Pass the research question as-is. Do NOT pre-bias with findings from other agents —
|
||||
the value of Gemini is independence.
|
||||
|
||||
### Phase 3 — Targeted follow-ups
|
||||
|
||||
Review all agent results. Identify knowledge gaps — areas where findings are thin,
|
||||
|
|
@ -187,7 +195,7 @@ When done, your output message should contain:
|
|||
**Brief:** {brief path}
|
||||
**Confidence:** {overall confidence 0.0-1.0}
|
||||
**Dimensions:** {N} researched
|
||||
**Agents:** {N} local + {N} external
|
||||
**Agents:** {N} local + {N} external + {gemini status}
|
||||
|
||||
### Key Findings
|
||||
- {Finding 1}
|
||||
|
|
@ -218,7 +226,7 @@ You can:
|
|||
- **Sources:** Every claim in the brief must cite a source (URL or file path).
|
||||
Never invent findings.
|
||||
- **Honesty:** If a question is trivially answerable, say so. Don't inflate research.
|
||||
- **Graceful degradation:** If MCP tools are unavailable (Tavily, MS Learn), proceed
|
||||
- **Graceful degradation:** If MCP tools are unavailable (Tavily, Gemini), proceed
|
||||
with available tools and note the limitation in the brief metadata.
|
||||
- **Independence:** Do not pre-bias external agents with local findings or vice versa.
|
||||
The value is in independent perspectives that are THEN triangulated.
|
||||
|
|
|
|||
|
|
@ -84,48 +84,31 @@ identical `(file, rule_key)` and `line == 0` collide.
|
|||
|
||||
### Pass 2 — HubSpot Judge filters (3 criteria)
|
||||
|
||||
Remove findings that fail ANY of these filters. **The `Removed as` column
|
||||
is load-bearing** — see *Suppression is two-valued* below:
|
||||
Drop findings that fail ANY of these filters:
|
||||
|
||||
| Filter | Test | Fails if | Removed as |
|
||||
|--------|------|----------|------------|
|
||||
| Succinctness | `title.length ≤ 100` and `detail.length ≤ 800` chars | Title is a paragraph or detail is a wall of text | `unverified` (`succinctness:title` / `succinctness:detail`) |
|
||||
| Accuracy | `file` resolves under the repo root AND `line` is plausible (≥ 0; ≤ file line count when known) | Path traversal escape, negative line, or impossibly large line number | **dropped** (`accuracy:refuted`) |
|
||||
| Actionability | `recommended_action` is non-empty AND begins with an imperative verb | Empty action, "consider …" hedges, or restating the title | `unverified` (`actionability:empty`) |
|
||||
| Filter | Test | Drop if |
|
||||
|--------|------|---------|
|
||||
| Succinctness | `title.length ≤ 100` and `detail.length ≤ 800` chars | Title is a paragraph or detail is a wall of text |
|
||||
| Accuracy | `file` resolves under the repo root AND `line` is plausible (≥ 0; ≤ file line count when known) | Path traversal escape, negative line, or impossibly large line number |
|
||||
| Actionability | `recommended_action` is non-empty AND begins with an imperative verb | Empty action, "consider …" hedges, or restating the title |
|
||||
|
||||
Succinctness and Actionability read the finding's *packaging*; neither
|
||||
examines the claim, so neither can establish the finding is unreal. Accuracy
|
||||
does: a citation that escapes the repo root refutes the finding as a claim
|
||||
about this codebase.
|
||||
|
||||
When removing a finding, preserve a one-line note in the
|
||||
When dropping a finding, preserve a one-line note in the
|
||||
`Suppressed Findings` body section so the user knows why the count
|
||||
shrank.
|
||||
|
||||
### Pass 3 — Cloudflare reasonableness (skipped in quick mode)
|
||||
|
||||
Remove findings that fail ANY of these tests:
|
||||
Drop findings that fail ANY of these tests:
|
||||
|
||||
- **No file:line citation** → **dropped** (`no-citation`). `file` is empty,
|
||||
or `line < 0`. Speculative "code might break somewhere" findings name no
|
||||
location, so they make no checkable claim at all.
|
||||
- **Unknown rule_key** → `unverified` (`unknown-rule_key`). `rule_key` is not
|
||||
in `RULE_CATALOGUE`. Reviewers occasionally emit ad-hoc rule keys; the
|
||||
catalogue is the contract, but a mislabelled finding is not a refuted one.
|
||||
*(High-effort mode does not reach this branch: Pass 3 is bypassed and the
|
||||
key is normalised to `PLAN_EXECUTE_DRIFT` and KEPT — see High-effort
|
||||
normalization below. The two fates never apply to the same input.)*
|
||||
- **No file:line citation.** `file` is empty, or `line < 0`. Speculative
|
||||
"code might break somewhere" findings have no anchor and are dropped.
|
||||
- **Unknown rule_key.** `rule_key` is not in `RULE_CATALOGUE`. Reviewers
|
||||
occasionally emit ad-hoc rule keys; the catalogue is the contract.
|
||||
- **Non-existent file.** `file` does not exist in the working tree AND
|
||||
the diff does not show it as `(new file)`. Use Glob to verify. **This test
|
||||
has three outcomes, not two:** Glob resolves and the file is absent from
|
||||
both tree and diff → **dropped** (`file-existence:refuted`); Glob resolves
|
||||
and the file is present → keep; **Glob cannot decide** (path outside the
|
||||
working tree, unreadable, or the tool errored) → `unverified`
|
||||
(`file-existence:indeterminate`). Never collapse *unresolvable* into
|
||||
*refuted*.
|
||||
the diff does not show it as `(new file)`. Use Glob to verify.
|
||||
- **Catalogue severity mismatch.** `severity` does not match the rule's
|
||||
catalogue tier (e.g., `MISSING_TEST` emitted as MINOR). Reset to the
|
||||
catalogue tier; this is a correction, neither a drop nor an unverified.
|
||||
catalogue tier; this is a correction, not a drop.
|
||||
|
||||
In `quick` mode, skip this pass entirely. Note the skip in the
|
||||
Executive Summary so the reader knows reasonableness was not applied.
|
||||
|
|
@ -142,32 +125,6 @@ purposes. This normalization happens BEFORE writing review.md,
|
|||
ensuring all `rule_key` values in the final review match the
|
||||
catalogue.
|
||||
|
||||
### Suppression is two-valued (fail-closed)
|
||||
|
||||
Every removal in Pass 2 and Pass 3 carries one of two fates, and the
|
||||
distinction decides whether the review may come back clean:
|
||||
|
||||
| Fate | Meaning | Weight in Pass 4 |
|
||||
|------|---------|------------------|
|
||||
| **dropped** | The test **refuted** the finding as a claim about this codebase. | None. It weighs nothing, correctly. |
|
||||
| **unverified** | The finding was removed **without** its claim ever being examined or settled. | Forbids `ALLOW`. |
|
||||
|
||||
The rule is one sentence: **a removal is `dropped` only when the test
|
||||
refuted the finding; every other removal is `unverified`.** A reason you
|
||||
cannot place is `unverified` — the default fails closed.
|
||||
|
||||
Why this exists: without it, a finding the coordinator could not
|
||||
substantiate is arithmetically identical to a finding that never existed,
|
||||
and both push the verdict toward `ALLOW`. The deterministic mirror of this
|
||||
rule, including the reason vocabulary, is
|
||||
`lib/review/coordinator-contract.mjs` (`classifySuppression`,
|
||||
`REFUTING_REASONS`, `UNVERIFIED_REASONS`) — prose and lib share one
|
||||
vocabulary on purpose.
|
||||
|
||||
**Unverified findings are not counted into a severity tier.** Their severity
|
||||
is reviewer-asserted and was never substantiated; counting it would let an
|
||||
unexamined finding *raise* the verdict, which is invention.
|
||||
|
||||
### Pass 4 — Compute verdict
|
||||
|
||||
Count findings by severity AFTER dedup and filtering. Verdict thresholds:
|
||||
|
|
@ -176,19 +133,7 @@ Count findings by severity AFTER dedup and filtering. Verdict thresholds:
|
|||
|--------|---------|
|
||||
| `BLOCKER ≥ 1` | `BLOCK` |
|
||||
| `BLOCKER == 0` AND `MAJOR ≥ 1` | `WARN` |
|
||||
| `BLOCKER == 0` AND `MAJOR == 0` AND nothing `unverified` AND every reviewer reported | `ALLOW` |
|
||||
| `BLOCKER == 0` AND `MAJOR == 0` AND (`unverified` non-empty OR a reviewer did not report) | `WARN` |
|
||||
|
||||
The fail-closed row never RAISES a verdict — it only withholds the clean
|
||||
one. The worst case of a false `unverified` is `WARN` plus a stated reason;
|
||||
the worst case of the old behaviour was a silent `ALLOW` over a live
|
||||
BLOCKER.
|
||||
|
||||
**When `ALLOW` is withheld, the Executive Summary's FIRST sentence must say
|
||||
so and name why** — e.g. "WARN: no blocking findings survived, but 1 finding
|
||||
could not be verified (succinctness:title) and brief-conformance-reviewer did
|
||||
not report." A withheld ALLOW that the reader cannot see is the same defect
|
||||
in a new place.
|
||||
| `BLOCKER == 0` AND `MAJOR == 0` | `ALLOW` |
|
||||
|
||||
Verdict is mechanical — never override. The verdict goes into the
|
||||
trailing JSON block AND the Executive Summary's first sentence.
|
||||
|
|
@ -236,10 +181,8 @@ prefix). Flow-style `findings: [a, b]` breaks the frontmatter parser.
|
|||
5. `## Findings (MAJOR)` — one subsection per MAJOR finding.
|
||||
6. `## Findings (MINOR)` — one subsection per MINOR finding.
|
||||
7. `## Findings (SUGGESTION)` — one subsection per SUGGESTION finding.
|
||||
8. `## Suppressed Findings` (optional) — one line per finding removed by
|
||||
Pass 2 or Pass 3, with the reason AND its fate, tagged `[dropped]` or
|
||||
`[unverified]`. Unverified lines come first: they are the ones that
|
||||
withheld `ALLOW`.
|
||||
8. `## Suppressed Findings` (optional) — one-line per finding dropped by
|
||||
Pass 2 or Pass 3, with the reason.
|
||||
9. `## Remediation Summary` — bullet count per severity + 1 sentence on
|
||||
what /trekplan will consume.
|
||||
|
||||
|
|
@ -261,7 +204,6 @@ The LAST fenced block in the file is a `json` block:
|
|||
{
|
||||
"verdict": "BLOCK | WARN | ALLOW",
|
||||
"counts": { "BLOCKER": N, "MAJOR": N, "MINOR": N, "SUGGESTION": N },
|
||||
"allow_blocked_by": ["unverified:succinctness:title (1)", "missing-reviewer:brief-conformance-reviewer", "unattributable-payload (1)"],
|
||||
"findings": [
|
||||
{
|
||||
"id": "<40-char-hex>",
|
||||
|
|
@ -301,14 +243,9 @@ for the ID list.
|
|||
the canonical 40-char SHA1 from `(file, line, rule_key, title)` using
|
||||
the algorithm in `lib/parsers/finding-id.mjs`. The frontmatter
|
||||
`findings:` list and the JSON block IDs must match.
|
||||
- **Suppressed findings are accountable.** When you remove a finding via
|
||||
Pass 2 or Pass 3, log it in `## Suppressed Findings` with the reason and
|
||||
its fate (`[dropped]` / `[unverified]`). Silent drops break the audit
|
||||
trail.
|
||||
- **Never spend an unexamined finding as evidence of a clean review.** If a
|
||||
removal did not refute the finding, it is `unverified` and `ALLOW` is off
|
||||
the table. This is the one place where you may not be minimal: when in
|
||||
doubt about a reason's fate, it is `unverified`.
|
||||
- **Suppressed findings are accountable.** When you drop a finding via
|
||||
Pass 2 or Pass 3, log it in `## Suppressed Findings` with the reason.
|
||||
Silent drops break the audit trail.
|
||||
- **No invention.** Never add a finding that did not appear in the
|
||||
reviewer outputs. Never escalate a finding's severity beyond what the
|
||||
catalogue specifies.
|
||||
|
|
|
|||
|
|
@ -386,8 +386,8 @@ The mapping table is canonical:
|
|||
- `high → {effort: high, model: opus}` (force opus for the high-confidence path)
|
||||
- `fable → {effort: high, model: fable}` (force Fable 5 for the max-quality path)
|
||||
|
||||
The fable tier reuses `effort: high` semantics — full swarm, contrarian
|
||||
always-on; `EFFORT_LEVELS` is unchanged (Voyage effort is orchestration
|
||||
The fable tier reuses `effort: high` semantics — full swarm, contrarian +
|
||||
gemini always-on; `EFFORT_LEVELS` is unchanged (Voyage effort is orchestration
|
||||
shape, not model reasoning effort). Model reasoning effort is inherited from
|
||||
the session: Fable 5's default effort is `high`, NOT xhigh. To run xhigh, the
|
||||
operator sets it at session level via `/effort xhigh`, the `effortLevel`
|
||||
|
|
@ -825,7 +825,7 @@ topic.
|
|||
> background orchestrator-agents could not spawn the research swarm —
|
||||
> before Claude Code 2.1.172 the harness did not expose the Agent tool to
|
||||
> sub-agents, so a background run silently degraded to single-context
|
||||
> reasoning without WebSearch / Tavily / WebFetch (see v2.4.0
|
||||
> reasoning without WebSearch / Tavily / WebFetch / Gemini (see v2.4.0
|
||||
> release notes). As of CC 2.1.172 sub-agents can spawn sub-agents (up to 5
|
||||
> levels deep), so a delegated redesign is under evaluation (see
|
||||
> `docs/cc-upgrade-2.1.181-decision-matrix.md`, W1/CC-26). Until then,
|
||||
|
|
|
|||
|
|
@ -419,46 +419,19 @@ to stderr but do NOT block the stop; `progress.json` is still authoritative.
|
|||
`status: stopped`) so the next-session producer-mismatch check has both
|
||||
candidates available. Use the same combined ESM block pattern as Phase 8.
|
||||
|
||||
### Check 2 — Plan file reaches every worktree
|
||||
### Check 2 — Plan file is tracked by git
|
||||
|
||||
Worktrees are created from HEAD, so tracking the plan file is the cheapest way
|
||||
to make it visible in each one. But the project directory may be **gitignored**
|
||||
— `.claude/projects/` is tool-managed and local-only, and a repo that ignores it
|
||||
is normal, not exotic. `git add -f` is **not** the answer there: it would push
|
||||
operator-local artifacts into history, and into whatever remote the repo
|
||||
publishes to. When the plan file is ignored, Phase 2.6 Step 2a' (which copies
|
||||
brief/plan/research into each worktree) is the delivery path, and this check
|
||||
must step aside instead of failing.
|
||||
Run `git ls-files --error-unmatch {plan-path} 2>/dev/null`. If the plan file is
|
||||
untracked (exit code != 0):
|
||||
|
||||
```bash
|
||||
PLAN_PATH="{plan-path}"
|
||||
if git ls-files --error-unmatch "$PLAN_PATH" >/dev/null 2>&1; then
|
||||
PLAN_TRACKING="tracked"
|
||||
else
|
||||
git check-ignore -q "$PLAN_PATH"
|
||||
case "$?" in
|
||||
0) PLAN_TRACKING="ignored" ;;
|
||||
1) PLAN_TRACKING="untracked" ;;
|
||||
*) echo "Error: git check-ignore failed on $PLAN_PATH - a fatal probe is not an answer about ignore status." >&2
|
||||
exit 1 ;;
|
||||
esac
|
||||
fi
|
||||
if [ "$PLAN_TRACKING" = "untracked" ]; then
|
||||
git add "$PLAN_PATH"
|
||||
git commit -m "chore: track plan file for parallel execution"
|
||||
fi
|
||||
git add {plan-path}
|
||||
git commit -m "chore: track plan file for parallel execution"
|
||||
```
|
||||
|
||||
Report by outcome:
|
||||
Report: `Plan file committed for worktree visibility.`
|
||||
|
||||
| `PLAN_TRACKING` | Report |
|
||||
|---|---|
|
||||
| `tracked` | `Plan file already tracked.` |
|
||||
| `untracked` | `Plan file committed for worktree visibility.` |
|
||||
| `ignored` | `Plan file is gitignored — not forced into history. Step 2a' copies it into each worktree.` |
|
||||
|
||||
Any other `git check-ignore` exit code is fatal and stops execution: a probe
|
||||
that failed is not a probe that answered "not ignored".
|
||||
This ensures every worktree created from HEAD will have the plan file.
|
||||
|
||||
### Check 3 — Scope fence overlap validation
|
||||
|
||||
|
|
@ -505,7 +478,7 @@ If cleanup fails, report the manual commands and stop.
|
|||
|
||||
After all 4 checks pass:
|
||||
```
|
||||
Pre-flight: PASS (clean tree, plan reaches worktrees, no overlaps, no stale worktrees)
|
||||
Pre-flight: PASS (clean tree, plan tracked, no overlaps, no stale worktrees)
|
||||
```
|
||||
|
||||
## Phase 2.6 — Multi-session orchestration (worktree-isolated)
|
||||
|
|
@ -631,45 +604,27 @@ Insert this block AFTER the worktree-creation loop and BEFORE wave dispatch
|
|||
|
||||
```bash
|
||||
PROJECT_SOURCE="$(realpath "${PROJECT_DIR}")"
|
||||
REPO_ROOT_REAL="$(realpath "${REPO_ROOT}")"
|
||||
# Compute destination relpath: PROJECT_DIR relative to REPO_ROOT.
|
||||
# This makes $wt/$PROJECT_REL valid regardless of whether the operator
|
||||
# passed --project as relative (.claude/projects/...) or absolute.
|
||||
# python3 + os.path.relpath is stdlib and portable - see the note below the
|
||||
# block for why no realpath flag may be used here.
|
||||
PROJECT_REL="$(python3 -c 'import os.path,sys; print(os.path.relpath(sys.argv[1], sys.argv[2]))' "$PROJECT_SOURCE" "$REPO_ROOT_REAL")"
|
||||
case "$PROJECT_REL" in
|
||||
""|..*)
|
||||
echo "Error: cannot derive a project relpath inside the repo ($PROJECT_SOURCE vs $REPO_ROOT_REAL)." >&2
|
||||
exit 1 ;;
|
||||
esac
|
||||
PROJECT_REL="$(realpath --relative-to="$REPO_ROOT" "$PROJECT_SOURCE")"
|
||||
for wt in "$WORKTREE_DIR"/session-*; do
|
||||
[ -d "$wt" ] || continue
|
||||
mkdir -p "$wt/$PROJECT_REL"
|
||||
cp "$PROJECT_SOURCE"/brief.md "$wt/$PROJECT_REL/"
|
||||
cp "$PROJECT_SOURCE"/plan.md "$wt/$PROJECT_REL/"
|
||||
if [ -d "$PROJECT_SOURCE/research" ]; then
|
||||
[ -d "$PROJECT_SOURCE/research" ] && \
|
||||
cp -r "$PROJECT_SOURCE/research" "$wt/$PROJECT_REL/"
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
Do not "improve" the relpath line into `realpath --relative-to=...`. That flag
|
||||
is GNU coreutils only; BSD `realpath` (the macOS default) rejects it, and the
|
||||
failure is **silent** — the command substitution leaves `PROJECT_REL` empty, so
|
||||
`mkdir -p "$wt/"` and `cp ... "$wt//"` both succeed and drop `brief.md`/`plan.md`
|
||||
at the worktree root, where no child session looks for them. Measured on an
|
||||
Intel Mac 2026-08-31: `realpath --relative-to=... ` → `realpath: illegal option
|
||||
-- -`, while bare `realpath <path>` works; a whole wave ran with zero steps
|
||||
executed. Both `realpath` calls above are bare path resolution, which BSD and
|
||||
GNU handle identically; resolving both operands before the relpath is what keeps
|
||||
it correct when one side goes through a symlink (macOS `/var` → `/private/var`).
|
||||
Note: `realpath --relative-to` is GNU coreutils. macOS users without
|
||||
`coreutils` (Homebrew `brew install coreutils` provides `grealpath`) may
|
||||
substitute a Python fallback:
|
||||
`python3 -c "import os.path,sys; print(os.path.relpath(sys.argv[1], sys.argv[2]))" "$PROJECT_SOURCE" "$REPO_ROOT"`.
|
||||
|
||||
Failure modes: an underivable relpath (empty, or outside the repo) aborts
|
||||
before anything is copied — better a loud stop than files delivered where no
|
||||
session reads them; any `cp` failure exits the wave non-zero, reported via
|
||||
Step 4 cleanup. A project without `research/` is not a failure.
|
||||
Source: brief Constraint 2.
|
||||
Failure mode: any `cp` failure exits the wave non-zero; reported via Step 4
|
||||
cleanup. Source: brief Constraint 2.
|
||||
|
||||
**2b. Launch sessions in this wave (each in its own worktree):**
|
||||
|
||||
|
|
|
|||
|
|
@ -898,33 +898,22 @@ validator-only; this surface just makes the friendly hint readable.
|
|||
|
||||
### High-effort behavior (v5.1.1)
|
||||
|
||||
When `phase_signal_result.effort == 'high'` for the `plan` phase, the
|
||||
additional adversarial Pass 2 on the post-revision plan is
|
||||
**unavailable, skipped**. Append the section header to plan.md before
|
||||
the trailing JSON block with exactly that status, so a reader can tell
|
||||
a deliberately dropped pass from one that failed:
|
||||
When `phase_signal_result.effort == 'high'` for the `plan` phase, after
|
||||
Phase 9 (plan-critic + scope-guardian dedup pass) runs to completion on
|
||||
the post-revision plan, run an ADDITIONAL `gemini-bridge` plan-review
|
||||
pass on the post-revision plan. Surface its findings as a separate
|
||||
`## Adversarial Pass 2 (gemini-bridge, v5.1.1 high-effort)` section
|
||||
appended to plan.md before the trailing JSON block.
|
||||
|
||||
```
|
||||
## Adversarial Pass 2 (v5.1.1 high-effort)
|
||||
Rationale (per risk-assessor finding + Decision B substitution
|
||||
2026-05-14): the originally-considered "extra plan-critic-iterasjon"
|
||||
risked a revision-loop because plan-critic dedup keys on
|
||||
`(file, line, rule_key)` triplets and post-revision line numbers shift.
|
||||
The gemini-bridge pass is independent (different agent, different
|
||||
perspective) and does not re-tread the same dedup space — it surfaces
|
||||
genuinely new findings rather than re-emitting closed ones.
|
||||
|
||||
**Status:** unavailable, skipped — the third-party bridge agent that
|
||||
ran this pass was dropped in v5.10.1 by operator directive. No
|
||||
substitute is wired. Phase 9 (plan-critic + scope-guardian) is the
|
||||
complete adversarial coverage for this plan.
|
||||
```
|
||||
|
||||
Do NOT fail, retry, or silently omit the section: a high-effort plan
|
||||
that carries no Pass 2 marker is indistinguishable from one whose Pass
|
||||
2 crashed.
|
||||
|
||||
History (why the slot exists): Decision B, 2026-05-14, substituted a
|
||||
bridge pass for an "extra plan-critic-iterasjon" because plan-critic
|
||||
dedup keys on `(file, line, rule_key)` triplets and post-revision line
|
||||
numbers shift, risking a revision-loop. Dropping the bridge does not
|
||||
revive that option — re-running plan-critic carries the same loop risk
|
||||
it always did.
|
||||
|
||||
Standard and low effort: do NOT emit the section at all.
|
||||
Standard and low effort: do NOT run the additional pass.
|
||||
|
||||
## Hard rules
|
||||
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ Supported flags:
|
|||
1. `--quick` — lightweight research, no agent swarm. The command itself does
|
||||
3-5 targeted searches inline. Set **mode = quick**.
|
||||
|
||||
2. `--local` — only codebase research. Skip external research agents.
|
||||
2. `--local` — only codebase research. Skip external agents and gemini bridge.
|
||||
Set **scope = local**.
|
||||
|
||||
3. `--external` — only external research. Skip codebase analysis agents.
|
||||
|
|
@ -87,14 +87,9 @@ Supported flags:
|
|||
behavior). `swarm` runs Voyage's own external-research agent swarm;
|
||||
`deep-research` delegates the external phase to Claude Code's built-in
|
||||
`/deep-research` dynamic workflow and adapts its report into the research-brief
|
||||
schema. **The delegation only works inside a closed version window:
|
||||
`2.1.154 <= CC < 2.1.218`** (dynamic workflows enabled). From Claude Code
|
||||
**2.1.218** `/deep-research` starts only when the operator invokes it, and the
|
||||
Skill tool refuses a model invocation outright (`disable-model-invocation`), so
|
||||
on 2.1.218 or newer the engine **always** falls back to `swarm`. The flag is
|
||||
kept as an additive opt-in and **never hard-fails**: outside the window it
|
||||
degrades to `swarm` and logs the reason. Orthogonal to
|
||||
`--profile`/`phase_signals`; only affects the external phase. Set
|
||||
schema (requires Claude Code 2.1.154+ and dynamic workflows enabled; falls back
|
||||
to `swarm` and notes the fallback if unavailable — never hard-fails). Orthogonal
|
||||
to `--profile`/`phase_signals`; only affects the external phase. Set
|
||||
**engine = {swarm|deep-research}** (the *requested* engine).
|
||||
|
||||
Flags can be combined:
|
||||
|
|
@ -121,7 +116,7 @@ Usage: /trekresearch <research question>
|
|||
Modes:
|
||||
default Interview → foreground research (local + external) → brief
|
||||
--quick Interview (short) → inline research (no agent swarm)
|
||||
--local Only codebase analysis agents (skip external research)
|
||||
--local Only codebase analysis agents (skip external + Gemini)
|
||||
--external Only external research agents (skip codebase analysis)
|
||||
--fg No-op alias (foreground is the only mode as of v2.4.0)
|
||||
--project Write brief into an trekbrief project folder (auto-indexed)
|
||||
|
|
@ -240,8 +235,8 @@ Then continue to the next phase inline.
|
|||
> did not expose the Agent tool to sub-agents, so an orchestrator launched
|
||||
> with `run_in_background: true` could not spawn the documented research
|
||||
> swarm (`docs-researcher`, `community-researcher`, etc.) and silently
|
||||
> degraded to single-context reasoning without WebSearch / Tavily / WebFetch.
|
||||
> As of CC 2.1.172 sub-agents can spawn sub-agents (up to 5 levels
|
||||
> degraded to single-context reasoning without WebSearch / Tavily / WebFetch
|
||||
> / Gemini. As of CC 2.1.172 sub-agents can spawn sub-agents (up to 5 levels
|
||||
> deep), so that block no longer holds — a delegated redesign is under
|
||||
> evaluation (see `docs/cc-upgrade-2.1.181-decision-matrix.md`, W1/CC-26).
|
||||
> Until then, running the phases inline in main context keeps the swarm
|
||||
|
|
@ -333,13 +328,9 @@ changes nothing (SC1). Keep the native-swarm anchors intact ("in parallel",
|
|||
|
||||
1. **Coarse pre-gate (best-effort, NOT a trust signal).** `Bash: claude --version`;
|
||||
parse the leading `X.Y.Z` (e.g. from `2.1.196 (Claude Code)`) and compare
|
||||
numerically against **both ends** of the supported window
|
||||
`2.1.154 <= version < 2.1.218` — split each on `.` and compare major, then minor,
|
||||
numerically against `2.1.154` — split each on `.` and compare major, then minor,
|
||||
then patch as integers (do NOT string-compare; lexical comparison mis-orders
|
||||
multi-digit patch numbers). If the version is `< 2.1.154` (below the
|
||||
dynamic-workflows floor), OR `>= 2.1.218` (the ceiling: `/deep-research` starts
|
||||
only when the operator invokes it, and the Skill tool refuses a model invocation
|
||||
with `disable-model-invocation` — see step 4's reason token), OR if
|
||||
multi-digit patch numbers). If the version is `< 2.1.154`, OR if
|
||||
`disableWorkflows: true` / `CLAUDE_CODE_DISABLE_WORKFLOWS=1` is set, skip to the
|
||||
fallback (step 4). **If `claude` is not on PATH inside the Bash tool (possible
|
||||
under `claude -p`) or the version cannot be parsed, treat the pre-gate as
|
||||
|
|
@ -356,18 +347,14 @@ changes nothing (SC1). Keep the native-swarm anchors intact ("in parallel",
|
|||
`/deep-research` report actually landed in context — substantive findings with
|
||||
citations, not an empty/denied/errored turn and not bare error text. This check
|
||||
must be **robust to all failure manifestations** (workflow disabled, approval
|
||||
denied, runtime error, empty output, or the Skill tool refusing with
|
||||
`disable-model-invocation` on CC 2.1.218+), because the disabled-headless
|
||||
behavior is undocumented: no recognizable cited report in context → fall back,
|
||||
regardless of how the failure surfaces.
|
||||
denied, runtime error, empty output), because the disabled-headless behavior is
|
||||
undocumented: no recognizable cited report in context → fall back, regardless of
|
||||
how the failure surfaces.
|
||||
|
||||
4. **On no real report (fallback):** set `effective_engine = swarm`, run the swarm
|
||||
blocks below, and **log the fallback at this decision point** — print
|
||||
`Engine: deep-research → swarm (fallback: <reason>)` and carry the reason into the
|
||||
Phase-8 Present summary and the brief's `## Executive Summary`. Known reason
|
||||
tokens: `disable-model-invocation` (CC >= 2.1.218 — the expected reason on any
|
||||
current Claude Code), `version-below-floor`, `workflows-disabled`,
|
||||
`no-cited-report`. **NEVER fabricate
|
||||
Phase-8 Present summary and the brief's `## Executive Summary`. **NEVER fabricate
|
||||
or synthesize a substitute report** — a structurally-valid-but-invented brief
|
||||
passes the structure-only validator and silently poisons `/trekplan`; that is the
|
||||
worst outcome of this feature.
|
||||
|
|
@ -391,6 +378,11 @@ Launch the new research-specialized agents:
|
|||
For each external agent, pass: the research question, specific dimensions to
|
||||
investigate, and any context from the interview.
|
||||
|
||||
### Bridge agent (scope = both or external, if enabled)
|
||||
|
||||
Launch `gemini-bridge` with the research question. Do NOT include findings from
|
||||
other agents — the value of Gemini is independence.
|
||||
|
||||
### Launch rules
|
||||
|
||||
- Launch ALL selected agents **in parallel** in a single message
|
||||
|
|
@ -751,7 +743,7 @@ Present a summary to the user:
|
|||
**Engine (effective):** {swarm | deep-research}{, with fallback reason if it fell back}
|
||||
**Confidence:** {overall confidence 0.0-1.0}
|
||||
**Dimensions:** {N} researched
|
||||
**Agents:** {N} local + {N} external
|
||||
**Agents:** {N} local + {N} external + {gemini: used | unavailable | skipped}
|
||||
|
||||
### Key Findings
|
||||
- {Finding 1}
|
||||
|
|
@ -796,7 +788,7 @@ Record format (one JSON line):
|
|||
"unique_sources": {N},
|
||||
"agents_local": {N},
|
||||
"agents_external": {N},
|
||||
"gemini_used": false,
|
||||
"gemini_used": {true|false},
|
||||
"confidence": {0.0-1.0},
|
||||
"contradictions": {N},
|
||||
"open_questions": {N}
|
||||
|
|
@ -887,16 +879,15 @@ commands surface, don't re-enforce.
|
|||
|
||||
When `phase_signal_result.effort == 'high'` for the `research` phase,
|
||||
run the FULL swarm regardless of normal triggering rules: 5 local
|
||||
agents + 4 external agents, AND force `contrarian-researcher` to
|
||||
always-on. Normally `contrarian-researcher` triggers conditionally when
|
||||
a leading recommendation is emerging from initial agents; in
|
||||
high-effort mode it runs unconditionally so the final brief always
|
||||
carries an adversarial counter-evidence pass.
|
||||
|
||||
The former third-party second-opinion bridge pass is **unavailable,
|
||||
skipped** (dropped in v5.10.1 by operator directive). High effort no
|
||||
longer has a bridge arm; `gemini_used` is a vestigial stats field
|
||||
pinned to `false`.
|
||||
agents + 4 external agents + 1 bridge agent, AND force
|
||||
`contrarian-researcher` AND `gemini-bridge` to always-on. Normally
|
||||
`contrarian-researcher` triggers conditionally when a leading
|
||||
recommendation is emerging from initial agents; in high-effort mode it
|
||||
runs unconditionally so the final brief always carries an adversarial
|
||||
counter-evidence pass. Similarly, `gemini-bridge` normally activates on
|
||||
significant architectural questions or when triangulation value is
|
||||
high; in high-effort mode it runs unconditionally to provide an
|
||||
independent second opinion.
|
||||
|
||||
High effort additionally arms the Phase 5 bounded follow-up loop (max 3
|
||||
turns per under-illuminated dimension, budgeted by
|
||||
|
|
@ -943,7 +934,7 @@ Low effort: inline research only, no agent swarm (existing
|
|||
local finding from steering an external agent's question. Attributing the bias
|
||||
risk to it left that risk with no control while the text read as though it had
|
||||
one.
|
||||
- **Graceful degradation:** If MCP tools are unavailable (Tavily, MS Learn),
|
||||
- **Graceful degradation:** If MCP tools are unavailable (Tavily, Gemini, MS Learn),
|
||||
proceed with available tools and note limitations in brief metadata.
|
||||
- **Cost:** Model resolution at Agent-spawn sites is a three-layer fallback:
|
||||
brief `phase_signals[<phase>].model` > `profile.phase_models[<phase>]` >
|
||||
|
|
|
|||
|
|
@ -249,37 +249,6 @@ do not feed unvalidated findings to the coordinator.
|
|||
In `quick` mode, launch only `code-correctness-reviewer`. The Executive
|
||||
Summary will note the brief-conformance pass was skipped.
|
||||
|
||||
### Reviewer accounting — every expected reviewer MUST report
|
||||
|
||||
Write down the expected reviewer set BEFORE the spawn: both reviewers in
|
||||
default mode, `code-correctness-reviewer` alone in `quick` mode. After the
|
||||
spawn, account for each one by name.
|
||||
|
||||
**Zero findings from a silent reviewer is indistinguishable from zero findings
|
||||
from a clean diff** — unless you check. A reviewer is *accounted for* only when
|
||||
it returned a payload that validated. Three ways it fails to:
|
||||
|
||||
| Failure | Handling |
|
||||
|---------|----------|
|
||||
| Output fails the schema after the 2 bounded re-asks | STOP (already specified above) |
|
||||
| Returned no final message at all | Re-ask that reviewer **once**. Still nothing → STOP. |
|
||||
| Was never launched (spawn error, wrong mode) | STOP. |
|
||||
|
||||
**On STOP: name the reviewer and the failure, and do not proceed to Phase 6.**
|
||||
Do not let the coordinator compute a verdict over a review one of whose
|
||||
reviewers never spoke — the count would be complete-looking and wrong. This is
|
||||
the same shape as the schema branch above ("do not feed unvalidated findings to
|
||||
the coordinator"), applied to the other two ways a reviewer can go missing.
|
||||
|
||||
A reviewer that ran but never delivered is most often the return-channel
|
||||
defect: check `~/.claude/projects/<proj>/<session>/subagents/agent-*.jsonl` for
|
||||
its final assistant block before re-asking, and confirm no `name` parameter was
|
||||
passed at the spawn (see the warning at the top of this phase).
|
||||
|
||||
If you proceed anyway under an explicit operator instruction, pass the expected
|
||||
set to the coordinator as `expectedReviewers` so the missing reviewer at least
|
||||
forbids `ALLOW` (`lib/review/coordinator-contract.mjs`, `missing_reviewers`).
|
||||
|
||||
## Phase 6 — Coordinator dedup + verdict
|
||||
|
||||
Launch `review-coordinator` (Agent tool) with the merged findings array
|
||||
|
|
@ -290,20 +259,10 @@ The coordinator runs the 4-pass process documented in
|
|||
|
||||
1. **Dedup** by `(file, line, rule_key)` triplet.
|
||||
2. **HubSpot Judge filters** — Succinctness, Accuracy, Actionability.
|
||||
3. **Cloudflare reasonableness** — remove speculative or catalogue-violating
|
||||
3. **Cloudflare reasonableness** — drop speculative or catalogue-violating
|
||||
findings (skipped in `quick` mode).
|
||||
4. **Verdict** — BLOCK / WARN / ALLOW per the threshold table.
|
||||
|
||||
**Fail-closed.** Every removal in Pass 2 and Pass 3 is either
|
||||
**dropped** (the test refuted the finding as a claim about this codebase) or
|
||||
**unverified** (the finding was removed without its claim ever being settled).
|
||||
A non-empty `unverified` bucket forbids `ALLOW`; the verdict becomes `WARN` and
|
||||
the Executive Summary's first sentence must say why. The fail-closed rule never
|
||||
raises a verdict — it only withholds the clean one. Fate table, reason
|
||||
vocabulary, and the `allow_blocked_by` field: `agents/review-coordinator.md`
|
||||
§*Suppression is two-valued*, mirrored deterministically in
|
||||
`lib/review/coordinator-contract.mjs`.
|
||||
|
||||
The coordinator's output is the full review.md content — frontmatter +
|
||||
body sections + trailing JSON block. Do NOT re-run the reviewers based
|
||||
on the coordinator's output.
|
||||
|
|
|
|||
|
|
@ -1,295 +0,0 @@
|
|||
# BRIEF — external assessment (VURDERING-V2 §5.12), verified against this repo
|
||||
|
||||
**Status:** assessment complete. **No implementation in this session** (scope-guard).
|
||||
**Source under review:** `claude-playlist-corpus/docs/VURDERING-V2.md` §5.12 (+ §3 G1–G8, §4),
|
||||
transferred via that repo's `docs/OVERFORING-V2.md` §5. It is an external recommendation
|
||||
built on 442 video analyses; its repo facts come from a subagent survey (2026-07-17) that
|
||||
is **not** re-verified and in which at least one error was already demonstrated.
|
||||
|
||||
**Method.** Every asserted GAP is marked **BEKREFTET** (confirmed) / **AVKREFTET**
|
||||
(refuted) / **ENDRET** (true in altered form) only after checking it against code in this
|
||||
repo. Every Claude Code *feature* claim is checked against the OKF bundle at
|
||||
`claude-code-llm-wiki/bundle/`, index-first — **where V2 and the bundle disagree, the
|
||||
bundle wins**. Absences are positive-controlled before being reported (a grep that finds
|
||||
nothing is a measurement, not a fact).
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
Of six recommended tiltak, **one is a real defect** (fail-open verdict computation),
|
||||
**one is unmeasured rather than unresolved** (the Workflow port), **three are already
|
||||
decided or already partly built**, and **one is a cheap confirmed gap**.
|
||||
|
||||
The single most consequential correction: V2 presents the Workflow port as the answer to
|
||||
this repo's "main-context relief is unmeasured" caveat. The bundle confirms the
|
||||
*mechanism*, but **this repo has still never measured main-context relief on a large
|
||||
fan-out** — the one bake-off that exists measured *subagent* tokens and says so. The port
|
||||
is therefore neither refuted nor justified. The next step is a measurement, not a port,
|
||||
and the measurement design already exists in `docs/T1-cc26-delegated-orchestration.md`.
|
||||
|
||||
---
|
||||
|
||||
## Tiltak 1 — Port orchestration to dynamic Workflows
|
||||
|
||||
**Verdict: split per command. Do not issue one global verdict.**
|
||||
|
||||
### `/trekreview` — **ENDRET** (already ported, and measured)
|
||||
|
||||
The port exists: opt-in `--workflow`, `scripts/trekreview-armB.workflow.mjs`, documented at
|
||||
`commands/trekreview.md:270` (*§ Phase 5–6 via the Workflow substrate*). The S10 bake-off
|
||||
(`docs/T2-bakeoff-results.md`, ≥3 runs/arm) found it **fidelity-equivalent**: verdict-match
|
||||
1.0, issue-coverage 100%, `(file,rule_key)` jaccard ≥ within-arm. It stays opt-in because
|
||||
the Workflow tool raises the consumer floor to Claude Code 2.1.154+, and the prose path
|
||||
keeps the lower floor.
|
||||
|
||||
### `/trekplan`, `/trekresearch` — **BEKREFTET as unported, UNMEASURED as beneficial**
|
||||
|
||||
Both still orchestrate their swarms from main context (`commands/trekplan.md:406`,
|
||||
`commands/trekresearch.md:388`). That part of V2 is correct.
|
||||
|
||||
What is **not** established is V2's stated effect — that Workflow answers the
|
||||
main-context-relief caveat. Two measurements exist in this repo and **neither measures it**:
|
||||
|
||||
| Measurement | What it actually measured | Bearing on the relief claim |
|
||||
|---|---|---|
|
||||
| `docs/T2-bakeoff-results.md:154` | **Subagent** tokens: Arm A median 92.8k vs Arm B 96.9k ⇒ **+4.4%** | None. The same line states Arm A's main-context hand-orchestration cost is **uncounted**. |
|
||||
| `docs/T1-synthesis-poc-results.md` | Δ **main-context** ≈ **0%** — for delegating *Phase 7 synthesis alone* | Negative, but out of scope: that doc itself names a **Phase-5 redesign** as the prerequisite for any relief. |
|
||||
|
||||
So the Phase-5 swarm — the case V2 is actually recommending — has never been measured in
|
||||
either arm, in either command.
|
||||
|
||||
**The measurement already has a design.** `docs/T1-cc26-delegated-orchestration.md:118`
|
||||
defines the gate metric (Δ main-context tokens resident in the main session at
|
||||
plan-complete, from stream-json `usage`) with pre-registered bars: adopt at ≥30%, reject
|
||||
below 15%. Running Arm A vs Arm B once against the trekplan swarm settles this.
|
||||
|
||||
**Cost framing must be corrected.** The bundle states plainly that a workflow run "can use
|
||||
meaningfully more tokens than working through the same task in conversation"
|
||||
(`bundle/concepts/docs-en-workflows.md:364`) — consistent with the measured +4.4%. The port
|
||||
cannot be justified as a cost saving. **Main-context headroom is the only defensible
|
||||
motive, and it is exactly the unmeasured quantity.**
|
||||
|
||||
**Recommendation:** measure before porting. Treat V2's tiltak 1 as a *hypothesis with a
|
||||
ready-made experiment*, not as an approved change.
|
||||
|
||||
---
|
||||
|
||||
## Tiltak 2 — Codify the holdout rules
|
||||
|
||||
**Verdict: (a) and (b) satisfied in practice but uncodified · (c) BEKREFTET, a real defect ·
|
||||
(d) ENDRET.** This is the tiltak with genuine substance.
|
||||
|
||||
### (a) Reviewers must never be forks — **AVKREFTET as a live defect, BEKREFTET as uncodified**
|
||||
|
||||
Reviewers are spawned as plain subagents (`voyage:brief-conformance-reviewer`,
|
||||
`voyage:code-correctness-reviewer`). `subagent_type: "fork"` appears nowhere in `commands/`
|
||||
or `agents/`; the only `subagent_type` in the tree is `"Explore"` at
|
||||
`commands/trekplan.md:501`, an exploration spawn, not a review spawn. The bundle confirms
|
||||
the rule is real — a forked subagent "inherits your full conversation context instead of
|
||||
starting fresh" (`bundle/concepts/docs-en-agents.md:37`). Voyage satisfies it by habit;
|
||||
nothing written forbids a future edit from breaking it.
|
||||
|
||||
### (b) Explicit "you shall not see" list — **AVKREFTET as a live defect, BEKREFTET as uncodified**
|
||||
|
||||
Phase 5 feeds reviewers exactly four things: the Phase-3 unified diff, the triage map, the
|
||||
brief path, and the rule catalogue (`commands/trekreview.md`, Phase 5 input list).
|
||||
Implementation plans and commit messages are **not** fed; `git log` appears only to compute
|
||||
the diff range and to print a suggested narrowing command (`commands/trekreview.md:132`,
|
||||
`:181`). Again: correct today, unprotected tomorrow.
|
||||
|
||||
### (c) Fail-closed verdicts — **BEKREFTET. This is the one real defect.**
|
||||
|
||||
`agents/review-coordinator.md` computes the verdict mechanically from the findings that
|
||||
*survive* filtering:
|
||||
|
||||
- **Pass 2** (HubSpot judge) and **Pass 3** (Cloudflare reasonableness) **drop** findings
|
||||
that fail their tests (`agents/review-coordinator.md:87`, `:101`).
|
||||
- **Pass 4** counts the survivors: `BLOCKER ≥ 1 → BLOCK`; else `MAJOR ≥ 1 → WARN`; else
|
||||
`ALLOW` (`:132`). "Verdict is mechanical — never override."
|
||||
|
||||
There is no third state. Three consequences follow:
|
||||
|
||||
1. **A finding that cannot be substantiated is dropped, and a dropped finding contributes
|
||||
zero — which moves the verdict toward ALLOW.** Pass 3's non-existent-file test ("`file`
|
||||
does not exist in the working tree AND the diff does not show it as `(new file)`") has
|
||||
no inconclusive branch: unresolvable and refuted are treated identically. This is
|
||||
structurally the failure V2 attributes to its source case, where "not E2E testable"
|
||||
passed as approved.
|
||||
2. **A reviewer that returns nothing produces the same result.** Phase 5 has no
|
||||
empty-return handling — grep for `empty|no findings|did not return|abort` over
|
||||
`commands/trekreview.md` returns 0 (positive control: `Phase 5` = 7 hits, `reviewer` =
|
||||
24 hits, so the file and the query are both live). Zero findings from a silent reviewer
|
||||
is indistinguishable from zero findings from a clean diff. This is the same surface as
|
||||
the idle-agent hole recorded in STATE's open decisions.
|
||||
3. **One fail-closed branch already exists, and it is the pattern to copy.** When reviewer
|
||||
output fails schema validation, Phase 5 allows 2 bounded re-asks and then *stops*: "do
|
||||
not feed unvalidated findings to the coordinator." That is the correct shape, applied to
|
||||
one failure mode only.
|
||||
|
||||
**The defect is codified, not merely prose — which makes it testable.** The four passes
|
||||
exist deterministically in `lib/review/coordinator-contract.mjs`, and
|
||||
`computeVerdict(findings)` (`:184`–`:193`) counts *only* the findings handed to it.
|
||||
`runContract` hands it `reasoned.kept` (`:206`); `suppressed` and `skipped` are returned in
|
||||
the result object but **carry no weight in the verdict**. So a dropped finding is
|
||||
arithmetically identical to a finding that never existed.
|
||||
|
||||
**Minimal fix shape (not implemented here):** an `unverified` bucket that is neither kept
|
||||
nor dropped, plus one rule — *a non-empty `unverified` bucket forbids ALLOW* — and a
|
||||
Phase-5 check that a reviewer actually returned. Both are additive; neither touches the
|
||||
severity catalogue or the existing thresholds. Because the logic is deterministic ("No LLM,
|
||||
no network, no time, no randomness", `:23`) and already has
|
||||
`tests/lib/coordinator-contract.test.mjs`, this can be driven test-first under the Iron
|
||||
Law — a failing test asserting that an unverifiable BLOCKER-severity finding cannot yield
|
||||
ALLOW is writable before any production change.
|
||||
|
||||
### (d) Presume-failure framing — **ENDRET**
|
||||
|
||||
Both reviewers are already adversarially framed: "Adversarial reviewer" in each
|
||||
`description`, "You never praise", "You never say 'looks good'"
|
||||
(`agents/code-correctness-reviewer.md:36`, `agents/brief-conformance-reviewer.md:35`). What
|
||||
is absent is specifically the presume-failure formulation V2 names ("this agent was lazy —
|
||||
find out why"). This is a wording change with no measurement behind it in the corpus; treat
|
||||
it as optional polish, not a gap.
|
||||
|
||||
---
|
||||
|
||||
## Tiltak 3 — Lightweight lane — **ENDRET** (narrower gap than stated, already tracked)
|
||||
|
||||
A lightweight path exists: `--quick` on both `/trekbrief` and `/trekplan`, and
|
||||
`/trekplan --quick` already skips the Phase-5 exploration swarm — "Skip agent swarm; use
|
||||
lightweight Glob/Grep scan and go directly to planning + adversarial review"
|
||||
(`commands/trekplan.md:122`). The same code-path is reachable a second way: a brief
|
||||
carrying `effort == 'low'` activates it without the flag (`:885`). What it does **not** do
|
||||
is bypass the 2.2 ceremony —
|
||||
`commands/trekbrief.md:126` states the framing question "is asked even in `--quick` mode",
|
||||
and a 2.2 brief still requires `framing` and a `## TL;DR` (`:472`, `:474`).
|
||||
|
||||
So the accurate gap is not "no lightweight path" but "the lightweight path still pays the
|
||||
2.2 toll". Already carried as open operator decision 1 in STATE — this is not a discovery.
|
||||
|
||||
---
|
||||
|
||||
## Tiltak 4 — Definition-of-done as an object — **BEKREFTET**
|
||||
|
||||
`commands/trekbrief.md` contains no notion of evidence, proof, verifier, residual risk, or
|
||||
next-step owner (grep for `evidence|proof|screenshot|verifier|residual risk|owner` → 0;
|
||||
positive control: `Success Criteri` = 5 hits). The gap is real.
|
||||
|
||||
**Cost note V2 does not carry:** the brief schema is **Handover 1**, a public contract
|
||||
(`docs/HANDOVER-CONTRACTS.md`). Adding required DoD fields is a breaking change for
|
||||
downstream consumers, i.e. a `brief_version` bump with a gate, not an edit. That moves it
|
||||
out of "cheap win" and into planned work.
|
||||
|
||||
---
|
||||
|
||||
## Tiltak 5 — Prune, don't build — **mostly ALREADY DECIDED**
|
||||
|
||||
- **`synthesis-agent`**: already dormant and labelled as such (`agents/synthesis-agent.md:3`),
|
||||
with the measurement that justified it (`docs/T1-synthesis-poc-results.md`, Δ≈0,
|
||||
DECLINED). Retiring versus keeping was decided in favour of keeping it as a
|
||||
re-measurable building block. V2 recommends a decision that has been made and recorded.
|
||||
- **24 all-opus agents**: 24 of 24 agent files carry `model: opus` (verified by count).
|
||||
This is an **operator pin**, not drift — commit `40d8742` "pin all sub-agents to Opus
|
||||
permanently (operator request)" — and the reconsideration V2 asks for already happened
|
||||
and is written down in `docs/voyage-vs-cc-balance-analysis.md` §10. Re-opening it is an
|
||||
operator decision, not an analysis task.
|
||||
- **`gemini-bridge`**: **BEKREFTET**, and the cheapest real item in this tiltak. It is
|
||||
already flagged `THIN_WRAP` / `DROP→NATIVE` in the same analysis (§V09, lines 81, 121,
|
||||
149) *and* it is broken at the engine (the `gemini-mcp` server fails deterministically on
|
||||
an SDK/API mismatch). Fix-or-drop is decidable today.
|
||||
|
||||
---
|
||||
|
||||
## Tiltak 6 — LSP as a harness component — **BEKREFTET**
|
||||
|
||||
`LSP` returns 0 hits across `commands/`, `agents/`, `lib/`, `docs/`. Voyage navigates with
|
||||
Glob/Grep, as V2 says. One qualifier: LSP is a harness-level capability available to
|
||||
agents, not a voyage feature — adoption is mostly permitting and prompting agents to use
|
||||
it, not code in this repo. Value is highest in large target repos, which is where voyage's
|
||||
exploration swarm actually runs.
|
||||
|
||||
---
|
||||
|
||||
## Bundle-gap
|
||||
|
||||
Checked against `bundle/concepts/docs-en-workflows.md`
|
||||
(`source_sha 363819ed9ec325275ca22023f6bb6b98fbbf6fcc12db0667478c5207e117751b`,
|
||||
timestamp 2026-08-15), plus `docs-en-agents.md` and `docs-en-sub-agents.md`.
|
||||
|
||||
**Confirmed by the bundle:**
|
||||
- The **2.1.154 floor** for dynamic workflows, on paid plans.
|
||||
- The **main-context relief mechanism**: "A workflow script holds the loop, the branching,
|
||||
and the intermediate results itself, so Claude's context holds only the final answer"
|
||||
(`:38`); "Intermediate results stay in script variables instead of landing in Claude's
|
||||
context" (`:316`). V2's mechanism claim is sound — it is the *magnitude* that is unmeasured.
|
||||
- **Per-stage model routing** (`:375`–`:378`).
|
||||
- **Cost caps**: agent caps, size guideline, and a `Large workflow` warning above 25 agents
|
||||
or 1.5M projected tokens (`:368`).
|
||||
|
||||
**Where V2 overstates, and the bundle wins:**
|
||||
- **"tokenbudsjetter" (token budgets).** The page documents *caps and size guidelines*, not
|
||||
a token budget — `budget` returns 0 hits on the page. V2's warning to "set an explicit
|
||||
token budget" has no documented primitive to point at in the official docs.
|
||||
- **"automatisk retry".** 0 hits. Not documented on the page.
|
||||
|
||||
**Limitation V2 omits, material for this repo:** resumability is **session-scoped** —
|
||||
"Resume works within the same Claude Code session. If you exit Claude Code while a workflow
|
||||
is running, the next session starts the workflow fresh" (`:360`). Further, stopping mid
|
||||
fan-out re-runs every agent that started after the stopped one (`:354`). Voyage is
|
||||
explicitly a *multi-session* tool, so resumability is a weaker argument here than V2 implies.
|
||||
|
||||
**Gap proper (feedback toward wiki v1.0):**
|
||||
1. The mirrored docs describe no programmatic **token-budget** primitive for workflows,
|
||||
while the Workflow tool's own runtime surface does expose a budget derived from an
|
||||
operator token directive. If that surface is real and stable, it is missing from the
|
||||
mirror. Flagged as a discrepancy, not asserted as a doc error.
|
||||
2. Nothing in the mirrored pages quantifies what a workflow's **final return** costs the
|
||||
orchestrating session. The relief mechanism is described qualitatively only, so a
|
||||
consumer cannot size the benefit from the docs alone — which is precisely why the
|
||||
measurement below cannot be replaced by reading.
|
||||
|
||||
---
|
||||
|
||||
## Recommended order (operator decides; nothing started)
|
||||
|
||||
1. **Fail-closed coordinator (tiltak 2c)** — smallest surface, highest value, no
|
||||
dependencies, and it closes a defect rather than adding a feature.
|
||||
2. **Measure Δ main-context on the `/trekplan` Phase-5 swarm (tiltak 1)** — Arm A vs Arm B
|
||||
against the pre-registered bars in `docs/T1-cc26-delegated-orchestration.md`. The port
|
||||
decision follows the number; it does not precede it.
|
||||
3. **`gemini-bridge` fix-or-drop (tiltak 5)** — already analysed, currently broken, cheap.
|
||||
|
||||
**Deferred with reasons:** tiltak 3 (already open operator decision 1) · tiltak 4 (requires
|
||||
a `brief_version` bump against a public contract) · tiltak 5's opus/synthesis items
|
||||
(decided; operator-pinned) · tiltak 6 (harness-level, not repo code).
|
||||
|
||||
---
|
||||
|
||||
## Verification log
|
||||
|
||||
| Claim | How verified |
|
||||
|---|---|
|
||||
| `/trekreview` Workflow port exists and was measured | `commands/trekreview.md:270`; `docs/T2-bakeoff-results.md` §Full run |
|
||||
| +4.4% is **subagent** tokens, main context uncounted | `docs/T2-bakeoff-results.md:154` (verbatim) |
|
||||
| Δ main-context ≈ 0 applies to Phase 7 only | `docs/T1-synthesis-poc-results.md:1`, `:100` |
|
||||
| Measurement design already exists | `docs/T1-cc26-delegated-orchestration.md:118`, `:125` |
|
||||
| Reviewers are not forks | `grep subagent_type\|fork commands/ agents/` → only `Explore` at `commands/trekplan.md:501` |
|
||||
| Plans/commit messages not fed to reviewers | Phase 5 input list; `commands/trekreview.md:132`, `:181` |
|
||||
| Verdict is computed from survivors only | `agents/review-coordinator.md:87`, `:101`, `:132` |
|
||||
| Same rule codified deterministically in `lib/` | `lib/review/coordinator-contract.mjs:184`–`:193`, `:206`; existing test `tests/lib/coordinator-contract.test.mjs` |
|
||||
| No empty-return handling in Phase 5 | grep → 0, positive control `Phase 5`=7, `reviewer`=24 |
|
||||
| Existing fail-closed branch on schema failure | `commands/trekreview.md`, Phase 5 bounded-retry paragraph |
|
||||
| `--quick` does not bypass 2.2 | `commands/trekbrief.md:126`, `:472`, `:474` |
|
||||
| No DoD fields in trekbrief | grep → 0, positive control `Success Criteri`=5 |
|
||||
| 24/24 agents pinned opus, by operator | file count 24/24; `git log 40d8742` |
|
||||
| gemini-bridge already flagged for drop | `docs/voyage-vs-cc-balance-analysis.md:81`, `:121`, `:149` |
|
||||
| No LSP usage | `grep -rn LSP commands/ agents/ lib/ docs/` → 0 |
|
||||
| Workflow floor, relief mechanism, model routing, caps | `bundle/concepts/docs-en-workflows.md:19`, `:38`, `:316`, `:375`, `:368` |
|
||||
| No documented token budget or retry in the bundle | `grep -i budget\|retry` on that page → 0 relevant |
|
||||
| Resume is session-scoped | `bundle/concepts/docs-en-workflows.md:354`, `:360` |
|
||||
| Fork inherits full conversation context | `bundle/concepts/docs-en-agents.md:37` |
|
||||
|
||||
**Not verified / stated as unverified:** V2's underlying video-corpus claims (the StrongDM
|
||||
bad-merge case, the 5.5-hour bake-off, the "$200 plan in 30 minutes" workflow) were not
|
||||
independently checked — they are cited here as V2's evidence, not as this repo's findings.
|
||||
|
|
@ -135,17 +135,6 @@ reused.
|
|||
|
||||
## 6. Cheaper PoC (audit-recommended, preferred over the full bake-off)
|
||||
|
||||
> **STATUS: RUN AND DECLINED (S12, 2026-06-18) — do not re-order as new work.**
|
||||
> Result: **Δ main-context (faithful flow) = 0.0%**, NEGATIVE against the 15% keep-floor.
|
||||
> Full method, fixtures and BASE-sweep: `docs/T1-synthesis-poc-results.md`;
|
||||
> reproduce with `node scripts/synthesis-measure.mjs`.
|
||||
> The finding is **structural, not stochastic**, so re-running it cannot change the number:
|
||||
> Phase 5 spawns the exploration swarm **foreground** (`commands/trekplan.md:158,338-341` —
|
||||
> "foreground is the only mode as of v2.4.0"), so the 6-10 outputs are **already resident in
|
||||
> main** before Phase 7 begins. Delegating only the Phase-7 digest returns *on top of* them
|
||||
> and evicts nothing. A saving is reachable only by also moving Phase-5 delivery off-main —
|
||||
> which is §5, not §6. `agents/synthesis-agent.md` therefore ships **dormant**.
|
||||
|
||||
Per `subagent-delegation-audit.md` §Recommendation, the lowest-risk way to test the delegation
|
||||
*premise* is **not** a wholesale orchestrator rewrite but **one narrow synthesis-agent**
|
||||
(intervention #1/#3): delegate only trekplan Phase 7 synthesis (the heaviest single inline read —
|
||||
|
|
@ -180,17 +169,5 @@ promising orchestration-substrate question and is untouched by this.
|
|||
only matters if a future nested pipeline approaches 5.
|
||||
2. Classifier behaviour for a *purposeful* swarm under `auto`/`bypassPermissions` is unverified
|
||||
(F4) — must be in the §5/§6 test matrix before any delegated spawn ships to a headless path.
|
||||
3. ~~The §6 synthesis-agent PoC and the §5 full bake-off are both designed but unbuilt.~~
|
||||
**CORRECTED 2026-09-03 (S95).** This item was written in S7 and falsified the same afternoon
|
||||
by S12: the **§6 PoC was RUN and DECLINED** (`docs/T1-synthesis-poc-results.md` — Δ_faithful
|
||||
= 0.0%, NEGATIVE). Only the **§5 full bake-off** remains unbuilt. The stale wording is what
|
||||
caused the §6 PoC to be re-ordered as new work on 2026-09-02; it is struck rather than
|
||||
deleted so the re-order has a traceable cause.
|
||||
4. **Environment premise updated (measured 2026-09-03, CC 2.1.259).** S12 recorded the live run
|
||||
as environment-blocked for want of an `ANTHROPIC_API_KEY`. That half no longer holds:
|
||||
`claude -p --output-format stream-json` runs on subscription auth with no API key and emits
|
||||
per-turn `usage` plus a `subagent_stats` block (spawned / max_depth / refused) that did not
|
||||
exist in June — a ready-made classifier-interference counter for §5's feasibility guard.
|
||||
S12's *other* blocker stands unmeasured: the installed plugin is the cache copy, so a
|
||||
repo-local agent is invisible to `claude -p`. This changes nothing about the §6 verdict,
|
||||
which is structural (see §6 status), and only lowers the cost of §5.
|
||||
3. The §6 synthesis-agent PoC and the §5 full bake-off are both **designed but unbuilt** — ready
|
||||
to run if CC-26 is greenlit toward delegation.
|
||||
|
|
|
|||
|
|
@ -167,18 +167,10 @@ arm produces correct final text at 66 lines and at 3730 lines alike, and only
|
|||
delivery fails, identically at both. The 38 recovered findings were re-used
|
||||
instead of re-run.
|
||||
|
||||
**A second external measurement closes the other half (akashic-intelligence
|
||||
S27, commit `f168630`).** The gap left above was that all four S25 cells were
|
||||
named, so the *returning* (unnamed) arm had no measurement above 66 lines.
|
||||
S27 supplies one: denominator **2 of 2 unnamed agents**, against a 4370-line
|
||||
plan; both returned, 30449 B and 10989 B, both valid JSON. The unnamed arm
|
||||
therefore returns at full scale as observed fact, not as inference.
|
||||
|
||||
Two caveats, kept at the strength akashic itself stated them. The measurement
|
||||
was taken by the repo that owns the finding, not by an independent third party.
|
||||
And byte-identity between the returned string and the file on disk was not
|
||||
proven — what is established is that a well-formed result of that size arrived,
|
||||
not that it arrived unaltered.
|
||||
What it does **not** close: their four cells were all named, so the *returning*
|
||||
(unnamed) arm still has no measurement above 66 lines. That a plain subagent
|
||||
returns a 21 KB result at that scale is inferred, not observed. Stated as
|
||||
inferred.
|
||||
|
||||
Their PONG control also carries the same lesson as cell 1 above, in a third
|
||||
repo: S25 reported that agent as having "gone idle without sending PONG". It
|
||||
|
|
|
|||
|
|
@ -98,7 +98,7 @@ Which native Claude Code primitive each pipeline step runs on today, and the alt
|
|||
| **continue** | Inline reads `.session-state.local.json` → zero-confirm resume | CC `--resume` (transcript replay, not typed work-state → insufficient) |
|
||||
| **cross-cutting** | 8 hook scripts: `pre-bash` + `pre-write` guards, `pre-agent-cap` loop-bound enforcement, `post-bash` stats, `session-title`, `pre-`/`post-compact` flush, **`Stop`→OTEL** export | — |
|
||||
|
||||
¹ MCP per research agent: `docs-researcher` → Microsoft Learn + Tavily · `community-`/`security-`/`contrarian-researcher` → Tavily (+ WebSearch/WebFetch). Graceful degradation when an MCP server is absent.
|
||||
¹ MCP per research agent: `docs-researcher` → Microsoft Learn + Tavily · `community-`/`security-`/`contrarian-researcher` → Tavily (+ WebSearch/WebFetch) · `gemini-bridge` → Gemini Deep Research MCP. Graceful degradation when an MCP server is absent.
|
||||
|
||||
**Legend:** *production* = wired and active · *dormant* = shipped but wired to nothing (`synthesis-agent`) · *opt-in* = behind a flag (`--workflow`) · *not wired* = considered, deferred (delegated orchestrator, the `AskUserQuestion` brief engine).
|
||||
|
||||
|
|
|
|||
|
|
@ -20,13 +20,13 @@ Always interactive. Phase 3 is a section-driven completeness loop (no hard cap o
|
|||
| _(default)_ | Interview + research (local + external) + synthesis + brief (foreground) |
|
||||
| `--project <dir>` | Write brief to `{dir}/research/{NN}-{slug}.md` (auto-incremented) |
|
||||
| `--quick` | Interview (short) + inline research (no agent swarm) |
|
||||
| `--local` | Only codebase analysis agents (skip external research) |
|
||||
| `--local` | Only codebase analysis agents (skip external + Gemini) |
|
||||
| `--external` | Only external research agents (skip codebase analysis) |
|
||||
| `--fg` | No-op alias (foreground is default since v2.4.0) |
|
||||
| `--gates {true\|false}` | (v3.4.0) Boolean autonomy-gate flag; present → gating on. Policy (`gates_mode`) detailed under `## Autonomy mode` in `docs/operations.md`. |
|
||||
| `--min-brief-version <ver>` | (S18) Warn — never block — if an attached `--project` brief declares a version below `<ver>` (e.g. `2.2`), i.e. sidesteps framing enforcement |
|
||||
| `--profile <name>` | (v4.1.0) Model profile for the research phase. |
|
||||
| `--engine {swarm\|deep-research}` | (deep-research-engine) Opt-in external-research engine; `deep-research` delegates the external phase to Claude Code's built-in `/deep-research` workflow, which only works on `2.1.154 <= CC < 2.1.218` — from **2.1.218** `/deep-research` is operator-invoked only (the Skill tool refuses with `disable-model-invocation`), so the engine always falls back to `swarm`. Never hard-fails. Default `swarm`. |
|
||||
| `--engine {swarm\|deep-research}` | (deep-research-engine) Opt-in external-research engine; `deep-research` delegates the external phase to Claude Code's built-in `/deep-research` workflow (CC 2.1.154+), falls back to `swarm`. Default `swarm`. |
|
||||
|
||||
Flags combine: `--project <dir> --local`, `--external --quick`.
|
||||
|
||||
|
|
|
|||
|
|
@ -19,10 +19,8 @@
|
|||
// What IS implemented, purely: Pass 1 (triplet dedup → highest-severity-wins
|
||||
// survivor + conformance tiebreak + detail concat + raised_by provenance),
|
||||
// Pass 2 succinctness + actionability-presence, Pass 3 reasonableness
|
||||
// (citation / unknown-rule_key suppression, severity-mismatch correction),
|
||||
// Pass 4 verdict thresholds — fail-closed: a suppression that did not REFUTE
|
||||
// the finding, and a reviewer that never reported, forbid ALLOW (see
|
||||
// classifySuppression). No LLM, no network, no time, no randomness.
|
||||
// (citation / unknown-rule_key drop, severity-mismatch correction), Pass 4
|
||||
// verdict thresholds. No LLM, no network, no time, no randomness.
|
||||
//
|
||||
// Reuses: SEVERITY_VALUES / RULE_KEYS / getRule (rule-catalogue.mjs),
|
||||
// computeFindingId (finding-id.mjs, triplet), validateFindings
|
||||
|
|
@ -37,71 +35,6 @@ import { validateFindings } from './findings-schema.mjs';
|
|||
export const JUDGE_TITLE_MAX = 100;
|
||||
export const JUDGE_DETAIL_MAX = 800;
|
||||
|
||||
// ---- Suppression classification (fail-closed) --------------------------------
|
||||
//
|
||||
// A removal is `dropped` ONLY when the test refuted the finding as a claim
|
||||
// about this codebase. Every other removal is `unverified`: the coordinator
|
||||
// took the finding out of the count without ever establishing it was unreal,
|
||||
// so it may not be spent as evidence of a clean review.
|
||||
|
||||
/**
|
||||
* Reasons that REFUTE. `no-citation` is the only one this deterministic subset
|
||||
* can emit: a finding whose `file` is empty or whose `line` is negative names
|
||||
* no location, so it makes no checkable claim at all
|
||||
* (agents/review-coordinator.md Pass 3 — "Speculative 'code might break
|
||||
* somewhere' findings have no anchor").
|
||||
*
|
||||
* `accuracy:refuted` (Pass 2 Accuracy — a citation escaping the repo root) and
|
||||
* `file-existence:refuted` (Pass 3 — absent from both working tree and diff)
|
||||
* are emitted by the LLM coordinator, whose fs/judgement branches this module
|
||||
* excludes. They are declared here anyway: the vocabulary is owned in one
|
||||
* place so prose and lib cannot drift.
|
||||
*/
|
||||
export const REFUTING_REASONS = Object.freeze(new Set([
|
||||
'no-citation',
|
||||
'accuracy:refuted',
|
||||
'file-existence:refuted',
|
||||
]));
|
||||
|
||||
/**
|
||||
* The reason vocabulary on the unverified side. `file-existence:indeterminate`
|
||||
* is emitted by the LLM coordinator's Pass 3 (which runs the fs Glob this
|
||||
* module deliberately excludes); the vocabulary is owned here so prose and lib
|
||||
* cannot drift.
|
||||
*/
|
||||
export const UNVERIFIED_REASONS = Object.freeze([
|
||||
'succinctness:title',
|
||||
'succinctness:detail',
|
||||
'actionability:empty',
|
||||
'unknown-rule_key',
|
||||
'file-existence:indeterminate',
|
||||
]);
|
||||
|
||||
/**
|
||||
* Classify a suppression reason. Anything not declared refuting is
|
||||
* `unverified` — the default is fail-CLOSED, so a reason introduced later
|
||||
* without a decision cannot silently move the verdict toward ALLOW.
|
||||
* @param {string} reason
|
||||
* @returns {'refuted'|'unverified'}
|
||||
*/
|
||||
export function classifySuppression(reason) {
|
||||
return REFUTING_REASONS.has(reason) ? 'refuted' : 'unverified';
|
||||
}
|
||||
|
||||
/**
|
||||
* Tag a finding with its suppression reason and route it to the refuted
|
||||
* (`dropped`) or the `unverified` bucket.
|
||||
* @param {object} finding
|
||||
* @param {string} reason
|
||||
* @param {object[]} dropped
|
||||
* @param {object[]} unverified
|
||||
*/
|
||||
function suppress(finding, reason, dropped, unverified) {
|
||||
const tagged = { ...finding, suppressed_reason: reason };
|
||||
if (classifySuppression(reason) === 'refuted') dropped.push(tagged);
|
||||
else unverified.push(tagged);
|
||||
}
|
||||
|
||||
/**
|
||||
* Catalogue-tier rank of a severity: lower number = higher severity.
|
||||
* BLOCKER=0 … SUGGESTION=3; an unknown severity ranks last.
|
||||
|
|
@ -189,18 +122,12 @@ export function dedupByTriplet(findings) {
|
|||
* (title > 100 or detail > 800 chars) and actionability (recommended_action,
|
||||
* when present, must be a non-empty string). The imperative-verb test is
|
||||
* excluded (LLM judgement).
|
||||
*
|
||||
* Both tests read a `.length`; neither examines the claim, so neither can
|
||||
* establish the finding is unreal. Both therefore route to `unverified`.
|
||||
* `dropped` stays in the signature for the refuting Pass-2 filter this subset
|
||||
* excludes (Accuracy: a path-traversal escape IS a refutation).
|
||||
* @param {object[]} findings
|
||||
* @returns {{ kept: object[], dropped: object[], unverified: object[] }}
|
||||
* @returns {{ kept: object[], dropped: object[] }}
|
||||
*/
|
||||
export function judgeFilter(findings) {
|
||||
const kept = [];
|
||||
const dropped = [];
|
||||
const unverified = [];
|
||||
for (const f of findings) {
|
||||
const titleLen = (f.title ?? '').length;
|
||||
const detailLen = (f.detail ?? '').length;
|
||||
|
|
@ -211,38 +138,31 @@ export function judgeFilter(findings) {
|
|||
(typeof f.recommended_action !== 'string' || f.recommended_action.trim().length === 0)) {
|
||||
reason = 'actionability:empty';
|
||||
}
|
||||
if (reason) suppress(f, reason, dropped, unverified);
|
||||
if (reason) dropped.push({ ...f, suppressed_reason: reason });
|
||||
else kept.push(f);
|
||||
}
|
||||
return { kept, dropped, unverified };
|
||||
return { kept, dropped };
|
||||
}
|
||||
|
||||
/**
|
||||
* Pass 3 — Cloudflare reasonableness (deterministic subset): drop findings
|
||||
* with no citation (empty file / line < 0) or an unknown rule_key; CORRECT a
|
||||
* severity that does not match the catalogue tier (a correction, not a drop).
|
||||
* The fs file-existence glob is excluded (I/O) — its indeterminate branch is
|
||||
* prose-side, tokenised as `file-existence:indeterminate`.
|
||||
*
|
||||
* `no-citation` REFUTES (the finding names no location, so it makes no
|
||||
* checkable claim) and is dropped. `unknown-rule_key` does not: an ad-hoc key
|
||||
* is a real defect wearing the wrong label — v5.1.1 high-effort mode already
|
||||
* KEEPS these, normalised to PLAN_EXECUTE_DRIFT — so it routes to `unverified`.
|
||||
* The fs file-existence glob is excluded (I/O).
|
||||
* @param {object[]} findings
|
||||
* @returns {{ kept: object[], dropped: object[], unverified: object[] }}
|
||||
* @returns {{ kept: object[], dropped: object[] }}
|
||||
*/
|
||||
export function reasonablenessFilter(findings) {
|
||||
const kept = [];
|
||||
const dropped = [];
|
||||
const unverified = [];
|
||||
for (const f of findings) {
|
||||
if (typeof f.file !== 'string' || f.file.length === 0 ||
|
||||
(typeof f.line === 'number' && f.line < 0)) {
|
||||
suppress(f, 'no-citation', dropped, unverified);
|
||||
dropped.push({ ...f, suppressed_reason: 'no-citation' });
|
||||
continue;
|
||||
}
|
||||
if (!RULE_KEYS.has(f.rule_key)) {
|
||||
suppress(f, 'unknown-rule_key', dropped, unverified);
|
||||
dropped.push({ ...f, suppressed_reason: 'unknown-rule_key' });
|
||||
continue;
|
||||
}
|
||||
const rule = getRule(f.rule_key);
|
||||
|
|
@ -252,114 +172,44 @@ export function reasonablenessFilter(findings) {
|
|||
kept.push(f);
|
||||
}
|
||||
}
|
||||
return { kept, dropped, unverified };
|
||||
return { kept, dropped };
|
||||
}
|
||||
|
||||
/**
|
||||
* Pass 4 — compute the verdict from severity counts (after dedup + filtering).
|
||||
* BLOCKER ≥ 1 → BLOCK; else MAJOR ≥ 1 → WARN; else ALLOW.
|
||||
*
|
||||
* FAIL-CLOSED: ALLOW additionally requires that nothing is `unverified` and
|
||||
* that every expected reviewer reported. Neither ever RAISES a verdict — the
|
||||
* severity thresholds are untouched — they only forbid the clean one, so the
|
||||
* worst case of a false unverified is WARN plus a stated reason, never a
|
||||
* silent pass. Unverified findings are NOT counted into a severity tier: their
|
||||
* severity is reviewer-asserted and was never substantiated.
|
||||
*
|
||||
* @param {object[]} findings
|
||||
* @param {{ unverified?: object[], missingReviewers?: string[], unattributablePayloads?: number }} [options]
|
||||
* @returns {{ verdict: 'BLOCK'|'WARN'|'ALLOW', counts: Record<string, number>, allow_blocked_by: string[] }}
|
||||
* @returns {{ verdict: 'BLOCK'|'WARN'|'ALLOW', counts: Record<string, number> }}
|
||||
*/
|
||||
export function computeVerdict(findings, options = {}) {
|
||||
export function computeVerdict(findings) {
|
||||
const counts = { BLOCKER: 0, MAJOR: 0, MINOR: 0, SUGGESTION: 0 };
|
||||
for (const f of findings) {
|
||||
if (counts[f.severity] !== undefined) counts[f.severity] += 1;
|
||||
}
|
||||
|
||||
const unverified = options.unverified ?? [];
|
||||
const missingReviewers = options.missingReviewers ?? [];
|
||||
const unattributablePayloads = options.unattributablePayloads ?? 0;
|
||||
const allow_blocked_by = [];
|
||||
const byReason = new Map();
|
||||
for (const f of unverified) {
|
||||
const reason = f?.suppressed_reason ?? 'unspecified';
|
||||
byReason.set(reason, (byReason.get(reason) ?? 0) + 1);
|
||||
}
|
||||
for (const [reason, n] of byReason) allow_blocked_by.push(`unverified:${reason} (${n})`);
|
||||
for (const r of missingReviewers) allow_blocked_by.push(`missing-reviewer:${r}`);
|
||||
if (unattributablePayloads > 0) allow_blocked_by.push(`unattributable-payload (${unattributablePayloads})`);
|
||||
|
||||
let verdict;
|
||||
if (counts.BLOCKER >= 1) verdict = 'BLOCK';
|
||||
else if (counts.MAJOR >= 1) verdict = 'WARN';
|
||||
else if (allow_blocked_by.length > 0) verdict = 'WARN';
|
||||
else verdict = 'ALLOW';
|
||||
return { verdict, counts, allow_blocked_by };
|
||||
return { verdict, counts };
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the full deterministic contract: ingest → Pass 1 → Pass 2 → Pass 3 → Pass 4.
|
||||
*
|
||||
* `options.expectedReviewers` names the reviewers this review was supposed to
|
||||
* hear from. A reviewer that is absent from the payloads, or whose named
|
||||
* payload failed schema validation and was thrown away at ingest, lands in
|
||||
* `missing_reviewers`; a payload that failed schema WITHOUT a reviewer name is
|
||||
* counted in `unattributable_payloads` instead. Either forbids ALLOW: an
|
||||
* unread reviewer is an absent one, and zero findings from a silent reviewer
|
||||
* must not read like zero findings from a clean diff.
|
||||
*
|
||||
* `suppressed` stays the UNION of `dropped` (refuted) and `unverified` so
|
||||
* existing consumers keep their meaning; `unverified` is the subset that
|
||||
* forbids ALLOW. Do not iterate both and count twice.
|
||||
*
|
||||
* @param {Array<{reviewer?: string, findings: object[]}>} reviewerPayloads
|
||||
* @param {{ expectedReviewers?: string[] }} [options]
|
||||
* @returns {{ verdict: string, counts: Record<string, number>, findings: object[], suppressed: object[], unverified: object[], skipped: object[], missing_reviewers: string[], unattributable_payloads: number, allow_blocked_by: string[] }}
|
||||
* @returns {{ verdict: string, counts: Record<string, number>, findings: object[], suppressed: object[], skipped: object[] }}
|
||||
*/
|
||||
export function runContract(reviewerPayloads, options = {}) {
|
||||
export function runContract(reviewerPayloads) {
|
||||
const { findings: ingested, skipped } = ingest(reviewerPayloads);
|
||||
const deduped = dedupByTriplet(ingested);
|
||||
const judged = judgeFilter(deduped);
|
||||
const reasoned = reasonablenessFilter(judged.kept);
|
||||
const unverified = [...judged.unverified, ...reasoned.unverified];
|
||||
|
||||
// A reviewer counts as REPORTED only when a payload carrying its name
|
||||
// validated. `validateFindings` merely warns on a missing `reviewer`, so a
|
||||
// payload can fail schema anonymously: that is an unattributable payload, not
|
||||
// a reviewer called "unnamed reviewer". Naming one would invent an agent
|
||||
// nobody launched, and would double-count with expectedReviewers when the two
|
||||
// are in fact the same failure.
|
||||
const skippedNames = new Set();
|
||||
let unattributable_payloads = 0;
|
||||
for (const s of skipped) {
|
||||
if (typeof s.reviewer === 'string' && s.reviewer.length > 0) skippedNames.add(s.reviewer);
|
||||
else unattributable_payloads += 1;
|
||||
}
|
||||
const reported = new Set();
|
||||
for (const payload of reviewerPayloads) {
|
||||
const name = payload?.reviewer;
|
||||
if (typeof name === 'string' && name.length > 0 && !skippedNames.has(name)) reported.add(name);
|
||||
}
|
||||
const missing_reviewers = [...skippedNames];
|
||||
for (const r of options.expectedReviewers ?? []) {
|
||||
if (!reported.has(r) && !missing_reviewers.includes(r)) missing_reviewers.push(r);
|
||||
}
|
||||
|
||||
const { verdict, counts, allow_blocked_by } = computeVerdict(reasoned.kept, {
|
||||
unverified,
|
||||
missingReviewers: missing_reviewers,
|
||||
unattributablePayloads: unattributable_payloads,
|
||||
});
|
||||
const { verdict, counts } = computeVerdict(reasoned.kept);
|
||||
return {
|
||||
verdict,
|
||||
counts,
|
||||
findings: reasoned.kept,
|
||||
suppressed: [...judged.dropped, ...judged.unverified, ...reasoned.dropped, ...reasoned.unverified],
|
||||
unverified,
|
||||
suppressed: [...judged.dropped, ...reasoned.dropped],
|
||||
skipped,
|
||||
missing_reviewers,
|
||||
unattributable_payloads,
|
||||
allow_blocked_by,
|
||||
};
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
{
|
||||
"name": "voyage",
|
||||
"version": "5.10.1",
|
||||
"private": true,
|
||||
"version": "5.10.0",
|
||||
"description": "Voyage — brief, research, plan, execute, review, continue. Contract-driven Claude Code pipeline. /trekbrief, /trekplan, and /trekreview each end by building a self-contained operator-annotation HTML (scripts/annotate.mjs, modelled on claude-code-100x): select text or click any heading/paragraph/list-item, pick intent (Fiks/Endre/Spørsmål), write comment, copy structured prompt, paste back, Claude revises the .md.",
|
||||
"type": "module",
|
||||
"engines": {
|
||||
|
|
|
|||
|
|
@ -14,6 +14,11 @@
|
|||
"trekresearch": {
|
||||
"defaultMode": "default",
|
||||
"maxDimensions": 8,
|
||||
"geminiBridge": {
|
||||
"enabled": true,
|
||||
"pollIntervalSeconds": 30,
|
||||
"timeoutMinutes": 25
|
||||
},
|
||||
"interview": {
|
||||
"maxQuestions": 4,
|
||||
"typicalQuestions": 3
|
||||
|
|
|
|||
|
|
@ -74,6 +74,15 @@ findings exist.*
|
|||
### Known Issues
|
||||
{Common pitfalls, gotchas, real-world problems from community-researcher}
|
||||
|
||||
## Gemini Second Opinion
|
||||
|
||||
*Independent research result from Gemini Deep Research. Provides a second
|
||||
perspective for triangulation. Omit this section if gemini-bridge was not used
|
||||
or was unavailable.*
|
||||
|
||||
{Gemini findings reformatted into key findings, sources cited, and areas of
|
||||
agreement/disagreement with other agents.}
|
||||
|
||||
## Synthesis
|
||||
|
||||
*Cross-cutting insights that emerge from combining local and external knowledge.
|
||||
|
|
@ -105,7 +114,7 @@ this section entirely.*
|
|||
|
||||
| # | Source | Type | Quality | Used in |
|
||||
|---|--------|------|---------|---------|
|
||||
| 1 | {URL or codebase path} | {official / community / codebase} | {high / medium / low} | {dimension name} |
|
||||
| 1 | {URL or codebase path} | {official / community / codebase / gemini} | {high / medium / low} | {dimension name} |
|
||||
|
||||
*Quality assessment:*
|
||||
- **high** — official documentation, verified codebase analysis, peer-reviewed
|
||||
|
|
|
|||
|
|
@ -129,7 +129,7 @@ the codebase and this brief contain sufficient context for planning."}
|
|||
- **Required for plan steps:** {which kinds of steps will consume this — e.g.,
|
||||
"migration strategy", "library selection", "threat model"}
|
||||
- **Confidence needed:** {high | medium | low}
|
||||
- **Estimated cost:** {quick — inline research | standard — agent swarm | deep — with contrarian}
|
||||
- **Estimated cost:** {quick — inline research | standard — agent swarm | deep — with contrarian + gemini}
|
||||
- **Scope hint:** {local | external | both}
|
||||
|
||||
### Topic 2: {Short title}
|
||||
|
|
|
|||
|
|
@ -1,349 +0,0 @@
|
|||
// tests/commands/trekexecute-parallel-portability.test.mjs
|
||||
//
|
||||
// Executable tests for the two shell blocks /trekexecute Phase 2.6 Step 2a' and
|
||||
// Phase 2.55 Check 2 tell the agent to run. Both blocks wrecked a real voyage on
|
||||
// macOS (order 20260831T214411Z-941965142-from-.claude):
|
||||
//
|
||||
// Defect 1: `realpath --relative-to` is GNU coreutils. On BSD realpath the
|
||||
// command substitution fails, PROJECT_REL becomes EMPTY, and `mkdir -p
|
||||
// "$wt/"` + `cp ... "$wt//"` both SUCCEED — brief.md/plan.md land at the
|
||||
// worktree root instead of the project relpath. Exit status stays 0; only
|
||||
// file location tells the truth. Every assertion here checks placement.
|
||||
// Defect 2: Check 2 ran `git add {plan-path}` unconditionally. When the
|
||||
// project directory is gitignored (normal — .claude/projects/ is tool-
|
||||
// managed and local-only) the add fails and the plan never reaches HEAD.
|
||||
//
|
||||
// The blocks are EXTRACTED from commands/trekexecute.md and executed, so the
|
||||
// test binds to what an agent actually copies, not to prose about it. Anchors
|
||||
// are grep-able strings, never line numbers.
|
||||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFileSync, writeFileSync, mkdirSync, mkdtempSync, rmSync, chmodSync } from 'node:fs';
|
||||
import { execFileSync, spawnSync } from 'node:child_process';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { realpathSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const ROOT = join(HERE, '..', '..');
|
||||
const COMMAND_FILE = join(ROOT, 'commands', 'trekexecute.md');
|
||||
|
||||
const COPY_BLOCK_ANCHOR = "**2a'. Copy gitignored project artifacts";
|
||||
const CHECK2_ANCHOR = '### Check 2 —';
|
||||
|
||||
// The pre-fix form, kept ONLY as the negative control for Defect 1.
|
||||
const LEGACY_GNU_LINE = 'PROJECT_REL="$(realpath --relative-to="$REPO_ROOT" "$PROJECT_SOURCE")"';
|
||||
|
||||
// --- helpers -------------------------------------------------------------
|
||||
|
||||
/** Extract the first ```bash fence that follows `anchor` in commands/trekexecute.md. */
|
||||
function extractBashBlock(anchor) {
|
||||
const text = readFileSync(COMMAND_FILE, 'utf8');
|
||||
const at = text.indexOf(anchor);
|
||||
assert.ok(at >= 0, `anchor not found in trekexecute.md: ${anchor}`);
|
||||
const fenceOpen = text.indexOf('```bash', at);
|
||||
assert.ok(fenceOpen >= 0, `no bash fence after anchor: ${anchor}`);
|
||||
const bodyStart = text.indexOf('\n', fenceOpen) + 1;
|
||||
const fenceClose = text.indexOf('```', bodyStart);
|
||||
assert.ok(fenceClose > bodyStart, `unterminated bash fence after anchor: ${anchor}`);
|
||||
return text.slice(bodyStart, fenceClose);
|
||||
}
|
||||
|
||||
/**
|
||||
* A PATH directory whose `realpath` behaves like BSD realpath: it rejects every
|
||||
* GNU long option and resolves bare paths correctly. Stubbed, never assumed —
|
||||
* the machine running the suite may or may not have GNU coreutils.
|
||||
*/
|
||||
function bsdRealpathStubDir() {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'trek-bsdstub-'));
|
||||
const stub = join(dir, 'realpath');
|
||||
writeFileSync(stub, [
|
||||
'#!/bin/sh',
|
||||
'# BSD realpath stand-in: no GNU long options.',
|
||||
'for a in "$@"; do',
|
||||
' case "$a" in',
|
||||
' --*) echo "realpath: illegal option -- -" >&2; exit 1 ;;',
|
||||
' esac',
|
||||
'done',
|
||||
"exec python3 -c 'import os,sys",
|
||||
'for p in sys.argv[1:]: print(os.path.realpath(p))',
|
||||
"' \"$@\"",
|
||||
'',
|
||||
].join('\n'));
|
||||
chmodSync(stub, 0o755);
|
||||
return dir;
|
||||
}
|
||||
|
||||
/** A PATH directory whose `git check-ignore` dies fatally (128), everything else real. */
|
||||
function fatalCheckIgnoreGitStubDir() {
|
||||
const realGit = execFileSync('/usr/bin/env', ['sh', '-c', 'command -v git'], { encoding: 'utf8' }).trim();
|
||||
assert.ok(realGit, 'git not on PATH — cannot build the check-ignore stub');
|
||||
const dir = mkdtempSync(join(tmpdir(), 'trek-gitstub-'));
|
||||
const stub = join(dir, 'git');
|
||||
writeFileSync(stub, [
|
||||
'#!/bin/sh',
|
||||
'# `git check-ignore` fatal (128): NOT an answer about ignore status.',
|
||||
'for a in "$@"; do',
|
||||
' if [ "$a" = "check-ignore" ]; then',
|
||||
' echo "fatal: simulated check-ignore failure" >&2',
|
||||
' exit 128',
|
||||
' fi',
|
||||
'done',
|
||||
`exec ${realGit} "$@"`,
|
||||
'',
|
||||
].join('\n'));
|
||||
chmodSync(stub, 0o755);
|
||||
return dir;
|
||||
}
|
||||
|
||||
function git(cwd, ...args) {
|
||||
return execFileSync('git', args, { cwd, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] });
|
||||
}
|
||||
|
||||
/**
|
||||
* A temp repo with a committed file plus a project directory holding brief/plan
|
||||
* (+ research unless opts.research === false). `gitignoreProject: true` adds the
|
||||
* .gitignore line that makes the project directory invisible to `git add` —
|
||||
* and, note, invisible to `git status --porcelain` too, which is exactly why
|
||||
* this topology sails through Check 1 and only trips on Check 2.
|
||||
*/
|
||||
function makeRepo(opts = {}) {
|
||||
const root = realpathSync(mkdtempSync(join(tmpdir(), 'trek-exec-')));
|
||||
git(root, 'init', '-q', '-b', 'main');
|
||||
git(root, 'config', 'user.email', 'test@example.invalid');
|
||||
git(root, 'config', 'user.name', 'Test');
|
||||
writeFileSync(join(root, 'README.md'), '# fixture\n');
|
||||
if (opts.gitignoreProject) writeFileSync(join(root, '.gitignore'), '.claude/projects/\n');
|
||||
git(root, 'add', 'README.md', ...(opts.gitignoreProject ? ['.gitignore'] : []));
|
||||
git(root, 'commit', '-qm', 'init');
|
||||
|
||||
const projectRel = join('.claude', 'projects', 'demo');
|
||||
const projectDir = join(root, projectRel);
|
||||
mkdirSync(projectDir, { recursive: true });
|
||||
writeFileSync(join(projectDir, 'brief.md'), '# brief\n');
|
||||
writeFileSync(join(projectDir, 'plan.md'), '# plan\n');
|
||||
if (opts.research !== false) {
|
||||
mkdirSync(join(projectDir, 'research'), { recursive: true });
|
||||
writeFileSync(join(projectDir, 'research', '01-x.md'), '# r\n');
|
||||
}
|
||||
|
||||
const worktreeDir = join(root, '.claude', 'trekplan-sessions', 'demo', 'worktrees');
|
||||
mkdirSync(join(worktreeDir, 'session-1'), { recursive: true });
|
||||
mkdirSync(join(worktreeDir, 'session-2'), { recursive: true });
|
||||
|
||||
return { root, projectRel, projectDir, worktreeDir };
|
||||
}
|
||||
|
||||
function runBlock(script, { cwd, env }) {
|
||||
return spawnSync('bash', ['-c', script], { cwd, env: { ...process.env, ...env }, encoding: 'utf8' });
|
||||
}
|
||||
|
||||
function copyBlockEnv(repo, pathPrefixDir) {
|
||||
return {
|
||||
REPO_ROOT: repo.root,
|
||||
PROJECT_DIR: repo.projectDir,
|
||||
WORKTREE_DIR: repo.worktreeDir,
|
||||
PATH: `${pathPrefixDir}:${process.env.PATH}`,
|
||||
};
|
||||
}
|
||||
|
||||
function exists(p) {
|
||||
try { readFileSync(p); return true; } catch { return false; }
|
||||
}
|
||||
|
||||
// --- Defect 1: portable relpath derivation -------------------------------
|
||||
|
||||
test("2a' — the BSD realpath stub is known-positive: bare path resolves, --relative-to is rejected", () => {
|
||||
const stubDir = bsdRealpathStubDir();
|
||||
try {
|
||||
const bare = runBlock('realpath "$HOME"', { cwd: ROOT, env: { PATH: `${stubDir}:${process.env.PATH}` } });
|
||||
assert.equal(bare.status, 0, 'stub must resolve a bare path (proves it can succeed)');
|
||||
assert.equal(bare.stdout.trim(), realpathSync(process.env.HOME));
|
||||
|
||||
const gnu = runBlock('realpath --relative-to=/ "$HOME"', { cwd: ROOT, env: { PATH: `${stubDir}:${process.env.PATH}` } });
|
||||
assert.notEqual(gnu.status, 0, 'stub must reject the GNU long option');
|
||||
assert.match(gnu.stderr, /illegal option/, 'stub must fail the way BSD realpath fails');
|
||||
assert.equal(gnu.stdout.trim(), '', 'no stdout — this is what leaves PROJECT_REL empty');
|
||||
} finally {
|
||||
rmSync(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("2a' — NEGATIVE CONTROL: the GNU form drops plan.md at the worktree ROOT under BSD realpath", () => {
|
||||
const repo = makeRepo();
|
||||
const stubDir = bsdRealpathStubDir();
|
||||
try {
|
||||
const legacy = [
|
||||
'PROJECT_SOURCE="$(realpath "${PROJECT_DIR}")"',
|
||||
LEGACY_GNU_LINE,
|
||||
'for wt in "$WORKTREE_DIR"/session-*; do',
|
||||
' [ -d "$wt" ] || continue',
|
||||
' mkdir -p "$wt/$PROJECT_REL"',
|
||||
' cp "$PROJECT_SOURCE"/brief.md "$wt/$PROJECT_REL/"',
|
||||
' cp "$PROJECT_SOURCE"/plan.md "$wt/$PROJECT_REL/"',
|
||||
'done',
|
||||
].join('\n');
|
||||
runBlock(legacy, { cwd: repo.root, env: copyBlockEnv(repo, stubDir) });
|
||||
|
||||
const wt = join(repo.worktreeDir, 'session-1');
|
||||
assert.equal(exists(join(wt, repo.projectRel, 'plan.md')), false,
|
||||
'the broken form must NOT put plan.md at the project relpath');
|
||||
assert.equal(exists(join(wt, 'plan.md')), true,
|
||||
'the broken form silently drops plan.md at the worktree root — the measured havari');
|
||||
} finally {
|
||||
rmSync(repo.root, { recursive: true, force: true });
|
||||
rmSync(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("2a' — the shipped block copies brief/plan/research to $wt/$PROJECT_REL without GNU realpath", () => {
|
||||
const repo = makeRepo();
|
||||
const stubDir = bsdRealpathStubDir();
|
||||
try {
|
||||
const r = runBlock(extractBashBlock(COPY_BLOCK_ANCHOR), { cwd: repo.root, env: copyBlockEnv(repo, stubDir) });
|
||||
assert.equal(r.status, 0, `block must succeed without GNU realpath. stderr: ${r.stderr}`);
|
||||
for (const s of ['session-1', 'session-2']) {
|
||||
const dest = join(repo.worktreeDir, s, repo.projectRel);
|
||||
assert.equal(exists(join(dest, 'plan.md')), true, `${s}: plan.md must reach $wt/$PROJECT_REL`);
|
||||
assert.equal(exists(join(dest, 'brief.md')), true, `${s}: brief.md must reach $wt/$PROJECT_REL`);
|
||||
assert.equal(exists(join(dest, 'research', '01-x.md')), true, `${s}: research/ must reach $wt/$PROJECT_REL`);
|
||||
assert.equal(exists(join(repo.worktreeDir, s, 'plan.md')), false,
|
||||
`${s}: nothing may land at the worktree root`);
|
||||
}
|
||||
} finally {
|
||||
rmSync(repo.root, { recursive: true, force: true });
|
||||
rmSync(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("2a' — a project without research/ is not a failure (block still exits 0)", () => {
|
||||
const repo = makeRepo({ research: false });
|
||||
const stubDir = bsdRealpathStubDir();
|
||||
try {
|
||||
const r = runBlock(extractBashBlock(COPY_BLOCK_ANCHOR), { cwd: repo.root, env: copyBlockEnv(repo, stubDir) });
|
||||
assert.equal(r.status, 0, `missing research/ must not fail the wave. stderr: ${r.stderr}`);
|
||||
assert.equal(exists(join(repo.worktreeDir, 'session-1', repo.projectRel, 'plan.md')), true);
|
||||
} finally {
|
||||
rmSync(repo.root, { recursive: true, force: true });
|
||||
rmSync(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("2a' — an underivable relpath fails LOUDLY instead of dropping files at the worktree root", () => {
|
||||
const repo = makeRepo();
|
||||
const stubDir = bsdRealpathStubDir();
|
||||
const outside = realpathSync(mkdtempSync(join(tmpdir(), 'trek-outside-')));
|
||||
try {
|
||||
mkdirSync(join(outside, 'p'), { recursive: true });
|
||||
writeFileSync(join(outside, 'p', 'brief.md'), 'b');
|
||||
writeFileSync(join(outside, 'p', 'plan.md'), 'p');
|
||||
const env = { ...copyBlockEnv(repo, stubDir), PROJECT_DIR: join(outside, 'p') };
|
||||
const r = runBlock(extractBashBlock(COPY_BLOCK_ANCHOR), { cwd: repo.root, env });
|
||||
assert.notEqual(r.status, 0, 'a project outside REPO_ROOT must abort the wave');
|
||||
assert.match(r.stderr, /relpath/i, 'the abort must name the cause');
|
||||
assert.equal(exists(join(repo.worktreeDir, 'session-1', 'plan.md')), false,
|
||||
'nothing may be dropped at the worktree root');
|
||||
} finally {
|
||||
rmSync(repo.root, { recursive: true, force: true });
|
||||
rmSync(outside, { recursive: true, force: true });
|
||||
rmSync(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("2a' — the GNU-only form is gone from the block agents copy", () => {
|
||||
const block = extractBashBlock(COPY_BLOCK_ANCHOR);
|
||||
assert.equal(block.includes('--relative-to'), false,
|
||||
'the copied block must not contain `realpath --relative-to` (GNU-only)');
|
||||
assert.ok(block.includes('os.path.relpath'), 'the copied block must derive the relpath portably');
|
||||
});
|
||||
|
||||
test("both copied blocks are ASCII-clean (bash 3.2 dies on a multibyte char under set -u)", () => {
|
||||
const nonAscii = (s) => s.split('\n')
|
||||
.map((line, i) => [i + 1, line])
|
||||
.filter(([, line]) => /[^\x00-\x7F]/.test(line));
|
||||
// Known-positive: the detector must actually fire on a multibyte char.
|
||||
assert.equal(nonAscii('echo "a — b"').length, 1, 'detector must find an em-dash');
|
||||
for (const anchor of [COPY_BLOCK_ANCHOR, CHECK2_ANCHOR]) {
|
||||
assert.deepEqual(nonAscii(extractBashBlock(anchor)), [],
|
||||
`non-ASCII inside the shell block after ${anchor} (prose outside the fence is fine)`);
|
||||
}
|
||||
});
|
||||
|
||||
// --- Defect 2: gitignored project directory ------------------------------
|
||||
|
||||
// The block carries the `{plan-path}` placeholder the way every other block in
|
||||
// trekexecute.md does. Substitute it exactly as an agent would — never inject
|
||||
// PLAN_PATH through the environment: that would supply what the doc must supply
|
||||
// itself, and a block that never assigns the variable would still pass.
|
||||
function check2Script(planPath) {
|
||||
const block = extractBashBlock(CHECK2_ANCHOR);
|
||||
assert.ok(block.includes('{plan-path}'),
|
||||
'Check 2 block must carry the {plan-path} placeholder for the agent to substitute');
|
||||
return block.replace('{plan-path}', planPath);
|
||||
}
|
||||
|
||||
function check2Env(repo, pathPrefixDir) {
|
||||
const env = { REPO_ROOT: repo.root, PLAN_PATH: '' };
|
||||
if (pathPrefixDir) env.PATH = `${pathPrefixDir}:${process.env.PATH}`;
|
||||
return env;
|
||||
}
|
||||
|
||||
test('Check 2 — gitignored project dir: no commit, no failure, and 2a\' still delivers the plan', () => {
|
||||
const repo = makeRepo({ gitignoreProject: true });
|
||||
const stubDir = bsdRealpathStubDir();
|
||||
try {
|
||||
// Known-positive on the premise: the plan file really is ignored here.
|
||||
const ci = spawnSync('git', ['check-ignore', '-v', join(repo.projectRel, 'plan.md')],
|
||||
{ cwd: repo.root, encoding: 'utf8' });
|
||||
assert.equal(ci.status, 0, 'fixture premise: the plan file must actually be gitignored');
|
||||
|
||||
const head = git(repo.root, 'rev-parse', 'HEAD').trim();
|
||||
const r = runBlock(check2Script(join(repo.projectRel, 'plan.md')),
|
||||
{ cwd: repo.root, env: check2Env(repo) });
|
||||
assert.equal(r.status, 0, `Check 2 must tolerate a gitignored plan file. stderr: ${r.stderr}`);
|
||||
assert.equal(git(repo.root, 'rev-parse', 'HEAD').trim(), head,
|
||||
'an ignored plan file must NOT be forced into history (origin is a public mirror)');
|
||||
|
||||
const copy = runBlock(extractBashBlock(COPY_BLOCK_ANCHOR), { cwd: repo.root, env: copyBlockEnv(repo, stubDir) });
|
||||
assert.equal(copy.status, 0, `copy step must succeed. stderr: ${copy.stderr}`);
|
||||
assert.equal(exists(join(repo.worktreeDir, 'session-1', repo.projectRel, 'plan.md')), true,
|
||||
'the plan must reach the worktree even though git never tracked it');
|
||||
} finally {
|
||||
rmSync(repo.root, { recursive: true, force: true });
|
||||
rmSync(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('Check 2 — KNOWN-POSITIVE: an untracked, NOT-ignored plan file is still added and committed', () => {
|
||||
const repo = makeRepo();
|
||||
try {
|
||||
const planPath = join(repo.projectRel, 'plan.md');
|
||||
const head = git(repo.root, 'rev-parse', 'HEAD').trim();
|
||||
const r = runBlock(check2Script(planPath), { cwd: repo.root, env: check2Env(repo) });
|
||||
assert.equal(r.status, 0, `Check 2 must succeed on a normal untracked plan. stderr: ${r.stderr}`);
|
||||
assert.notEqual(git(repo.root, 'rev-parse', 'HEAD').trim(), head,
|
||||
'a trackable plan file must still be committed for worktree visibility');
|
||||
const ls = spawnSync('git', ['ls-files', '--error-unmatch', planPath], { cwd: repo.root, encoding: 'utf8' });
|
||||
assert.equal(ls.status, 0, 'the plan file must now be tracked');
|
||||
} finally {
|
||||
rmSync(repo.root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('Check 2 — a FATAL git check-ignore (128) is not read as "not ignored"', () => {
|
||||
const repo = makeRepo({ gitignoreProject: true });
|
||||
const stubDir = fatalCheckIgnoreGitStubDir();
|
||||
try {
|
||||
const head = git(repo.root, 'rev-parse', 'HEAD').trim();
|
||||
const r = runBlock(check2Script(join(repo.projectRel, 'plan.md')),
|
||||
{ cwd: repo.root, env: check2Env(repo, stubDir) });
|
||||
assert.notEqual(r.status, 0, 'a fatal check-ignore must stop, not fall through to git add');
|
||||
assert.match(r.stderr, /check-ignore/, 'the stop must name the failing probe');
|
||||
assert.equal(git(repo.root, 'rev-parse', 'HEAD').trim(), head, 'no commit may be made on a fatal probe');
|
||||
} finally {
|
||||
rmSync(repo.root, { recursive: true, force: true });
|
||||
rmSync(stubDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
|
@ -152,18 +152,14 @@ test('no agents/*.md frontmatter contains an <example> block (M4: examples live
|
|||
}
|
||||
});
|
||||
|
||||
// Floor lowered 34 -> 32 in v5.10.1: `gemini-bridge` (2 examples) was DELETED as an
|
||||
// agent by operator directive, not relocated. The pin guards M4's "relocation moves,
|
||||
// never deletes" invariant, so it must move only when an agent legitimately leaves the
|
||||
// inventory — never to absorb examples that went missing from a surviving agent.
|
||||
test('agent bodies retain at least 32 <example> blocks (M4: relocation moves, never deletes)', () => {
|
||||
test('agent bodies retain at least 34 <example> blocks (M4: relocation moves, never deletes)', () => {
|
||||
let total = 0;
|
||||
for (const f of agentFiles) {
|
||||
total += (bodyOf(read(`agents/${f}`)).match(/<example>/g) || []).length;
|
||||
}
|
||||
assert.ok(
|
||||
total >= 32,
|
||||
`expected >= 32 <example> blocks across agent bodies (16 agents x 2), got ${total} ` +
|
||||
total >= 34,
|
||||
`expected >= 34 <example> blocks across agent bodies (17 agents x 2), got ${total} ` +
|
||||
`— examples may have been deleted instead of relocated (M4)`,
|
||||
);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -6,9 +6,6 @@
|
|||
|
||||
import { test } from 'node:test';
|
||||
import { strict as assert } from 'node:assert';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
severityRank,
|
||||
ingest,
|
||||
|
|
@ -16,14 +13,9 @@ import {
|
|||
judgeFilter,
|
||||
reasonablenessFilter,
|
||||
computeVerdict,
|
||||
classifySuppression,
|
||||
REFUTING_REASONS,
|
||||
UNVERIFIED_REASONS,
|
||||
runContract,
|
||||
} from '../../lib/review/coordinator-contract.mjs';
|
||||
|
||||
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
||||
|
||||
// ---- Pass 1 — dedup --------------------------------------------------------
|
||||
|
||||
test('dedupByTriplet — genuine cross-reviewer collapse (identical triplet) → 1, raised_by both', () => {
|
||||
|
|
@ -106,41 +98,29 @@ test('computeVerdict — counts each severity tier', () => {
|
|||
|
||||
// ---- Pass 3 — reasonableness -----------------------------------------------
|
||||
|
||||
test('reasonablenessFilter — citation-less is REFUTED, unknown rule_key is UNVERIFIED, severity mismatch corrected', () => {
|
||||
// Contract change (fail-closed): only `no-citation` refutes. An ad-hoc
|
||||
// rule_key is a real defect wearing the wrong label — v5.1.1 high-effort mode
|
||||
// already keeps those, normalised to PLAN_EXECUTE_DRIFT.
|
||||
test('reasonablenessFilter — drops unknown rule_key + citation-less, corrects severity mismatch', () => {
|
||||
const r = reasonablenessFilter([
|
||||
{ file: 'x.mjs', line: 1, rule_key: 'NOPE_KEY', severity: 'BLOCKER' }, // unknown → unverified
|
||||
{ file: 'x.mjs', line: 1, rule_key: 'NOPE_KEY', severity: 'BLOCKER' }, // unknown → drop
|
||||
{ file: '', line: 1, rule_key: 'MISSING_TEST', severity: 'MAJOR' }, // no file → drop
|
||||
{ file: 'x.mjs', line: -1, rule_key: 'MISSING_TEST', severity: 'MAJOR' }, // line < 0 → drop
|
||||
{ file: 'x.mjs', line: 1, rule_key: 'MISSING_TEST', severity: 'MINOR' }, // catalogue is MAJOR → correct, keep
|
||||
]);
|
||||
assert.equal(r.kept.length, 1);
|
||||
assert.equal(r.dropped.length, 2);
|
||||
assert.deepEqual(r.dropped.map((f) => f.suppressed_reason), ['no-citation', 'no-citation']);
|
||||
assert.equal(r.unverified.length, 1);
|
||||
assert.equal(r.unverified[0].suppressed_reason, 'unknown-rule_key');
|
||||
assert.equal(r.dropped.length, 3);
|
||||
assert.equal(r.kept[0].severity, 'MAJOR');
|
||||
assert.equal(r.kept[0].original_severity, 'MINOR');
|
||||
});
|
||||
|
||||
// ---- Pass 2 — judge --------------------------------------------------------
|
||||
|
||||
test('judgeFilter — over-long title and empty recommended_action are UNVERIFIED, not dropped', () => {
|
||||
// Contract change (fail-closed): both implemented Pass 2 tests read a
|
||||
// `.length` and never examine the claim, so neither refutes the finding.
|
||||
// `dropped` is empty here on purpose — the refuting Pass 2 filter (Accuracy)
|
||||
// is the one this deterministic subset excludes.
|
||||
test('judgeFilter — drops over-long title and empty recommended_action', () => {
|
||||
const j = judgeFilter([
|
||||
{ file: 'x.mjs', line: 1, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'x'.repeat(101) }, // too long → unverified
|
||||
{ file: 'x.mjs', line: 2, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'ok', recommended_action: ' ' }, // empty action → unverified
|
||||
{ file: 'x.mjs', line: 1, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'x'.repeat(101) }, // too long → drop
|
||||
{ file: 'x.mjs', line: 2, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'ok', recommended_action: ' ' }, // empty action → drop
|
||||
{ file: 'x.mjs', line: 3, rule_key: 'MISSING_TEST', severity: 'MAJOR', title: 'ok' }, // keep (no action field is fine)
|
||||
]);
|
||||
assert.equal(j.kept.length, 1);
|
||||
assert.equal(j.dropped.length, 0);
|
||||
assert.equal(j.unverified.length, 2);
|
||||
assert.deepEqual(j.unverified.map((f) => f.suppressed_reason), ['succinctness:title', 'actionability:empty']);
|
||||
assert.equal(j.dropped.length, 2);
|
||||
});
|
||||
|
||||
// ---- ingest ----------------------------------------------------------------
|
||||
|
|
@ -181,155 +161,3 @@ test('runContract — deterministic: identical input yields identical output', (
|
|||
];
|
||||
assert.deepEqual(runContract(input), runContract(input));
|
||||
});
|
||||
|
||||
// ---- Fail-closed: the `unverified` bucket (ORDRE 834432937) -----------------
|
||||
//
|
||||
// The defect: a finding REMOVED by Pass 2/Pass 3, and a reviewer whose payload
|
||||
// was thrown away or never arrived, are all arithmetically identical to a
|
||||
// finding that never existed -- they push the verdict toward ALLOW. Measured
|
||||
// before the fix (probe, 2026-09-01): an over-long-title BLOCKER -> ALLOW; a
|
||||
// payload with one ad-hoc rule_key -> the whole payload skipped, its valid
|
||||
// BLOCKER sibling gone -> ALLOW.
|
||||
//
|
||||
// The rule under test: a removal is `dropped` ONLY when the test refutes the
|
||||
// finding as a claim about this codebase. Every other removal is `unverified`,
|
||||
// and a non-empty `unverified` -- or a reviewer that did not report -- forbids
|
||||
// ALLOW.
|
||||
|
||||
test('classifySuppression — only no-citation refutes; form and taxonomy failures are unverified', () => {
|
||||
assert.equal(classifySuppression('no-citation'), 'refuted',
|
||||
'a finding that names no location makes no checkable claim');
|
||||
assert.equal(classifySuppression('succinctness:title'), 'unverified');
|
||||
assert.equal(classifySuppression('succinctness:detail'), 'unverified');
|
||||
assert.equal(classifySuppression('actionability:empty'), 'unverified');
|
||||
assert.equal(classifySuppression('unknown-rule_key'), 'unverified');
|
||||
assert.equal(classifySuppression('file-existence:indeterminate'), 'unverified');
|
||||
assert.equal(classifySuppression('something-nobody-declared'), 'unverified',
|
||||
'an unclassified reason must fail CLOSED, not open');
|
||||
});
|
||||
|
||||
test('computeVerdict — non-empty unverified forbids ALLOW but never downgrades BLOCK or WARN', () => {
|
||||
const u = [{ file: 'x.mjs', line: 1, rule_key: 'MISSING_TEST', severity: 'BLOCKER' }];
|
||||
const withUnverified = computeVerdict([], { unverified: u });
|
||||
assert.equal(withUnverified.verdict, 'WARN', 'ALLOW is forbidden while anything is unverified');
|
||||
assert.deepEqual(withUnverified.counts, { BLOCKER: 0, MAJOR: 0, MINOR: 0, SUGGESTION: 0 },
|
||||
'the unverified finding is NOT counted into a severity tier');
|
||||
assert.ok(withUnverified.allow_blocked_by.length > 0);
|
||||
|
||||
assert.equal(computeVerdict([{ severity: 'BLOCKER' }], { unverified: u }).verdict, 'BLOCK',
|
||||
'BLOCK stands regardless of the unverified bucket');
|
||||
assert.equal(computeVerdict([{ severity: 'MAJOR' }], { unverified: u }).verdict, 'WARN');
|
||||
assert.equal(computeVerdict([], { unverified: [] }).verdict, 'ALLOW',
|
||||
'known-positive control: an empty unverified bucket still allows ALLOW');
|
||||
});
|
||||
|
||||
test('computeVerdict — a reviewer that did not report forbids ALLOW', () => {
|
||||
const r = computeVerdict([], { missingReviewers: ['brief-conformance-reviewer'] });
|
||||
assert.equal(r.verdict, 'WARN');
|
||||
assert.ok(r.allow_blocked_by.some((x) => x.includes('brief-conformance-reviewer')));
|
||||
});
|
||||
|
||||
test('runContract — a BLOCKER dropped for an over-long title cannot yield ALLOW', () => {
|
||||
// Pass 2 succinctness reads `.length`. It never examines the claim, so it
|
||||
// cannot establish the finding is unreal -- it is unverified, not refuted.
|
||||
const result = runContract([
|
||||
{ reviewer: 'code-correctness-reviewer', findings: [
|
||||
{ file: 'lib/auth/jwt.mjs', line: 19, rule_key: 'SECURITY_INJECTION', severity: 'BLOCKER', title: 'x'.repeat(101), detail: 'algo taken from the JWT header' },
|
||||
] },
|
||||
]);
|
||||
assert.notEqual(result.verdict, 'ALLOW', 'an unsubstantiated BLOCKER must never clear the review');
|
||||
assert.equal(result.findings.length, 0, 'it is still not a kept finding');
|
||||
assert.equal(result.unverified.length, 1);
|
||||
assert.equal(result.unverified[0].suppressed_reason, 'succinctness:title');
|
||||
assert.equal(result.suppressed.length, 1, 'suppressed stays the union of dropped + unverified');
|
||||
});
|
||||
|
||||
test('runContract — a schema-invalid payload cannot yield ALLOW (an unread reviewer is an absent one)', () => {
|
||||
// Measured: one ad-hoc rule_key invalidates the WHOLE payload at ingest, so a
|
||||
// valid BLOCKER sibling disappears with it. That must not read as "clean".
|
||||
const result = runContract([
|
||||
{ reviewer: 'code-correctness-reviewer', findings: [
|
||||
{ file: 'lib/auth/jwt.mjs', line: 19, rule_key: 'SECURITY_INJECTION', severity: 'BLOCKER', title: 'real' },
|
||||
{ file: 'x.mjs', line: 1, rule_key: 'NOPE_KEY', severity: 'MINOR', title: 'ad-hoc key' },
|
||||
] },
|
||||
]);
|
||||
assert.equal(result.skipped.length, 1);
|
||||
assert.notEqual(result.verdict, 'ALLOW');
|
||||
assert.ok(result.allow_blocked_by.some((x) => x.includes('code-correctness-reviewer')));
|
||||
});
|
||||
|
||||
test('runContract — a reviewer named in expectedReviewers that never reported cannot yield ALLOW', () => {
|
||||
const result = runContract(
|
||||
[{ reviewer: 'code-correctness-reviewer', findings: [] }],
|
||||
{ expectedReviewers: ['code-correctness-reviewer', 'brief-conformance-reviewer'] },
|
||||
);
|
||||
assert.deepEqual(result.missing_reviewers, ['brief-conformance-reviewer']);
|
||||
assert.notEqual(result.verdict, 'ALLOW');
|
||||
});
|
||||
|
||||
test('runContract — known-positive control: every reviewer reported, nothing suppressed → ALLOW', () => {
|
||||
// Proves ALLOW is still REACHABLE. Without this, "no ALLOW" is not a
|
||||
// fail-closed contract, only a broken one.
|
||||
const result = runContract(
|
||||
[
|
||||
{ reviewer: 'code-correctness-reviewer', findings: [
|
||||
{ file: 'a.mjs', line: 1, rule_key: 'MISSING_ERROR_HANDLING', severity: 'MINOR', title: 'unguarded await', recommended_action: 'Wrap the await in a try/catch.' },
|
||||
] },
|
||||
{ reviewer: 'brief-conformance-reviewer', findings: [] },
|
||||
],
|
||||
{ expectedReviewers: ['code-correctness-reviewer', 'brief-conformance-reviewer'] },
|
||||
);
|
||||
assert.equal(result.verdict, 'ALLOW');
|
||||
assert.equal(result.unverified.length, 0);
|
||||
assert.deepEqual(result.missing_reviewers, []);
|
||||
assert.deepEqual(result.allow_blocked_by, []);
|
||||
});
|
||||
|
||||
test('classifySuppression — the refuting reasons the LLM coordinator emits are declared here too', () => {
|
||||
// agents/review-coordinator.md Pass 2 "Accuracy" and Pass 3 "Non-existent
|
||||
// file" DO refute (a citation outside the repo root, a file absent from both
|
||||
// tree and diff). Both are fs/judgement branches this deterministic subset
|
||||
// excludes, but the vocabulary is owned here so prose and lib cannot drift.
|
||||
assert.equal(classifySuppression('accuracy:refuted'), 'refuted');
|
||||
assert.equal(classifySuppression('file-existence:refuted'), 'refuted');
|
||||
assert.equal(classifySuppression('file-existence:indeterminate'), 'unverified',
|
||||
'unresolvable must never collapse into refuted');
|
||||
});
|
||||
|
||||
test('suppression vocabulary — the two sets are disjoint and every reason is documented in the prose', () => {
|
||||
const refuting = [...REFUTING_REASONS];
|
||||
const overlap = refuting.filter((r) => UNVERIFIED_REASONS.includes(r));
|
||||
assert.deepEqual(overlap, [], 'a reason cannot be both refuting and unverified');
|
||||
|
||||
const prose = readFileSync(join(ROOT, 'agents/review-coordinator.md'), 'utf-8');
|
||||
assert.ok(prose.includes('review-coordinator'), 'known-positive control: the prose file loaded');
|
||||
for (const reason of [...refuting, ...UNVERIFIED_REASONS]) {
|
||||
assert.ok(prose.includes(reason),
|
||||
`reason "${reason}" is declared in the lib but never documented in agents/review-coordinator.md`);
|
||||
}
|
||||
});
|
||||
|
||||
test('runContract — an anonymous invalid payload is unattributable, not a reviewer named "unnamed reviewer"', () => {
|
||||
// `validateFindings` only WARNS on a missing `reviewer`, so a payload can
|
||||
// fail schema while carrying no name. Reporting it as a reviewer name
|
||||
// invents an agent nobody launched, and double-counts with expectedReviewers
|
||||
// when they are in fact the same failure.
|
||||
const result = runContract(
|
||||
[{ findings: [{ file: 'x.mjs', line: 1, rule_key: 'NOPE', severity: 'MAJOR' }] }],
|
||||
{ expectedReviewers: ['code-correctness-reviewer'] },
|
||||
);
|
||||
assert.deepEqual(result.missing_reviewers, ['code-correctness-reviewer'],
|
||||
'missing_reviewers carries real names only');
|
||||
assert.equal(result.unattributable_payloads, 1);
|
||||
assert.ok(result.allow_blocked_by.some((x) => x.startsWith('unattributable-payload')));
|
||||
assert.ok(!result.allow_blocked_by.some((x) => x.includes('unnamed reviewer')));
|
||||
});
|
||||
|
||||
test('runContract — an anonymous invalid payload forbids ALLOW on its own, with no expectedReviewers', () => {
|
||||
// The fail-closed floor must not depend on the caller passing an expected
|
||||
// set: without this, dropping the name from a payload would restore ALLOW.
|
||||
const result = runContract([{ findings: [{ file: 'x.mjs', line: 1, rule_key: 'NOPE', severity: 'MAJOR' }] }]);
|
||||
assert.deepEqual(result.missing_reviewers, []);
|
||||
assert.equal(result.unattributable_payloads, 1);
|
||||
assert.notEqual(result.verdict, 'ALLOW');
|
||||
});
|
||||
|
|
|
|||
|
|
@ -701,23 +701,6 @@ test('producing commands emit file:// link in final report (operator-UX contract
|
|||
}
|
||||
});
|
||||
|
||||
test('package.json is marked private — the npm name `voyage` belongs to a third party', () => {
|
||||
// D-census 2026-08-20/21: registry.npmjs.org/voyage answers 200, but that is
|
||||
// NOT this package — it is "Advanced HTTP Routing System for Node.js"
|
||||
// (NEURS/voyage, v0.0.1, 2022, maintainer kevin.martin). This manifest is test
|
||||
// tooling for a Claude Code plugin distributed through the marketplace catalogue
|
||||
// (`ref: vX.Y.Z`), never through npm. Without `private`, nothing in the manifest
|
||||
// stops an accidental `npm publish` against a name someone else owns — a one-way
|
||||
// action. Sibling non-package manifests (okr, repo-mailbox, repo-standard) all
|
||||
// carry the flag; voyage was the exception. Operator decision (S93): option A.
|
||||
const pkg = JSON.parse(read('package.json'));
|
||||
assert.equal(
|
||||
pkg.private,
|
||||
true,
|
||||
'package.json must declare "private": true — the npm name `voyage` is owned by a third party',
|
||||
);
|
||||
});
|
||||
|
||||
test('package.json still has no "npm run render" script (removed in v5.0.1)', () => {
|
||||
const pkg = JSON.parse(read('package.json'));
|
||||
assert.equal(
|
||||
|
|
@ -1134,57 +1117,6 @@ test('deep-research-engine: --engine is documented + consistent across surfaces'
|
|||
);
|
||||
});
|
||||
|
||||
test('deep-research-engine: the CC 2.1.218 operator-only ceiling is documented on every surface', () => {
|
||||
// CC 2.1.218 changed /deep-research to "start only when invoked manually"; from
|
||||
// there the Skill tool refuses a model invocation outright with
|
||||
// `disable-model-invocation`. The flag stays (additive opt-in, always degrades to
|
||||
// swarm), but no surface may keep promising a path Claude Code has removed, and
|
||||
// none may leave `2.1.154+` standing alone as the engine's requirement.
|
||||
const SURFACES = ['commands/trekresearch.md', 'docs/command-modes.md', 'README.md'];
|
||||
for (const f of SURFACES) {
|
||||
const body = read(f);
|
||||
assert.ok(
|
||||
body.includes('2.1.218'),
|
||||
`${f} must document the CC 2.1.218 ceiling for the deep-research engine`,
|
||||
);
|
||||
assert.ok(
|
||||
body.includes('disable-model-invocation'),
|
||||
`${f} must name disable-model-invocation as the fallback reason from CC 2.1.218`,
|
||||
);
|
||||
}
|
||||
|
||||
// The one-line reference rows carry the whole truth on their own line — an
|
||||
// operator reading the flag table never scrolls to the command prose.
|
||||
const ROWS = [
|
||||
['docs/command-modes.md', /^.*`--engine \{swarm\\\|deep-research\}`.*$/m],
|
||||
['README.md', /^\|\s*\*\*Engine\*\*.*$/m],
|
||||
];
|
||||
for (const [f, re] of ROWS) {
|
||||
const m = read(f).match(re);
|
||||
assert.ok(m, `${f} must still carry the --engine reference row`);
|
||||
assert.ok(
|
||||
m[0].includes('2.1.218') && m[0].includes('disable-model-invocation'),
|
||||
`${f} --engine row must state the real window (2.1.218 + disable-model-invocation)`,
|
||||
);
|
||||
}
|
||||
|
||||
// Both load-bearing regions of the command itself: the flag bullet and the
|
||||
// pre-gate that decides whether the delegation is attempted at all.
|
||||
const research = read('commands/trekresearch.md');
|
||||
const bullet = research.match(/8\. `--engine <name>`[\s\S]*?Flags can be combined/);
|
||||
assert.ok(bullet, 'trekresearch.md must still carry the --engine flag bullet');
|
||||
assert.ok(
|
||||
bullet[0].includes('2.1.218') && bullet[0].includes('disable-model-invocation'),
|
||||
'trekresearch.md --engine bullet must state the real window, not just the 2.1.154 floor',
|
||||
);
|
||||
const pregate = research.match(/\*\*Coarse pre-gate[\s\S]*?authoritative guard\./);
|
||||
assert.ok(pregate, 'trekresearch.md must still carry the deep-research pre-gate');
|
||||
assert.ok(
|
||||
pregate[0].includes('2.1.218'),
|
||||
'the pre-gate must carry an upper ceiling at 2.1.218, not only the 2.1.154 floor',
|
||||
);
|
||||
});
|
||||
|
||||
// ── STORM bounded loop — env-vars documented across the four surfaces ──────
|
||||
// Same cross-doc shape as the --engine pin above. An operator-facing switch
|
||||
// documented on one surface is a switch most operators never find; and the
|
||||
|
|
@ -1702,94 +1634,3 @@ test('v5.9: no commands/*.md frontmatter carries a model: key (session inheritan
|
|||
assert.deepEqual(offenders, [],
|
||||
`command frontmatter must omit model: (orchestrator follows the session model); offenders: ${offenders.join(', ')}`);
|
||||
});
|
||||
|
||||
// --- S95 (③): gemini-bridge is DROPPED from the pipeline (operator directive) ---
|
||||
//
|
||||
// Operator directive, three repetitions: gemini-mcp and voyage:gemini-bridge are
|
||||
// never to be used. The drop is permanent, not a wait-for-upgrade — so the pin is
|
||||
// on the LIVE pipeline surfaces only. Historical records (CHANGELOG entries,
|
||||
// measurement/decision docs) must KEEP their references: they say what a past
|
||||
// version did or what was measured then, and rewriting them is the same defect
|
||||
// class as bumping a version string inside a measurement doc.
|
||||
//
|
||||
// `gemini_used` is deliberately NOT banned. It is a vestigial trekresearch stats
|
||||
// field, pinned to false by the drop and kept so the observability export schema
|
||||
// does not break for existing consumers (see TREKRESEARCH_ALLOWED).
|
||||
|
||||
// Live = anything that can still steer a run. Explicit, so a new file forces a decision.
|
||||
const GEMINI_LIVE_SURFACES = [
|
||||
'README.md',
|
||||
'CLAUDE.md',
|
||||
'settings.json',
|
||||
'docs/architecture.md',
|
||||
'docs/command-modes.md',
|
||||
'templates/research-brief-template.md',
|
||||
'templates/trekbrief-template.md',
|
||||
'tests/fixtures/jsonl-schemas.md',
|
||||
...listMd('commands').map((f) => `commands/${f}`),
|
||||
...listMd('agents').map((f) => `agents/${f}`),
|
||||
];
|
||||
|
||||
// Historical/measurement records that MUST keep their gemini references.
|
||||
const GEMINI_HISTORICAL = [
|
||||
'CHANGELOG.md',
|
||||
'docs/cc-upgrade-2.1.181-decision-matrix.md',
|
||||
'docs/voyage-vs-cc-balance-analysis.md',
|
||||
'docs/BRIEF-vurdering-v2.md',
|
||||
'docs/balance-backlog-plan.md',
|
||||
'docs/agent-description-token-trim-brief.md',
|
||||
'docs/S22-happy-path-dogfood.md',
|
||||
'docs/deep-research-engine-brief.md',
|
||||
'examples/02-real-cli/REGENERATED.md',
|
||||
];
|
||||
|
||||
test('S95: agents/gemini-bridge.md is removed from the agent inventory', () => {
|
||||
assert.ok(
|
||||
!listMd('agents').includes('gemini-bridge.md'),
|
||||
'gemini-bridge is dropped by operator directive — agents/gemini-bridge.md must not exist',
|
||||
);
|
||||
});
|
||||
|
||||
test('S95: no live pipeline surface still names the gemini-bridge agent', () => {
|
||||
const offenders = [];
|
||||
for (const rel of GEMINI_LIVE_SURFACES) {
|
||||
// Everything gemini EXCEPT the vestigial gemini_used stats field.
|
||||
const hits = (read(rel).match(/gemini[\w-]*/gi) || []).filter(
|
||||
(m) => m.toLowerCase() !== 'gemini_used',
|
||||
);
|
||||
if (hits.length) offenders.push(`${rel} (${[...new Set(hits)].join(', ')})`);
|
||||
}
|
||||
assert.deepEqual(
|
||||
offenders, [],
|
||||
`gemini-bridge is dropped — these live surfaces still reference it: ${offenders.join(' · ')}`,
|
||||
);
|
||||
});
|
||||
|
||||
test('S95: known-positive control — the historical records still DO carry gemini references', () => {
|
||||
// Verifiseringsloven ansikt 4: a scanner that finds nothing anywhere proves
|
||||
// nothing. This pins that the same query CAN find, so the empty result above
|
||||
// is a measurement and not a broken query.
|
||||
const silent = GEMINI_HISTORICAL.filter((rel) => !/gemini/i.test(read(rel)));
|
||||
assert.deepEqual(
|
||||
silent, [],
|
||||
`these historical records lost their gemini references — history must not be rewritten: ${silent.join(', ')}`,
|
||||
);
|
||||
});
|
||||
|
||||
test('S95: trekplan high-effort Pass 2 degrades explicitly instead of failing', () => {
|
||||
const t = read('commands/trekplan.md');
|
||||
const section = t.split('\n### High-effort behavior (v5.1.1)\n')[1];
|
||||
assert.ok(section, 'trekplan.md must still carry the High-effort behavior sub-section');
|
||||
const head = section.split('\n## ')[0];
|
||||
assert.match(
|
||||
head, /unavailable, skipped/,
|
||||
'the dropped Pass 2 must read "unavailable, skipped" explicitly — a removed pass that is silent is indistinguishable from one that failed',
|
||||
);
|
||||
});
|
||||
|
||||
test('S95: gemini_used survives as a vestigial stats field (export schema unbroken)', () => {
|
||||
assert.ok(
|
||||
TREKRESEARCH_ALLOWED.has('gemini_used'),
|
||||
'gemini_used stays in the trekresearch export allowlist — the agent is dropped, the field is pinned false, and removing it would break existing consumers',
|
||||
);
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue