voyage/docs/devils-advocate-plan.md

Devil's-advocate session — adversarial cold audit of Voyage (Dynamic Workflow)

Status: Planned 2026-06-18 (post-S13). Executes in the NEXT session (after the operator's /clear). Operator mandate (2026-06-18, verbatim): «Planlegg en "devils advocate" sesjon … hvor du nå oppdaterer STATE.md og forklarer at det skal brukes en Dynamic Workflow for analysen. … denne pluginen er veldig viktig for meg.» Opt-in: The operator explicitly requested a Dynamic Workflow for the analysis — the next session is authorized to call the Workflow tool (multi-agent orchestration). This is a durable opt-in for that session.

Why this, why now

v5.5.0 just shipped. Today's reflection surfaced an uncomfortable fact: Voyage was not used to build Voyage — the whole upgrade arc was hand-orchestrated with bespoke planning docs. Before investing further, subject the plugin to a cold, hostile audit that actively tries to prove it is over-engineered, wrong, or unmaintainable — and see what survives. The operator cares about this plugin; the kindest thing is an honest adversary, not applause.

Objective (what the session must produce)

A severity-tagged, adversarially-verified critique written to docs/devils-advocate-results.md: where Voyage is genuinely weak, where its claims outrun its code, and what (if anything) should change. This is a steelman attack plus an honest rebuttal pass — not a feature wishlist, not vibes. Every finding cites file:line.

Why a Dynamic Workflow (not inline, not the Voyage pipeline)

• The audit is a fan-out → verify → synthesize shape across independent adversarial angles — exactly the Workflow tool's review/red-team pattern. Independent skeptics that can't see each other's reasoning catch more than one context reasoning sequentially, and the rebuttal pass kills plausible-but-unfair critiques.
• It is deliberately not run through Voyage's own /trek* pipeline: the point is to judge Voyage from the outside, cold — not to dogfood it (that is a separate exercise the prior session flagged).
• The operator asked for a Dynamic Workflow explicitly.

Adversarial dimensions — one finder each, build the STRONGEST case AGAINST

Each finder reads the actual code/docs (cite file:line); no claim without evidence.

• D1 — Ceremony vs. value (existential). Steelman: "Voyage is a sophistication trap — Plan mode + /code-review + a TODO gets 90% of the outcome at 10% of the cognitive/token cost. 7 commands, 24 agents, 697 tests are overhead, not value." Exhibit A: today's non-dogfood.
• D2 — Today's decisions were rationalizations, not conclusions. Steelman each: CC-26/27 "narrow hybrid not wholesale" (genuine analysis or status-quo bias? the bake-offs were largely designed-not-run or deterministic estimates); NW2 "POSITIVE" at +4.4% tokens (is that positive? was fidelity real?); NW3 Δ=0% decline (the measurement was chars/4 with swept, not API-measured BASE — is the decline robust or an unfalsifiable-estimate artifact?); v5.5.0 versioning (skipping 5.2–5.4 as "unreleased internal milestones" — coherent for consumers, or post-hoc tidy-up?).
• D3 — Brief-as-PUBLIC-CONTRACT is fragile. Steelman: the framing-enforcement invariant ("brief framing must match operator intent") is itself an admission that the pipeline structurally polishes wrong premises. Do the three 2.2 defensive layers actually prevent that failure, or just add friction + a checkbox?
• D4 — Orchestration architecture rests on shifting harness behavior. Steelman: the v2.4.0 "inline orchestration" migration was forced by a premise ("harness doesn't expose Agent to sub-agents") that turned out false (CC-01/S7). The architecture depends on harness behavior the plugin can't control; the dormant synthesis-agent + prose-vs-Workflow indecision are symptoms.
• D5 — Maintainability / rot / solo-maintainer risk. Steelman: 24 agents (one dormant), 7 commands, many docs, 697 tests — a large share pinning prose (doc-consistency). Is that testing value or testing wording? Is this past the complexity event horizon for one fork-and-own maintainer?
• D6 — Claims vs. reality (honesty audit). Steelman: README/CLAUDE.md make strong claims (context engineering, adversarial review, disciplined execution). Take the load-bearing ones and check whether the code delivers or whether it's aspirational prose. (Turn Voyage's own conformance discipline back on its self-description.)

Workflow shape (pipeline by default — refine the script on run)

export const meta = {
  name: 'voyage-devils-advocate',
  description: "Adversarial cold audit of Voyage — steelman attacks + honest rebuttals",
  phases: [
    { title: 'Attack',     detail: 'one skeptic per dimension builds the strongest case against' },
    { title: 'Rebut',      detail: 'steelman the defense; verdict STANDS / WEAKENED / REFUTED' },
    { title: 'Synthesize', detail: 'prioritize survivors + completeness critic' },
  ],
}
const DIMENSIONS = [ /* D1..D6, each {key, prompt} from the section above */ ]
const audited = await pipeline(
  DIMENSIONS,
  d => agent(`Devil's advocate on Voyage, dimension "${d.key}". ${d.prompt} Read the real code/docs, cite file:line, build the STRONGEST case AGAINST. No vibes.`,
        { label:`attack:${d.key}`, phase:'Attack', schema: FINDING_SCHEMA, effort:'high' }),
  (finding, d) => agent(`Steelman the DEFENSE against this critique, then judge each claim FATAL/MAJOR/MINOR and STANDS/WEAKENED/REFUTED under the best rebuttal: ${JSON.stringify(finding)}`,
        { label:`rebut:${d.key}`, phase:'Rebut', schema: VERDICT_SCHEMA, effort:'high' }),
)
const synthesis = await agent(`Synthesize this adversarial audit into a prioritized verdict + a "what this audit might have missed" section. Findings+rebuttals: ${JSON.stringify(audited.filter(Boolean))}`,
  { phase:'Synthesize', schema: SYNTHESIS_SCHEMA, effort:'high' })
return synthesis

• FINDING_SCHEMA: { dimension, findings: [{ claim, severity: FATAL|MAJOR|MINOR, evidence: [file:line], why_it_matters }] }
• VERDICT_SCHEMA: { dimension, verdicts: [{ claim, strongest_rebuttal, verdict: STANDS|WEAKENED|REFUTED, residual_severity }] }
• SYNTHESIS_SCHEMA: { survivors: [...ranked], refuted: [...], top_changes: [...], audit_blind_spots: [...] }

Scale, models, cost

~6 attack + 6 rebut + 1 synthesis ≈ 13 agents. effort: high on attack + synthesis (adversarial reasoning); Opus is the default (inherit session model) — pure-retrieval sub-steps may drop to sonnet. Before launching, announce the plan + rough agent count so the operator sees the scope (Workflows are token-heavy).

Success criteria (verifiseringsplikt)

1. docs/devils-advocate-results.md written, with ≥1 finding per dimension, each carrying file:line evidence and an explicit STANDS / WEAKENED / REFUTED verdict after the rebuttal pass.
2. A synthesis that names which critiques are REAL (worth acting on, prioritized by severity × actionability) vs. survived-as-fine.
3. A "what this audit might have missed" section (completeness critic).
4. No finding asserted without evidence; rebuttals are genuine defenses, not strawmen.
5. The audit critiques this very plan too (the dimensions may be wrong/incomplete — say so).

Scope guard

This is an audit that produces findings + recommendations, NOT an implementation session. Do not start changing Voyage based on the results without a fresh operator go-ahead. Stop at the results doc + a short summary.