From f971db8231c383542d18be3ab8a39f6130650e30 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Thu, 18 Jun 2026 20:28:57 +0200 Subject: [PATCH] =?UTF-8?q?docs(voyage):=20S14=20addendum=20=E2=80=94=20ve?= =?UTF-8?q?rify=20finding=20#1,=20surface=20default-profile=20bug?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operator challenged audit finding #1 (README 'cheap Sonnet' vs agents opus). Verified against code: per-phase model system is real (operator correct), but no default profile makes exploration/review sonnet (balanced+premium both set plan/review=opus; only opt-in economy=sonnet) -> finding #1 STANDS. NEW defect audit missed: resolveProfile() defaults to 'premium' but README:759 + profiles.md say 'balanced' — code-vs-docs mismatch, unguarded. Corrected fix recorded; keeps Opus-as-default. --- docs/devils-advocate-results.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/devils-advocate-results.md b/docs/devils-advocate-results.md index a936a20..1918428 100644 --- a/docs/devils-advocate-results.md +++ b/docs/devils-advocate-results.md @@ -128,3 +128,17 @@ The audit was instructed to critique its own plan. It did: ## Scope note This is an **audit** — findings + recommendations only. Per the plan's scope guard and the operator's run mode, **no Voyage code or docs were changed** on the basis of these results (the one exception: STATE.md's stale `697` test count is corrected at session end, since STATE is continuity state, not plugin surface, and propagating a now-verified-wrong number would be its own defect). Acting on the top changes above requires a fresh operator go-ahead. + +--- + +## Addendum — post-audit verification of finding #1 (2026-06-18, operator-prompted) + +The operator challenged finding #1 ("README sells cheap Sonnet but agents are opus"), correctly recalling that Voyage has a **per-phase model-selection system** (profiles + `phase_signals.model`). Main context verified the actual model-resolution path against the code. Result: **the operator's recollection is right, the finding still STANDS, and verification surfaced a NEW defect the audit missed.** + +1. **The per-phase system is real.** `lib/profiles/resolver.mjs#resolveProfile()` + `phase-signal-resolver.mjs` resolve a model per *phase* (= command). At Agent-spawn sites, the resolved phase model is used if set; otherwise the agent's `model: opus` frontmatter (trekplan.md "Cost" hard rule). The model is **uniform per phase** (orchestrator + swarm share it) — there is **no** mechanism to run the orchestrator on opus and the swarm on sonnet within one phase, so the README cost-prose's "Opus orchestrates / Sonnet runs the swarms" split is **not an achievable configuration**, independent of the count defect. + +2. **No default profile makes exploration/review Sonnet.** `balanced` and `premium` both set `plan → opus` and `review → opus` (verified: `lib/profiles/balanced.yaml`, `premium.yaml`). Only opt-in `economy` sets `plan/review → sonnet`. So under every *default*-candidate profile the exploration AND review swarms run on **Opus**. README's unconditional "6–8 Sonnet exploration agents" / "Sonnet runs the exploration and review swarms" (195/223/266/785/857) is false for every default. **Finding #1 STANDS and is reinforced.** + +3. **NEW defect (audit missed it): code default profile ≠ documented default.** `resolveProfile()` returns `{ profile: 'premium', profile_source: 'default' }` (resolver.mjs:156; docstring: "Order: --profile flag > VOYAGE_PROFILE env > 'premium'"). But README.md:759 ("`balanced` (default)") + the lookup-order list + `docs/profiles.md:15,18,121` ("`balanced` is the v4.1 default", "the default tier is locked") all say the default is **`balanced`**. **Code says premium; all docs say balanced** — a real code-vs-docs contradiction with cost/behavior impact (premium runs brief+execute on opus and turns external research ON; balanced does not), unguarded by `doc-consistency.test.mjs`. Severity: **MAJOR**. Most likely resolution: docs reflect design intent (balanced), so the code default is the bug — but confirm against git history before fixing. + +**Corrected fix for finding #1 (supersedes "flip vs rewrite"):** (a) resolve the default-profile mismatch so code + README + `docs/profiles.md` agree on one default (likely `balanced`); (b) rewrite the README cost narrative to describe the *actual* per-phase-profile mechanism (default runs plan/review on Opus; `economy` runs all-Sonnet; no orchestrator-vs-swarm split); (c) add doc-consistency pins for the default-profile name and the per-phase model claims. This keeps Opus-as-default and does **not** require flipping any agent frontmatter.