voyage/docs/BRIEF-vurdering-v2.md

BRIEF — external assessment (VURDERING-V2 §5.12), verified against this repo

Status: assessment complete. No implementation in this session (scope-guard). Source under review: claude-playlist-corpus/docs/VURDERING-V2.md §5.12 (+ §3 G1–G8, §4), transferred via that repo's docs/OVERFORING-V2.md §5. It is an external recommendation built on 442 video analyses; its repo facts come from a subagent survey (2026-07-17) that is not re-verified and in which at least one error was already demonstrated.

Method. Every asserted GAP is marked BEKREFTET (confirmed) / AVKREFTET (refuted) / ENDRET (true in altered form) only after checking it against code in this repo. Every Claude Code feature claim is checked against the OKF bundle at claude-code-llm-wiki/bundle/, index-first — where V2 and the bundle disagree, the bundle wins. Absences are positive-controlled before being reported (a grep that finds nothing is a measurement, not a fact).

---

TL;DR

Of six recommended tiltak, one is a real defect (fail-open verdict computation), one is unmeasured rather than unresolved (the Workflow port), three are already decided or already partly built, and one is a cheap confirmed gap.

The single most consequential correction: V2 presents the Workflow port as the answer to this repo's "main-context relief is unmeasured" caveat. The bundle confirms the mechanism, but this repo has still never measured main-context relief on a large fan-out — the one bake-off that exists measured subagent tokens and says so. The port is therefore neither refuted nor justified. The next step is a measurement, not a port, and the measurement design already exists in docs/T1-cc26-delegated-orchestration.md.

---

Tiltak 1 — Port orchestration to dynamic Workflows

Verdict: split per command. Do not issue one global verdict.

/trekreview — ENDRET (already ported, and measured)

The port exists: opt-in --workflow, scripts/trekreview-armB.workflow.mjs, documented at commands/trekreview.md:270 (§ Phase 5–6 via the Workflow substrate). The S10 bake-off (docs/T2-bakeoff-results.md, ≥3 runs/arm) found it fidelity-equivalent: verdict-match 1.0, issue-coverage 100%, (file,rule_key) jaccard ≥ within-arm. It stays opt-in because the Workflow tool raises the consumer floor to Claude Code 2.1.154+, and the prose path keeps the lower floor.

/trekplan, /trekresearch — BEKREFTET as unported, UNMEASURED as beneficial

Both still orchestrate their swarms from main context (commands/trekplan.md:406, commands/trekresearch.md:388). That part of V2 is correct.

What is not established is V2's stated effect — that Workflow answers the main-context-relief caveat. Two measurements exist in this repo and neither measures it:

Measurement	What it actually measured	Bearing on the relief claim
docs/T2-bakeoff-results.md:154	Subagent tokens: Arm A median 92.8k vs Arm B 96.9k ⇒ +4.4%	None. The same line states Arm A's main-context hand-orchestration cost is uncounted.
docs/T1-synthesis-poc-results.md	Δ main-context ≈ 0% — for delegating Phase 7 synthesis alone	Negative, but out of scope: that doc itself names a Phase-5 redesign as the prerequisite for any relief.

So the Phase-5 swarm — the case V2 is actually recommending — has never been measured in either arm, in either command.

The measurement already has a design. docs/T1-cc26-delegated-orchestration.md:118 defines the gate metric (Δ main-context tokens resident in the main session at plan-complete, from stream-json usage) with pre-registered bars: adopt at ≥30%, reject below 15%. Running Arm A vs Arm B once against the trekplan swarm settles this.

Cost framing must be corrected. The bundle states plainly that a workflow run "can use meaningfully more tokens than working through the same task in conversation" (bundle/concepts/docs-en-workflows.md:364) — consistent with the measured +4.4%. The port cannot be justified as a cost saving. Main-context headroom is the only defensible motive, and it is exactly the unmeasured quantity.

Recommendation: measure before porting. Treat V2's tiltak 1 as a hypothesis with a ready-made experiment, not as an approved change.

---

Tiltak 2 — Codify the holdout rules

Verdict: (a) and (b) satisfied in practice but uncodified · (c) BEKREFTET, a real defect · (d) ENDRET. This is the tiltak with genuine substance.

(a) Reviewers must never be forks — AVKREFTET as a live defect, BEKREFTET as uncodified

Reviewers are spawned as plain subagents (voyage:brief-conformance-reviewer, voyage:code-correctness-reviewer). subagent_type: "fork" appears nowhere in commands/ or agents/; the only subagent_type in the tree is "Explore" at commands/trekplan.md:501, an exploration spawn, not a review spawn. The bundle confirms the rule is real — a forked subagent "inherits your full conversation context instead of starting fresh" (bundle/concepts/docs-en-agents.md:37). Voyage satisfies it by habit; nothing written forbids a future edit from breaking it.

(b) Explicit "you shall not see" list — AVKREFTET as a live defect, BEKREFTET as uncodified

Phase 5 feeds reviewers exactly four things: the Phase-3 unified diff, the triage map, the brief path, and the rule catalogue (commands/trekreview.md, Phase 5 input list). Implementation plans and commit messages are not fed; git log appears only to compute the diff range and to print a suggested narrowing command (commands/trekreview.md:132, :181). Again: correct today, unprotected tomorrow.

(c) Fail-closed verdicts — BEKREFTET. This is the one real defect.

agents/review-coordinator.md computes the verdict mechanically from the findings that survive filtering:

• Pass 2 (HubSpot judge) and Pass 3 (Cloudflare reasonableness) drop findings that fail their tests (agents/review-coordinator.md:87, :101).
• Pass 4 counts the survivors: BLOCKER ≥ 1 → BLOCK; else MAJOR ≥ 1 → WARN; else ALLOW (:132). "Verdict is mechanical — never override."

There is no third state. Three consequences follow:

1. A finding that cannot be substantiated is dropped, and a dropped finding contributes zero — which moves the verdict toward ALLOW. Pass 3's non-existent-file test ("file does not exist in the working tree AND the diff does not show it as (new file)") has no inconclusive branch: unresolvable and refuted are treated identically. This is structurally the failure V2 attributes to its source case, where "not E2E testable" passed as approved.
2. A reviewer that returns nothing produces the same result. Phase 5 has no empty-return handling — grep for empty|no findings|did not return|abort over commands/trekreview.md returns 0 (positive control: Phase 5 = 7 hits, reviewer = 24 hits, so the file and the query are both live). Zero findings from a silent reviewer is indistinguishable from zero findings from a clean diff. This is the same surface as the idle-agent hole recorded in STATE's open decisions.
3. One fail-closed branch already exists, and it is the pattern to copy. When reviewer output fails schema validation, Phase 5 allows 2 bounded re-asks and then stops: "do not feed unvalidated findings to the coordinator." That is the correct shape, applied to one failure mode only.

The defect is codified, not merely prose — which makes it testable. The four passes exist deterministically in lib/review/coordinator-contract.mjs, and computeVerdict(findings) (:184–:193) counts only the findings handed to it. runContract hands it reasoned.kept (:206); suppressed and skipped are returned in the result object but carry no weight in the verdict. So a dropped finding is arithmetically identical to a finding that never existed.

Minimal fix shape (not implemented here): an unverified bucket that is neither kept nor dropped, plus one rule — a non-empty unverified bucket forbids ALLOW — and a Phase-5 check that a reviewer actually returned. Both are additive; neither touches the severity catalogue or the existing thresholds. Because the logic is deterministic ("No LLM, no network, no time, no randomness", :23) and already has tests/lib/coordinator-contract.test.mjs, this can be driven test-first under the Iron Law — a failing test asserting that an unverifiable BLOCKER-severity finding cannot yield ALLOW is writable before any production change.

(d) Presume-failure framing — ENDRET

Both reviewers are already adversarially framed: "Adversarial reviewer" in each description, "You never praise", "You never say 'looks good'" (agents/code-correctness-reviewer.md:36, agents/brief-conformance-reviewer.md:35). What is absent is specifically the presume-failure formulation V2 names ("this agent was lazy — find out why"). This is a wording change with no measurement behind it in the corpus; treat it as optional polish, not a gap.

---

Tiltak 3 — Lightweight lane — ENDRET (narrower gap than stated, already tracked)

A lightweight path exists: --quick on both /trekbrief and /trekplan, and /trekplan --quick already skips the Phase-5 exploration swarm (commands/trekplan.md:122, :885). What it does not do is bypass the 2.2 ceremony — commands/trekbrief.md:126 states the framing question "is asked even in --quick mode", and a 2.2 brief still requires framing and a ## TL;DR (:472, :474).

So the accurate gap is not "no lightweight path" but "the lightweight path still pays the 2.2 toll". Already carried as open operator decision 1 in STATE — this is not a discovery.

---

Tiltak 4 — Definition-of-done as an object — BEKREFTET

commands/trekbrief.md contains no notion of evidence, proof, verifier, residual risk, or next-step owner (grep for evidence|proof|screenshot|verifier|residual risk|owner → 0; positive control: Success Criteri = 5 hits). The gap is real.

Cost note V2 does not carry: the brief schema is Handover 1, a public contract (docs/HANDOVER-CONTRACTS.md). Adding required DoD fields is a breaking change for downstream consumers, i.e. a brief_version bump with a gate, not an edit. That moves it out of "cheap win" and into planned work.

---

Tiltak 5 — Prune, don't build — mostly ALREADY DECIDED

• synthesis-agent: already dormant and labelled as such (agents/synthesis-agent.md:3), with the measurement that justified it (docs/T1-synthesis-poc-results.md, Δ≈0, DECLINED). Retiring versus keeping was decided in favour of keeping it as a re-measurable building block. V2 recommends a decision that has been made and recorded.
• 24 all-opus agents: 24 of 24 agent files carry model: opus (verified by count). This is an operator pin, not drift — commit 40d8742 "pin all sub-agents to Opus permanently (operator request)" — and the reconsideration V2 asks for already happened and is written down in docs/voyage-vs-cc-balance-analysis.md §10. Re-opening it is an operator decision, not an analysis task.
• gemini-bridge: BEKREFTET, and the cheapest real item in this tiltak. It is already flagged THIN_WRAP / DROP→NATIVE in the same analysis (§V09, lines 81, 121, 149) and it is broken at the engine (the gemini-mcp server fails deterministically on an SDK/API mismatch). Fix-or-drop is decidable today.

---

Tiltak 6 — LSP as a harness component — BEKREFTET

LSP returns 0 hits across commands/, agents/, lib/, docs/. Voyage navigates with Glob/Grep, as V2 says. One qualifier: LSP is a harness-level capability available to agents, not a voyage feature — adoption is mostly permitting and prompting agents to use it, not code in this repo. Value is highest in large target repos, which is where voyage's exploration swarm actually runs.

---

Bundle-gap

Checked against bundle/concepts/docs-en-workflows.md (source_sha 363819ed9ec325275ca22023f6bb6b98fbbf6fcc12db0667478c5207e117751b, timestamp 2026-08-15), plus docs-en-agents.md and docs-en-sub-agents.md.

Confirmed by the bundle:

• The 2.1.154 floor for dynamic workflows, on paid plans.
• The main-context relief mechanism: "A workflow script holds the loop, the branching, and the intermediate results itself, so Claude's context holds only the final answer" (:38); "Intermediate results stay in script variables instead of landing in Claude's context" (:316). V2's mechanism claim is sound — it is the magnitude that is unmeasured.
• Per-stage model routing (:375–:378).
• Cost caps: agent caps, size guideline, and a Large workflow warning above 25 agents or 1.5M projected tokens (:368).

Where V2 overstates, and the bundle wins:

• "tokenbudsjetter" (token budgets). The page documents caps and size guidelines, not a token budget — budget returns 0 hits on the page. V2's warning to "set an explicit token budget" has no documented primitive to point at in the official docs.
• "automatisk retry". 0 hits. Not documented on the page.

Limitation V2 omits, material for this repo: resumability is session-scoped — "Resume works within the same Claude Code session. If you exit Claude Code while a workflow is running, the next session starts the workflow fresh" (:360). Further, stopping mid fan-out re-runs every agent that started after the stopped one (:354). Voyage is explicitly a multi-session tool, so resumability is a weaker argument here than V2 implies.

Gap proper (feedback toward wiki v1.0):

1. The mirrored docs describe no programmatic token-budget primitive for workflows, while the Workflow tool's own runtime surface does expose a budget derived from an operator token directive. If that surface is real and stable, it is missing from the mirror. Flagged as a discrepancy, not asserted as a doc error.
2. Nothing in the mirrored pages quantifies what a workflow's final return costs the orchestrating session. The relief mechanism is described qualitatively only, so a consumer cannot size the benefit from the docs alone — which is precisely why the measurement below cannot be replaced by reading.

---

Recommended order (operator decides; nothing started)

1. Fail-closed coordinator (tiltak 2c) — smallest surface, highest value, no dependencies, and it closes a defect rather than adding a feature.
2. Measure Δ main-context on the /trekplan Phase-5 swarm (tiltak 1) — Arm A vs Arm B against the pre-registered bars in docs/T1-cc26-delegated-orchestration.md. The port decision follows the number; it does not precede it.
3. gemini-bridge fix-or-drop (tiltak 5) — already analysed, currently broken, cheap.

Deferred with reasons: tiltak 3 (already open operator decision 1) · tiltak 4 (requires a brief_version bump against a public contract) · tiltak 5's opus/synthesis items (decided; operator-pinned) · tiltak 6 (harness-level, not repo code).

---

Verification log

Claim	How verified
/trekreview Workflow port exists and was measured	commands/trekreview.md:270; docs/T2-bakeoff-results.md §Full run
+4.4% is subagent tokens, main context uncounted	docs/T2-bakeoff-results.md:154 (verbatim)
Δ main-context ≈ 0 applies to Phase 7 only	docs/T1-synthesis-poc-results.md:1, :100
Measurement design already exists	docs/T1-cc26-delegated-orchestration.md:118, :125
Reviewers are not forks	grep subagent_type|fork commands/ agents/ → only Explore at commands/trekplan.md:501
Plans/commit messages not fed to reviewers	Phase 5 input list; commands/trekreview.md:132, :181
Verdict is computed from survivors only	agents/review-coordinator.md:87, :101, :132
Same rule codified deterministically in lib/	lib/review/coordinator-contract.mjs:184–:193, :206; existing test tests/lib/coordinator-contract.test.mjs
No empty-return handling in Phase 5	grep → 0, positive control Phase 5=7, reviewer=24
Existing fail-closed branch on schema failure	commands/trekreview.md, Phase 5 bounded-retry paragraph
--quick does not bypass 2.2	commands/trekbrief.md:126, :472, :474
No DoD fields in trekbrief	grep → 0, positive control Success Criteri=5
24/24 agents pinned opus, by operator	file count 24/24; git log 40d8742
gemini-bridge already flagged for drop	docs/voyage-vs-cc-balance-analysis.md:81, :121, :149
No LSP usage	grep -rn LSP commands/ agents/ lib/ docs/ → 0
Workflow floor, relief mechanism, model routing, caps	bundle/concepts/docs-en-workflows.md:19, :38, :316, :375, :368
No documented token budget or retry in the bundle	grep -i budget|retry on that page → 0 relevant
Resume is session-scoped	bundle/concepts/docs-en-workflows.md:354, :360
Fork inherits full conversation context	bundle/concepts/docs-en-agents.md:37

Not verified / stated as unverified: V2's underlying video-corpus claims (the StrongDM bad-merge case, the 5.5-hour bake-off, the "$200 plan in 30 minutes" workflow) were not independently checked — they are cited here as V2's evidence, not as this repo's findings.