# BRIEF — external assessment (VURDERING-V2 §5.12), verified against this repo **Status:** assessment complete. **No implementation in this session** (scope-guard). **Source under review:** `claude-playlist-corpus/docs/VURDERING-V2.md` §5.12 (+ §3 G1–G8, §4), transferred via that repo's `docs/OVERFORING-V2.md` §5. It is an external recommendation built on 442 video analyses; its repo facts come from a subagent survey (2026-07-17) that is **not** re-verified and in which at least one error was already demonstrated. **Method.** Every asserted GAP is marked **BEKREFTET** (confirmed) / **AVKREFTET** (refuted) / **ENDRET** (true in altered form) only after checking it against code in this repo. Every Claude Code *feature* claim is checked against the OKF bundle at `claude-code-llm-wiki/bundle/`, index-first — **where V2 and the bundle disagree, the bundle wins**. Absences are positive-controlled before being reported (a grep that finds nothing is a measurement, not a fact). --- ## TL;DR Of six recommended tiltak, **one is a real defect** (fail-open verdict computation), **one is unmeasured rather than unresolved** (the Workflow port), **three are already decided or already partly built**, and **one is a cheap confirmed gap**. The single most consequential correction: V2 presents the Workflow port as the answer to this repo's "main-context relief is unmeasured" caveat. The bundle confirms the *mechanism*, but **this repo has still never measured main-context relief on a large fan-out** — the one bake-off that exists measured *subagent* tokens and says so. The port is therefore neither refuted nor justified. The next step is a measurement, not a port, and the measurement design already exists in `docs/T1-cc26-delegated-orchestration.md`. --- ## Tiltak 1 — Port orchestration to dynamic Workflows **Verdict: split per command. Do not issue one global verdict.** ### `/trekreview` — **ENDRET** (already ported, and measured) The port exists: opt-in `--workflow`, `scripts/trekreview-armB.workflow.mjs`, documented at `commands/trekreview.md:270` (*§ Phase 5–6 via the Workflow substrate*). The S10 bake-off (`docs/T2-bakeoff-results.md`, ≥3 runs/arm) found it **fidelity-equivalent**: verdict-match 1.0, issue-coverage 100%, `(file,rule_key)` jaccard ≥ within-arm. It stays opt-in because the Workflow tool raises the consumer floor to Claude Code 2.1.154+, and the prose path keeps the lower floor. ### `/trekplan`, `/trekresearch` — **BEKREFTET as unported, UNMEASURED as beneficial** Both still orchestrate their swarms from main context (`commands/trekplan.md:406`, `commands/trekresearch.md:388`). That part of V2 is correct. What is **not** established is V2's stated effect — that Workflow answers the main-context-relief caveat. Two measurements exist in this repo and **neither measures it**: | Measurement | What it actually measured | Bearing on the relief claim | |---|---|---| | `docs/T2-bakeoff-results.md:154` | **Subagent** tokens: Arm A median 92.8k vs Arm B 96.9k ⇒ **+4.4%** | None. The same line states Arm A's main-context hand-orchestration cost is **uncounted**. | | `docs/T1-synthesis-poc-results.md` | Δ **main-context** ≈ **0%** — for delegating *Phase 7 synthesis alone* | Negative, but out of scope: that doc itself names a **Phase-5 redesign** as the prerequisite for any relief. | So the Phase-5 swarm — the case V2 is actually recommending — has never been measured in either arm, in either command. **The measurement already has a design.** `docs/T1-cc26-delegated-orchestration.md:118` defines the gate metric (Δ main-context tokens resident in the main session at plan-complete, from stream-json `usage`) with pre-registered bars: adopt at ≥30%, reject below 15%. Running Arm A vs Arm B once against the trekplan swarm settles this. **Cost framing must be corrected.** The bundle states plainly that a workflow run "can use meaningfully more tokens than working through the same task in conversation" (`bundle/concepts/docs-en-workflows.md:364`) — consistent with the measured +4.4%. The port cannot be justified as a cost saving. **Main-context headroom is the only defensible motive, and it is exactly the unmeasured quantity.** **Recommendation:** measure before porting. Treat V2's tiltak 1 as a *hypothesis with a ready-made experiment*, not as an approved change. --- ## Tiltak 2 — Codify the holdout rules **Verdict: (a) and (b) satisfied in practice but uncodified · (c) BEKREFTET, a real defect · (d) ENDRET.** This is the tiltak with genuine substance. ### (a) Reviewers must never be forks — **AVKREFTET as a live defect, BEKREFTET as uncodified** Reviewers are spawned as plain subagents (`voyage:brief-conformance-reviewer`, `voyage:code-correctness-reviewer`). `subagent_type: "fork"` appears nowhere in `commands/` or `agents/`; the only `subagent_type` in the tree is `"Explore"` at `commands/trekplan.md:501`, an exploration spawn, not a review spawn. The bundle confirms the rule is real — a forked subagent "inherits your full conversation context instead of starting fresh" (`bundle/concepts/docs-en-agents.md:37`). Voyage satisfies it by habit; nothing written forbids a future edit from breaking it. ### (b) Explicit "you shall not see" list — **AVKREFTET as a live defect, BEKREFTET as uncodified** Phase 5 feeds reviewers exactly four things: the Phase-3 unified diff, the triage map, the brief path, and the rule catalogue (`commands/trekreview.md`, Phase 5 input list). Implementation plans and commit messages are **not** fed; `git log` appears only to compute the diff range and to print a suggested narrowing command (`commands/trekreview.md:132`, `:181`). Again: correct today, unprotected tomorrow. ### (c) Fail-closed verdicts — **BEKREFTET. This is the one real defect.** `agents/review-coordinator.md` computes the verdict mechanically from the findings that *survive* filtering: - **Pass 2** (HubSpot judge) and **Pass 3** (Cloudflare reasonableness) **drop** findings that fail their tests (`agents/review-coordinator.md:87`, `:101`). - **Pass 4** counts the survivors: `BLOCKER ≥ 1 → BLOCK`; else `MAJOR ≥ 1 → WARN`; else `ALLOW` (`:132`). "Verdict is mechanical — never override." There is no third state. Three consequences follow: 1. **A finding that cannot be substantiated is dropped, and a dropped finding contributes zero — which moves the verdict toward ALLOW.** Pass 3's non-existent-file test ("`file` does not exist in the working tree AND the diff does not show it as `(new file)`") has no inconclusive branch: unresolvable and refuted are treated identically. This is structurally the failure V2 attributes to its source case, where "not E2E testable" passed as approved. 2. **A reviewer that returns nothing produces the same result.** Phase 5 has no empty-return handling — grep for `empty|no findings|did not return|abort` over `commands/trekreview.md` returns 0 (positive control: `Phase 5` = 7 hits, `reviewer` = 24 hits, so the file and the query are both live). Zero findings from a silent reviewer is indistinguishable from zero findings from a clean diff. This is the same surface as the idle-agent hole recorded in STATE's open decisions. 3. **One fail-closed branch already exists, and it is the pattern to copy.** When reviewer output fails schema validation, Phase 5 allows 2 bounded re-asks and then *stops*: "do not feed unvalidated findings to the coordinator." That is the correct shape, applied to one failure mode only. **The defect is codified, not merely prose — which makes it testable.** The four passes exist deterministically in `lib/review/coordinator-contract.mjs`, and `computeVerdict(findings)` (`:184`–`:193`) counts *only* the findings handed to it. `runContract` hands it `reasoned.kept` (`:206`); `suppressed` and `skipped` are returned in the result object but **carry no weight in the verdict**. So a dropped finding is arithmetically identical to a finding that never existed. **Minimal fix shape (not implemented here):** an `unverified` bucket that is neither kept nor dropped, plus one rule — *a non-empty `unverified` bucket forbids ALLOW* — and a Phase-5 check that a reviewer actually returned. Both are additive; neither touches the severity catalogue or the existing thresholds. Because the logic is deterministic ("No LLM, no network, no time, no randomness", `:23`) and already has `tests/lib/coordinator-contract.test.mjs`, this can be driven test-first under the Iron Law — a failing test asserting that an unverifiable BLOCKER-severity finding cannot yield ALLOW is writable before any production change. ### (d) Presume-failure framing — **ENDRET** Both reviewers are already adversarially framed: "Adversarial reviewer" in each `description`, "You never praise", "You never say 'looks good'" (`agents/code-correctness-reviewer.md:36`, `agents/brief-conformance-reviewer.md:35`). What is absent is specifically the presume-failure formulation V2 names ("this agent was lazy — find out why"). This is a wording change with no measurement behind it in the corpus; treat it as optional polish, not a gap. --- ## Tiltak 3 — Lightweight lane — **ENDRET** (narrower gap than stated, already tracked) A lightweight path exists: `--quick` on both `/trekbrief` and `/trekplan`, and `/trekplan --quick` already skips the Phase-5 exploration swarm — "Skip agent swarm; use lightweight Glob/Grep scan and go directly to planning + adversarial review" (`commands/trekplan.md:122`). The same code-path is reachable a second way: a brief carrying `effort == 'low'` activates it without the flag (`:885`). What it does **not** do is bypass the 2.2 ceremony — `commands/trekbrief.md:126` states the framing question "is asked even in `--quick` mode", and a 2.2 brief still requires `framing` and a `## TL;DR` (`:472`, `:474`). So the accurate gap is not "no lightweight path" but "the lightweight path still pays the 2.2 toll". Already carried as open operator decision 1 in STATE — this is not a discovery. --- ## Tiltak 4 — Definition-of-done as an object — **BEKREFTET** `commands/trekbrief.md` contains no notion of evidence, proof, verifier, residual risk, or next-step owner (grep for `evidence|proof|screenshot|verifier|residual risk|owner` → 0; positive control: `Success Criteri` = 5 hits). The gap is real. **Cost note V2 does not carry:** the brief schema is **Handover 1**, a public contract (`docs/HANDOVER-CONTRACTS.md`). Adding required DoD fields is a breaking change for downstream consumers, i.e. a `brief_version` bump with a gate, not an edit. That moves it out of "cheap win" and into planned work. --- ## Tiltak 5 — Prune, don't build — **mostly ALREADY DECIDED** - **`synthesis-agent`**: already dormant and labelled as such (`agents/synthesis-agent.md:3`), with the measurement that justified it (`docs/T1-synthesis-poc-results.md`, Δ≈0, DECLINED). Retiring versus keeping was decided in favour of keeping it as a re-measurable building block. V2 recommends a decision that has been made and recorded. - **24 all-opus agents**: 24 of 24 agent files carry `model: opus` (verified by count). This is an **operator pin**, not drift — commit `40d8742` "pin all sub-agents to Opus permanently (operator request)" — and the reconsideration V2 asks for already happened and is written down in `docs/voyage-vs-cc-balance-analysis.md` §10. Re-opening it is an operator decision, not an analysis task. - **`gemini-bridge`**: **BEKREFTET**, and the cheapest real item in this tiltak. It is already flagged `THIN_WRAP` / `DROP→NATIVE` in the same analysis (§V09, lines 81, 121, 149) *and* it is broken at the engine (the `gemini-mcp` server fails deterministically on an SDK/API mismatch). Fix-or-drop is decidable today. --- ## Tiltak 6 — LSP as a harness component — **BEKREFTET** `LSP` returns 0 hits across `commands/`, `agents/`, `lib/`, `docs/`. Voyage navigates with Glob/Grep, as V2 says. One qualifier: LSP is a harness-level capability available to agents, not a voyage feature — adoption is mostly permitting and prompting agents to use it, not code in this repo. Value is highest in large target repos, which is where voyage's exploration swarm actually runs. --- ## Bundle-gap Checked against `bundle/concepts/docs-en-workflows.md` (`source_sha 363819ed9ec325275ca22023f6bb6b98fbbf6fcc12db0667478c5207e117751b`, timestamp 2026-08-15), plus `docs-en-agents.md` and `docs-en-sub-agents.md`. **Confirmed by the bundle:** - The **2.1.154 floor** for dynamic workflows, on paid plans. - The **main-context relief mechanism**: "A workflow script holds the loop, the branching, and the intermediate results itself, so Claude's context holds only the final answer" (`:38`); "Intermediate results stay in script variables instead of landing in Claude's context" (`:316`). V2's mechanism claim is sound — it is the *magnitude* that is unmeasured. - **Per-stage model routing** (`:375`–`:378`). - **Cost caps**: agent caps, size guideline, and a `Large workflow` warning above 25 agents or 1.5M projected tokens (`:368`). **Where V2 overstates, and the bundle wins:** - **"tokenbudsjetter" (token budgets).** The page documents *caps and size guidelines*, not a token budget — `budget` returns 0 hits on the page. V2's warning to "set an explicit token budget" has no documented primitive to point at in the official docs. - **"automatisk retry".** 0 hits. Not documented on the page. **Limitation V2 omits, material for this repo:** resumability is **session-scoped** — "Resume works within the same Claude Code session. If you exit Claude Code while a workflow is running, the next session starts the workflow fresh" (`:360`). Further, stopping mid fan-out re-runs every agent that started after the stopped one (`:354`). Voyage is explicitly a *multi-session* tool, so resumability is a weaker argument here than V2 implies. **Gap proper (feedback toward wiki v1.0):** 1. The mirrored docs describe no programmatic **token-budget** primitive for workflows, while the Workflow tool's own runtime surface does expose a budget derived from an operator token directive. If that surface is real and stable, it is missing from the mirror. Flagged as a discrepancy, not asserted as a doc error. 2. Nothing in the mirrored pages quantifies what a workflow's **final return** costs the orchestrating session. The relief mechanism is described qualitatively only, so a consumer cannot size the benefit from the docs alone — which is precisely why the measurement below cannot be replaced by reading. --- ## Recommended order (operator decides; nothing started) 1. **Fail-closed coordinator (tiltak 2c)** — smallest surface, highest value, no dependencies, and it closes a defect rather than adding a feature. 2. **Measure Δ main-context on the `/trekplan` Phase-5 swarm (tiltak 1)** — Arm A vs Arm B against the pre-registered bars in `docs/T1-cc26-delegated-orchestration.md`. The port decision follows the number; it does not precede it. 3. **`gemini-bridge` fix-or-drop (tiltak 5)** — already analysed, currently broken, cheap. **Deferred with reasons:** tiltak 3 (already open operator decision 1) · tiltak 4 (requires a `brief_version` bump against a public contract) · tiltak 5's opus/synthesis items (decided; operator-pinned) · tiltak 6 (harness-level, not repo code). --- ## Verification log | Claim | How verified | |---|---| | `/trekreview` Workflow port exists and was measured | `commands/trekreview.md:270`; `docs/T2-bakeoff-results.md` §Full run | | +4.4% is **subagent** tokens, main context uncounted | `docs/T2-bakeoff-results.md:154` (verbatim) | | Δ main-context ≈ 0 applies to Phase 7 only | `docs/T1-synthesis-poc-results.md:1`, `:100` | | Measurement design already exists | `docs/T1-cc26-delegated-orchestration.md:118`, `:125` | | Reviewers are not forks | `grep subagent_type\|fork commands/ agents/` → only `Explore` at `commands/trekplan.md:501` | | Plans/commit messages not fed to reviewers | Phase 5 input list; `commands/trekreview.md:132`, `:181` | | Verdict is computed from survivors only | `agents/review-coordinator.md:87`, `:101`, `:132` | | Same rule codified deterministically in `lib/` | `lib/review/coordinator-contract.mjs:184`–`:193`, `:206`; existing test `tests/lib/coordinator-contract.test.mjs` | | No empty-return handling in Phase 5 | grep → 0, positive control `Phase 5`=7, `reviewer`=24 | | Existing fail-closed branch on schema failure | `commands/trekreview.md`, Phase 5 bounded-retry paragraph | | `--quick` does not bypass 2.2 | `commands/trekbrief.md:126`, `:472`, `:474` | | No DoD fields in trekbrief | grep → 0, positive control `Success Criteri`=5 | | 24/24 agents pinned opus, by operator | file count 24/24; `git log 40d8742` | | gemini-bridge already flagged for drop | `docs/voyage-vs-cc-balance-analysis.md:81`, `:121`, `:149` | | No LSP usage | `grep -rn LSP commands/ agents/ lib/ docs/` → 0 | | Workflow floor, relief mechanism, model routing, caps | `bundle/concepts/docs-en-workflows.md:19`, `:38`, `:316`, `:375`, `:368` | | No documented token budget or retry in the bundle | `grep -i budget\|retry` on that page → 0 relevant | | Resume is session-scoped | `bundle/concepts/docs-en-workflows.md:354`, `:360` | | Fork inherits full conversation context | `bundle/concepts/docs-en-agents.md:37` | **Not verified / stated as unverified:** V2's underlying video-corpus claims (the StrongDM bad-merge case, the 5.5-hour bake-off, the "$200 plan in 30 minutes" workflow) were not independently checked — they are cited here as V2's evidence, not as this repo's findings.