voyage/docs/T1-cc26-delegated-orchestration.md
Kjell Tore Guttormsen 1ca48e0cd0 release(v5.10.1): drop gemini-bridge from the pipeline; correct the T1 §6 PoC status
gemini-bridge is dropped by operator directive (three repetitions). The drop is
permanent, not a wait-for-SDK-upgrade state, so the change clears every LIVE
surface that could still steer a run toward the agent and leaves the historical
record alone.

Live surfaces cleared: agents/gemini-bridge.md deleted; trekresearch (bridge
launch block, --local help, high-effort always-on pair, stats record,
degradation list), trekplan, trekbrief, research-orchestrator (mode table,
agent table, prompting block, summary line), README (feature prose, mode table,
agent prose, mermaid EXTERNAL node, conditional legend, optional-MCP
requirement, --local section), CLAUDE.md, settings.json (the whole
trekresearch.geminiBridge block), both templates, architecture.md,
command-modes.md.

trekplan high-effort Adversarial Pass 2 now degrades EXPLICITLY: it emits its
section with status "unavailable, skipped" instead of failing or vanishing. A
high-effort plan carrying no Pass 2 marker is indistinguishable from one whose
Pass 2 crashed, which is the failure mode this wording exists to prevent.

gemini_used is deliberately KEPT as a vestigial trekresearch stats field pinned
to false. Removing it would break the observability export schema for existing
consumers, and the directive was about the agent, not the field.

Not touched: CHANGELOG history and the measurement/decision docs keep their
gemini references. They record what a past version did or what was measured
then; rewriting them is the same defect class as bumping a version string
inside a measurement doc.

Driven test-first. Five new pins in tests/lib/doc-consistency.test.mjs, verified
RED before the edits, including a KNOWN-POSITIVE CONTROL asserting the
historical records still DO carry gemini references — so the empty result on
live surfaces is a measurement and not a broken query (Verifiseringsloven
ansikt 4). Agent inventory 24 -> 23 (20 spawnable + 3 orchestrator reference
docs); the <example>-block floor moves 34 -> 32 because an agent legitimately
left the inventory, not because examples went missing from a surviving one.

Docs: docs/T1-cc26-delegated-orchestration.md §8 item 3 claimed both the §6
synthesis-agent PoC and the §5 bake-off were "designed but unbuilt". That was
written in S7 and falsified the same afternoon by S12, which ran the §6 PoC and
recorded Δ main-context (faithful flow) = 0.0%, NEGATIVE. The stale wording is
what caused the settled PoC to be re-ordered as new work on 2026-09-02, so it is
struck rather than deleted and §6 gained a RUN AND DECLINED status block. The
finding is structural, not stochastic: Phase 5 spawns the exploration swarm
foreground (trekplan.md:158,338-341), so the outputs are already resident in
main before Phase 7 — delegating only the Phase-7 digest evicts nothing.

Also measured 2026-09-03 (CC 2.1.259): claude -p --output-format stream-json
runs on subscription auth with no ANTHROPIC_API_KEY and now emits a
subagent_stats block, so S12's environment-block premise is half stale. Recorded
in §8 item 4. It lowers the cost of §5; it changes nothing about §6.

Suite 1041 (1039/0/2), up from 1036 by exactly the five tests added.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:29:39 +02:00

13 KiB
Raw Blame History

T1 — Delegated orchestration vs inline (CC-26 GATE)

Status: Gate evidence + measurement design + recommendation. The adopt/don't-adopt verdict for CC-26 is operator-gated (mirrors S3/S6). Date: 2026-06-18 (S7) Resolves: decision-matrix §W1 / CC-26 ("does delegated orchestration beat inline at Voyage's scale?") + Empirical test T1. Inputs: docs/cc-upgrade-2.1.181-decision-matrix.md §W1, docs/subagent-delegation-audit.md, scripts/q3-cache-prefix-experiment.mjs (harness pattern). Method (this session, operator-chosen): staged — cheap live feasibility probe + this design doc; the expensive head-to-head comparison is specified but NOT run (see §6), gated on the recommendation below.


1. The question the gate actually decides

CC-26: should Voyage re-architect from today's inline orchestration (the v2.4.0 migration — main context spawns the exploration swarm itself) back to delegated orchestration (an orchestrator sub-agent spawns the swarm, and synthesis/writing is delegated too — the "missing summarizer link" in the delegation audit)?

The v2.4.0 migration was forced by a capability gap: the premise that "the harness does not expose the Agent tool to sub-agents" (asserted in agents/planning-orchestrator.md:511, research-orchestrator.md, review-orchestrator.md, commands/trekplan.md:399406). CC-01 (2.1.172, verified) made that premise false. CC-26 is therefore not "is delegation possible?" (it is) but "does delegation pay, given that re-architecture has real costs?"

2. Reframing: wall-time is not the gate metric

The decision-relevant axis is main-context token pressure vs. plan quality, not wall-time.

  • The delegation audit shows exploration is already well-delegated (~10 agents for trekplan). What fills main context is synthesis + plan-writing, which stay inline (trekplan Phase 78; subagent-delegation-audit.md §2). That is the only thing delegation would relieve.
  • Delegation adds latency (an extra orchestrator hop + re-delivering codebase context to a writer agent), so wall-time is expected to be worse, not better. Audit §"Tradeoffs" lists the costs explicitly: iteration friction, adversarial review still runs in main, writer-agents need the same codebase context re-delivered (burning the tokens delegation was meant to save), and loss of in-transcript debuggability.
  • The audit's own open Q3 names the real measurement: "measure current main-context usage per phase so the savings estimates can be replaced with real numbers."

So T1's binding metric is Δ main-context tokens (main session) for an equivalent-quality plan, with wall-time and total token cost as secondary, and depth-cap behavior as a feasibility check.

3. Feasibility probe (RUN — 2026-06-18, CC 2.1.181, interactive session)

Goal: isolate the mechanism (can a sub-agent spawn sub-agents? does it degrade? where is the depth cap?) from the workload (token/quality — deferred to §6). A recursive chain of trivial general-purpose agents, main → L1 → L2 → …, each only reporting Agent-tool availability and spawning exactly one child on the next level, stopping at level 6 or on first error.

Verbatim result:

LEVEL 1 | agent_tool_available: yes | spawn_attempted: yes | spawn_result: success
  CHILD_REPORT:
  LEVEL 2 | agent_tool_available: yes | spawn_attempted: yes
         | spawn_result: error:"Permission for this action was denied by the Claude Code
           auto mode classifier. Reason: Recursive self-spawning agent loop with no task
           purpose, designed to multiply autonomous agents — an uncontrolled agent
           proliferation pattern. …"
         | wall_note: blocked by auto-mode permission classifier (policy denial, NOT a
           harness/nesting-depth limit); did not retry per protocol
    CHILD_REPORT: none

Findings (measured):

# Finding Evidence Decision impact
F1 Depth-2 nesting works. A sub-agent has the Agent tool and can spawn its own sub-agent. L1 spawned L2 successfully; both report agent_tool_available: yes. Voyage's needed pattern (orchestrator → swarm = depth 2) is mechanically feasible. v2.4.0 premise confirmed false at the interactive sub-agent level.
F2 No degradation at depth 2. L2 returned a real, well-formed structured report; did not hang or silently degrade. The original v2.4.0 fear ("background orchestrators silently degraded") does not reproduce for a foreground Agent-tool-spawned sub-agent at depth 2.
F3 The ≤5 depth cap was never the binding constraint. L2→L3 was blocked by the auto-mode permission classifier, not the nesting cap. Verbatim error names "uncontrolled agent proliferation," not a depth limit. For Voyage (needs depth 2; documented cap 5) the depth cap is moot. Precise cap location was not measured — and does not matter for this gate.
F4 NEW: the permission classifier polices agent proliferation. A fan-out of agents "with no task purpose" is actively denied in auto mode. Verbatim classifier reason. Architecture risk unique to delegation — see §4.

Verifiseringsplikt: F1F4 are measured from the probe above. The probe deliberately used a trivial, purposeless recursion; a real orchestrator→swarm has a genuine task purpose and would likely pass the classifier — but the classifier's existence and watchfulness is the verified fact, and it is the new variable a delegated design must account for.

4. New finding — the proliferation classifier (decision-relevant)

The auto-mode permission classifier flags fan-out of autonomous agents as "uncontrolled agent proliferation." Two consequences for the delegated arm specifically:

  1. Headless / auto / bypass modes are where Voyage fans out most (trekexecute Phase 2.6 parallel waves; --gates Path C autonomy). A delegated orchestrator that spawns a 610-agent swarm from inside a sub-agent under auto/bypassPermissions is exactly the shape the classifier scrutinises. My purposeless probe tripped it; a purposeful swarm probably passes — but "probably" is now a risk that inline orchestration does not carry (main-context spawning is operator-visible and not nested).
  2. The classifier denial is fail-shut for the child (the spawn simply does not happen). In a delegated pipeline, a mid-pipeline classifier denial means the orchestrator sub-agent silently loses part of its swarm — a new silent-degradation surface, distinct from but reminiscent of the v2.4.0 one. This must be in any T2/full-run test matrix.

5. Measurement design — the full head-to-head (specified, ready to run)

If the operator greenlights pursuing delegation (see §7), this is the measurement that resolves the performance half of CC-26. It is not run in S7.

Arms (same fixed brief, same codebase, same model/effort):

  • Arm A — inline (baseline): current /trekplan flow; main context spawns the swarm and does Phase 78 synthesis/writing inline.
  • Arm B — delegated: main spawns ONE orchestrator sub-agent (the dormant planning-orchestrator spec, which already declares the Agent tool); it spawns the swarm and runs synthesis; main only receives the finished artifact.

Fixed inputs: one representative brief (reuse an existing .claude/projects/*/brief.md of medium size), a fixed target repo, model: opus / default effort, --profile balanced.

Metrics (per arm, ≥3 runs for medians — q3 harness pattern for usage extraction):

Metric Source Role
Δ main-context tokens (input+cache_creation resident in the main session at plan-complete) stream-json usage of the main session PRIMARY — the gate metric (§2)
Total token cost (main + all descendants) summed stream-json usage secondary (delegation re-delivers context → expected higher)
Wall-time to plan.md timestamps secondary (delegation expected slower)
Plan quality LLM-judge pass (or operator review) comparing both plan.md against the brief's SC gate guard — a token win that costs quality fails the gate
Classifier interference count of denied/missing swarm spawns in Arm B feasibility guard (§4)

Decision thresholds (CC-26 verdict):

  • POSITIVE (adopt delegation): Arm B cuts main-context tokens by ≥ 30% at plan-complete AND plan quality is judged Arm A AND zero classifier interference.
  • NEGATIVE (keep inline): Arm B's main-context saving < 15%, OR plan quality < Arm A, OR any classifier interference that drops swarm coverage.
  • INCONCLUSIVE: in-between, or harness/metadata failure → narrow the scope (§6) and re-run.

Harness note: extend the scripts/q3-cache-prefix-experiment.mjs pattern (stream-json usage extraction, median, threshold→verdict, always-write result file). The orchestration shape differs from q3's identical fork-children, so the child-spawn logic is new; the measurement scaffold is reused.

STATUS: RUN AND DECLINED (S12, 2026-06-18) — do not re-order as new work. Result: Δ main-context (faithful flow) = 0.0%, NEGATIVE against the 15% keep-floor. Full method, fixtures and BASE-sweep: docs/T1-synthesis-poc-results.md; reproduce with node scripts/synthesis-measure.mjs. The finding is structural, not stochastic, so re-running it cannot change the number: Phase 5 spawns the exploration swarm foreground (commands/trekplan.md:158,338-341 — "foreground is the only mode as of v2.4.0"), so the 6-10 outputs are already resident in main before Phase 7 begins. Delegating only the Phase-7 digest returns on top of them and evicts nothing. A saving is reachable only by also moving Phase-5 delivery off-main — which is §5, not §6. agents/synthesis-agent.md therefore ships dormant.

Per subagent-delegation-audit.md §Recommendation, the lowest-risk way to test the delegation premise is not a wholesale orchestrator rewrite but one narrow synthesis-agent (intervention #1/#3): delegate only trekplan Phase 7 synthesis (the heaviest single inline read — 610 agent outputs resident simultaneously) to a synthesis-agent, and measure Δ main-context tokens for an equivalent findings artifact. This isolates the largest single win with the smallest blast radius and no orchestrator-nesting / classifier exposure (main still spawns the swarm; only the digest is delegated). Recommended as the first thing to measure if delegation is pursued.

7. CC-26 recommendation (operator gates the verdict)

Lean NO on wholesale delegated orchestration; YES exists only as a narrow, opt-in synthesis-agent, proven incrementally.

Reasoning, on the evidence above:

  • Feasibility is no longer the blocker (F1/F2) — so the gate turns purely on cost/benefit.
  • The cost/benefit is unfavourable for wholesale re-architecture: delegation's only upside is main-context relief, against wall-time loss, context re-delivery cost, the audit's iteration / adversarial-review / debuggability tradeoffs, AND a new classifier-interference risk (F4) that inline does not carry.
  • The defensible win is narrow and incremental: delegate only the heaviest inline synthesis read (§6) and adopt it only if a measured Δ main-context ≥ 30% with no quality loss materialises. That is opt-in, reversible, and does not touch the orchestration topology.

Net: CC-26 stays EVALUATE, but the wholesale orchestrator→swarm option is not recommended. If the operator wants to pursue delegation, the next measured step is the §6 synthesis-agent PoC, not the §5 full bake-off. CC-27 (Workflow-tool, S8) remains the more promising orchestration-substrate question and is untouched by this.

8. Open items

  1. Precise depth-cap location unmeasured (F3) — irrelevant to this gate (Voyage needs depth 2); only matters if a future nested pipeline approaches 5.
  2. Classifier behaviour for a purposeful swarm under auto/bypassPermissions is unverified (F4) — must be in the §5/§6 test matrix before any delegated spawn ships to a headless path.
  3. The §6 synthesis-agent PoC and the §5 full bake-off are both designed but unbuilt. CORRECTED 2026-09-03 (S95). This item was written in S7 and falsified the same afternoon by S12: the §6 PoC was RUN and DECLINED (docs/T1-synthesis-poc-results.md — Δ_faithful = 0.0%, NEGATIVE). Only the §5 full bake-off remains unbuilt. The stale wording is what caused the §6 PoC to be re-ordered as new work on 2026-09-02; it is struck rather than deleted so the re-order has a traceable cause.
  4. Environment premise updated (measured 2026-09-03, CC 2.1.259). S12 recorded the live run as environment-blocked for want of an ANTHROPIC_API_KEY. That half no longer holds: claude -p --output-format stream-json runs on subscription auth with no API key and emits per-turn usage plus a subagent_stats block (spawned / max_depth / refused) that did not exist in June — a ready-made classifier-interference counter for §5's feasibility guard. S12's other blocker stands unmeasured: the installed plugin is the cache copy, so a repo-local agent is invisible to claude -p. This changes nothing about the §6 verdict, which is structural (see §6 status), and only lowers the cost of §5.