voyage/docs/T1-cc26-delegated-orchestration.md
Kjell Tore Guttormsen 1ca48e0cd0 release(v5.10.1): drop gemini-bridge from the pipeline; correct the T1 §6 PoC status
gemini-bridge is dropped by operator directive (three repetitions). The drop is
permanent, not a wait-for-SDK-upgrade state, so the change clears every LIVE
surface that could still steer a run toward the agent and leaves the historical
record alone.

Live surfaces cleared: agents/gemini-bridge.md deleted; trekresearch (bridge
launch block, --local help, high-effort always-on pair, stats record,
degradation list), trekplan, trekbrief, research-orchestrator (mode table,
agent table, prompting block, summary line), README (feature prose, mode table,
agent prose, mermaid EXTERNAL node, conditional legend, optional-MCP
requirement, --local section), CLAUDE.md, settings.json (the whole
trekresearch.geminiBridge block), both templates, architecture.md,
command-modes.md.

trekplan high-effort Adversarial Pass 2 now degrades EXPLICITLY: it emits its
section with status "unavailable, skipped" instead of failing or vanishing. A
high-effort plan carrying no Pass 2 marker is indistinguishable from one whose
Pass 2 crashed, which is the failure mode this wording exists to prevent.

gemini_used is deliberately KEPT as a vestigial trekresearch stats field pinned
to false. Removing it would break the observability export schema for existing
consumers, and the directive was about the agent, not the field.

Not touched: CHANGELOG history and the measurement/decision docs keep their
gemini references. They record what a past version did or what was measured
then; rewriting them is the same defect class as bumping a version string
inside a measurement doc.

Driven test-first. Five new pins in tests/lib/doc-consistency.test.mjs, verified
RED before the edits, including a KNOWN-POSITIVE CONTROL asserting the
historical records still DO carry gemini references — so the empty result on
live surfaces is a measurement and not a broken query (Verifiseringsloven
ansikt 4). Agent inventory 24 -> 23 (20 spawnable + 3 orchestrator reference
docs); the <example>-block floor moves 34 -> 32 because an agent legitimately
left the inventory, not because examples went missing from a surviving one.

Docs: docs/T1-cc26-delegated-orchestration.md §8 item 3 claimed both the §6
synthesis-agent PoC and the §5 bake-off were "designed but unbuilt". That was
written in S7 and falsified the same afternoon by S12, which ran the §6 PoC and
recorded Δ main-context (faithful flow) = 0.0%, NEGATIVE. The stale wording is
what caused the settled PoC to be re-ordered as new work on 2026-09-02, so it is
struck rather than deleted and §6 gained a RUN AND DECLINED status block. The
finding is structural, not stochastic: Phase 5 spawns the exploration swarm
foreground (trekplan.md:158,338-341), so the outputs are already resident in
main before Phase 7 — delegating only the Phase-7 digest evicts nothing.

Also measured 2026-09-03 (CC 2.1.259): claude -p --output-format stream-json
runs on subscription auth with no ANTHROPIC_API_KEY and now emits a
subagent_stats block, so S12's environment-block premise is half stale. Recorded
in §8 item 4. It lowers the cost of §5; it changes nothing about §6.

Suite 1041 (1039/0/2), up from 1036 by exactly the five tests added.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 20:29:39 +02:00

196 lines
13 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# T1 — Delegated orchestration vs inline (CC-26 GATE)
**Status:** Gate evidence + measurement design + recommendation. The adopt/don't-adopt
verdict for CC-26 is **operator-gated** (mirrors S3/S6).
**Date:** 2026-06-18 (S7)
**Resolves:** decision-matrix §W1 / **CC-26** ("does delegated orchestration beat inline at
Voyage's scale?") + Empirical test **T1**.
**Inputs:** `docs/cc-upgrade-2.1.181-decision-matrix.md` §W1, `docs/subagent-delegation-audit.md`,
`scripts/q3-cache-prefix-experiment.mjs` (harness pattern).
**Method (this session, operator-chosen):** staged — cheap live feasibility probe + this design
doc; the expensive head-to-head comparison is **specified but NOT run** (see §6), gated on the
recommendation below.
---
## 1. The question the gate actually decides
CC-26: should Voyage re-architect from today's **inline** orchestration (the v2.4.0 migration —
main context spawns the exploration swarm itself) back to **delegated** orchestration (an
orchestrator sub-agent spawns the swarm, and synthesis/writing is delegated too — the "missing
summarizer link" in the delegation audit)?
The v2.4.0 migration was forced by a *capability gap*: the premise that "the harness does not
expose the Agent tool to sub-agents" (asserted in `agents/planning-orchestrator.md:511`,
`research-orchestrator.md`, `review-orchestrator.md`, `commands/trekplan.md:399406`). CC-01
(2.1.172, verified) made that premise false. CC-26 is therefore **not** "is delegation possible?"
(it is) but "**does delegation pay**, given that re-architecture has real costs?"
## 2. Reframing: wall-time is not the gate metric
The decision-relevant axis is **main-context token pressure vs. plan quality**, not wall-time.
- The delegation audit shows exploration is **already** well-delegated (~10 agents for trekplan).
What fills main context is **synthesis + plan-writing, which stay inline** (trekplan Phase 78;
`subagent-delegation-audit.md` §2). That is the only thing delegation would relieve.
- Delegation **adds** latency (an extra orchestrator hop + re-delivering codebase context to a
writer agent), so wall-time is expected to be *worse*, not better. Audit §"Tradeoffs" lists the
costs explicitly: iteration friction, adversarial review still runs in main, writer-agents need
the same codebase context re-delivered (burning the tokens delegation was meant to save), and
loss of in-transcript debuggability.
- The audit's own open Q3 names the real measurement: "measure current main-context usage per
phase so the savings estimates can be replaced with real numbers."
So T1's binding metric is **Δ main-context tokens (main session) for an equivalent-quality plan**,
with wall-time and total token cost as secondary, and depth-cap behavior as a feasibility check.
## 3. Feasibility probe (RUN — 2026-06-18, CC 2.1.181, interactive session)
**Goal:** isolate the *mechanism* (can a sub-agent spawn sub-agents? does it degrade? where is the
depth cap?) from the *workload* (token/quality — deferred to §6). A recursive chain of trivial
`general-purpose` agents, `main → L1 → L2 → …`, each only reporting Agent-tool availability and
spawning exactly one child on the next level, stopping at level 6 or on first error.
**Verbatim result:**
```
LEVEL 1 | agent_tool_available: yes | spawn_attempted: yes | spawn_result: success
CHILD_REPORT:
LEVEL 2 | agent_tool_available: yes | spawn_attempted: yes
| spawn_result: error:"Permission for this action was denied by the Claude Code
auto mode classifier. Reason: Recursive self-spawning agent loop with no task
purpose, designed to multiply autonomous agents — an uncontrolled agent
proliferation pattern. …"
| wall_note: blocked by auto-mode permission classifier (policy denial, NOT a
harness/nesting-depth limit); did not retry per protocol
CHILD_REPORT: none
```
**Findings (measured):**
| # | Finding | Evidence | Decision impact |
|---|---------|----------|-----------------|
| F1 | **Depth-2 nesting works.** A sub-agent has the Agent tool and can spawn its own sub-agent. | L1 spawned L2 successfully; both report `agent_tool_available: yes`. | Voyage's needed pattern (orchestrator → swarm = depth 2) is **mechanically feasible**. v2.4.0 premise confirmed false at the interactive sub-agent level. |
| F2 | **No degradation at depth 2.** | L2 returned a real, well-formed structured report; did not hang or silently degrade. | The original v2.4.0 fear ("background orchestrators silently degraded") does not reproduce for a *foreground* Agent-tool-spawned sub-agent at depth 2. |
| F3 | **The ≤5 depth cap was never the binding constraint.** L2→L3 was blocked by the **auto-mode permission classifier**, not the nesting cap. | Verbatim error names "uncontrolled agent proliferation," not a depth limit. | For Voyage (needs depth 2; documented cap 5) the depth cap is **moot**. Precise cap location was not measured — and does not matter for this gate. |
| F4 | **NEW: the permission classifier polices agent proliferation.** A fan-out of agents "with no task purpose" is actively denied in auto mode. | Verbatim classifier reason. | **Architecture risk unique to delegation** — see §4. |
> Verifiseringsplikt: F1F4 are *measured* from the probe above. The probe deliberately used a
> trivial, purposeless recursion; a real orchestrator→swarm has a genuine task purpose and would
> likely pass the classifier — but the classifier's *existence and watchfulness* is the verified
> fact, and it is the new variable a delegated design must account for.
## 4. New finding — the proliferation classifier (decision-relevant)
The auto-mode permission classifier flags fan-out of autonomous agents as "uncontrolled agent
proliferation." Two consequences for the delegated arm specifically:
1. **Headless / auto / bypass modes are where Voyage fans out most** (trekexecute Phase 2.6
parallel waves; `--gates` Path C autonomy). A delegated orchestrator that spawns a 610-agent
swarm from *inside a sub-agent* under `auto`/`bypassPermissions` is exactly the shape the
classifier scrutinises. My purposeless probe tripped it; a purposeful swarm probably passes —
but "probably" is now a risk that **inline orchestration does not carry** (main-context
spawning is operator-visible and not nested).
2. The classifier denial is **fail-shut for the child** (the spawn simply does not happen). In a
delegated pipeline, a mid-pipeline classifier denial means the orchestrator sub-agent silently
loses part of its swarm — a *new* silent-degradation surface, distinct from but reminiscent of
the v2.4.0 one. This must be in any T2/full-run test matrix.
## 5. Measurement design — the full head-to-head (specified, ready to run)
If the operator greenlights pursuing delegation (see §7), this is the measurement that resolves the
*performance* half of CC-26. It is **not run** in S7.
**Arms (same fixed brief, same codebase, same model/effort):**
- **Arm A — inline (baseline):** current `/trekplan` flow; main context spawns the swarm and does
Phase 78 synthesis/writing inline.
- **Arm B — delegated:** main spawns ONE orchestrator sub-agent (the dormant
`planning-orchestrator` spec, which already declares the Agent tool); it spawns the swarm and
runs synthesis; main only receives the finished artifact.
**Fixed inputs:** one representative brief (reuse an existing `.claude/projects/*/brief.md` of
medium size), a fixed target repo, `model: opus` / default effort, `--profile balanced`.
**Metrics (per arm, ≥3 runs for medians — q3 harness pattern for usage extraction):**
| Metric | Source | Role |
|--------|--------|------|
| **Δ main-context tokens** (input+cache_creation resident in the *main* session at plan-complete) | stream-json `usage` of the main session | **PRIMARY** — the gate metric (§2) |
| Total token cost (main + all descendants) | summed stream-json `usage` | secondary (delegation re-delivers context → expected higher) |
| Wall-time to `plan.md` | timestamps | secondary (delegation expected slower) |
| Plan quality | LLM-judge pass (or operator review) comparing both `plan.md` against the brief's SC | **gate guard** — a token win that costs quality fails the gate |
| Classifier interference | count of denied/missing swarm spawns in Arm B | feasibility guard (§4) |
**Decision thresholds (CC-26 verdict):**
- **POSITIVE (adopt delegation):** Arm B cuts main-context tokens by **≥ 30%** at plan-complete
AND plan quality is judged **≥** Arm A AND zero classifier interference.
- **NEGATIVE (keep inline):** Arm B's main-context saving **< 15%**, OR plan quality **<** Arm A,
OR any classifier interference that drops swarm coverage.
- **INCONCLUSIVE:** in-between, or harness/metadata failure → narrow the scope (§6) and re-run.
**Harness note:** extend the `scripts/q3-cache-prefix-experiment.mjs` pattern (stream-json `usage`
extraction, median, threshold→verdict, always-write result file). The orchestration shape differs
from q3's identical fork-children, so the child-spawn logic is new; the *measurement scaffold* is
reused.
## 6. Cheaper PoC (audit-recommended, preferred over the full bake-off)
> **STATUS: RUN AND DECLINED (S12, 2026-06-18) — do not re-order as new work.**
> Result: **Δ main-context (faithful flow) = 0.0%**, NEGATIVE against the 15% keep-floor.
> Full method, fixtures and BASE-sweep: `docs/T1-synthesis-poc-results.md`;
> reproduce with `node scripts/synthesis-measure.mjs`.
> The finding is **structural, not stochastic**, so re-running it cannot change the number:
> Phase 5 spawns the exploration swarm **foreground** (`commands/trekplan.md:158,338-341` —
> "foreground is the only mode as of v2.4.0"), so the 6-10 outputs are **already resident in
> main** before Phase 7 begins. Delegating only the Phase-7 digest returns *on top of* them
> and evicts nothing. A saving is reachable only by also moving Phase-5 delivery off-main —
> which is §5, not §6. `agents/synthesis-agent.md` therefore ships **dormant**.
Per `subagent-delegation-audit.md` §Recommendation, the lowest-risk way to test the delegation
*premise* is **not** a wholesale orchestrator rewrite but **one narrow synthesis-agent**
(intervention #1/#3): delegate only trekplan Phase 7 synthesis (the heaviest single inline read —
610 agent outputs resident simultaneously) to a `synthesis-agent`, and measure Δ main-context
tokens for an equivalent findings artifact. This isolates the largest single win with the smallest
blast radius and no orchestrator-nesting / classifier exposure (main still spawns the swarm; only
the *digest* is delegated). Recommended as the **first** thing to measure if delegation is pursued.
## 7. CC-26 recommendation (operator gates the verdict)
**Lean NO on wholesale delegated orchestration; YES exists only as a narrow, opt-in
synthesis-agent, proven incrementally.**
Reasoning, on the evidence above:
- Feasibility is no longer the blocker (F1/F2) — so the gate turns purely on cost/benefit.
- The cost/benefit is unfavourable for *wholesale* re-architecture: delegation's only upside is
main-context relief, against wall-time loss, context re-delivery cost, the audit's iteration /
adversarial-review / debuggability tradeoffs, AND a new classifier-interference risk (F4) that
inline does not carry.
- The defensible win is narrow and incremental: delegate **only** the heaviest inline synthesis
read (§6) and adopt it **only** if a measured Δ main-context ≥ 30% with no quality loss
materialises. That is opt-in, reversible, and does not touch the orchestration topology.
**Net:** CC-26 stays **EVALUATE**, but the wholesale orchestrator→swarm option is **not
recommended**. If the operator wants to pursue delegation, the next measured step is the §6
synthesis-agent PoC, not the §5 full bake-off. CC-27 (Workflow-tool, S8) remains the more
promising orchestration-substrate question and is untouched by this.
## 8. Open items
1. Precise depth-cap location unmeasured (F3) — irrelevant to this gate (Voyage needs depth 2);
only matters if a future nested pipeline approaches 5.
2. Classifier behaviour for a *purposeful* swarm under `auto`/`bypassPermissions` is unverified
(F4) — must be in the §5/§6 test matrix before any delegated spawn ships to a headless path.
3. ~~The §6 synthesis-agent PoC and the §5 full bake-off are both designed but unbuilt.~~
**CORRECTED 2026-09-03 (S95).** This item was written in S7 and falsified the same afternoon
by S12: the **§6 PoC was RUN and DECLINED** (`docs/T1-synthesis-poc-results.md` — Δ_faithful
= 0.0%, NEGATIVE). Only the **§5 full bake-off** remains unbuilt. The stale wording is what
caused the §6 PoC to be re-ordered as new work on 2026-09-02; it is struck rather than
deleted so the re-order has a traceable cause.
4. **Environment premise updated (measured 2026-09-03, CC 2.1.259).** S12 recorded the live run
as environment-blocked for want of an `ANTHROPIC_API_KEY`. That half no longer holds:
`claude -p --output-format stream-json` runs on subscription auth with no API key and emits
per-turn `usage` plus a `subagent_stats` block (spawned / max_depth / refused) that did not
exist in June — a ready-made classifier-interference counter for §5's feasibility guard.
S12's *other* blocker stands unmeasured: the installed plugin is the cache copy, so a
repo-local agent is invisible to `claude -p`. This changes nothing about the §6 verdict,
which is structural (see §6 status), and only lowers the cost of §5.