voyage/docs/T1-synthesis-poc-results.md

T1 — Synthesis-agent PoC: Δ main-context measurement (NW3 / S12)

Status: Measurement complete — verdict below. Method: deterministic token- accounting over real exploration fixtures (the live ≥3-run bake-off of T1 §5 is the stronger instrument but is (a) environment-blocked here — no ANTHROPIC_API_KEY, and the installed plugin is a cache copy so a fresh synthesis-agent is invisible to claude -p; and (b) unnecessary, because the binding answer is STRUCTURAL, not stochastic). Resolves: decision-matrix §W1 / CC-26 narrow PoC (docs/T1-cc26-delegated-orchestration.md §6). Reproduce: node scripts/synthesis-measure.mjs (regenerates this file).

Verifiseringsplikt: token figures are an explicit chars/4 estimate (labelled), not a tokenizer count. The gate turns on the RATIO Δ%, in which the per-token constant cancels for the out term. BASE (the fixed main-session baseline) is environment-dependent and was NOT API-measured this session → swept across a documented band, not asserted.

1. The decisive structural finding (BASE-independent)

trekplan runs the Phase 5 exploration swarm foreground (foreground is the only mode since v2.4.0; commands/trekplan.md). Foreground Agent/Task results are delivered back into the main transcript, so after Phase 5 the 6–10 exploration outputs are already resident in main. Raw outputs are never written to disk (trekplan.md:569 reserves the "do NOT write to disk" rule for the synthesis text only). Phase 7 synthesis therefore reasons over already-resident context. Delegating only the Phase-7 read to a synthesis-agent — "main still spawns the swarm; only the digest is delegated" (T1 §6) — cannot evict those outputs from main; the digest simply returns on top of them.

⇒ Δ main-context (faithful flow) ≈ 0 — independent of every token count below. The ≥30% saving is only realizable by ALSO moving Phase-5 delivery off-main (swarm-writes-to- disk, or a nested orchestrator owning the swarm), which is the wholesale change T1 §7 explicitly declined and is OUT of NW3 scope.

2. Fixtures (measured)

• Exploration dir: tests/fixtures/synthesis/exploration
• Digest: tests/fixtures/synthesis/digest.json

exploration output	chars	est. tokens
architecture-mapper.md	4774	1194
convention-scanner.md	1704	426
dependency-tracer.md	3470	868
git-historian.md	1703	426
risk-assessor.md	1950	488
task-finder.md	1927	482
test-strategist.md	1720	430
OUT (Σ resident in main)	—	4314
digest (DIG)	—	624

3. Δ main-context — both framings, swept over BASE

inline = base+out+dig · delegated (faithful) = base+out+dig (out already resident) · delegated (disk-potential) = base+dig (out off-main).

BASE (est.)	inline	faithful Δ	faithful verdict	disk-potential Δ	disk verdict
30000	34938	0.0%	NEGATIVE	12.3%	NEGATIVE
50000	54938	0.0%	NEGATIVE	7.9%	NEGATIVE
80000	84938	0.0%	NEGATIVE	5.1%	NEGATIVE
120000	124938	0.0%	NEGATIVE	3.5%	NEGATIVE

Break-even BASE for the disk-potential upper bound to reach the 30% adopt bar: ~9,442 tokens (below this BASE the hypothetical disk path would clear 30%; at/above it, even the upper bound fails). A real Voyage main session's BASE (CC system prompt + plugin command/agent/skill listings + CLAUDE.md) is large, so the disk upper bound is itself fragile.

Fixture-independent break-even (so the verdict does not hinge on fixture size)

disk-potential Δ = out/(base+out+dig), so it clears the 30% adopt bar iff out / base > 0.30/0.70 ≈ 0.43 — the combined exploration output must exceed ~43% of the fixed main baseline. The table below sweeps OUT at an illustrative typical BASE = 60,000 (independent of this run's fixtures):

OUT (Σ exploration tokens)	disk-potential Δ @ ref BASE	clears 30%?
5,000	7.6%	no
10,000	14.2%	no
20,000	24.8%	no
30,000	33.1%	yes
40,000	39.8%	yes

This run's fixtures total OUT = 4314 tokens across 7 concise representative outputs — one concrete point on the curve. Even a generously large real swarm (OUT in the tens of thousands) only clears 30% when the main baseline is unusually small, and never in the faithful flow (Δ=0). The verdict is therefore robust to fixture size.

4. Quality

The digest-output contract (lib/plan/synthesis-digest-schema.mjs) pins the same Phase-7 synthesis dimensions main produces inline (task, architecture_model, reusable_code, contradictions, risks, gaps, source-tagged findings). A delegated digest that validates is structurally quality-equivalent to the inline one — but quality is moot here: the faithful Δ is ~0, so there is no token win for quality to defend.

5. Verdict

DECLINED per measurement. NW3-as-scoped yields Δ main-context ≈ 0% (faithful flow, structural — the Phase-5 foreground swarm already makes the outputs resident; delegating Phase 7 evicts nothing). The disk-potential upper bound is reachable only via an out-of- scope Phase-5 change and is itself BASE-fragile.

RESULT: NEGATIVE (Δ_faithful = 0.0% < 15.0% adopt-floor)

6. Disposition

• agents/synthesis-agent.md ships dormant (a documented, schema-conformant deliverable); commands/trekplan.md Phase 7 is NOT wired to it.
• If main-context relief is later wanted, the prerequisite is a Phase-5 redesign (swarm- writes-to-disk / nested orchestrator) — a separate, larger decision (re-open CC-26 §7).
• The dormant agent + this harness make that future step cheap to re-measure: drop new fixtures in and re-run.