docs(voyage): S8 (W1/CC-27 gate) — T2 Workflow-substrate probe + measurement design
Second W1 gate, same staged execution as S7 (operator-chosen): cheap live feasibility probe + design doc; the prose-vs-Workflow bake-off specified but NOT run. Probe (CC 2.1.181 interactive): a minimal trekreview-shaped Workflow — parallel([reviewerA, reviewerB]) with a findings schema -> agent(coordinator) with a verdict schema — ran end-to-end. F1 core ports natively; F2 structured schemas retire the JSON-parse fragility at trekreview.md:202-204; F3 result returns to main; F4 a small purposeful fan-out did NOT trip the S7 proliferation classifier. 3 agents / 85461 tokens / 13.8s. Reframe: 'substrate swap' is a false binary — a /trek* command is ~80% non-orchestration glue, so Workflow can only replace the fan-out->synthesize core (hybrid). CC-27 recommendation (operator gates verdict): selective hybrid, NOT wholesale swap. Tier 1 ship a prose schema contract (the F2 win, no Workflow dep); tier 2 port trekreview Phase 5-6 to a Workflow only if the designed bake-off shows fidelity-equivalent output + acceptable control/cost; tier 3 wholesale swap declined (portability floor 2.1.154+, opt-in UX, mid-flow visibility loss). Open risk inherited from S7: classifier at large fan-out under auto/bypass still unverified. New: docs/T2-cc27-workflow-substrate.md (gate evidence F0-F4 + bake-off design with thresholds + no-Workflow schema-contract PoC). Matrix: CC-27 row + S8 resolutions + open-question/T2 pointers updated. Docs-only; no code/schema. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
This commit is contained in:
parent
cccc535a13
commit
b5f3d4a932
2 changed files with 278 additions and 3 deletions
|
|
@ -123,7 +123,7 @@ S4 implemented the four scope items above. TDD: a failing test was written first
|
|||
| ID | Change (version) | Type | Voyage relevance | Decision | Rationale |
|
||||
|----|------------------|------|------------------|----------|-----------|
|
||||
| **CC-26** | Sub-agents spawn sub-agents ≤5 deep (**2.1.172**, verified) — the design response to CC-01 | NEW | Could restore *delegated* orchestration: an orchestrator sub-agent spawns the swarm; synthesis/writing delegated (the "missing summarizer link" in `docs/subagent-delegation-audit.md`). Frees main-context tokens | **EVALUATE → lean NO (S7)** | "Can spawn 5 deep" ≠ "Voyage's orchestrator→6-agent-swarm pattern performs well." **S7 (2026-06-18, GATE):** feasibility probed cheaply — depth-2 nesting works, no degradation; depth cap moot (Voyage needs depth 2); NEW finding = auto-mode proliferation classifier polices agent fan-out (risk unique to delegation). On cost/benefit, **wholesale delegation NOT recommended**; the only defensible path is a narrow opt-in synthesis-agent PoC, proven by Δ main-context tokens. Operator gates verdict. Full bake-off designed but NOT run. See `docs/T1-cc26-delegated-orchestration.md` + §S7. |
|
||||
| **CC-27** | Dynamic Workflows / Workflow tool — orchestrates tens–hundreds of agents (**2.1.154**, verified); keyword `workflow`→`ultracode` (2.1.160); `agent()` attribution headers (2.1.174) | NEW | Voyage **hand-rolls** swarm/wave/pipeline orchestration in command prose — the Workflow tool is a native primitive for exactly this | **EVALUATE** (strategic) | The biggest identity decision: adopt Workflow as Voyage's execution substrate, or stay prose-orchestrated for portability/control? Tradeoffs: native concurrency + pipelining + budget control vs. dependency on a newer primitive + loss of fine-grained prose control + opt-in/billing semantics. Prototype one pipeline (e.g. `/trekreview`'s reviewer swarm) as a Workflow and compare. |
|
||||
| **CC-27** | Dynamic Workflows / Workflow tool — orchestrates tens–hundreds of agents (**2.1.154**, verified); keyword `workflow`→`ultracode` (2.1.160); `agent()` attribution headers (2.1.174) | NEW | Voyage **hand-rolls** swarm/wave/pipeline orchestration in command prose — the Workflow tool is a native primitive for exactly this | **EVALUATE → selective hybrid (S8)** | The biggest identity decision: adopt Workflow as substrate, or stay prose-orchestrated? **S8 (2026-06-18, GATE):** probed cheaply — a minimal trekreview-shaped `parallel()`→`agent()` Workflow ran end-to-end (F1), structured schemas retire the JSON-parse fragility at `trekreview.md:202–204` (F2), result returns to main (F3), and a small purposeful fan-out did **not** trip the S7 proliferation classifier (F4). Reframe: "substrate swap" is a false binary — a `/trek*` command is ~80% non-orchestration glue, so Workflow can only ever replace the fan-out→synthesize *core* (hybrid). **Recommendation: selective hybrid, NOT wholesale swap** — tier 1 ship a prose schema contract (the F2 win, no Workflow dep); tier 2 port trekreview Phase 5–6 to a Workflow only if the designed bake-off shows fidelity-equivalent output + acceptable control/cost; tier 3 wholesale swap declined (portability floor 2.1.154+, opt-in UX, visibility loss). Operator gates verdict. Full bake-off designed, NOT run. See `docs/T2-cc27-workflow-substrate.md` + §S8. |
|
||||
| **CC-28** | `TaskCreate` reliability — auto-repairs malformed input, schema in errors (2.1.163/169) | FIX | `TaskCreate`/`TaskUpdate` are in `trekplan`/orchestrator frontmatter but not actively used in command logic | **DEFER** | Becomes relevant only if W1 adopts task-graph orchestration. Tie to CC-26/27 outcome. |
|
||||
| **CC-29** | `subagent_type` matching now case/separator-insensitive (2.1.140); multiple `Agent(...)` types in `tools:` no longer dropped (2.1.147); subagent transcript/backgrounding fixes (2.1.178) | FIX | improves DX of any delegated-orchestration design | **SHIP** (verify) | Free robustness. Confirm Voyage's agent `tools:` grants (none currently declare multiple `Agent(...)` types) and `subagent_type` references are unaffected. |
|
||||
| **CC-31** | Worktree-isolation guard now applies in background sessions (2.1.154); `worktree.bgIsolation:"none"` (2.1.143); `EnterWorktree` switching mid-session (2.1.157) | CHANGE | `trekexecute` Phase 2.6 parallel waves + `trekplan` "execute with team" use git worktrees / `TeamCreate isolation:"worktree"` | **EVALUATE** | Verify Voyage's worktree-based parallel execution still behaves under the tightened bg-isolation guard. Affects the multi-session headless path. |
|
||||
|
|
@ -160,6 +160,41 @@ measurement-design doc; the expensive head-to-head comparison was specified but
|
|||
bake-off design (thresholds POSITIVE/NEGATIVE/INCONCLUSIVE), and the §6 synthesis-agent PoC, all
|
||||
ready to run if the operator greenlights pursuing delegation.
|
||||
|
||||
## S8 resolutions (W1 / CC-27 gate — 2026-06-18, operator-gated)
|
||||
|
||||
S8 was the second W1 gate — the orchestration-substrate identity decision. Operator chose the same
|
||||
**staged** execution as S7: cheap live feasibility probe + measurement-design doc; the head-to-head
|
||||
prose-vs-Workflow bake-off was specified but **not run**.
|
||||
|
||||
- **Feasibility (measured).** A minimal trekreview-shaped Workflow — `parallel([reviewerA,
|
||||
reviewerB])` with a findings schema → `agent(coordinator)` with a verdict schema, trivial agents,
|
||||
synthetic input — ran end-to-end in this CC 2.1.181 interactive session: both reviewers returned
|
||||
(F1), structured schemas delivered typed findings with no JSON-parse step (F2), the synthesizer
|
||||
reproduced Phase-6 dedup+verdict behavior and the result returned to main (F3), and the small
|
||||
purposeful fan-out did **not** trip the S7 proliferation classifier (F4). _3 agents · 85 461
|
||||
tokens · 13.8 s._
|
||||
- **Reframe — "substrate swap" is a false binary.** A `/trek*` command is ~80% non-orchestration
|
||||
glue (mode parsing, triage, validators, stats, HTML) and ~20% agent fan-out. The Workflow tool
|
||||
can only ever replace the fan-out→synthesize *core* (trekreview Phase 5–6), so the real decision
|
||||
is a **scoped hybrid per core**, not a wholesale identity swap. (Mirrors S7's "wall-time is not
|
||||
the gate metric" reframe.)
|
||||
- **The one concrete win is schema robustness (F2)** — and it is capturable *without* the Workflow
|
||||
tool: codify reviewer-output JSON as a validated prose schema contract, retiring the fragile
|
||||
"collect trailing JSON / re-ask on parse error" at `trekreview.md:202–204`.
|
||||
- **CC-27 recommendation (operator gates verdict).** **Selective hybrid, NOT wholesale swap**, in
|
||||
three tiers: (1) ship a prose schema contract regardless — the F2 win, zero new dependency; (2)
|
||||
port trekreview Phase 5–6 to a Workflow *only if* the designed bake-off shows fidelity-equivalent
|
||||
output + acceptable control/cost — best-case first port; (3) wholesale substrate swap **declined**
|
||||
(portability floor 2.1.154+, opt-in/billing UX on every invocation, loss of mid-flow operator
|
||||
visibility for an operator-gated review tool). CC-26 (S7) and CC-27 (S8) are now both resolved to
|
||||
"narrow/selective, operator-gated, not wholesale."
|
||||
- **Open risk inherited from S7.** Classifier behavior at *large* fan-out (trekplan's 6–10-agent
|
||||
swarm) under `auto`/`bypass` is still unverified — trekreview's 2–3 agents are below threshold,
|
||||
but a later trekplan port must measure it first.
|
||||
- **Artifact.** `docs/T2-cc27-workflow-substrate.md` — full gate evidence (F0–F4), §5 bake-off
|
||||
design (thresholds POSITIVE/NEGATIVE/INCONCLUSIVE), §6 no-Workflow schema-contract PoC, all ready
|
||||
to run if the operator greenlights tier 1 and/or tier 2.
|
||||
|
||||
## Sequencing
|
||||
|
||||
```
|
||||
|
|
@ -180,7 +215,7 @@ Incremental (W3): CC-11/CC-14/CC-15 as small, independently-shippable hardenin
|
|||
|
||||
## Open questions (need operator or empirical answer)
|
||||
|
||||
1. **W1 identity:** does Voyage adopt the Workflow tool as substrate, or stay prose-orchestrated? (CC-27)
|
||||
1. **W1 identity:** does Voyage adopt the Workflow tool as substrate, or stay prose-orchestrated? (CC-27) — **S8: false binary; lean selective hybrid (port the fan-out→synthesize core only), NOT wholesale swap; see §S8.**
|
||||
2. **W1 perf:** does delegated orchestration (orchestrator sub-agent → swarm) beat inline at Voyage's scale? (CC-26 — empirical) — **S7: feasibility YES, but lean NO on wholesale adoption; see §S7.**
|
||||
3. **W2 effort model:** map `phase_signals` onto native `effort:`, or keep bespoke? (CC-22 — gates v5.4)
|
||||
4. **CC-08:** is GH #36071 (hooks in headless) fixed? Determines whether the safety-preamble can relax.
|
||||
|
|
@ -188,7 +223,7 @@ Incremental (W3): CC-11/CC-14/CC-15 as small, independently-shippable hardenin
|
|||
## Empirical tests required
|
||||
|
||||
- **T1 (CC-26):** orchestrator-sub-agent spawns the planning swarm vs. inline baseline — wall-time, quality, token cost, depth-cap behavior. Harness: extend `scripts/q3-cache-prefix-experiment.mjs` pattern. — **S7: feasibility half RUN** (depth-2 works, no degradation, cap moot, proliferation-classifier risk found); **perf half DESIGNED, NOT run** (full bake-off + cheaper synthesis-agent PoC specified in `docs/T1-cc26-delegated-orchestration.md` §5/§6).
|
||||
- **T2 (CC-27):** reimplement `/trekreview`'s reviewer swarm as a Workflow; compare control, cost, and output fidelity vs. prose orchestration.
|
||||
- **T2 (CC-27):** reimplement `/trekreview`'s reviewer swarm as a Workflow; compare control, cost, and output fidelity vs. prose orchestration. — **S8: feasibility RUN** (probe: core ports natively, schemas retire JSON fragility, small fan-out classifier-clean); **fidelity/control/cost bake-off DESIGNED, NOT run** (`docs/T2-cc27-workflow-substrate.md` §5/§6).
|
||||
- **T3 (CC-04/CC-31):** research-agent MCP degradation under `--strict-mcp-config`; worktree parallel-wave behavior under tightened bg-isolation.
|
||||
|
||||
---
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue