docs(voyage): S8 (W1/CC-27 gate) — T2 Workflow-substrate probe + measurement design
Second W1 gate, same staged execution as S7 (operator-chosen): cheap live feasibility probe + design doc; the prose-vs-Workflow bake-off specified but NOT run. Probe (CC 2.1.181 interactive): a minimal trekreview-shaped Workflow — parallel([reviewerA, reviewerB]) with a findings schema -> agent(coordinator) with a verdict schema — ran end-to-end. F1 core ports natively; F2 structured schemas retire the JSON-parse fragility at trekreview.md:202-204; F3 result returns to main; F4 a small purposeful fan-out did NOT trip the S7 proliferation classifier. 3 agents / 85461 tokens / 13.8s. Reframe: 'substrate swap' is a false binary — a /trek* command is ~80% non-orchestration glue, so Workflow can only replace the fan-out->synthesize core (hybrid). CC-27 recommendation (operator gates verdict): selective hybrid, NOT wholesale swap. Tier 1 ship a prose schema contract (the F2 win, no Workflow dep); tier 2 port trekreview Phase 5-6 to a Workflow only if the designed bake-off shows fidelity-equivalent output + acceptable control/cost; tier 3 wholesale swap declined (portability floor 2.1.154+, opt-in UX, mid-flow visibility loss). Open risk inherited from S7: classifier at large fan-out under auto/bypass still unverified. New: docs/T2-cc27-workflow-substrate.md (gate evidence F0-F4 + bake-off design with thresholds + no-Workflow schema-contract PoC). Matrix: CC-27 row + S8 resolutions + open-question/T2 pointers updated. Docs-only; no code/schema. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
This commit is contained in:
parent
cccc535a13
commit
b5f3d4a932
2 changed files with 278 additions and 3 deletions
240
docs/T2-cc27-workflow-substrate.md
Normal file
240
docs/T2-cc27-workflow-substrate.md
Normal file
|
|
@ -0,0 +1,240 @@
|
|||
# T2 — Workflow tool as orchestration substrate (CC-27 GATE)
|
||||
|
||||
**Status:** Gate evidence + measurement design + recommendation. The adopt/don't-adopt
|
||||
verdict for CC-27 is **operator-gated** (mirrors S3/S6/S7).
|
||||
**Date:** 2026-06-18 (S8)
|
||||
**Resolves:** decision-matrix §W1 / **CC-27** ("does Voyage adopt the Workflow tool as its
|
||||
execution substrate, or stay prose-orchestrated?") + Empirical test **T2**.
|
||||
**Inputs:** `docs/cc-upgrade-2.1.181-decision-matrix.md` §W1 / CC-27, `commands/trekreview.md`
|
||||
(Phases 5–6, the prototype target), `docs/T1-cc26-delegated-orchestration.md` §4 (the S7
|
||||
proliferation-classifier handoff), the Workflow-tool reference (CC 2.1.154+).
|
||||
**Method (this session, operator-chosen):** staged — cheap live feasibility probe + this design
|
||||
doc; the expensive head-to-head bake-off is **specified but NOT run** (see §5), gated on the
|
||||
recommendation below. Same shape as S7.
|
||||
|
||||
---
|
||||
|
||||
## 1. The question the gate actually decides
|
||||
|
||||
CC-27: Voyage **hand-rolls** its swarm / wave / pipeline orchestration in command prose — the main
|
||||
session reads a `/trek*` command, interprets its phase prose, and spawns agents via the Agent tool
|
||||
itself. The Workflow tool (2.1.154+) is a native primitive for exactly this: a JS-scripted
|
||||
orchestrator with `parallel()` / `pipeline()` / `agent({schema})`, background execution, budget
|
||||
control, and resume/journaling.
|
||||
|
||||
The decision is framed as the **biggest identity choice**: adopt Workflow as Voyage's execution
|
||||
substrate, or stay prose-orchestrated for portability and fine-grained control? But that framing is
|
||||
a trap — see §2.
|
||||
|
||||
## 2. Reframing: "substrate swap" is a false binary; the axis is selective hybrid
|
||||
|
||||
The decision-relevant axis is **not** "all-Workflow vs all-prose." It is **"does wrapping a
|
||||
fan-out→synthesize *core* in a Workflow call earn its keep, against the portability / opt-in /
|
||||
in-transcript-visibility costs?"** — measured per core, not per pipeline.
|
||||
|
||||
Why the binary is false, using `/trekreview` (the named prototype) as the worked example:
|
||||
|
||||
- A `/trek*` command is **~80% non-orchestration glue** and **~20% agent fan-out.** trekreview's
|
||||
450 lines are: mode parsing (Phase 1), brief validation (Phase 2), SHA-range discovery (Phase 3),
|
||||
a deterministic path-pattern triage classifier (Phase 4), the strict validator + repair-in-place
|
||||
+ stats JSONL + HTML annotation (Phase 8), and validate-only mode (Phase 8.5). The Workflow tool
|
||||
addresses **none** of that — it orchestrates *agents*, not bash calls, file writes, validators,
|
||||
and operator-facing HTML.
|
||||
- The part Workflow actually replaces is **Phase 5 (parallel reviewers) + Phase 6 (coordinator
|
||||
synthesis)** — the clean fan-out→barrier→synthesize core. That is the 20%.
|
||||
- So "adopt Workflow as substrate" can only ever mean **embed a Workflow call for the core, keep
|
||||
prose for the glue** — i.e. a *hybrid*, not a substrate swap. The honest question is therefore
|
||||
scoped: is the hybrid worth it for *this core*?
|
||||
|
||||
This mirrors the S7 reframing of CC-26 ("wall-time is not the gate metric; Δ main-context tokens
|
||||
is"). For CC-27 the reframe is: **the gate metric is output-fidelity-preserving control/cost on a
|
||||
single clean-barrier core, not a wholesale substrate identity.**
|
||||
|
||||
The trekreview core, for reference (the shape that gets ported):
|
||||
|
||||
```
|
||||
Phase 5: [ brief-conformance-reviewer ∥ code-correctness-reviewer ] parallel fan-out
|
||||
↓ merge findings arrays
|
||||
Phase 6: review-coordinator (dedup → HubSpot Judge → Cloudflare reasonableness → verdict)
|
||||
↓
|
||||
Phase 7: write review.md ← stays prose (file I/O, atomic write, frontmatter rules)
|
||||
```
|
||||
|
||||
## 3. Feasibility probe (RUN — 2026-06-18, CC 2.1.181, interactive session)
|
||||
|
||||
**Goal:** isolate the *mechanism + semantics* (does the fan-out→synthesize shape execute and return
|
||||
to main? do structured schemas remove the JSON-parse fragility? does the S7 proliferation classifier
|
||||
bite a Workflow fan-out? is the tool even invocable here?) from the *workload* (real reviewers on a
|
||||
real diff — deferred to §5). A minimal trekreview-shaped Workflow: `parallel([reviewerA, reviewerB])`
|
||||
with a findings schema → `agent(coordinator)` with a verdict schema, run on a one-line synthetic
|
||||
input, trivial agents, no file reads.
|
||||
|
||||
**Verbatim result (Workflow return value, surfaced to main via task-notification):**
|
||||
|
||||
```json
|
||||
{
|
||||
"reviewers_returned": 2,
|
||||
"merged_findings_count": 2,
|
||||
"sample_findings": [
|
||||
{"file":"foo.js","line":10,"rule_key":"SC_UNTRACED","severity":"MAJOR"},
|
||||
{"file":"foo.js","line":10,"rule_key":"ERR_UNGUARDED_PARSE","severity":"MAJOR"}
|
||||
],
|
||||
"verdict": {"verdict":"BLOCK","deduped_count":2,
|
||||
"note":"Two distinct findings at foo.js:10 — rule_keys differ, so the dedup key
|
||||
(file,line,rule_key) keeps both. … verdict BLOCK on the conservative
|
||||
interpretation that one or more MAJOR findings remain unresolved."}
|
||||
}
|
||||
```
|
||||
_Usage: 3 agents · 85 461 tokens · 13.8 s wall._
|
||||
|
||||
**Findings (measured):**
|
||||
|
||||
| # | Finding | Evidence | Decision impact |
|
||||
|---|---------|----------|-----------------|
|
||||
| F0 | **The Workflow tool is invocable here without a hard opt-in block.** | Launch returned a task ID; the operator's STATE directive (S8 = "reimplement the trekreview swarm as one Workflow") satisfied the opt-in gate. | The opt-in/billing gate is real but a command/operator directive **can** satisfy it. The general UX question (does every `/trekreview` invocation cleanly count as opt-in?) is unresolved — §4. |
|
||||
| F1 | **The fan-out→barrier→synthesize core executes end-to-end.** Both parallel reviewers returned; the synthesizer ran on the merged result. | `reviewers_returned: 2`; the coordinator produced a verdict over both. | trekreview Phase 5→6 ports **natively** to `parallel()` → `agent()`. The shape is a 1:1 fit. |
|
||||
| F2 | **Structured-output schemas remove the JSON-parse fragility.** Findings returned as typed, validated objects — no "collect trailing JSON block / `JSON.parse` / re-ask on error" dance. | `sample_findings` are schema-shaped `{file,line,rule_key,severity}`; no parse step in the script. | **The single most concrete win.** Directly retires the fragile contract at `commands/trekreview.md:202–204`. Validation+retry happens at the tool layer. |
|
||||
| F3 | **The finished artifact returns to main as structured JSON.** | The notification `<result>` carried the full return object; main received the digest, not a transcript to re-parse. | Main gets a clean handoff — but only the *final* object (see F4-control). |
|
||||
| F4 | **A small purposeful fan-out did NOT trip the S7 proliferation classifier.** | 2-way fan-out + 1 synthesizer ran with zero denials; contrast S7's *purposeless recursive* chain, which the auto-mode classifier denied at L2→L3. | For trekreview specifically (2–3 agents) the classifier risk is **low**. The classifier's behavior at *large* Workflow fan-out (trekplan's 6–10-agent swarm) under `auto`/`bypass` remains **unverified** — same caveat shape as S7. |
|
||||
|
||||
> Verifiseringsplikt: F0–F4 are *measured* from the probe above. The probe used trivial agents and a
|
||||
> small (2-way) purposeful fan-out in an interactive session. It does **not** establish (a) classifier
|
||||
> behavior for a large fan-out under auto/bypass, (b) output fidelity against a *real* diff, or (c)
|
||||
> token cost at production context size — all deferred to §5. What it establishes is mechanism,
|
||||
> schema-robustness, return-to-main, and small-scale classifier tolerance.
|
||||
|
||||
## 4. Decision-relevant analysis — the honest ledger
|
||||
|
||||
### 4.1 What the Workflow substrate demonstrably wins (measured / structural)
|
||||
|
||||
1. **Schema-validated reviewer contracts (F2)** — the one concrete, measured win. trekreview today
|
||||
"collects each reviewer's trailing JSON block … on parse error, ask the agent to re-emit"
|
||||
(`trekreview.md:202–204`). `agent({schema})` makes that a tool-layer guarantee. *This win is
|
||||
capturable even without the Workflow tool — see §6.*
|
||||
2. **Deterministic control flow (F1/F3)** — fan-out, merge (`flatMap`), dedup-by-triplet can be
|
||||
plain JS, not prose the model re-interprets each run. The Phase-6 dedup pass (`(file,line,
|
||||
rule_key)`) is pure data manipulation that does not need an agent at all in a Workflow port;
|
||||
only the *judgment* passes (HubSpot Judge, Cloudflare reasonableness) stay agent calls.
|
||||
3. **Native pipelining / budget control** — `pipeline()` (no-barrier streaming) and `budget.*`
|
||||
exist. Minor for trekreview's single barrier; potentially relevant for trekplan's longer chain.
|
||||
|
||||
### 4.2 What the Workflow substrate costs (measured / structural)
|
||||
|
||||
1. **Loss of in-transcript operator visibility (F3, structural).** Workflow runs in the background;
|
||||
intermediate reviewer findings appear in `/workflows`, not the main conversation. trekreview is
|
||||
an **adversarial-review tool whose verdict the operator gates** — in-line visibility of each
|
||||
finding as it lands, and the ability to interrupt mid-swarm, is a real property prose has and
|
||||
Workflow trades away. (The final `review.md` artifact is still operator-gated, so this is a
|
||||
degradation of *mid-flow* visibility, not of the gate itself.)
|
||||
2. **Portability floor.** Workflow is CC **2.1.154+**. Voyage ships as a plugin; making a command
|
||||
require the Workflow tool raises the consumer's CC floor. Prose commands run on any recent CC.
|
||||
3. **Opt-in / billing semantics (F0).** Invocable here via the operator directive, but the Workflow
|
||||
tool is explicitly gated on opt-in ("ONLY when the user has explicitly opted into multi-agent
|
||||
orchestration"). A `/trekreview` invocation would have to *count as* that opt-in cleanly, or the
|
||||
UX gains a second gate. Unresolved.
|
||||
4. **Per-agent context floor is real but NOT a Workflow-specific tax.** 85k tokens for 3 trivial
|
||||
agents (~28k/agent) is the fresh-context floor each spawned agent pays — but **prose
|
||||
orchestration spawns the same 3 agents** (2 reviewers + coordinator) and pays the same floor.
|
||||
So token cost is **roughly a wash** for equal agent count; it is *not* a strong differentiator
|
||||
either way. (This corrects the instinct to count it as a Workflow con.)
|
||||
5. **Loss of ad-hoc mid-flow model judgment.** Prose lets main *read* a malformed reviewer output
|
||||
and decide to re-ask; a Workflow handles that via schema-retry (better for JSON) but cannot make
|
||||
the unscripted judgment calls a prose-driven main session can.
|
||||
|
||||
### 4.3 The classifier handoff from S7 (resolved-partial)
|
||||
|
||||
S7's open item — "classifier behavior for a *purposeful* swarm under auto/bypass is unverified" —
|
||||
is **partially closed**: a small purposeful Workflow fan-out is tolerated (F4). It remains open for
|
||||
*large* fan-out (≥6 agents) under auto/bypass, which is trekplan's shape, not trekreview's. trekreview's
|
||||
2–3-agent core is below the risk threshold; the bake-off (§5) must still measure it for the larger swarms.
|
||||
|
||||
## 5. Measurement design — the full prose-vs-Workflow bake-off (specified, ready to run)
|
||||
|
||||
If the operator greenlights a scoped port (§7), this resolves the *fidelity/control/cost* half of
|
||||
CC-27. **Not run** in S8.
|
||||
|
||||
**Arms (same brief, same delivered diff, same model/effort; Phases 1–4 + 7–8 stay prose in both):**
|
||||
- **Arm A — prose (baseline):** current `/trekreview` Phase 5–6 — main spawns the two reviewers in
|
||||
parallel, collects + parses their JSON, spawns `review-coordinator`.
|
||||
- **Arm B — Workflow:** Phase 5–6 reimplemented as one Workflow —
|
||||
`parallel([conformance, correctness])` with a findings schema → `agent(coordinator)` with a
|
||||
verdict schema; the dedup-by-triplet pass moved to plain JS; main receives the `review.md` body.
|
||||
|
||||
**Fixed inputs:** one representative delivered project (reuse an existing `.claude/projects/*/` with
|
||||
a real diff + brief of medium size), `model: opus` / default effort, `--profile balanced`.
|
||||
|
||||
**Metrics (per arm, ≥3 runs for medians — q3 harness pattern for usage extraction):**
|
||||
|
||||
| Metric | Source | Role |
|
||||
|--------|--------|------|
|
||||
| **Output fidelity** — same verdict + equivalent finding set (IDs, severities, rule_keys) | diff the two `review.md` | **PRIMARY** — a substrate that changes the verdict/findings fails the gate |
|
||||
| JSON-robustness — parse-error/re-ask events (Arm A) vs schema-retries (Arm B) | transcript/script logs | **the concrete win** — quantifies the fragility removed (F2) |
|
||||
| Control / operator visibility | qualitative: intermediate findings in-transcript? interruptible mid-swarm? | **gate guard** — review is operator-gated (§4.2.1) |
|
||||
| Classifier interference | count of denied/missing spawns (Arm B, repeat under `auto`/`bypass`) | feasibility guard (F4 / S7) |
|
||||
| Total token cost (main + descendants) | summed stream-json `usage` | secondary — expected ≈ wash (§4.2.4) |
|
||||
| Wall-time to `review.md` | timestamps | secondary |
|
||||
|
||||
**Decision thresholds (CC-27 verdict for *this core*):**
|
||||
- **POSITIVE (adopt scoped hybrid):** output fidelity ≡ Arm A (same verdict; finding set within
|
||||
tolerance) **AND** JSON-robustness strictly better **AND** zero classifier interference **AND**
|
||||
token cost within +15% **AND** operator visibility judged acceptable (the `review.md` gate
|
||||
survives).
|
||||
- **NEGATIVE (keep prose):** verdict/findings diverge, **OR** any classifier interference drops a
|
||||
reviewer, **OR** token cost > +30%, **OR** loss of mid-flow visibility judged unacceptable.
|
||||
- **INCONCLUSIVE:** in-between → fall back to §6 (schema contract in prose, no Workflow) and re-measure.
|
||||
|
||||
**Harness note:** extend the `scripts/q3-cache-prefix-experiment.mjs` pattern (stream-json `usage`
|
||||
extraction, median, threshold→verdict, always-write result file). The fidelity diff (review.md ↔
|
||||
review.md) is new; the usage scaffold is reused.
|
||||
|
||||
## 6. Cheaper first step (preferred over the full bake-off) — schema contract without Workflow
|
||||
|
||||
The single concrete win (F2) is **capturable without adopting the Workflow tool at all.** The
|
||||
reviewer-output fragility lives in prose ("collect trailing JSON block; on parse error, ask the
|
||||
agent to re-emit"). The narrowest, zero-dependency fix is to **codify the reviewer findings JSON as
|
||||
a validated schema contract in prose Phase 5** — main validates each reviewer's JSON against a
|
||||
findings JSON-schema (Voyage already has `lib/validators/` + `lib/util/frontmatter.mjs` parsers) and
|
||||
re-asks on schema failure, not just on parse failure.
|
||||
|
||||
This captures the robustness win with **no portability floor, no opt-in gate, no loss of
|
||||
in-transcript visibility** — the substrate stays prose. It is the S8 analog of S7 §6's "narrow
|
||||
synthesis-agent PoC preferred over the full bake-off": isolate the largest win with the smallest
|
||||
blast radius. **Recommended as the first thing to ship if the operator wants the win without the
|
||||
substrate commitment.**
|
||||
|
||||
## 7. CC-27 recommendation (operator gates the verdict)
|
||||
|
||||
**Three tiers, in increasing commitment:**
|
||||
|
||||
1. **Ship regardless (cheapest, no Workflow):** codify reviewer-output JSON as a schema-validated
|
||||
contract in prose Phase 5 (§6). Captures the one measured win (F2 robustness) with zero new
|
||||
dependency, zero opt-in friction, zero visibility loss. Low risk, high value.
|
||||
2. **Scoped hybrid — a measured YES candidate, gated on the §5 bake-off:** port trekreview Phase 5–6
|
||||
to a Workflow **if** the bake-off shows fidelity-equivalent output + acceptable control/cost.
|
||||
trekreview's clean-barrier core is the **best-case first port**; trekplan's swarm is a later,
|
||||
classifier-sensitive candidate (its larger fan-out under auto/bypass is the unverified risk).
|
||||
3. **Wholesale substrate swap — NOT recommended.** Portability floor (2.1.154+), opt-in/billing UX
|
||||
on every invocation, 80%-of-each-command is non-orchestration glue Workflow does not touch, and
|
||||
loss of mid-flow operator visibility for tools whose verdict the operator gates. The identity
|
||||
framing ("adopt Workflow as substrate") over-claims what the primitive can replace.
|
||||
|
||||
**Net:** CC-27 stays **EVALUATE**, resolving toward **selective hybrid, not substrate swap** — a
|
||||
slightly more YES-leaning posture than S7's CC-26, because here both feasibility (F1) *and* a
|
||||
concrete win (F2) are confirmed, where CC-26's only upside (main-context relief) was speculative and
|
||||
counterweighted. The wholesale-substrate option is declined. The first action, if the operator wants
|
||||
movement, is the §6 prose schema contract; the first Workflow port, if pursued, is trekreview
|
||||
Phase 5–6 via the §5 bake-off. CC-26 (delegated orchestration, S7) and CC-27 are now both resolved
|
||||
to "narrow/selective, operator-gated, not wholesale."
|
||||
|
||||
## 8. Open items
|
||||
|
||||
1. **Output fidelity unmeasured** — the probe used a synthetic input; a real-diff prose-vs-Workflow
|
||||
`review.md` comparison (§5 PRIMARY metric) is designed but unrun.
|
||||
2. **Classifier behavior at large fan-out under auto/bypass unverified** (F4) — trekreview's 2–3
|
||||
agents are below threshold; trekplan's 6–10-agent swarm is not, and must be measured before any
|
||||
Workflow port reaches a headless/auto path. Inherits directly from S7 §4.
|
||||
3. **Opt-in UX for command-invoked Workflows unresolved** (F0) — whether a bare `/trekreview`
|
||||
invocation cleanly satisfies the Workflow opt-in gate for an end user (not just via an operator
|
||||
STATE directive) needs a real end-user test.
|
||||
4. **§6 schema contract and §5 bake-off are both designed but unbuilt** — ready to run if CC-27 is
|
||||
greenlit toward the schema contract (tier 1) and/or the scoped port (tier 2).
|
||||
|
|
@ -123,7 +123,7 @@ S4 implemented the four scope items above. TDD: a failing test was written first
|
|||
| ID | Change (version) | Type | Voyage relevance | Decision | Rationale |
|
||||
|----|------------------|------|------------------|----------|-----------|
|
||||
| **CC-26** | Sub-agents spawn sub-agents ≤5 deep (**2.1.172**, verified) — the design response to CC-01 | NEW | Could restore *delegated* orchestration: an orchestrator sub-agent spawns the swarm; synthesis/writing delegated (the "missing summarizer link" in `docs/subagent-delegation-audit.md`). Frees main-context tokens | **EVALUATE → lean NO (S7)** | "Can spawn 5 deep" ≠ "Voyage's orchestrator→6-agent-swarm pattern performs well." **S7 (2026-06-18, GATE):** feasibility probed cheaply — depth-2 nesting works, no degradation; depth cap moot (Voyage needs depth 2); NEW finding = auto-mode proliferation classifier polices agent fan-out (risk unique to delegation). On cost/benefit, **wholesale delegation NOT recommended**; the only defensible path is a narrow opt-in synthesis-agent PoC, proven by Δ main-context tokens. Operator gates verdict. Full bake-off designed but NOT run. See `docs/T1-cc26-delegated-orchestration.md` + §S7. |
|
||||
| **CC-27** | Dynamic Workflows / Workflow tool — orchestrates tens–hundreds of agents (**2.1.154**, verified); keyword `workflow`→`ultracode` (2.1.160); `agent()` attribution headers (2.1.174) | NEW | Voyage **hand-rolls** swarm/wave/pipeline orchestration in command prose — the Workflow tool is a native primitive for exactly this | **EVALUATE** (strategic) | The biggest identity decision: adopt Workflow as Voyage's execution substrate, or stay prose-orchestrated for portability/control? Tradeoffs: native concurrency + pipelining + budget control vs. dependency on a newer primitive + loss of fine-grained prose control + opt-in/billing semantics. Prototype one pipeline (e.g. `/trekreview`'s reviewer swarm) as a Workflow and compare. |
|
||||
| **CC-27** | Dynamic Workflows / Workflow tool — orchestrates tens–hundreds of agents (**2.1.154**, verified); keyword `workflow`→`ultracode` (2.1.160); `agent()` attribution headers (2.1.174) | NEW | Voyage **hand-rolls** swarm/wave/pipeline orchestration in command prose — the Workflow tool is a native primitive for exactly this | **EVALUATE → selective hybrid (S8)** | The biggest identity decision: adopt Workflow as substrate, or stay prose-orchestrated? **S8 (2026-06-18, GATE):** probed cheaply — a minimal trekreview-shaped `parallel()`→`agent()` Workflow ran end-to-end (F1), structured schemas retire the JSON-parse fragility at `trekreview.md:202–204` (F2), result returns to main (F3), and a small purposeful fan-out did **not** trip the S7 proliferation classifier (F4). Reframe: "substrate swap" is a false binary — a `/trek*` command is ~80% non-orchestration glue, so Workflow can only ever replace the fan-out→synthesize *core* (hybrid). **Recommendation: selective hybrid, NOT wholesale swap** — tier 1 ship a prose schema contract (the F2 win, no Workflow dep); tier 2 port trekreview Phase 5–6 to a Workflow only if the designed bake-off shows fidelity-equivalent output + acceptable control/cost; tier 3 wholesale swap declined (portability floor 2.1.154+, opt-in UX, visibility loss). Operator gates verdict. Full bake-off designed, NOT run. See `docs/T2-cc27-workflow-substrate.md` + §S8. |
|
||||
| **CC-28** | `TaskCreate` reliability — auto-repairs malformed input, schema in errors (2.1.163/169) | FIX | `TaskCreate`/`TaskUpdate` are in `trekplan`/orchestrator frontmatter but not actively used in command logic | **DEFER** | Becomes relevant only if W1 adopts task-graph orchestration. Tie to CC-26/27 outcome. |
|
||||
| **CC-29** | `subagent_type` matching now case/separator-insensitive (2.1.140); multiple `Agent(...)` types in `tools:` no longer dropped (2.1.147); subagent transcript/backgrounding fixes (2.1.178) | FIX | improves DX of any delegated-orchestration design | **SHIP** (verify) | Free robustness. Confirm Voyage's agent `tools:` grants (none currently declare multiple `Agent(...)` types) and `subagent_type` references are unaffected. |
|
||||
| **CC-31** | Worktree-isolation guard now applies in background sessions (2.1.154); `worktree.bgIsolation:"none"` (2.1.143); `EnterWorktree` switching mid-session (2.1.157) | CHANGE | `trekexecute` Phase 2.6 parallel waves + `trekplan` "execute with team" use git worktrees / `TeamCreate isolation:"worktree"` | **EVALUATE** | Verify Voyage's worktree-based parallel execution still behaves under the tightened bg-isolation guard. Affects the multi-session headless path. |
|
||||
|
|
@ -160,6 +160,41 @@ measurement-design doc; the expensive head-to-head comparison was specified but
|
|||
bake-off design (thresholds POSITIVE/NEGATIVE/INCONCLUSIVE), and the §6 synthesis-agent PoC, all
|
||||
ready to run if the operator greenlights pursuing delegation.
|
||||
|
||||
## S8 resolutions (W1 / CC-27 gate — 2026-06-18, operator-gated)
|
||||
|
||||
S8 was the second W1 gate — the orchestration-substrate identity decision. Operator chose the same
|
||||
**staged** execution as S7: cheap live feasibility probe + measurement-design doc; the head-to-head
|
||||
prose-vs-Workflow bake-off was specified but **not run**.
|
||||
|
||||
- **Feasibility (measured).** A minimal trekreview-shaped Workflow — `parallel([reviewerA,
|
||||
reviewerB])` with a findings schema → `agent(coordinator)` with a verdict schema, trivial agents,
|
||||
synthetic input — ran end-to-end in this CC 2.1.181 interactive session: both reviewers returned
|
||||
(F1), structured schemas delivered typed findings with no JSON-parse step (F2), the synthesizer
|
||||
reproduced Phase-6 dedup+verdict behavior and the result returned to main (F3), and the small
|
||||
purposeful fan-out did **not** trip the S7 proliferation classifier (F4). _3 agents · 85 461
|
||||
tokens · 13.8 s._
|
||||
- **Reframe — "substrate swap" is a false binary.** A `/trek*` command is ~80% non-orchestration
|
||||
glue (mode parsing, triage, validators, stats, HTML) and ~20% agent fan-out. The Workflow tool
|
||||
can only ever replace the fan-out→synthesize *core* (trekreview Phase 5–6), so the real decision
|
||||
is a **scoped hybrid per core**, not a wholesale identity swap. (Mirrors S7's "wall-time is not
|
||||
the gate metric" reframe.)
|
||||
- **The one concrete win is schema robustness (F2)** — and it is capturable *without* the Workflow
|
||||
tool: codify reviewer-output JSON as a validated prose schema contract, retiring the fragile
|
||||
"collect trailing JSON / re-ask on parse error" at `trekreview.md:202–204`.
|
||||
- **CC-27 recommendation (operator gates verdict).** **Selective hybrid, NOT wholesale swap**, in
|
||||
three tiers: (1) ship a prose schema contract regardless — the F2 win, zero new dependency; (2)
|
||||
port trekreview Phase 5–6 to a Workflow *only if* the designed bake-off shows fidelity-equivalent
|
||||
output + acceptable control/cost — best-case first port; (3) wholesale substrate swap **declined**
|
||||
(portability floor 2.1.154+, opt-in/billing UX on every invocation, loss of mid-flow operator
|
||||
visibility for an operator-gated review tool). CC-26 (S7) and CC-27 (S8) are now both resolved to
|
||||
"narrow/selective, operator-gated, not wholesale."
|
||||
- **Open risk inherited from S7.** Classifier behavior at *large* fan-out (trekplan's 6–10-agent
|
||||
swarm) under `auto`/`bypass` is still unverified — trekreview's 2–3 agents are below threshold,
|
||||
but a later trekplan port must measure it first.
|
||||
- **Artifact.** `docs/T2-cc27-workflow-substrate.md` — full gate evidence (F0–F4), §5 bake-off
|
||||
design (thresholds POSITIVE/NEGATIVE/INCONCLUSIVE), §6 no-Workflow schema-contract PoC, all ready
|
||||
to run if the operator greenlights tier 1 and/or tier 2.
|
||||
|
||||
## Sequencing
|
||||
|
||||
```
|
||||
|
|
@ -180,7 +215,7 @@ Incremental (W3): CC-11/CC-14/CC-15 as small, independently-shippable hardenin
|
|||
|
||||
## Open questions (need operator or empirical answer)
|
||||
|
||||
1. **W1 identity:** does Voyage adopt the Workflow tool as substrate, or stay prose-orchestrated? (CC-27)
|
||||
1. **W1 identity:** does Voyage adopt the Workflow tool as substrate, or stay prose-orchestrated? (CC-27) — **S8: false binary; lean selective hybrid (port the fan-out→synthesize core only), NOT wholesale swap; see §S8.**
|
||||
2. **W1 perf:** does delegated orchestration (orchestrator sub-agent → swarm) beat inline at Voyage's scale? (CC-26 — empirical) — **S7: feasibility YES, but lean NO on wholesale adoption; see §S7.**
|
||||
3. **W2 effort model:** map `phase_signals` onto native `effort:`, or keep bespoke? (CC-22 — gates v5.4)
|
||||
4. **CC-08:** is GH #36071 (hooks in headless) fixed? Determines whether the safety-preamble can relax.
|
||||
|
|
@ -188,7 +223,7 @@ Incremental (W3): CC-11/CC-14/CC-15 as small, independently-shippable hardenin
|
|||
## Empirical tests required
|
||||
|
||||
- **T1 (CC-26):** orchestrator-sub-agent spawns the planning swarm vs. inline baseline — wall-time, quality, token cost, depth-cap behavior. Harness: extend `scripts/q3-cache-prefix-experiment.mjs` pattern. — **S7: feasibility half RUN** (depth-2 works, no degradation, cap moot, proliferation-classifier risk found); **perf half DESIGNED, NOT run** (full bake-off + cheaper synthesis-agent PoC specified in `docs/T1-cc26-delegated-orchestration.md` §5/§6).
|
||||
- **T2 (CC-27):** reimplement `/trekreview`'s reviewer swarm as a Workflow; compare control, cost, and output fidelity vs. prose orchestration.
|
||||
- **T2 (CC-27):** reimplement `/trekreview`'s reviewer swarm as a Workflow; compare control, cost, and output fidelity vs. prose orchestration. — **S8: feasibility RUN** (probe: core ports natively, schemas retire JSON fragility, small fan-out classifier-clean); **fidelity/control/cost bake-off DESIGNED, NOT run** (`docs/T2-cc27-workflow-substrate.md` §5/§6).
|
||||
- **T3 (CC-04/CC-31):** research-agent MCP degradation under `--strict-mcp-config`; worktree parallel-wave behavior under tightened bg-isolation.
|
||||
|
||||
---
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue