feat(fase1): spike B - magentic unbounded + concurrent state isolation [skip-docs]
This commit is contained in:
parent
9b9a17e2ed
commit
44111113fb
3 changed files with 213 additions and 0 deletions
52
docs/fase1-spikes/findings-b.md
Normal file
52
docs/fase1-spikes/findings-b.md
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
# Spike B findings — Magentic unbounded footgun (G1/B4) + fan-out state isolation (G2/B7)
|
||||
|
||||
**Assumptions:** (G1/B4) Magentic with `limits=None` does NOT self-terminate, so an
|
||||
explicit limit is mandatory; (G2/B7) reusing a shared fan-out builder/workflow instance
|
||||
corrupts state, while a fresh-instance helper prevents it.
|
||||
|
||||
This spike drives the **real** GA builders with the fake client (de-risked by the Step 2
|
||||
builder smoke), so it runs entirely in the quality gate — **no live LLM**.
|
||||
|
||||
## (a) Magentic unbounded termination — CONFIRMED
|
||||
|
||||
- A `StandardMagenticManager` whose progress ledger always reports *not satisfied /
|
||||
progress being made* never finalizes.
|
||||
- With **`max_round_count=None`** the workflow streams orchestration events indefinitely;
|
||||
only the shared harness round/iteration guard (`Budget` + `TokenMeter`) stops it.
|
||||
`unbounded_magentic_self_terminates()` returns **False** → it does **not** self-terminate.
|
||||
- With an explicit **`max_round_count=2`** the same manager stops cleanly
|
||||
("Magentic Orchestrator: Max round count reached") and yields an output.
|
||||
- **Implication (B4):** a stop-criterion/round cap is mandatory at startup for any
|
||||
Magentic use — never rely on natural termination. (Magentic stays experimental and is
|
||||
exercised here ONLY to confirm the footgun, never as the debate default — that remains
|
||||
Group Chat maker-checker, A2/G8.)
|
||||
|
||||
### Minor MAF observation (recorded)
|
||||
|
||||
Cutting an *instrumented* Magentic stream short — the only way to observe an unbounded run
|
||||
— makes OpenTelemetry log a benign `Failed to detach context` on generator close, and the
|
||||
in-flight fake response surfaces a `RuntimeWarning: coroutine was never awaited`. Both are
|
||||
cosmetic (no functional effect); the spike silences the OTel context logger and filters the
|
||||
warning. Worth knowing if Fase 2 observes streamed orchestrations.
|
||||
|
||||
## (b) Fan-out state isolation — CONFIRMED
|
||||
|
||||
- **Shared instance:** one `ConcurrentBuilder` workflow reused across the three reference
|
||||
projects → the shared fake clients accumulate calls (max call count == 3) → **state
|
||||
bleed**.
|
||||
- **Fresh instance:** `fresh_workflow()` builds a new workflow + fresh clients per run →
|
||||
each run's clients see exactly one call (`[1, 1, 1]`) → **zero bleed** across all 3
|
||||
projects (B7).
|
||||
- **Implication:** Fase 2 fan-out must build a fresh workflow/executor instance per
|
||||
project run (a factory like `fresh_workflow()`), never reuse a shared instance.
|
||||
|
||||
## Token use
|
||||
|
||||
**0 — no live LLM.** All behavior is driven by the deterministic `FakeChatClient`; its
|
||||
"tokens" are word-counts of canned replies and are not meaningful cost numbers here.
|
||||
|
||||
## Implication for Fase 2
|
||||
|
||||
Both footguns behave exactly as the §15 register predicted. The framework MUST: (1) require
|
||||
explicit round/stop caps for any Magentic-style loop (fail-fast at startup, D6/B4); and
|
||||
(2) use a fresh-instance-per-run factory for fan-out to avoid state corruption (B7).
|
||||
Loading…
Add table
Add a link
Reference in a new issue