portfolio-optimiser/docs/fase1-spikes/findings-b.md

2.9 KiB

Spike B findings — Magentic unbounded footgun (G1/B4) + fan-out state isolation (G2/B7)

Assumptions: (G1/B4) Magentic with limits=None does NOT self-terminate, so an explicit limit is mandatory; (G2/B7) reusing a shared fan-out builder/workflow instance corrupts state, while a fresh-instance helper prevents it.

This spike drives the real GA builders with the fake client (de-risked by the Step 2 builder smoke), so it runs entirely in the quality gate — no live LLM.

(a) Magentic unbounded termination — CONFIRMED

  • A StandardMagenticManager whose progress ledger always reports not satisfied / progress being made never finalizes.
  • With max_round_count=None the workflow streams orchestration events indefinitely; only the shared harness round/iteration guard (Budget + TokenMeter) stops it. unbounded_magentic_self_terminates() returns False → it does not self-terminate.
  • With an explicit max_round_count=2 the same manager stops cleanly ("Magentic Orchestrator: Max round count reached") and yields an output.
  • Implication (B4): a stop-criterion/round cap is mandatory at startup for any Magentic use — never rely on natural termination. (Magentic stays experimental and is exercised here ONLY to confirm the footgun, never as the debate default — that remains Group Chat maker-checker, A2/G8.)

Minor MAF observation (recorded)

Cutting an instrumented Magentic stream short — the only way to observe an unbounded run — makes OpenTelemetry log a benign Failed to detach context on generator close, and the in-flight fake response surfaces a RuntimeWarning: coroutine was never awaited. Both are cosmetic (no functional effect); the spike silences the OTel context logger and filters the warning. Worth knowing if Fase 2 observes streamed orchestrations.

(b) Fan-out state isolation — CONFIRMED

  • Shared instance: one ConcurrentBuilder workflow reused across the three reference projects → the shared fake clients accumulate calls (max call count == 3) → state bleed.
  • Fresh instance: fresh_workflow() builds a new workflow + fresh clients per run → each run's clients see exactly one call ([1, 1, 1]) → zero bleed across all 3 projects (B7).
  • Implication: Fase 2 fan-out must build a fresh workflow/executor instance per project run (a factory like fresh_workflow()), never reuse a shared instance.

Token use

0 — no live LLM. All behavior is driven by the deterministic FakeChatClient; its "tokens" are word-counts of canned replies and are not meaningful cost numbers here.

Implication for Fase 2

Both footguns behave exactly as the §15 register predicted. The framework MUST: (1) require explicit round/stop caps for any Magentic-style loop (fail-fast at startup, D6/B4); and (2) use a fresh-instance-per-run factory for fan-out to avoid state corruption (B7).