portfolio-optimiser/docs/fase1-spikes
2026-06-24 10:13:23 +02:00
..
findings-a.md feat(fase1): spike A - group chat maker-checker vs single-agent [skip-docs] 2026-06-24 10:13:23 +02:00
README.md build(fase1): add dev orchestration + solver + async deps, scaffold spikes 2026-06-24 09:57:57 +02:00

Fase 1 — De-risk spikes (AD)

Throwaway spikes. Their only job is to turn the research §15 register's most dangerous documented-but-unverified assumptions into measured facts before the Fase 2 vertical slice. Code here is expected to be discarded once the findings are recorded — see Disposal.

Why these four

Before committing to the full architecture in Fase 2, we empirically de-risk the four assumptions that — if wrong — force a redesign:

Spike Assumption (register ref) What it measures
A Group Chat maker-checker beats a single-agent baseline by enough to justify its multiplicative token cost (U3 / G7) convergence rounds, stall frequency, token use — maker-checker vs single-agent, with a cheaper/better verdict
B The known MAF footguns behave as predicted and our guards hold: Magentic unbounded termination when limits=None (G1/B4); shared-builder / fan-out state corruption (G2/B7) guard fires on unbounded Magentic; zero state-bleed with the fresh-instance helper
C A blocking deterministic hybrid-validator (B1) can structurally block an out-of-range proposal structural rejection of an out-of-range proposal; P10/P50/P90 for a valid one; capped self-repair
D ExpeL retrieval (B2) surfaces a relevant prior verdict for a similar new proposal top-K retrieval returns the structurally-similar verdict over surface-text decoys

Each spike produces a short findings note (findings-{a,b,c,d}.md) with a confirmed/refuted verdict and a token-use line (including 0 — no live LLM where the spike runs without an endpoint).

Resolved version gate (premise-verified at planning time)

  • Installed agent-framework-core = 1.9.0 (introspection / uv pip show).
  • The orchestration builders (GroupChatBuilder, ConcurrentBuilder, MagenticBuilder, StandardMagenticManager, TerminationCondition) live in the separate package agent-framework-orchestrations, GA 1.0.0 (requires agent-framework-core<2,>=1.9.0 — exactly our installed core).
  • Because the spikes are throwaway, this package + pulp (Spike C solver) + pytest-asyncio (MAF orchestrations are async) are pinned in the dev optional-dependency group — not core. Promotion to core is a Fase 2 decision.

Gate-green contract

Each spike splits into:

  • a logic layer — pure functions/classes we author, always exercised by the quality gate (no live endpoint, no full MAF workflow run); and
  • an integration/live layer — drives the real MAF builders and/or a real LOCAL LLM; runs when available, otherwise skips (except Spike B, where driving the builders with a fake client IS the de-risk).

The headline empirical claims (Spike A's better/cheaper verdict; live token numbers) live in the integration/live layer and are honestly reported as endpoint-dependent. The gate stays green from the logic layer alone.

Status

Spike Assumption Result Confirmed/Refuted Implication for Fase 2
A maker-checker > single-agent (U3/G7) pending
B Magentic unbounded + fan-out bleed (G1/G2) pending
C blocking hybrid-validator (B1) pending
D ExpeL retrieval (B2) pending

(Consolidated in Step 7 once each spike's findings note is written.)

Disposal

These spikes are throwaway. To remove them completely after the findings are recorded:

rm -rf spikes tests/spikes docs/fase1-spikes

Then revert the three pyproject.toml edits that supported them:

  1. Remove the dev deps agent-framework-orchestrations, pulp, pytest-asyncio from [project.optional-dependencies].dev.
  2. Restore [tool.ruff] src = ["src", "tests"] (drop "spikes").
  3. Restore [tool.pytest.ini_options] pythonpath = ["src"] and remove asyncio_mode = "auto".

Then uv sync --extra dev to regenerate uv.lock. Nothing in src/ is touched by the spikes, so disposal leaves the framework core exactly as Fase 0 left it.