diff --git a/docs/fase1-spikes/README.md b/docs/fase1-spikes/README.md index ee8a3ab..8fbe505 100644 --- a/docs/fase1-spikes/README.md +++ b/docs/fase1-spikes/README.md @@ -45,16 +45,21 @@ The headline empirical claims (Spike A's better/cheaper verdict; live token numb live in the integration/live layer and are honestly reported as **endpoint-dependent**. The gate stays green from the logic layer alone. -## Status +## Status — all four de-risked (gate green) -| Spike | Assumption | Result | Confirmed/Refuted | Implication for Fase 2 | -|-------|-----------|--------|-------------------|------------------------| -| A | maker-checker > single-agent (U3/G7) | _pending_ | — | — | -| B | Magentic unbounded + fan-out bleed (G1/G2) | _pending_ | — | — | -| C | blocking hybrid-validator (B1) | _pending_ | — | — | -| D | ExpeL retrieval (B2) | _pending_ | — | — | +| Spike | Assumption | Result | Verdict | Token use | Implication for Fase 2 | +|-------|-----------|--------|---------|-----------|------------------------| +| **A** | maker-checker > single-agent (U3/G7) | verdict logic green; `GroupChatBuilder` drivable; cheaper/better is endpoint-dependent | **CONFIRMED (logic)** — empirical arm gated | word-count proxy; live arm not run (no endpoint) | Keep the codified `verdict` (better ∧ ≤3× tokens); measure the empirical cost/benefit on a LOCAL endpoint before locking the debate default | +| **B** | Magentic unbounded + fan-out bleed (G1/G2) | unbounded `max_round_count=None` needs an external guard; shared fan-out instance bleeds, fresh does not | **CONFIRMED** | 0 — no live LLM | Require explicit round/stop caps for any Magentic loop (fail-fast); use a fresh-instance-per-run factory for fan-out (B7) | +| **C** | blocking hybrid-validator (B1) | typed IR + real CBC solve + Monte-Carlo P10/P50/P90; out-of-range → `Rejection` (distinct type, no percentiles) | **CONFIRMED** | 0 — deterministic; live gen gated | Keep the `Rejection`/`ValidatedProposal` type split (structural block) + CBC-absent escalate; migrate to `pulp[cbc]`/`COIN_CMD` for PuLP 4.0 | +| **D** | ExpeL retrieval (B2) | structural similarity (codes + measure + magnitude) returns the true match as top-1 over surface-text decoys; deterministic | **CONFIRMED** | 0 — deterministic retrieval | Keep structured similarity as the baseline; add embeddings only if it proves insufficient on real data | -_(Consolidated in Step 7 once each spike's findings note is written.)_ +**Builder de-risk (Step 2):** `FakeChatClient` drives the GA `GroupChatBuilder`, +`ConcurrentBuilder`, and `MagenticBuilder` — confirmed by the builder smoke + probes, so +Spikes A/B never needed the escalate fallback. Per-spike detail: `findings-{a,b,c,d}.md`. + +**Quality gate at Fase 1 close:** `ruff check .` ✓, `ruff format --check .` ✓, +`mypy src` ✓, `pytest` ✓ (48 passed, 1 skipped = the gated live arm). `src/` untouched. ## Disposal