open/portfolio-optimiser
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions

docs(fase1): consolidate spike findings + confirm green quality gate

Browse source
This commit is contained in:
Kjell Tore Guttormsen 2026-06-24 10:35:33 +02:00
commit b81e22b637
1 changed files with 13 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

21
docs/fase1-spikes/README.md
View file

@ -45,16 +45,21 @@ The headline empirical claims (Spike A's better/cheaper verdict; live token numb
live in the integration/live layer and are honestly reported as **endpoint-dependent**.
The gate stays green from the logic layer alone.
## Status
## Status — all four de-risked (gate green)
| Spike | Assumption | Result | Confirmed/Refuted | Implication for Fase 2 |
|-------|-----------|--------|-------------------|------------------------|
| A | maker-checker > single-agent (U3/G7) | _pending_ | — | — |
| B | Magentic unbounded + fan-out bleed (G1/G2) | _pending_ | — | — |
| C | blocking hybrid-validator (B1) | _pending_ | — | — |
| D | ExpeL retrieval (B2) | _pending_ | — | — |
| Spike | Assumption | Result | Verdict | Token use | Implication for Fase 2 |
|-------|-----------|--------|---------|-----------|------------------------|
| **A** | maker-checker > single-agent (U3/G7) | verdict logic green; `GroupChatBuilder` drivable; cheaper/better is endpoint-dependent | **CONFIRMED (logic)** — empirical arm gated | word-count proxy; live arm not run (no endpoint) | Keep the codified `verdict` (better ∧ ≤3× tokens); measure the empirical cost/benefit on a LOCAL endpoint before locking the debate default |
| **B** | Magentic unbounded + fan-out bleed (G1/G2) | unbounded `max_round_count=None` needs an external guard; shared fan-out instance bleeds, fresh does not | **CONFIRMED** | 0 — no live LLM | Require explicit round/stop caps for any Magentic loop (fail-fast); use a fresh-instance-per-run factory for fan-out (B7) |
| **C** | blocking hybrid-validator (B1) | typed IR + real CBC solve + Monte-Carlo P10/P50/P90; out-of-range → `Rejection` (distinct type, no percentiles) | **CONFIRMED** | 0 — deterministic; live gen gated | Keep the `Rejection`/`ValidatedProposal` type split (structural block) + CBC-absent escalate; migrate to `pulp[cbc]`/`COIN_CMD` for PuLP 4.0 |
| **D** | ExpeL retrieval (B2) | structural similarity (codes + measure + magnitude) returns the true match as top-1 over surface-text decoys; deterministic | **CONFIRMED** | 0 — deterministic retrieval | Keep structured similarity as the baseline; add embeddings only if it proves insufficient on real data |
_(Consolidated in Step 7 once each spike's findings note is written.)_
**Builder de-risk (Step 2):** `FakeChatClient` drives the GA `GroupChatBuilder`,
`ConcurrentBuilder`, and `MagenticBuilder` — confirmed by the builder smoke + probes, so
Spikes A/B never needed the escalate fallback. Per-spike detail: `findings-{a,b,c,d}.md`.
**Quality gate at Fase 1 close:** `ruff check .` ✓, `ruff format --check .` ✓,
`mypy src` ✓, `pytest` ✓ (48 passed, 1 skipped = the gated live arm). `src/` untouched.
## Disposal