76 lines
4.1 KiB
Markdown
76 lines
4.1 KiB
Markdown
# Fase 1 — De-risk spikes (A–D)
|
||
|
||
> **Throwaway** spikes. Their only job is to turn the research §15 register's most
|
||
> dangerous *documented-but-unverified* assumptions into **measured facts** before
|
||
> the Fase 2 vertical slice. Code here is expected to be discarded once the
|
||
> findings are recorded — see [Disposal](#disposal).
|
||
|
||
## Why these four
|
||
|
||
Before committing to the full architecture in **Fase 2**, we empirically de-risk
|
||
the four assumptions that — if wrong — force a redesign:
|
||
|
||
| Spike | Assumption (register ref) | What it measures |
|
||
|-------|---------------------------|------------------|
|
||
| **A** | Group Chat maker-checker beats a single-agent baseline by enough to justify its multiplicative token cost (U3 / G7) | convergence rounds, stall frequency, token use — maker-checker vs single-agent, with a cheaper/better verdict |
|
||
| **B** | The known MAF footguns behave as predicted and our guards hold: Magentic unbounded termination when `limits=None` (G1/B4); shared-builder / fan-out state corruption (G2/B7) | guard fires on unbounded Magentic; zero state-bleed with the fresh-instance helper |
|
||
| **C** | A blocking deterministic hybrid-validator (B1) can *structurally* block an out-of-range proposal | structural rejection of an out-of-range proposal; P10/P50/P90 for a valid one; capped self-repair |
|
||
| **D** | ExpeL retrieval (B2) surfaces a relevant prior verdict for a similar new proposal | top-K retrieval returns the structurally-similar verdict over surface-text decoys |
|
||
|
||
Each spike produces a short findings note (`findings-{a,b,c,d}.md`) with a
|
||
**confirmed/refuted** verdict and a **token-use line** (including `0 — no live LLM`
|
||
where the spike runs without an endpoint).
|
||
|
||
## Resolved version gate (premise-verified at planning time)
|
||
|
||
- Installed `agent-framework-core` = **1.9.0** (introspection / `uv pip show`).
|
||
- The orchestration builders (`GroupChatBuilder`, `ConcurrentBuilder`,
|
||
`MagenticBuilder`, `StandardMagenticManager`, `TerminationCondition`) live in the
|
||
separate package **`agent-framework-orchestrations`**, GA **1.0.0**
|
||
(`requires agent-framework-core<2,>=1.9.0` — exactly our installed core).
|
||
- Because the spikes are **throwaway**, this package + `pulp` (Spike C solver) +
|
||
`pytest-asyncio` (MAF orchestrations are async) are pinned in the **`dev`**
|
||
optional-dependency group — **not** core. Promotion to core is a Fase 2 decision.
|
||
|
||
## Gate-green contract
|
||
|
||
Each spike splits into:
|
||
- a **logic layer** — pure functions/classes we author, always exercised by the
|
||
quality gate (no live endpoint, no full MAF workflow run); and
|
||
- an **integration/live layer** — drives the real MAF builders and/or a real LOCAL
|
||
LLM; runs when available, otherwise `skip`s (except Spike B, where driving the
|
||
builders with a fake client IS the de-risk).
|
||
|
||
The headline empirical claims (Spike A's better/cheaper verdict; live token numbers)
|
||
live in the integration/live layer and are honestly reported as **endpoint-dependent**.
|
||
The gate stays green from the logic layer alone.
|
||
|
||
## Status
|
||
|
||
| Spike | Assumption | Result | Confirmed/Refuted | Implication for Fase 2 |
|
||
|-------|-----------|--------|-------------------|------------------------|
|
||
| A | maker-checker > single-agent (U3/G7) | _pending_ | — | — |
|
||
| B | Magentic unbounded + fan-out bleed (G1/G2) | _pending_ | — | — |
|
||
| C | blocking hybrid-validator (B1) | _pending_ | — | — |
|
||
| D | ExpeL retrieval (B2) | _pending_ | — | — |
|
||
|
||
_(Consolidated in Step 7 once each spike's findings note is written.)_
|
||
|
||
## Disposal
|
||
|
||
These spikes are throwaway. To remove them completely after the findings are recorded:
|
||
|
||
```bash
|
||
rm -rf spikes tests/spikes docs/fase1-spikes
|
||
```
|
||
|
||
Then revert the three `pyproject.toml` edits that supported them:
|
||
|
||
1. Remove the dev deps `agent-framework-orchestrations`, `pulp`, `pytest-asyncio`
|
||
from `[project.optional-dependencies].dev`.
|
||
2. Restore `[tool.ruff] src = ["src", "tests"]` (drop `"spikes"`).
|
||
3. Restore `[tool.pytest.ini_options] pythonpath = ["src"]` and remove
|
||
`asyncio_mode = "auto"`.
|
||
|
||
Then `uv sync --extra dev` to regenerate `uv.lock`. Nothing in `src/` is touched by
|
||
the spikes, so disposal leaves the framework core exactly as Fase 0 left it.
|