open/portfolio-optimiser
1
0
Fork 0
Code Issues Pull requests Releases Activity

feat(fase1): spike D - verdictstore + expel retrieval [skip-docs]

Browse source
This commit is contained in:
Kjell Tore Guttormsen 2026-06-24 10:32:39 +02:00
commit f7a36b59ac
3 changed files with 302 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

47
docs/fase1-spikes/findings-d.md Normal file
View file

@ -0,0 +1,47 @@
# Spike D findings — VerdictStore + ExpeL retrieval (B2)
**Assumption:** retrieval surfaces a relevant *prior* verdict for a similar new proposal —
the substrate for the framework's learning loop.
## Result — CONFIRMED (deterministic, no endpoint)
A minimal in-memory `VerdictStore` holds 12 synthetic verdicts seeded from the reference
domain's cost codes, measure types, magnitudes, and decisions (B2's "10–20").
**Similarity is structural, not textual (reviewer refinement #2):** a weighted score over
*structured* fields — Jaccard on the affected cost-code set (0.60) + a `measure_type`
match (0.25) + a magnitude-bucket match on the claimed saving (0.15). Raw description text
is **deliberately ignored**.
`retrieve(query, k)` is the guaranteed SC-D unit. The test is **non-tautological by
construction**: the true match shares the structured fields with the query but uses
*different wording*, while two decoys share the query's *surface text* but differ
structurally (disjoint codes, different measure type, different magnitude bucket). The
structural retriever returns the **true match as top-1** and ranks the surface-text decoys
last — a text-matching retriever would be fooled. Ordering is **deterministic** (ties
break by verdict id).
**ExpeL injection:** a thin `ExpeLContextProvider` subclasses the real
`agent_framework.ContextProvider` and, in `before_run`, injects the retrieved verdicts as
few-shot instructions via `SessionContext.extend_instructions` — asserted against the
introspected interface. The `retrieve` ranking remains the deliverable regardless of the
MAF session surface.
## Out of scope (Fase 2 option)
The **embedding-based** similarity path is intentionally not built — it needs a live
endpoint, is non-deterministic, and serves no SC for a throwaway spike. Structured-field
similarity is sufficient to confirm B2. Embeddings (or a hybrid structured+embedding score)
are a Fase 2 option for the durable VerdictStore.
## Token use
**0 — deterministic retrieval.** No model is called; similarity is pure arithmetic over
structured fields. The ExpeL provider only *formats* retrieved verdicts into few-shot text
— the actual model call that would consume tokens is a Fase 2 concern.
## Implication for Fase 2
The learning loop's retrieval is realizable with a simple, deterministic, structural
similarity — good enough to surface relevant prior verdicts. Fase 2 can keep this as the
baseline and add embeddings only if structured similarity proves insufficient on real data.