feat(fase2): wire Step-1 ExpeL retrieval into the hypothesis prompt

Closes maalbilde §5 gap #1 (the one missing "feedback-into-prompt" dataflow)
for the OKF-bundle path. Before, ExpeL was computed AFTER generation into a
discarded SessionContext, so a prior verdict could not influence any hypothesis
(context_providers=0).

- New okf.py: framework-neutral OKF bundle navigation (index + frontmatter +
  cross-links), pure stdlib, no agent_framework/mcp (D7-portable), enforced by
  test_okf_is_maf_free.
- verdicts.py: seed_store_from_bundle + bundle_candidate_features build the
  ExpeL substrate + the pre-hypothesis query key from a bundle.
- run_project(bundle_dir=...): folds the candidate's prior verdicts into the
  generation context BEFORE generate_via_llm; the road path is unchanged.

Load-bearing (maalbilde §7): test_step1_expel_loadbearing proves a prior verdict
reaches the hypothesis prompt and goes RED when the fold is detached (shown via
TDD red->green). The marker is the minted verdict id (content hash) because
docs_dir==bundle_dir lets keyword chunk-stuffing leak the realization rate;
clean layer separation is Fase 2b.

Suite 121->133 passed; mypy + ruff check clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MHR8iKxJRxDiDfNw8HZmWE
This commit is contained in:
Kjell Tore Guttormsen 2026-06-29 10:56:48 +02:00
commit d6d83d42b5
9 changed files with 529 additions and 7 deletions

View file

@ -25,6 +25,10 @@ Python ≥3.10. MAF (`agent-framework-core` 1.9.0). Pakkehåndtering: `uv`. To b
- **Rent teknisk rammeverk:** deployer eier DPIA/ROS/behandlingsformål. Bygg IKKE compliance-funksjoner — kun tekniske forutsetninger (lokal-only, provenance, ingen stille egress) + disclaimer.
- **90%-prinsipp:** bygg den generiske kjernen + tydelige extension points; jakt IKKE de siste 10 %.
- **Deterministisk validator er obligatorisk og blokkerende** — aldri valgfri plugin.
- **Framework-nøytral kontekst-søm:** OKF-bundle-navigasjon (`okf.py`) og den delte
`shared/`-kjernen er ren stdlib — null `agent_framework`/`mcp`-import, så samme bundles
konsumeres uendret av begge stacker (D7-portabel). Håndhevet av
`tests/test_okf.py::test_okf_is_maf_free`; importér aldri MAF inn i kontekst-laget.
- **Stoppkriterier + budsjett-tak påkrevd ved oppstart** (fail-fast, aldri ubegrenset loop).
- **Group Chat maker-checker** som debatt-default (IKKE Magentic, som er eksperimentell).
- **Kostnadsdisiplin:** utvikle primært på lokal profil (gratis); Foundry/Azure (privat tenant finnes) kun til målrettet, minimal verifisering; billigste modeller + små syntetiske data + harde token-tak. Ingen tunge test-kjøringer.

View file

@ -2,7 +2,7 @@
Generic, open framework on **Microsoft Agent Framework (MAF)** for finding cost-savings / efficiency proposals *within* each project of a portfolio of independent projects. Multiple agents collaborate to generate candidate proposals; a mandatory deterministic validator (solver + Monte Carlo) decides the numbers; domain experts review via human-in-the-loop, and the system learns from their verdicts.
> **Status:** Early development (plan phase). Not yet usable.
> **Status:** Early development. The deterministic backbone is solid; the agentic learning loop is being wired one load-bearing seam at a time — **Step 1 (OKF context → hypothesis) is wired** (see below). Not yet end-to-end usable.
> **Disclaimer — technical framework only.** This project is a *technical framework*. Organizations that deploy it are themselves responsible for ensuring a valid processing purpose and for any required assessments (DPIA, risk/ROS, security reviews, etc.). The framework ships technical affordances (local-only mode, provenance/audit logging, no silent data egress) to *enable* compliant use, but makes no compliance guarantees.
@ -10,6 +10,13 @@ Generic, open framework on **Microsoft Agent Framework (MAF)** for finding cost-
The result will never fit any single customer 100%. The goal is a **~90% genuinely generic core plus clear extension points**, so competent people can configure the last mile per customer. We deliberately do not chase the final 10%.
## Agentic loop — wiring status
The mandatory deterministic backbone (validator + budget meter + provenance) is solid and load-bearing. The agentic learning loop (see the [target picture](docs/plan/2026-06-26-maalbilde-agentic-loop.md) §11) is wired one seam at a time:
- **Step 1 — OKF context → hypothesis (wired).** `run_project(..., bundle_dir=...)` navigates a project's [OKF bundle](shared/examples/bygg-energi-mikro/) and folds the candidate's prior expert verdicts (ExpeL retrieval) into the hypothesis prompt *before* generation — so a prior verdict provably influences the next hypothesis. Guarded by a load-bearing test that fails when the seam is detached (`tests/test_step1_expel_loadbearing.py`).
- **Steps 38** (checker gating, informed refinement, async file feedback, gated wiki promotion) — not yet wired.
## Docs
- [`docs/plan/2026-06-26-maalbilde-agentic-loop.md`](docs/plan/2026-06-26-maalbilde-agentic-loop.md) — target picture: the agentic cost-saving loop + OKF knowledge architecture (north star).

View file

@ -0,0 +1,135 @@
"""OKF (Open Knowledge Format) bundle navigation — framework-neutral, D7-portable context seam.
Reads a bundle the way OKF intends (progressive disclosure): start at ``index.md``, follow
intra-bundle cross-links, parse each file's YAML frontmatter, classify by the one required
``type`` field. **NO** ``agent_framework``, **NO** ``mcp`` pure stdlib, so the SAME navigation
serves both the MAF and the Claude-SDK implementations unchanged (målbilde §4 vendor-neutrality).
Robustness is part of the spec (OKF SPEC §4): consumers MUST tolerate broken links and unknown
fields. A link to a missing file or one escaping the bundle is silently skipped, never raised.
Path-safety reuses ``retrieval.safe_resolve`` (also pure stdlib): each cross-link is canonicalised
and boundary-checked against the bundle dir, fail-closed.
"""
from __future__ import annotations
import json
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from portfolio_optimiser.retrieval import PathSecurityError, safe_resolve
_INDEX_NAME = "index.md"
_IR_PROJECTION = "validator-input.json"
# Intra-bundle markdown cross-links: ``](target.md)``. Targets with a path separator (``/``) are
# treated as out-of-bundle and skipped (only same-dir bundle files are navigated).
_LINK_RE = re.compile(r"\]\(([^)]+\.md)\)")
def parse_frontmatter(path: str | Path) -> dict[str, str]:
"""Read the leading ``---``-delimited YAML frontmatter block as key:value strings.
Minimal by design (no ``yaml`` dependency): enough for the one required ``type`` field and the
verdict's scalar fields. List values (``tags: [...]``) are kept verbatim; unknown fields are
preserved (OKF SPEC §4). Returns ``{}`` when there is no frontmatter block."""
lines = Path(path).read_text(encoding="utf-8").splitlines()
if not lines or lines[0].strip() != "---":
return {}
fm: dict[str, str] = {}
for line in lines[1:]:
if line.strip() == "---":
break
key, sep, val = line.partition(":")
if sep:
fm[key.strip()] = val.strip()
return fm
def _read_body(path: Path) -> str:
"""The markdown body after the frontmatter block (or the whole file if there is none)."""
lines = path.read_text(encoding="utf-8").splitlines(keepends=True)
if lines and lines[0].strip() == "---":
for i in range(1, len(lines)):
if lines[i].strip() == "---":
return "".join(lines[i + 1 :]).lstrip("\n")
return "".join(lines)
@dataclass(frozen=True)
class BundleFile:
"""One OKF file: its name, declared ``type`` (``""`` if absent), frontmatter, and body."""
name: str
type: str
frontmatter: dict[str, str]
body: str
@dataclass(frozen=True)
class Bundle:
"""A navigated OKF bundle: ``index.md`` plus every cross-linked file that resolves."""
dir: str
files: tuple[BundleFile, ...]
@property
def index_summary(self) -> str:
"""The index body — the progressive-disclosure entry point (not whole-bundle stuffing)."""
return next((f.body for f in self.files if f.name == _INDEX_NAME), "")
@property
def verdicts(self) -> list[BundleFile]:
"""Every ``type: verdict`` file (the ExpeL seeds the Step-1 wiring retrieves)."""
return [f for f in self.files if f.type == "verdict"]
@property
def hypothesis(self) -> BundleFile | None:
"""The candidate ``type: hypothesis`` file, if present."""
return next((f for f in self.files if f.type == "hypothesis"), None)
def _load_file(bundle_dir: str, name: str) -> BundleFile | None:
"""Resolve ``name`` within ``bundle_dir`` and read it, or ``None`` if missing / escaping the
bundle (OKF §4 broken-link tolerance + fail-closed path-safety)."""
try:
resolved = Path(safe_resolve(bundle_dir, name))
except PathSecurityError:
return None
if not resolved.is_file():
return None
fm = parse_frontmatter(resolved)
return BundleFile(name=name, type=fm.get("type", ""), frontmatter=fm, body=_read_body(resolved))
def navigate_bundle(bundle_dir: str) -> Bundle:
"""Navigate the OKF bundle from ``index.md``: parse the index, follow its intra-bundle ``.md``
cross-links, and read each linked file's frontmatter + body. Deterministic: index first, then
links in first-seen order, de-duplicated. Broken / escaping links are skipped (§4). Raises
``ValueError`` only when ``index.md`` itself is unreadable (a bundle has no entry point)."""
index = _load_file(bundle_dir, _INDEX_NAME)
if index is None:
raise ValueError(f"OKF bundle has no readable {_INDEX_NAME}: {bundle_dir!r}")
files: list[BundleFile] = [index]
seen = {_INDEX_NAME}
for target in _LINK_RE.findall(index.body):
if "/" in target or target in seen:
continue # only same-dir bundle files; de-dup repeated links
seen.add(target)
linked = _load_file(bundle_dir, target)
if linked is not None:
files.append(linked)
return Bundle(dir=bundle_dir, files=tuple(files))
def load_ir_projection(bundle_dir: str, name: str = _IR_PROJECTION) -> dict[str, Any]:
"""Load the bundle's IR projection (``validator-input.json`` by default): the candidate
measure's cost-IR (``measure``, ``affected_items``, ``claimed_saving_nok``) — the
pre-hypothesis ExpeL query-key source. Raises if missing / escaping the bundle (fail-fast: it
is required input, not an optional cross-link)."""
resolved = Path(safe_resolve(bundle_dir, name))
if not resolved.is_file():
raise FileNotFoundError(f"IR projection not found in bundle: {name!r}")
data: dict[str, Any] = json.loads(resolved.read_text(encoding="utf-8"))
return data

View file

@ -44,11 +44,13 @@ from portfolio_optimiser.ir import SavingsProposal
from portfolio_optimiser.provenance import ProvenanceStamp
from portfolio_optimiser.reference_domain import Project, load_reference_projects
from portfolio_optimiser.validator import Rejection, ValidatedProposal
from portfolio_optimiser import okf
from portfolio_optimiser.verdicts import (
ExpeLContextProvider,
ProposalFeatures,
Verdict,
VerdictStore,
bundle_candidate_features,
capture_verdict,
)
from portfolio_optimiser.workflow import fresh_workflow
@ -114,6 +116,34 @@ def _project_by_id(project_id: str) -> Project:
raise ValueError(f"unknown project_id: {project_id!r}")
def _project_from_bundle(bundle_dir: str, project_id: str) -> Project:
"""Derive a minimal ``Project`` from an OKF bundle (so a bundle the loop runs need NOT be a
road reference-domain project). Only ``id`` + ``name`` reach the generation prompt
(``generate._build_messages``), so ``cost_items`` is empty and ``verdict_input`` is unused here
(the Layer-2 decision flows via the ``verdict_input`` argument). Fail-fast: the bundle's IR
``project_id`` must match the requested id."""
ir = okf.load_ir_projection(bundle_dir)
if ir["project_id"] != project_id:
raise ValueError(f"bundle project_id {ir['project_id']!r} != requested {project_id!r}")
project_file = next(
(f for f in okf.navigate_bundle(bundle_dir).files if f.type == "project"), None
)
name = (
project_file.frontmatter.get("title", project_id).strip('"')
if project_file is not None
else project_id
)
return Project(
id=project_id,
name=name,
description="",
currency="NOK",
cost_items=(),
docs_dir=bundle_dir,
verdict_input={},
)
def _features_of(proposal: SavingsProposal) -> ProposalFeatures:
return ProposalFeatures(
affected_codes=frozenset(item.code for item in proposal.affected_items),
@ -136,6 +166,7 @@ async def run_project(
*,
docs_dir: str,
verdict_input: dict[str, str],
bundle_dir: str | None = None,
store: VerdictStore | None = None,
client_factory: Callable[[str], BaseChatClient] | None = None,
max_rounds: int = 3,
@ -147,8 +178,11 @@ async def run_project(
) -> RunResult:
"""Run the vertical slice for ONE project. ``client_factory`` is the test-injection seam
(defaults to the real backend). ``verdict_input`` carries the expert decision/rationale
(Layer-2). Raises ``pydantic.ValidationError`` on a bad contract and ``BudgetExceeded``
when the token/round cap is crossed."""
(Layer-2). ``bundle_dir`` (Fase 2a) makes the run OKF-bundle-driven: the project is derived
from the bundle and, before generation, the candidate's prior verdicts in ``store`` are folded
into the hypothesis prompt (Step-1 ExpeL wiring, målbilde §5/§7). Raises
``pydantic.ValidationError`` on a bad contract and ``BudgetExceeded`` when the token/round cap
is crossed."""
# 1. Fail-fast: validate ALL contracts (incl. the verdict-feedback shape) before any client.
load_contracts(
{"docs_dir": docs_dir, "top_k": top_k},
@ -156,8 +190,13 @@ async def run_project(
verdict_input,
)
# 2-3. Project + cited chunks (first-class provenance citations).
project = _project_by_id(project_id)
# 2-3. Project + cited chunks (first-class provenance citations). An OKF-bundle run derives its
# project from the bundle (need not be a road reference-domain project).
project = (
_project_from_bundle(bundle_dir, project_id)
if bundle_dir is not None
else _project_by_id(project_id)
)
chunks = retrieve_chunks("cost saving measure", docs_dir, top_k)
citations = [chunk_dict_to_citation(c) for c in chunks]
if not citations:
@ -189,6 +228,17 @@ async def run_project(
debate_output = _debate_text(result)
gen_context = debate_output or context
# Step-1 ExpeL wiring (Fase 2a, målbilde §5/§7): fold the candidate's prior verdicts INTO the
# hypothesis context BEFORE generation, keyed on the OKF bundle's candidate features (available
# pre-hypothesis). THIS is the one missing dataflow — previously ExpeL was computed
# post-generation into a discarded SessionContext (step 7 below), so a prior verdict could not
# reach the next hypothesis. Bundle-driven path with a populated store only; the road path is
# untouched (its post-hoc, proposal-keyed retrieval below is unchanged).
if bundle_dir is not None and store is not None and store.verdicts:
expel_query = bundle_candidate_features(bundle_dir)
fewshot = ExpeLContextProvider(store, expel_query, k=top_k).format_fewshot()
gen_context = f"{fewshot}\n\n{gen_context}"
# 5. Structured candidate -> blocking validation; token bound = the meter in this loop.
proposer_client = factory("proposer")
outcome = await generate_via_llm(proposer_client, project, gen_context, meter)
@ -211,8 +261,11 @@ async def run_project(
token_usage=meter.tokens,
)
# 7. ExpeL: surface prior verdicts for this proposal (exercises the two-arg
# extend_instructions injection on a real SessionContext — the learning loop).
# 7. ExpeL (regression guard + traceability): exercises the two-arg extend_instructions
# injection on a REAL SessionContext (the Critical Fase-1 GA-signature guard), and surfaces
# the proposal-keyed retrieval for RunResult.retrieved. On the bundle path the load-bearing
# ExpeL->prompt dataflow already happened pre-generation (above); this block's SessionContext
# is NOT what reaches the prompt.
store = store if store is not None else VerdictStore(verdicts=[])
features = _features_of(proposal)
provider = ExpeLContextProvider(store, features, k=top_k)

View file

@ -21,9 +21,12 @@ from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from typing import Any
from agent_framework import ContextProvider, SessionContext
from portfolio_optimiser import okf
# Weights: the affected cost-code overlap dominates, then measure type, then magnitude.
_W_CODES, _W_MEASURE, _W_MAGNITUDE = 0.60, 0.25, 0.15
_MAGNITUDE_BUCKETS = [(0.0, 1e5), (1e5, 5e5), (5e5, 1e6), (1e6, float("inf"))]
@ -196,3 +199,58 @@ def seed_store() -> VerdictStore:
for vid, codes, mtype, saving, decision, desc in rows
]
)
# --- OKF-bundle seeding (Fase 2a): turn a project's bundle into the ExpeL substrate ---
def _features_from_ir(ir: dict[str, Any]) -> ProposalFeatures:
"""Map a bundle's IR projection (``validator-input.json``) to the structural features the
store ranks on: the affected cost-code set, the measure string, and the claimed magnitude."""
return ProposalFeatures(
affected_codes=frozenset(item["code"] for item in ir["affected_items"]),
measure_type=ir["measure"],
claimed_saving_nok=ir["claimed_saving_nok"],
description=ir.get("measure", ""),
)
def bundle_candidate_features(bundle_dir: str) -> ProposalFeatures:
"""The pre-hypothesis ExpeL query key: the candidate measure's structural features, read from
the OKF bundle's IR projection. Available BEFORE any proposal is generated — which is what lets
Step-1 retrieve prior verdicts and fold them into the hypothesis prompt (målbilde §2 step 1)."""
return _features_from_ir(okf.load_ir_projection(bundle_dir))
def _verdict_rationale(fm: dict[str, str]) -> str:
"""Build the few-shot rationale from a ``type: verdict`` file's frontmatter, carrying the
learning signal the deterministic validator cannot compute (the realization rate + expected
actual). This is the ExpeL signal that must reach the next hypothesis."""
base = fm.get("description", "")
signal = [
f"{label}={fm[key]}"
for key, label in (
("realization_rate", "realiseringsgrad"),
("expected_actual_saving_nok", "forventet_faktisk_NOK"),
)
if fm.get(key)
]
return f"{base} [{'; '.join(signal)}]" if signal else base
def seed_store_from_bundle(bundle_dir: str) -> VerdictStore:
"""Build a ``VerdictStore`` from an OKF bundle's ``type: verdict`` files. Each verdict is keyed
on the bundle's candidate features (so it retrieves for that measure) and carries the
realization signal in its rationale. The seed verdict stands in for the durable HITL verdict a
real expert would supply via the same folder interface (målbilde §3)."""
features = bundle_candidate_features(bundle_dir)
bundle = okf.navigate_bundle(bundle_dir)
verdicts = [
capture_verdict(
features,
vf.frontmatter.get("decision", "approved"),
_verdict_rationale(vf.frontmatter),
)
for vf in bundle.verdicts
]
return VerdictStore(verdicts=verdicts)

View file

@ -163,6 +163,43 @@ def make_portfolio_client_factory() -> Callable[..., Callable[[str], BaseChatCli
return _make
class _RecordingChatClient(SyntheticUsageChatClient):
"""Records the incoming prompt blob per call into a SHARED sink, then returns a fixed valid
reply. Lets a test assert exactly what text reached the prompt the probe the Step-1 ExpeL
wiring is made load-bearing against (does a prior verdict reach the hypothesis prompt?)."""
def __init__(self, sink: list[str], reply: str, *, tokens_per_reply: int = 8) -> None:
super().__init__(default_reply=reply, tokens_per_reply=tokens_per_reply)
self._sink = sink
def _inner_get_response(
self, *, messages: Sequence[Message], stream: bool, options: Any, **kwargs: Any
) -> Any:
self._sink.append(" ".join(getattr(m, "text", "") or "" for m in messages))
return super()._inner_get_response(
messages=messages, stream=stream, options=options, **kwargs
)
@pytest.fixture()
def make_recording_client_factory() -> Callable[
[str], tuple[Callable[[str], BaseChatClient], list[str]]
]:
"""Return a maker that builds a per-role client factory recording every prompt blob into a
shared list. Returns ``(factory, recorded_prompts)`` so the test inspects what reached the
prompt across the whole run (debate rounds + generation)."""
def _make(reply: str) -> tuple[Callable[[str], BaseChatClient], list[str]]:
sink: list[str] = []
def factory(role: str) -> BaseChatClient:
return _RecordingChatClient(sink, reply)
return factory, sink
return _make
@pytest.fixture()
def fresh_store() -> VerdictStore:
return VerdictStore(verdicts=[])

96
tests/test_okf.py Normal file
View file

@ -0,0 +1,96 @@
"""OKF bundle navigation (okf.py) — the framework-neutral, D7-portable context seam.
These tests pin the minimal navigation the Step-1 ExpeL wiring depends on: read ``index.md``,
follow intra-bundle cross-links, parse each file's frontmatter, classify by ``type``, and locate
the candidate IR projection. Robustness (OKF SPEC §4): broken links are tolerated, never raised.
No ``agent_framework``/``mcp`` import is allowed in ``okf`` guarded by ``test_okf_is_maf_free``.
"""
from __future__ import annotations
from pathlib import Path
from portfolio_optimiser import okf
BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
def test_navigate_classifies_every_typed_file() -> None:
"""The energi bundle resolves to its six typed OKF files (index + project + hypothesis +
methodology + reference + verdict), each carrying its declared ``type``."""
bundle = okf.navigate_bundle(str(BUNDLE_DIR))
types = {f.name: f.type for f in bundle.files}
assert types == {
"index.md": "index",
"bygg-kontor-nord.md": "project",
"tiltak-led-retrofit.md": "hypothesis",
"metode-ipmvp-a.md": "methodology",
"kilder-realiseringsgap.md": "reference",
"verdict-led-fro.md": "verdict",
}
def test_navigate_exposes_verdicts_and_hypothesis() -> None:
"""The navigation surfaces exactly one ``type: verdict`` file (the ExpeL seed) and the
candidate hypothesis the two the Step-1 wiring keys on."""
bundle = okf.navigate_bundle(str(BUNDLE_DIR))
assert [f.name for f in bundle.verdicts] == ["verdict-led-fro.md"]
assert bundle.verdicts[0].frontmatter["realization_rate"] == "0.82"
assert bundle.hypothesis is not None
assert bundle.hypothesis.name == "tiltak-led-retrofit.md"
def test_navigate_index_summary_is_progressive_disclosure() -> None:
"""``index_summary`` is the index body (the progressive-disclosure entry point), not stuffing
the whole bundle."""
bundle = okf.navigate_bundle(str(BUNDLE_DIR))
assert "progressiv disclosure" in bundle.index_summary.lower()
def test_navigate_tolerates_broken_links(tmp_path) -> None:
"""OKF SPEC §4: a consumer MUST tolerate broken links. An index linking a missing file
navigates without raising, simply omitting the absent target."""
(tmp_path / "index.md").write_text(
"---\ntype: index\n---\n\nSee [gone](missing.md) and [here](real.md).\n",
encoding="utf-8",
)
(tmp_path / "real.md").write_text("---\ntype: project\n---\n\nbody\n", encoding="utf-8")
bundle = okf.navigate_bundle(str(tmp_path))
names = {f.name for f in bundle.files}
assert names == {"index.md", "real.md"} # missing.md silently skipped, no raise
def test_load_ir_projection_returns_candidate_ir() -> None:
"""The bundle's IR projection (``validator-input.json``) is the candidate measure's cost-IR —
the pre-hypothesis ExpeL query key source."""
ir = okf.load_ir_projection(str(BUNDLE_DIR))
assert ir["project_id"] == "BYGG-KONTOR-NORD"
assert [a["code"] for a in ir["affected_items"]] == ["ENERGI-TOTAL-EL"]
assert ir["claimed_saving_nok"] == 30000
def test_parse_frontmatter_reads_scalar_fields() -> None:
"""The minimal frontmatter reader returns the leading ``---`` block as key:value strings."""
fm = okf.parse_frontmatter(BUNDLE_DIR / "verdict-led-fro.md")
assert fm["type"] == "verdict"
assert fm["decision"] == "approved_with_adjustment"
def test_okf_is_maf_free() -> None:
"""D7 portability: ``okf.py`` IMPORTS no ``agent_framework`` / ``mcp`` (the docstring may name
them to document the constraint, exactly as ``retrieval.py`` does) checked via the AST, not a
raw substring, so the prose claim doesn't trip the guard."""
import ast
src = (
Path(__file__).resolve().parents[1] / "src" / "portfolio_optimiser" / "okf.py"
).read_text(encoding="utf-8")
imported: list[str] = []
for node in ast.walk(ast.parse(src)):
if isinstance(node, ast.Import):
imported += [a.name for a in node.names]
elif isinstance(node, ast.ImportFrom):
imported.append(node.module or "")
forbidden = [m for m in imported if m.split(".")[0] in {"agent_framework", "mcp"}]
assert forbidden == [], f"okf.py must not import MAF/mcp, found: {forbidden}"

View file

@ -0,0 +1,93 @@
"""Step-1 load-bearing seam (målbilde §5 / §7): a prior expert verdict MUST reach the next
hypothesis prompt.
The diagnosis (fot-i-bakken, verified in code): ExpeL retrieval was computed AFTER generation and
written into a discarded ``SessionContext`` so the seed verdict could not influence any
hypothesis (``context_providers`` = 0). Fase 2a wires it: before ``generate_via_llm``, the
candidate's prior verdicts are folded into the generation context.
These two tests are a load-bearing PAIR, not a smoke test (per the Fase-2 green-but-dead trap):
- the positive test asserts the seed verdict's realization signal ("0.82") IS in the generation
prompt it goes RED the moment the fold is detached;
- the empty-store test asserts the signal is ABSENT without a seed proving the signal's presence
is CAUSED by the seam, not incidental to the fixture.
"""
from __future__ import annotations
from pathlib import Path
from portfolio_optimiser.run import run_project
from portfolio_optimiser.validator import ValidatedProposal
from portfolio_optimiser.verdicts import VerdictStore, seed_store_from_bundle
BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
# A VALID BYGG-KONTOR-NORD proposal: affected total = 300000 x 1.0; no assumptions -> degenerate
# Monte Carlo P90 = 0.30 x 300000 = 90000 >= claimed 30000 -> validates on the first attempt.
_ENERGY_REPLY = (
'{"measure":"LED-retrofit av kontorbelysning","affected_items":'
'[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":30000}'
)
_VERDICT_INPUT = {"decision": "approved", "rationale": "expert reviewed (sim)"}
# The load-bearing marker is the seed verdict's MINTED id (a 16-char content hash that
# ``ExpeLContextProvider.format_fewshot`` emits as ``- [<id>] ...``). It cannot appear in the
# bundle's raw text, so its presence in the prompt isolates the STRUCTURAL ExpeL path. (The
# realization rate "0.82" is NOT a usable marker here: ``docs_dir == bundle_dir``, so keyword
# chunk-stuffing reads the verdict file and would leak "0.82" regardless of the fold. Layer
# separation — verdicts out of the keyword docs folder — is Fase 2b.)
_SEED_ID = seed_store_from_bundle(str(BUNDLE_DIR)).verdicts[0].id
def _generation_prompts(sink: list[str]) -> list[str]:
"""The generation-call prompts (``generate._build_messages`` embeds 'SavingsProposal'),
isolated from the debate-round prompts also captured in the sink."""
return [p for p in sink if "SavingsProposal" in p]
async def test_prior_verdict_reaches_hypothesis_prompt(make_recording_client_factory) -> None:
"""LOAD-BEARING: with the energi bundle's seed verdict in the store, the realization signal
reaches the hypothesis-generation prompt. Detach the fold in ``run.py`` and this goes red."""
store = seed_store_from_bundle(str(BUNDLE_DIR))
assert store.verdicts, "precondition: the bundle seeds exactly one verdict"
factory, recorded = make_recording_client_factory(_ENERGY_REPLY)
result = await run_project(
"BYGG-KONTOR-NORD",
"local",
docs_dir=str(BUNDLE_DIR),
bundle_dir=str(BUNDLE_DIR),
verdict_input=_VERDICT_INPUT,
store=store,
client_factory=factory,
)
assert isinstance(result.outcome, ValidatedProposal)
gen_prompts = _generation_prompts(recorded)
assert gen_prompts, "the generation call must have happened"
assert any(_SEED_ID in p for p in gen_prompts), (
"the seed verdict did not reach the hypothesis prompt — the ExpeL->prompt fold is detached"
)
async def test_empty_store_leaves_no_signal_in_prompt(make_recording_client_factory) -> None:
"""CAUSALITY CONTROL: the same run with an EMPTY store carries no realization signal into the
prompt proving the signal above is caused by the seam, making the positive test load-bearing
(not merely green against the fixture's mere presence)."""
factory, recorded = make_recording_client_factory(_ENERGY_REPLY)
await run_project(
"BYGG-KONTOR-NORD",
"local",
docs_dir=str(BUNDLE_DIR),
bundle_dir=str(BUNDLE_DIR),
verdict_input=_VERDICT_INPUT,
store=VerdictStore(verdicts=[]),
client_factory=factory,
)
assert all(_SEED_ID not in p for p in recorded), (
"no seed verdict in the store, yet its id appeared in a prompt — "
"the assertion is not actually load-bearing"
)

View file

@ -7,6 +7,8 @@ genuine two-arg ``extend_instructions(source_id, instructions)`` GA signature
Critical Fase 1 risk. Pattern: tests/spikes/test_d_verdictstore.py + real SessionContext.
"""
from pathlib import Path
import pytest
from agent_framework import SessionContext
@ -15,9 +17,13 @@ from portfolio_optimiser.verdicts import (
ProposalFeatures,
Verdict,
VerdictStore,
bundle_candidate_features,
capture_verdict,
seed_store_from_bundle,
)
_BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
_QUERY = ProposalFeatures(
affected_codes=frozenset({"05.2", "03.1"}),
measure_type="scope_reduction",
@ -106,3 +112,36 @@ async def test_before_run_populates_real_sessioncontext_two_arg() -> None:
ctx = SessionContext(input_messages=[], instructions=[])
await provider.before_run(agent=None, session=None, context=ctx, state={})
assert any(true_id in instr for instr in ctx.instructions)
# --- OKF-bundle seeding (Fase 2a): the pre-hypothesis ExpeL query key + the seed store ---
def test_bundle_candidate_features_keys_on_the_ir_projection() -> None:
"""The pre-hypothesis ExpeL query is the candidate measure's cost-IR features (from the
bundle's ``validator-input.json``) — available BEFORE any proposal is generated."""
features = bundle_candidate_features(str(_BUNDLE_DIR))
assert features.affected_codes == frozenset({"ENERGI-TOTAL-EL"})
assert "LED-retrofit" in features.measure_type
assert features.claimed_saving_nok == 30000
def test_seed_store_from_bundle_carries_the_realization_signal() -> None:
"""Each ``type: verdict`` file becomes a structurally-keyed ``Verdict`` whose rationale carries
the learning signal the validator cannot compute (the realization rate 0.82)."""
store = seed_store_from_bundle(str(_BUNDLE_DIR))
assert len(store.verdicts) == 1
seed = store.verdicts[0]
assert seed.proposal_features.affected_codes == frozenset({"ENERGI-TOTAL-EL"})
assert "approved" in seed.decision
assert "0.82" in seed.rationale
def test_seed_store_retrieval_matches_the_candidate() -> None:
"""A3: seed and query derive from the SAME IR -> similarity 1.0 -> the lone seed is retrieved
for the candidate (the structural match the Step-1 wiring relies on)."""
store = seed_store_from_bundle(str(_BUNDLE_DIR))
query = bundle_candidate_features(str(_BUNDLE_DIR))
hits = store.retrieve(query, k=3)
assert len(hits) == 1
assert "0.82" in hits[0].rationale