diff --git a/CLAUDE.md b/CLAUDE.md index 1ee1ce0..243af49 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -88,6 +88,18 @@ Python ≥3.10. MAF (`agent-framework-core` 1.9.0). Pakkehåndtering: `uv`. To b dom er navigerbar (RØD når `link_in_index` detaches); promotert signal holdes ute av `bundle_context` (RØD når en beskrivende index-label lekker det inn). Index-RMW er ikke-atomisk (enprosess-MVP). - **Kostnadsdisiplin:** utvikle primært på lokal profil (gratis); Foundry/Azure (privat tenant finnes) kun til målrettet, minimal verifisering; billigste modeller + små syntetiske data + harde token-tak. Ingen tunge test-kjøringer. +- **Offline simulering = primært metode-bevis (kostnadsdrevet, erstatter §11.8):** operatøren kjører + IKKE MAF mot ekte modell (verken Azure/Foundry eller Ollama — API for begge repoene er for kostbart + privat). `portfolio_optimiser.simulation` driver `run_project` med en SKRIPTET syntetisk chat-klient + (`ScriptedChatClient` på `OpenAIChatCompletionClient` — IKKE bare `BaseChatClient`, ellers no-op-er + `BudgetMiddleware`) over to kjøringer adskilt av en promotering, og viser at læringssløyfa lukkes: + Run A's godkjente persona-dom (markør fraværende fra bundelen) → `promote_verdict` → re-seed → Run B's + hypotese-prompt bærer markøren (tom-wiki-kontroll på Run A beviser kausalitet). **Ærlighet (§1, + ufravikelig):** beviser plumbing + deterministisk ryggrad + at dataflyten lukkes — IKKE at en levende + LLM ville produsert forslaget/dommen (skriptede stand-ins). Den genuine modell-atferd-sammenligningen + lever på Claude-SDK-siden (minimal API-kjøring). Skriptet klient = MAF-side stillas, IKKE delt + (`shared/` forblir framework-nøytralt). Kjøres `uv run python -m portfolio_optimiser.simulation`. + Load-bearing: `tests/test_simulation_loadbearing.py` blir RØD når promoteringen detaches. - **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet. - Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase. diff --git a/README.md b/README.md index 267bf6e..c205c87 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,26 @@ The mandatory deterministic backbone (validator + budget meter + provenance) is - **Step 7 — async file feedback loop (wired).** `run_project(..., verdict_dir=...)` adds the long feedback timescale (target picture §3/§7): an expert/persona drops a verdict file (plain JSON — the raw output layer, §10 R2) into an inbox folder *after* a run, and a separate, later run ingests it — merged into the store *before* the Step-1 fold — so a verdict that landed out of band reaches the next hypothesis. The loop is fully resumable across runs separated in time; no live session is assumed. The system *reads* the folder, the expert/persona *writes* it (§3 role split), so `run_project` deliberately does **not** persist its own captured verdict back (that is the outbox / Step-8 concern). Ingestion is tolerant (a missing folder, foreign or half-written files are skipped, not raised) and **merges** (never replaces), preserving `run_portfolio`'s cross-project store. Reachable from the CLI via `--bundle-dir --verdict-dir`. Load-bearing: `tests/test_step7_async_loop_loadbearing.py` — a verdict dropped after run A must reach run B's prompt (run B uses a *fresh* store, so the transfer is the file loop, not in-memory carryover), with an empty-inbox control proving causality. - **Step 8 — gated wiki promotion (wired).** When an expert/persona **approves** an outcome, `verdicts.promote_verdict` lifts it from the raw output layer into the context layer (the OKF bundle) as a `type: verdict` concept file, navigable by the next run's `seed_store_from_bundle` (target picture §3/§6/§7). The **gate** is fail-closed: a verdict whose decision is not an approval raises `PromotionRefused` and writes/links nothing — only human/persona-approved knowledge enters the wiki, never raw agent output (self-contamination). The promotion is provenance-stamped (who approved / which experiment / when — `timestamp` is a required keyword, no wall-clock default). The OKF writer lives in `okf.py` and stays pure stdlib (D7-portable, MAF-free). **R4 = optional + gated:** `promote_verdict` is a public opt-in primitive, deliberately **not** wired into `run_project` (mirrors `write_verdict` — the system reads context; the gate/persona promotes). Two honesty limits: the promoted file is *minimal* (it carries the learning signal only as `description`/body prose — it does not reproduce the hand-authored seed's structured `realization_rate` etc.), and because the verdict id is the learning key, two approvals about the same candidate share a filename (last-write-wins, like `write_verdict`) — the wiki grows one curated file per distinct candidate, not per verdict event. Load-bearing trio (`tests/test_step8_promotion_loadbearing.py`): the gate refuses a non-approved verdict (red if the gate is removed), the approved verdict is navigable (red if `link_in_index` is detached), and the promoted signal stays out of `bundle_context` — reaching a prompt only via the gated ExpeL fold (red if a descriptive index label leaks it into the read-context). +## Offline simulation — the end-to-end proof + +The loop is proven end to end **offline**, with no real model. `portfolio_optimiser.simulation` +drives `run_project` with a **scripted** synthetic chat client (network-free) across two runs +separated by a promotion, and shows the learning loop close: Run A produces a validated, +persona-approved verdict carrying a realization marker absent from the bundle; `promote_verdict` +lifts it into the OKF wiki; a re-seed picks it up; **Run B's hypothesis prompt then carries the +marker** (an empty-wiki control on Run A proves causality). Run it: + +``` +uv run python -m portfolio_optimiser.simulation +``` + +This is a deliberate, cost-driven substitution for a real-model run (target picture §11 step 8): +it proves the plumbing, the deterministic spine, and that the **learning dataflow closes**. It does +**not** prove that a live LLM would *produce* the proposal or verdict — those are scripted stand-ins +for the swarm and the expert persona (honesty per §1). The scripted client is MAF-side scaffolding, +not part of the framework-neutral `shared/` core. Load-bearing: `tests/test_simulation_loadbearing.py` +goes red the moment promotion is detached (the marker never crosses into Run B). + ## Docs - [`docs/plan/2026-06-26-maalbilde-agentic-loop.md`](docs/plan/2026-06-26-maalbilde-agentic-loop.md) — target picture: the agentic cost-saving loop + OKF knowledge architecture (north star). diff --git a/src/portfolio_optimiser/simulation.py b/src/portfolio_optimiser/simulation.py new file mode 100644 index 0000000..de50e9d --- /dev/null +++ b/src/portfolio_optimiser/simulation.py @@ -0,0 +1,280 @@ +"""Offline simulation of the full agentic loop — the end-to-end method proof (replaces målbilde +§11.8's real-model run). + +**Operator decision (pragmatic, cost-driven):** MAF is NOT run against a real model — neither +Azure/Foundry nor local Ollama — because paying for API runs across both repos (MAF + the +Claude-SDK sibling) is too costly privately. This module is the primary proof instead: it drives +``run_project`` with a **scripted** synthetic chat client (no network, no model) and demonstrates +that the loop's dataflow closes end to end across two runs separated by a promotion: + + context -> hypothesis -> maker/checker debate -> deterministic validator -> persona verdict + -> PROMOTION into the OKF wiki -> the next run's hypothesis is informed by it. + +**What this proves:** the plumbing, the deterministic spine, and that the learning loop closes — +a verdict approved in Run A reaches Run B's hypothesis prompt purely through the file-backed wiki. +**What it does NOT prove (honesty, målbilde §1):** that a live LLM would *produce* the proposal or +the verdict unprompted — those are scripted stand-ins for the swarm and the expert persona. The +genuine model-behaviour comparison lives on the Claude-SDK side (a minimal API run). The scripted +client is MAF-side scaffolding; it is NOT part of the framework-neutral ``shared/`` core. +""" + +from __future__ import annotations + +import shutil +from collections.abc import Awaitable, Callable, Mapping, Sequence +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from agent_framework import ( + BaseChatClient, + ChatResponse, + ChatResponseUpdate, + Message, + ResponseStream, + UsageDetails, +) +from agent_framework_openai import OpenAIChatCompletionClient + +from portfolio_optimiser.run import RunResult, run_project +from portfolio_optimiser.validator import ValidatedProposal +from portfolio_optimiser.verdicts import VerdictStore, promote_verdict, seed_store_from_bundle + +_BUNDLE_DIR = Path(__file__).resolve().parents[2] / "shared" / "examples" / "bygg-energi-mikro" +_PROJECT_ID = "BYGG-KONTOR-NORD" + +# A VALID SavingsProposal for BYGG-KONTOR-NORD: total = 300000 x 1.0, P90 = 0.30 x 300000 = 90000, +# claimed 30000 <= 90000 -> validates on the first attempt (no `assumptions` -> degenerate MC). +_VALID_PROPOSAL = ( + '{"measure":"LED-retrofit av kontorbelysning","affected_items":' + '[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":30000}' +) +# The checker's debate turn ends with the gate marker the run parses (run._checker_verdict). +_CHECKER_APPROVE = "Tallene er innenfor feasibelt område og resonnementet holder. VERDICT: APPROVE" + +# The persona's verdict: an APPROVE that carries NEW realization knowledge the validator cannot +# compute. The marker (a realization rate ABSENT from the bundle — the seed is 0.82) is the payload +# we trace from Run A's persona judgement, through promotion, into Run B's hypothesis prompt. +_DEFAULT_MARKER = "realiseringsgrad=0.79" +_DEFAULT_PERSONA_RATIONALE = ( + "Godkjent. I drift realiseres erfaringsvis ~79% av en timeplan-stipulert LED-besparelse i " + f"kontorbygg ({_DEFAULT_MARKER}) pga overestimerte driftstimer; forventet faktisk ca 23700 NOK/aar." +) + + +class ScriptedChatClient(OpenAIChatCompletionClient): + """A network-free, SCRIPTED chat client: it returns a fixed ``reply`` and records every prompt + it receives into a shared ``sink`` (the observation probe). It is a stand-in for a real model — + it proves the loop's plumbing, NOT model behaviour. + + Subclasses the LAYERED ``OpenAIChatCompletionClient`` (not the minimal ``BaseChatClient``) so the + always-attached ``BudgetMiddleware`` is not silently no-op'd (verified). Construction is offline + (loopback ``base_url`` + dummy key); ``_inner_get_response`` intercepts before any HTTP.""" + + OTEL_PROVIDER_NAME = "synthetic" + + def __init__(self, reply: str, sink: list[str], *, tokens_per_reply: int = 8) -> None: + super().__init__(model="synthetic", api_key="synthetic", base_url="http://127.0.0.1:9/v1") + self._reply = reply + self._sink = sink + self._tokens = tokens_per_reply + + def _inner_get_response( + self, + *, + messages: Sequence[Message], + options: Mapping[str, Any], + stream: bool = False, + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + self._sink.append(" ".join(getattr(m, "text", "") or "" for m in messages)) + usage = UsageDetails(total_token_count=self._tokens) + if stream: + + async def _agen() -> Any: + # The framework accepts a {"type": "text", ...} dict here (its types under-specify it). + yield ChatResponseUpdate( + role="assistant", + contents=[{"type": "text", "text": self._reply}], # type: ignore[list-item] + ) + + return self._build_response_stream(_agen()) + + async def _coro() -> ChatResponse: + return ChatResponse( + messages=[Message(role="assistant", contents=[self._reply])], + response_id="synthetic", + usage_details=usage, + ) + + return _coro() + + +def scripted_factory(replies: dict[str, str], sink: list[str]) -> Callable[[str], BaseChatClient]: + """A role-keyed client factory: ``factory("proposer")`` and ``factory("checker")`` each return a + fresh ``ScriptedChatClient`` with that role's reply, all sharing ONE ``sink``. MAF stamps the + proposer/checker identity from the agent name, so role-keyed stateless replies suffice (no + per-turn counter); the shared ``sink`` spans the debate turns and the generation call.""" + + def factory(role: str) -> BaseChatClient: + return ScriptedChatClient(replies[role], sink) + + return factory + + +@dataclass(frozen=True) +class LearningSimulationResult: + """The trace of one two-run learning simulation. ``marker_in_run_b_prompt`` true while + ``marker_in_run_a_prompt`` false is the closed loop: the persona knowledge approved in Run A + reached Run B's hypothesis only via promotion into the wiki.""" + + run_a: RunResult + run_b: RunResult + promoted_path: Path + marker: str + marker_in_run_a_prompt: bool + marker_in_run_b_prompt: bool + run_a_generation_prompts: list[str] + run_b_generation_prompts: list[str] + + +def _generation_prompts(sink: list[str]) -> list[str]: + """The generation-call prompts (``generate._build_messages`` embeds 'SavingsProposal'), isolated + from the debate-round prompts also captured in the shared sink.""" + return [p for p in sink if "SavingsProposal" in p] + + +async def simulate_learning_loop( + bundle_dir: str, + work_dir: str, + *, + persona_rationale: str = _DEFAULT_PERSONA_RATIONALE, + marker: str = _DEFAULT_MARKER, + timestamp: str = "2026-06-30", + max_rounds: int = 3, +) -> LearningSimulationResult: + """Run the loop twice on a throwaway COPY of the bundle (the shared fixture is never mutated), + with a promotion in between, and trace whether the persona's approved knowledge crosses runs. + + Run A: a fresh (empty) wiki -> an uninformed hypothesis; the persona approves with NEW realization + knowledge (``marker`` in ``persona_rationale``). ``promote_verdict`` lifts that verdict into the + wiki; ``seed_store_from_bundle`` re-reads the wiki; Run B's Step-1 ExpeL fold then carries the + marker into its hypothesis prompt. The two runs use SEPARATE sinks so each prompt set is + inspected independently.""" + if marker not in persona_rationale: + raise ValueError("marker must be a substring of persona_rationale (the carried payload)") + + copy = Path(work_dir) / "bundle" + shutil.copytree(bundle_dir, copy) + copy_s = str(copy) + replies = {"proposer": _VALID_PROPOSAL, "checker": _CHECKER_APPROVE} + verdict_input = {"decision": "approved", "rationale": persona_rationale} + + # Run A — empty wiki isolates the persona's NEW knowledge. + sink_a: list[str] = [] + run_a = await run_project( + _PROJECT_ID, + "local", + docs_dir=copy_s, + bundle_dir=copy_s, + verdict_input=verdict_input, + store=VerdictStore(verdicts=[]), + client_factory=scripted_factory(replies, sink_a), + max_rounds=max_rounds, + ) + + # Gate-promote the persona verdict from the raw output layer into the OKF wiki (Steg 8). + promoted_path = promote_verdict( + copy_s, + run_a.verdict, + approver="ekspert-persona (sim)", + experiment="sim-run-A", + timestamp=timestamp, + ) + + # Re-seed the wiki: the promoted verdict is now navigable and folds into the next run. + store_b = seed_store_from_bundle(copy_s) + + # Run B — a separate, later run reads the updated wiki. + sink_b: list[str] = [] + run_b = await run_project( + _PROJECT_ID, + "local", + docs_dir=copy_s, + bundle_dir=copy_s, + verdict_input=verdict_input, + store=store_b, + client_factory=scripted_factory(replies, sink_b), + max_rounds=max_rounds, + ) + + gen_a = _generation_prompts(sink_a) + gen_b = _generation_prompts(sink_b) + return LearningSimulationResult( + run_a=run_a, + run_b=run_b, + promoted_path=promoted_path, + marker=marker, + marker_in_run_a_prompt=any(marker in p for p in gen_a), + marker_in_run_b_prompt=any(marker in p for p in gen_b), + run_a_generation_prompts=gen_a, + run_b_generation_prompts=gen_b, + ) + + +def _outcome_line(result: RunResult) -> str: + o = result.outcome + if isinstance(o, ValidatedProposal): + return ( + f"VALIDATED (claimed {o.proposal.claimed_saving_nok:.0f} <= P90 {o.p90:.0f} NOK; " + f"measure: {o.proposal.measure})" + ) + return f"REJECTED ({o.reason})" + + +def main(argv: list[str] | None = None) -> int: # pragma: no cover - console trace + """Run the simulation against the energi bundle in a throwaway temp dir and print an honest, + readable trace. Invoke: ``uv run python -m portfolio_optimiser.simulation``.""" + import asyncio + import tempfile + + work = tempfile.mkdtemp(prefix="po-sim-") + result = asyncio.run(simulate_learning_loop(str(_BUNDLE_DIR), work)) + + print("=" * 78) + print("OFFLINE SIMULATION — scripted agent replies, NO real model.") + print("Proves the loop's dataflow + deterministic spine + that the learning loop closes.") + print("Does NOT prove a live LLM would produce these — proposal/verdict are scripted.") + print("=" * 78) + + print("\nRUN A (fresh wiki — no prior verdicts)") + print(f" validator : {_outcome_line(result.run_a)}") + print(f" checker : VERDICT={result.run_a.checker_verdict.upper()}") + print(f" persona : {result.run_a.verdict.decision} -> {result.run_a.verdict.rationale}") + print( + f" prompt has marker '{result.marker}': {result.marker_in_run_a_prompt} (expected False)" + ) + + print("\nPROMOTE (gated wiki-promotion, Steg 8)") + print(f" wrote : {result.promoted_path.name} (linked into index.md, neutral label)") + + print("\nRUN B (re-seeded wiki — reads the promoted verdict)") + print(f" validator : {_outcome_line(result.run_b)}") + print( + f" prompt has marker '{result.marker}': {result.marker_in_run_b_prompt} (expected True)" + ) + + closed = result.marker_in_run_b_prompt and not result.marker_in_run_a_prompt + print("\n" + "-" * 78) + if closed: + print("LEARNING LOOP CLOSED: the persona knowledge approved in Run A reached Run B's") + print("hypothesis purely via the file-backed OKF wiki (promote -> re-seed -> ExpeL fold).") + else: + print("LEARNING LOOP NOT CLOSED — the marker did not cross runs as expected.") + print("-" * 78) + print(f"\n(working copy: {work})") + return 0 if closed else 1 + + +if __name__ == "__main__": # pragma: no cover - console entry + raise SystemExit(main()) diff --git a/tests/test_simulation_loadbearing.py b/tests/test_simulation_loadbearing.py new file mode 100644 index 0000000..dbc617c --- /dev/null +++ b/tests/test_simulation_loadbearing.py @@ -0,0 +1,53 @@ +"""Offline simulation — the end-to-end method proof (replaces målbilde §11.8's real-model run). + +Operator decision: MAF is NOT run against a real model (Azure/Foundry or local Ollama) — API for +both repos is too costly privately. Instead this offline simulation, driven by SCRIPTED synthetic +agent replies, is the primary proof that the agentic loop's dataflow closes end to end: +context -> hypothesis -> maker/checker debate -> validator -> verdict -> PROMOTION -> next run's +hypothesis. It proves the plumbing + the deterministic spine + that the learning loop closes; it +does NOT prove a live LLM would produce the proposal (that is scripted) — honesty per målbilde §1. + +This load-bearing test runs the actual two-run cycle: Run A (fresh wiki) produces a validated, +persona-approved verdict carrying a realization marker absent from the bundle; `promote_verdict` +lifts it into the OKF wiki; a re-seed picks it up; Run B's hypothesis prompt then carries the +marker. The empty-store control (Run A carries no marker) proves causality — the signal reaches +Run B only via promotion. RED the moment promotion is detached. +""" + +from __future__ import annotations + +from pathlib import Path + +from portfolio_optimiser.simulation import simulate_learning_loop +from portfolio_optimiser.validator import ValidatedProposal + +BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro" + + +async def test_simulation_closes_the_learning_loop(tmp_path) -> None: + """LOAD-BEARING: a persona verdict approved in Run A reaches Run B's hypothesis prompt purely + via the file-backed OKF wiki (promote -> re-seed -> ExpeL fold). Goes RED if ``promote_verdict`` + is detached from ``simulate_learning_loop`` (Run B's store then lacks the marker).""" + result = await simulate_learning_loop(str(BUNDLE_DIR), str(tmp_path)) + + # The loop ran end to end on both runs (scripted proposal validates: P90=90000 >= 30000). + assert isinstance(result.run_a.outcome, ValidatedProposal) + assert isinstance(result.run_b.outcome, ValidatedProposal) + # A1: the synthetic client is layered, so the always-attached BudgetMiddleware metered real-shaped + # token usage (a bare BaseChatClient would silently no-op). + assert result.run_a.provenance.token_usage > 0 + + # The promoted verdict landed in the wiki. + assert result.promoted_path.exists() + assert result.promoted_path.name.startswith("promoted-verdict-") + + # Causality control: Run A (empty wiki) carries no marker into its hypothesis prompt. + assert not result.marker_in_run_a_prompt, ( + "Run A carried the marker with an empty wiki — the positive result would not be caused by " + "promotion" + ) + # The learning loop closed: the promoted persona knowledge reached Run B's hypothesis. + assert result.marker_in_run_b_prompt, ( + "the persona verdict approved in Run A did not reach Run B's hypothesis prompt — the " + "promote -> re-seed -> fold learning loop is not closed" + )