portfolio-optimiser/README.md

portfolio-optimiser

Generic, open framework on Microsoft Agent Framework (MAF) for finding cost-savings / efficiency proposals within each project of a portfolio of independent projects. Multiple agents collaborate to generate candidate proposals; a mandatory deterministic validator (solver + Monte Carlo) decides the numbers; domain experts review via human-in-the-loop, and the system learns from their verdicts.

Status: Early development. The deterministic backbone is solid; the agentic learning loop is being wired one load-bearing seam at a time — Steps 1, 3/4, 5, 7 and 8 are wired (OKF-navigated context, checker gate, informed refinement, the async file feedback loop, and gated wiki promotion; see below). Not yet end-to-end usable.

Disclaimer — technical framework only. This project is a technical framework. Organizations that deploy it are themselves responsible for ensuring a valid processing purpose and for any required assessments (DPIA, risk/ROS, security reviews, etc.). The framework ships technical affordances (local-only mode, provenance/audit logging, no silent data egress) to enable compliant use, but makes no compliance guarantees.

Design philosophy

The result will never fit any single customer 100%. The goal is a ~90% genuinely generic core plus clear extension points, so competent people can configure the last mile per customer. We deliberately do not chase the final 10%.

Agentic loop — wiring status

The mandatory deterministic backbone (validator + budget meter + provenance) is solid and load-bearing. The agentic learning loop (see the target picture §11) is wired one seam at a time:

• Step 1 — OKF context → hypothesis (wired). run_project(..., bundle_dir=...) wires the first agentic seam in two halves:
  • Context by navigation, not stuffing. The agent read-context is built by navigating the project's OKF bundle (index.md + frontmatter + cross-links, progressive disclosure) — never keyword chunk-stuffing (target picture §2/§4).
  • Verdict layer gated out of context. The type: verdict layer is excluded from that context; the candidate's prior expert verdicts reach the hypothesis prompt only through the gated ExpeL fold (folded in before generation), so a prior verdict provably — and exclusively — influences the next hypothesis.
  • Two load-bearing tests fail when the seam is detached: the realization signal must reach the prompt via the fold (tests/test_step1_expel_loadbearing.py), and must never leak via context (tests/test_okf.py::test_bundle_context_excludes_verdict_layer).
• Steps 3/4 — checker gates the reasoning (wired). Two falsifiers now act on the same candidate: the deterministic validator gates the numbers (blocking, as before), and the maker-checker's checker gates the reasoning (target picture §2/§6). The checker ends its turn with a VERDICT: APPROVE / VERDICT: REJECT — <reason> line; an explicit reject blocks an otherwise-validated proposal (run_project surfaces both debate participants via output_from=agents and overrides the outcome to a checker-sourced Rejection). The gate is opt-in-reject (fail-open on a missing marker), and provenance.validator_decision stays honest — it reflects the validator only, never the checker. Load-bearing: tests/test_checker_gate_loadbearing.py goes red on either detach (revert output_from, or drop the override).
• Step 5 — informed refinement (wired). The proposer's bounded retry is no longer blind: the validator's previous Rejection.reason is fed into the next attempt's prompt (generate.py generate_via_llm → _build_messages(prior_rejection=...)), so the model corrects against the falsification instead of re-answering identically (target picture §5/§7). It carries only the most-recent reason (never an accumulated history, never the rejected proposal JSON) and runs under the existing max_attempts + token-meter cap — no new loop, so "improve until good enough" without a ceiling stays impossible. The only per-attempt falsifier here is the validator; seeding generation with the checker's critique is a run-level, separately-scoped concern and is deliberately not done here. Load-bearing: tests/test_step5_refine_loadbearing.py goes red when the reason is detached from the prompt (the outcome never flips and the verbatim-reason assertion fails), with a bounded control proving the loop still stops at max_attempts.
• Step 7 — async file feedback loop (wired). run_project(..., verdict_dir=...) adds the long feedback timescale (target picture §3/§7): an expert/persona drops a verdict file (plain JSON — the raw output layer, §10 R2) into an inbox folder after a run, and a separate, later run ingests it — merged into the store before the Step-1 fold — so a verdict that landed out of band reaches the next hypothesis. The loop is fully resumable across runs separated in time; no live session is assumed. The system reads the folder, the expert/persona writes it (§3 role split), so run_project deliberately does not persist its own captured verdict back (that is the outbox / Step-8 concern). Ingestion is tolerant (a missing folder, foreign or half-written files are skipped, not raised) and merges (never replaces), preserving run_portfolio's cross-project store. Reachable from the CLI via --bundle-dir --verdict-dir. Load-bearing: tests/test_step7_async_loop_loadbearing.py — a verdict dropped after run A must reach run B's prompt (run B uses a fresh store, so the transfer is the file loop, not in-memory carryover), with an empty-inbox control proving causality.
• Step 8 — gated wiki promotion (wired). When an expert/persona approves an outcome, verdicts.promote_verdict lifts it from the raw output layer into the context layer (the OKF bundle) as a type: verdict concept file, navigable by the next run's seed_store_from_bundle (target picture §3/§6/§7). The gate is fail-closed: a verdict whose decision is not an approval raises PromotionRefused and writes/links nothing — only human/persona-approved knowledge enters the wiki, never raw agent output (self-contamination). The promotion is provenance-stamped (who approved / which experiment / when — timestamp is a required keyword, no wall-clock default). The OKF writer lives in okf.py and stays pure stdlib (D7-portable, MAF-free). R4 = optional + gated: promote_verdict is a public opt-in primitive, deliberately not wired into run_project (mirrors write_verdict — the system reads context; the gate/persona promotes). Two honesty limits: the promoted file is minimal (it carries the learning signal only as description/body prose — it does not reproduce the hand-authored seed's structured realization_rate etc.), and because the verdict id is the learning key, two approvals about the same candidate share a filename (last-write-wins, like write_verdict) — the wiki grows one curated file per distinct candidate, not per verdict event. Load-bearing trio (tests/test_step8_promotion_loadbearing.py): the gate refuses a non-approved verdict (red if the gate is removed), the approved verdict is navigable (red if link_in_index is detached), and the promoted signal stays out of bundle_context — reaching a prompt only via the gated ExpeL fold (red if a descriptive index label leaks it into the read-context).

Offline simulation — the end-to-end proof

The loop is proven end to end offline, with no real model. portfolio_optimiser.simulation drives run_project with a scripted synthetic chat client (network-free) across two runs separated by a promotion, and shows the learning loop close: Run A produces a validated, persona-approved verdict carrying a realization marker absent from the bundle; promote_verdict lifts it into the OKF wiki; a re-seed picks it up; Run B's hypothesis prompt then carries the marker (an empty-wiki control on Run A proves causality). Run it:

uv run python -m portfolio_optimiser.simulation

This is a deliberate, cost-driven substitution for a real-model run (target picture §11 step 8): it proves the plumbing, the deterministic spine, and that the learning dataflow closes. It does not prove that a live LLM would produce the proposal or verdict — those are scripted stand-ins for the swarm and the expert persona (honesty per §1). The scripted client is MAF-side scaffolding, not part of the framework-neutral shared/ core. Load-bearing: tests/test_simulation_loadbearing.py goes red the moment promotion is detached (the marker never crosses into Run B).

Shared expert-reviewer persona (§8)

The expert reviewer is a framework-neutral Agent Skill in shared/skills/expert-reviewer/ — a SKILL.md persona prompt (the energy-advisor / M&V role, the realization-gap methodology the validator cannot compute) plus a canonical references/example-verdict.json. It is the shared artifact both reference implementations consume to instantiate the reviewer; shared/ stays pure data, so the MAF side reads it via portfolio_optimiser.persona.load_persona_example and the Claude-SDK sibling reads the same JSON with its own loader. This de-stubs the simulation: its persona judgement (decision + rationale + traced marker) is now sourced from the artifact at call time, not a hardcoded literal — so the shared persona is genuinely consumed and cannot rot silently. The persona's decision is binary (approved / rejected, the feedback contract the run path accepts); the realization correction lives in the rationale prose. Load-bearing trio (tests/test_persona_skill_loadbearing.py): structure + framework-neutrality (red on a framework import), the example is valid pipeline input (red on schema/contract drift), and the simulation's marker follows the artifact file (red the moment the persona is re-inlined).

Docs

• docs/plan/2026-06-26-maalbilde-agentic-loop.md — target picture: the agentic cost-saving loop + OKF knowledge architecture (north star).
• docs/research/2026-06-23-prior-art-platform.md — prior-art & platform research (incl. implementation register §15).
• docs/plan/2026-06-23-incremental-plan.md — incremental delivery plan (deterministic backbone).
• shared/ — framework-neutral shared core (concept + example OKF knowledge bundles + the expert-reviewer persona skill), reused unchanged by both reference implementations.

Stack

Python ≥3.10 · MAF (agent-framework) · uv. Backend profiles: Azure/Foundry (full) + local (fallback).

Develop

uv sync
uv run pytest
uv run ruff check .