portfolio-optimiser-claude/README.md

portfolio-optimiser-claude

Sibling implementation of the portfolio-optimiser method on the Claude Agent SDK (decision D7). An open, generic Python framework that finds cost savings inside each project in a portfolio of independent projects: agents generate candidate measures, a mandatory deterministic validator gates the numbers, domain experts judge via human-in-the-loop, and the system learns from the verdicts.

Status: the D7 build (S5–S10) is complete, and the deterministic ingest layer (CSV and SQL source types) has since been added in front of the loop. The deterministic backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by seam, each proven by load-bearing tests (562 tests, all running offline without an API key). The programme's single budgeted live model run has been executed and validated — its artifacts are committed under runs/s10/ (see below).

Disclaimer — technical framework only. The deployer owns DPIA, risk assessment, and the legal basis for any processing. The framework ships only the technical preconditions: local-only operation, first-class provenance, no silent data egress.

Built from the spec, not the sibling

The method itself is framework-neutral and lives in portfolio-optimiser-commons (consumed here as a git subtree under shared/): the normative method spec (RFC 2119), the OKF bundle-navigation contract, the golden/conformance suite — the only oracle for the validator — and the shared expert-reviewer persona skill. This repo implements that spec on the Claude Agent SDK; it deliberately does not reverse-engineer the MAF sibling (open/portfolio-optimiser). Two independent implementations of one spec, compared afterwards, is the point of D7.

Architecture — the seams

Everything below the run layer is pure config/file logic and runs deterministically, offline. Module by module:

Deterministic backbone (method-spec §3 step 4, §7–§10)

• ir.py — the typed cost-IR of a candidate measure (§7.1).
• validator.py — the deterministic validator; blocking, and frozen by the shared golden suite (§7.2), which is the only fasit it answers to.
• provenance.py — the first-class provenance stamp (§9); authoritative data, not after-the-fact logging.
• contracts.py — fail-fast startup contracts (§10): stop criteria and budget caps are required at startup, and the model map (data/model_map.json, role → Claude model id per backend profile) is validated before anything runs.

Context seam (§3 step 1)

• okf.py — read-context built by navigating the project's OKF bundle (index.md + frontmatter + cross-links, progressive disclosure) — never keyword chunk-stuffing. The type: verdict layer is excluded from the read-context.
• experience.py — the ExpeL-style experience seam: store, structural retrieval, and the gated fold. A prior expert verdict reaches the next hypothesis only through the fold, never by leaking through context.

Agentic loop (§3 steps 2–5, §8)

• budget.py — the budget meter: no unbounded loop exists anywhere in the framework. On top of the post-charge token/round caps sits an optional pre-call run-total USD belt that refuses the next model call once the run has crossed its USD budget.
• loop.py — generate, maker–checker debate, gate, and informed refinement: the validator's previous rejection reason is fed into the next bounded attempt, so the model corrects against the falsification instead of re-answering identically.

Learning loop (§3 steps 7–8, §4–§6)

• inbox.py — the async verdict-file contract: an expert drops a plain-JSON verdict into an inbox folder after a run; a later run ingests it tolerantly and merges it before the fold. The shippable entrance for that later run is run.py: uv run python -m portfolio_optimiser_claude.run --bundle <dir> --inbox <dir>.
• promotion.py — the promotion gate, fail-closed: only an approved verdict is lifted into the OKF context layer; anything else raises and writes nothing.
• persona.py — the expert-reviewer persona sourced from the shared artifact in shared/skills/expert-reviewer/ at call time, so the shared persona is genuinely consumed and cannot rot silently.

Value layer (stack-local contract — mirrored from the sibling plan's capability description, never from its code)

• ledger.py — the typed savings ledger: realized savings enter the book only through the fail-closed expert gate (an approved verdict + a named expert + an explicit timestamp), and the sum key is dimension-free, so the same realized saving surfaced via two dimensions is never double-counted. Persistence is deterministic JSON.
• goals.py — the goal contract: absolute savings target, hard/soft. A hard goal reached raises a structured stop event, never a silent stop; the percent-goal baseline is D-E-gated and refused explicitly.

Run layer (the only part that touches the network)

• sdk_client.py — the Claude Agent SDK client, isolated from local configuration (setting_sources=[]) so no user/project config can leak into a run.
• artifacts.py — §9 citations plus deterministic run-artifact persistence, including on structured stops (a budget stop still leaves artifacts behind).
• outbox.py — the outbox output layer (S2.1): each completed run persists a run_id-named proposal/outcome pair — the system's own output, which it writes freely (the role split governs the inbox and wiki, not this). The outcome carries the inbox join key (verdict_id, minted the same way the inbox mints an expert verdict's id) so outstanding verdicts can be tracked and live artifacts captured; bytes reuse the deterministic house JSON writer, and the S10 artifact formats are untouched.
• run.py — the generic run entrance: composes merge-inbox → seed → fold (§5) and drives the loop under the budget meter, persisting artifacts on both outcomes — a structured budget stop included. The model client is injected, so the offline suite proves the same orchestration with a scripted client; only the CLI's default constructs the SDK client. --live-dry-run is the live-run drill (K8): it builds everything a real run would (contracts → compose → client construction → preflight) and captures a run_id-named runconfig + preflight pair to the outbox (model-id, parameters, caps — no wall-clock, so the bytes stay deterministic), then stops before the first model call. It exits 0 when the preflight is clear and non-zero when it refused, but captures the artifacts and makes zero model calls either way — a future operator-gated live run is rigged and rehearsed offline, with no spend.
• portfolio.py — the sequential multi-project run and learning loop: run_portfolio drives N projects from a schema-validated reference config, composing each project's context afresh (re-entrant, fresh debate state per run) and collecting one result per project in config order. Two things are deliberately shared portfolio-wide: the §8 budget meter (the cap) and a single learning VerdictStore — a verdict available when project k composes survives into project k+1's fold (cross-project threading, §5), and an optional verdict_dir is the portfolio-level expert inbox the system reads before each fold. The default failure policy raises (a stack-local choice until D-D flips it to collect-and-continue).
• run_s10.py — the programme's ONE live run (cost discipline D6); run-path only.
• costsim.py — pre-run cost simulation (offline — the one Run-layer module that never touches the network): a deterministic UPPER-BOUND USD estimate for a (portfolio-)run before any spend, a what-if over the models in model_map.json × effort levels. Pricing is schema-validated config (data/pricing.example.json): a per-Mtok rate per model, each with a required source + date so a stale rate is visible, never silent. A model configured with no price fails fast — there is no hardcoded rate anywhere (a grep-guard proves it), and the figure is marked ESTIMAT (the whole cap billed at the rate is an upper bound; real runs cost less). uv run python -m portfolio_optimiser_claude.costsim.
• preflight.py — the SDK/API preflight (offline — the boundary the operator crosses before any spend): everything that can be validated without a model call is checked here, so a broken config stops cheaply instead of on the first billed call. Four checks — a credential is present and not a placeholder form (ANTHROPIC_API_KEY, or the bundled CLI's own CLAUDE_CODE_OAUTH_TOKEN; the value itself is never validated online), the model_map profile exists and every id it resolves to is real, claude_agent_sdk imports and its bundled CLI is present on disk, and the §8 stop/budget contract is set. It never calls the API — a green preflight implies no more than that (§1). Each deficiency is a structured, actionable refusal. uv run python -m portfolio_optimiser_claude.preflight.
• hitl.py — the operator's view of the long feedback loop (offline, read-only): which proposals still await an expert verdict, and who should judge each. A pure file-based id-join across the three layers it READS and never writes — the outbox (K5, the persisted verdict_id join key), the inbox (a §4.2-valid verdict settles a proposal), and optionally a bundle's promoted verdicts (§6). Routing maps a proposal's measure (a config-string key now; K13 formalizes the dimension catalog) to an expert via a schema-validated table (nøkkel→ekspert, fail-fast) with an optional default; an unmatched measure is UNROUTED. uv run python -m portfolio_optimiser_claude.hitl pending|route.
• notify.py — deliverable notification sinks that never break the no-silent-egress invariant (§8): console and file deliver locally, webhook is the one transport that leaves the machine and fires ONLY behind an explicit per-run opt-in flag (--allow-webhook-egress) — mirroring the ingest-spec §8 rule that the flag is a run argument, never a config field, so the config cannot grant itself network access. The webhook transport is injected: the suite passes a canned transport (no socket is ever opened), and the real transport lives behind one seam function that the suite never calls (a grep-guard proves no network path exists elsewhere in the module). run.py (on both outcomes — a budget stop notifies too) and hitl.py (read-only preserved) share the same opt-in-gated CLI seam, refusing a webhook-without-opt-in before any spend. The payload shape is stack-local (no shared notification spec across the siblings).
• valuereport.py — what the loop actually delivered (offline, read-only): a deterministic projection of the outbox, the inbox and the ledger into three distinct columns — modelled (what the system claimed), expert-corrected (what the §4.2 verdict makes of that claim), realized (what passed the expert gate into the book) — plus goal progress, a quantified learning effect and cost against value. No model call, no clock, no new state. The honesty rule (§1) sets its shape: a figure the layers do not carry is reported UNMARKED, never back-filled from the stage before it. An approved_with_adjustment verdict changes the amount but carries none in the §4.2 shape, so its corrected value is unquantified — and a project nobody has judged has no realized figure at all, rather than a zero that reads as a judgment. Learning is measured, not asserted: settled proposals split by run_id order into an earlier and a later cohort, and a rising approval share is only reported alongside the modelled→corrected gap that shrank behind it. Cost (USD, itself a K6 upper bound) and value (NOK) are printed side by side and never divided — this repo carries no sourced exchange rate, and a ratio would invent one. Available standalone (uv run python -m portfolio_optimiser_claude.valuereport --outbox <dir> --inbox <dir> [--ledger <file>]) and as an opt-in side product of a run (run.py --value-report <file>, which requires --outbox and is refused before any spend without one; the report is written on both run outcomes and never rewrites the run's own exit code).

Load-bearing tests (§11)

Every seam is proven by a test that goes red when the seam is detached — green-but-dead is the failure mode the rule exists for. Among them: test_step1_expel_loadbearing.py (the verdict signal reaches the prompt via the fold, and only via the fold), test_checker_gate_loadbearing.py (an explicit checker reject blocks a validated proposal), test_step5_refine_loadbearing.py (the rejection reason verifiably reaches the retry prompt, and the loop still stops at the cap), test_step7_async_loop_loadbearing.py (a verdict dropped after run A reaches run B's prompt through the file loop, with an empty-inbox control), test_step8_promotion_loadbearing.py (the gate refuses non-approved verdicts; the promoted signal stays out of the read-context), test_portfolio_learning_loadbearing.py (a verdict available at project k survives into project k+1's fold via the shared store, with a marker-absent control), test_outbox_loadbearing.py (a completed run's run_id-named outbox pair is written on the entrance path, with a no-outbox control, and the outcome carries the inbox join key), test_preflight.py (a missing credential and a placeholder model id are each refused before any spend, and the preflight carries no network path of its own), test_dry_run_loadbearing.py (the live-run drill captures its runconfig + preflight artifacts and stops before the first model call — a call-counting client proves zero calls, red the moment the stop seam is detached), test_hitl_loadbearing.py (a proposal with no verdict is listed pending and disappears once an inbox or promoted verdict shares its id — red the moment the id-join filter is detached — and hitl never writes any layer, proven by a before/after byte snapshot), test_notify_loadbearing.py and test_notify_seam_loadbearing.py (a webhook without the per-run opt-in flag refuses fail-fast and its transport never fires — red the moment the gate is detached — the canned transport receives the structured payload, an AST grep-guard proves no network path lives outside the one injectable seam function, and the run/hitl entrances emit on their outcomes while hitl stays read-only), test_valuereport_loadbearing.py and test_valuereport_seam_loadbearing.py (an unjudged project's realized value stays unmarked and never mirrors the modelled claim — red the moment that boundary is detached — a rising approval share is not reported as learning without the gap arithmetic behind it, the projection writes no byte into the three layers it reads, and the run entrance produces the report on both outcomes while leaving the run's verdict alone), and test_sdk_isolation.py (local config cannot capture the checker).

The ingest layer — CSV and SQL, in front of the loop

The method spec forbids query-time retrieval against the bundle (§3 Step 1), so data reaches the model only via OKF bundles. The ingest layer is the deterministic step that satisfies that: a connector reads a real source, and the extract is materialized as an OKF bundle the existing 8-step loop then consumes unchanged. It makes zero model calls, touches no network, and ingest.py imports nothing from the SDK — it is pure standard library. Built from the shared ingest-spec.md alone.

D7 implements the two conformance-required source types:

• file — a local CSV catalogue; extraction paths are boundary-checked fail-closed against the source root (the OKF path rule).
• sql — a local SQLite database, opened read-only (mode=ro), one SELECT per extraction; the connection location is resolved at run time from a named environment variable (connection_ref), never stored in the manifest.

Both are frozen by byte-identical golden extractions (examples/ingest-golden-file/, examples/ingest-golden-sql/) and by the load-bearing seam tests (provenance stamping, navigability through the unchanged okf.py, the reserved verdict layer, and re-ingest safety over a promoted verdict).

Honesty rule (§1): the http source type is an optional extension point, not built in D7 — a manifest naming it is rejected fail-fast at validation (test_malformed_manifest_is_rejected), never silently accepted. The HTTP/MCP extension point is demonstrated only in the MAF sibling (against a local mock, behind an opt-in network flag); this repo ships no network connector and no live-source integration. How the layer works and how one would extend it is documented in docs/extending.md.

The live run — S10, executed and validated

The loop's closure is proven offline by a scripted two-run simulation (simulation.py: run A → persona verdict → §6 promotion gate → run B on a fresh store — the marker crosses runs via the gate, and never without it). On top of that offline proof sits the programme's single budgeted real run (D6: exactly one live API run in the whole programme), executed 2026-07-03 against the micro bundle shared/examples/bygg-energi-mikro/:

• exit 0 · validator validated · checker approve on the first attempt · 2 of 12 rounds · 36 791 of 150 000 budgeted tokens · cost $0.127514 (Haiku 4.5, per the model map), under a first-class max_budget_usd cap.
• The proposal claimed a deliberately conservative 30 000 NOK saving against the bundle's p10–p90 band of 68.5k–121k — and validates.
• All four artifacts are committed as fixed reference output in runs/s10/: proposal.json, provenance.json (with §9 citations), run_result.json, usage.json.

Honesty rule (§1): everything else in the repo is deterministic and offline; nothing here claims more live behaviour than that one documented run.

Stack

Python ≥3.10 · claude-agent-sdk ≥0.2 (bundles the Claude Code CLI; an API key is needed only at actual query() time) · uv · Pydantic for contract validation.

Development

uv sync          # install dependencies
uv run pytest    # 562 tests — run without any API key and without network
uv run ruff check . && uv run ruff format --check .
uv run mypy src  # strict

The offline invariant is deliberate: everything below the run layer is pure config/file logic, so the full suite (including every load-bearing seam proof) runs with no key and no network.