portfolio-optimiser-claude/README.md

# portfolio-optimiser-claude

Claude Agent SDK sibling implementation of the portfolio-optimiser method: an open Python framework that finds cost savings inside each project in a portfolio.

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-%E2%89%A53.10-blue.svg)](pyproject.toml)
[![Built on Claude Agent SDK](https://img.shields.io/badge/built%20on-Claude%20Agent%20SDK-D97757.svg)](https://github.com/anthropics/claude-agent-sdk-python)

The projects in the portfolio are independent, and the saving is found *inside* each of
them: agents generate candidate measures, a mandatory deterministic validator gates the
numbers, domain experts judge via human-in-the-loop, and the system learns from the
verdicts. This is the **Claude Agent SDK** implementation of that method (decision D7).

> **Status:** the D7 build (S5–S10) is complete, and the deterministic **ingest layer**
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
> backbone, the agentic loop, the learning loop, the value layer, and the ingest connectors
> are wired seam by seam, each proven by load-bearing tests (603 at the time of writing, all
> running offline without an API key — `uv run pytest` is the source of truth). The
> programme's single budgeted **live model run has been executed and validated** — its
> artifacts are committed under [`runs/s10/`](runs/s10/) (see below).

> **Disclaimer — technical framework only.** The deployer owns DPIA, risk assessment, and
> the legal basis for any processing. The framework ships only the technical
> preconditions: local-only operation, first-class provenance, no silent data egress.

## Install

Not published to a package index — this repo is consumed from source, with `uv`:

```bash
git clone https://git.fromaitochitta.com/open/portfolio-optimiser-claude.git
cd portfolio-optimiser-claude
uv sync
uv run pytest    # the full suite, offline, no API key
```

`uv sync` resolves the one non-PyPI dependency (`llm-ingestion-okf`) from a pinned tag on
the same forge, declared in [`pyproject.toml`](pyproject.toml) — no extra index or
credentials are needed. Nothing runs the suite automatically: this forge has no CI runner,
so that one command from a clean clone is the verification, and there is no CI badge above
pretending otherwise.

## Non-goals

- **Not a portfolio reallocator.** It finds savings *inside* each project; it never moves
  budget between them, and it does not rank projects against each other.
- **Not a compliance product.** The deployer owns DPIA, risk assessment, and the legal
  basis for any processing (see the disclaimer above); the framework ships technical
  preconditions only.
- **No network or live-source connectors.** The `http` source type is a documented
  extension point that is *not* built here — a manifest naming it is rejected fail-fast,
  never silently accepted.
- **No query-time retrieval over the bundle.** Data reaches the model only as a
  pre-materialized OKF bundle (method-spec §3 Step 1); there is no RAG index and no
  keyword chunk-stuffing.
- **Not a port of the MAF sibling.** It is built from the shared spec alone; the two
  implementations exist to be compared, not to converge by copying.
- **Not a continuously running service.** Everything below the run layer is deterministic
  and offline, and the programme budgeted exactly one live model run (S10, below).

## Built from the spec, not the sibling

The method itself is framework-neutral and lives in
[`portfolio-optimiser-commons`](https://git.fromaitochitta.com/ktg/portfolio-optimiser-commons)
(consumed here as a git subtree under [`shared/`](shared/)): the normative method spec
(RFC 2119), the OKF bundle-navigation contract, the golden/conformance suite — the *only*
oracle for the validator — and the shared expert-reviewer persona skill. This repo
implements that spec on the Claude Agent SDK; it deliberately does **not**
reverse-engineer the MAF sibling
([`open/portfolio-optimiser`](https://git.fromaitochitta.com/open/portfolio-optimiser)).
Two independent implementations of one spec, compared afterwards, is the point of D7.

## Architecture — the seams

Everything below the run layer is pure config/file logic and runs deterministically,
offline. Module by module:

**Deterministic backbone** (method-spec §3 step 4, §7–§10)
- `ir.py` — the typed cost-IR of a candidate measure (§7.1).
- `validator.py` — the deterministic validator; blocking, and frozen by the shared golden
  suite (§7.2), which is the only fasit it answers to.
- `provenance.py` — the first-class provenance stamp (§9); authoritative data, not
  after-the-fact logging. It also records **which SDK build produced the run**, taken from
  the producing client and never from the environment: the SDK's reported cost is a
  client-side estimate against a price table frozen at that build, so the figure is only
  checkable later if the run says which build computed it. A run driven by anything else
  (the scripted stand-in, an injected client) reports `null` rather than borrowing the
  installed version it never used.
- `contracts.py` — fail-fast startup contracts (§10): stop criteria and budget caps are
  required at startup, and the model map (`data/model_map.json`, role → Claude model id
  per backend profile) is validated before anything runs.

**Context seam** (§3 step 1)
- `okf.py` — read-context built by **navigating** the project's OKF bundle (`index.md` +
  frontmatter + cross-links, progressive disclosure) — never keyword chunk-stuffing. The
  `type: verdict` layer is excluded from the read-context.
- `experience.py` — the ExpeL-style experience seam: store, structural retrieval, and the
  gated fold. A prior expert verdict reaches the next hypothesis *only* through the fold,
  never by leaking through context.

**Agentic loop** (§3 steps 2–5, §8)
- `budget.py` — the budget meter: no unbounded loop exists anywhere in the framework. On
  top of the post-charge token/round caps sits an optional pre-call run-total USD belt that
  refuses the next model call once the run has crossed its USD budget.
- `loop.py` — generate, maker–checker debate, gate, and informed refinement: the
  validator's previous rejection reason is fed into the next bounded attempt, so the
  model corrects against the falsification instead of re-answering identically.

**Learning loop** (§3 steps 7–8, §4–§6)
- `inbox.py` — the async verdict-file contract: an expert drops a plain-JSON verdict into
  an inbox folder after a run; a later run ingests it tolerantly and merges it before the
  fold. The shippable entrance for that later run is `run.py`:
  `uv run python -m portfolio_optimiser_claude.run --bundle <dir> --inbox <dir>`.
- `promotion.py` — the promotion gate, **fail-closed**: only an approved verdict is
  lifted into the OKF context layer; anything else raises and writes nothing.
- `persona.py` — the expert-reviewer persona sourced from the shared artifact in
  [`shared/skills/expert-reviewer/`](shared/skills/expert-reviewer/) at call time, so the
  shared persona is genuinely consumed and cannot rot silently.

**Value layer** (stack-local contract — mirrored from the sibling plan's capability
description, never from its code)
- `ledger.py` — the typed savings ledger: realized savings enter the book **only** through
  the fail-closed expert gate (an approved verdict + a named expert + an explicit
  timestamp), and the sum key is dimension-free, so the same realized saving surfaced via
  two dimensions is never double-counted. Persistence is deterministic JSON.
- `goals.py` — the goal contract: absolute savings target, hard/soft. A hard goal reached
  raises a structured stop event, never a silent stop; the percent-goal baseline is
  D-E-gated and refused explicitly. On the command line (`run.py --goals <file> --ledger
  <file>`) the goal is checked against the ledger's realized sum **before the first model
  call**: the budget caps bound spend, the goal bounds achievement, so a target already met
  stops the run at exit 4 without buying anything. A soft goal reached is a flag and the run
  continues; an absent `--ledger` is an empty book, so the goal is still evaluated — never
  skipped.

**Run layer** (the only part that touches the network)
- `sdk_client.py` — the Claude Agent SDK client, isolated from local configuration
  (`setting_sources=[]`) so no user/project config can leak into a run.
- `artifacts.py` — §9 citations plus deterministic run-artifact persistence, including on
  structured stops (a budget stop still leaves artifacts behind).
- `outbox.py` — the outbox output layer (S2.1): each completed run persists a `run_id`-named
  proposal/outcome pair — the system's own output, which it writes freely (the role split
  governs the inbox and wiki, not this). The outcome carries the inbox join key (`verdict_id`,
  minted the same way the inbox mints an expert verdict's id) so outstanding verdicts can be
  tracked and live artifacts captured; bytes reuse the deterministic house JSON writer, and
  the S10 artifact formats are untouched.
- `run.py` — the generic run entrance and the operator's collecting surface: composes
  merge-inbox → seed → fold (§5) and drives the loop under the budget meter, persisting
  artifacts on both outcomes — a structured budget stop included. It runs either **one**
  project (`--bundle`) or a **portfolio** (`--portfolio`, with `--verdict-dir` as the
  portfolio-level expert inbox); exactly one of the two is required, and asking for both is
  refused. The model client is injected, so the offline suite proves the
  same orchestration with a scripted client; only the CLI's default constructs the SDK
  client. `--live-dry-run` is the **live-run drill** (K8): it builds everything a real run
  would (contracts → compose → client construction → preflight) and captures a `run_id`-named
  `runconfig` + `preflight` pair to the outbox (model-id, parameters, caps, and the SDK build
  read from the client the drill constructed — no wall-clock, so the bytes stay
  deterministic), then **stops before the first model call**. It exits 0 when
  the preflight is clear and non-zero when it refused, but captures the artifacts and makes
  zero model calls either way — a future operator-gated live run is rigged and rehearsed
  offline, with no spend.
- `portfolio.py` — the sequential multi-project run and learning loop: `run_portfolio` drives
  N projects from a schema-validated reference config, composing each project's context afresh
  (re-entrant, fresh debate state per run) and collecting one result per project in config
  order. Two things are deliberately shared portfolio-wide: the §8 budget meter (the cap) and a
  single learning `VerdictStore` — a verdict available when project k composes survives into
  project k+1's fold (cross-project threading, §5), and an optional `verdict_dir` is the
  portfolio-level expert inbox the system reads before each fold. The default failure policy
  raises (a stack-local choice until D-D flips it to collect-and-continue). Reachable from the
  command line as `run.py --portfolio <file> [--verdict-dir <dir>]`. That path **persists
  nothing** — it returns typed results and prints one line per project, because the outbox
  names its pairs by `run_id` and a portfolio pass has none of its own. Rather than accept
  `--outbox`/`--out`/`--value-report` and quietly ignore them, the entrance refuses them
  there and points at the per-project `--bundle` runs (§1). The refusal is an **allowlist**:
  it names the flags the portfolio pass acts on, so a flag added later and classified nowhere
  is refused rather than silently ignored — fail-closed, not fail-quiet.
- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only.
- `costsim.py` — pre-run cost simulation (**offline** — the one Run-layer module that never
  touches the network): a deterministic UPPER-BOUND USD estimate for a (portfolio-)run
  *before* any spend, a what-if over the models in `model_map.json` × effort levels. Pricing
  is schema-validated config (`data/pricing.example.json`): a per-Mtok rate per model, each
  with a required source + date so a stale rate is visible, never silent. A model configured
  with no price fails fast — there is no hardcoded rate anywhere (a grep-guard proves it), and
  the figure is marked `ESTIMAT` (the whole cap billed at the rate is an upper bound; real runs
  cost less). `uv run python -m portfolio_optimiser_claude.costsim`.
- `preflight.py` — the SDK/API preflight (**offline** — the boundary the operator crosses
  *before* any spend): everything that can be validated without a model call is checked here,
  so a broken config stops cheaply instead of on the first billed call. Four checks — a
  credential is present and not a placeholder form (`ANTHROPIC_API_KEY`, or the bundled CLI's
  own `CLAUDE_CODE_OAUTH_TOKEN`; the value itself is *never* validated online), the model_map
  profile exists and every id it resolves to is real, `claude_agent_sdk` imports and its
  bundled CLI is present on disk, and the §8 stop/budget contract is set. It never calls the
  API — a green preflight implies no more than that (§1). Each deficiency is a structured,
  actionable refusal. `uv run python -m portfolio_optimiser_claude.preflight`.
- `hitl.py` — the operator's view of the long feedback loop (**offline, read-only**): which
  proposals still *await* an expert verdict, and who should judge each. A pure file-based
  id-join across the three layers it READS and never writes — the outbox (K5, the persisted
  `verdict_id` join key), the inbox (a §4.2-valid verdict settles a proposal), and optionally a
  bundle's promoted verdicts (§6). Routing maps a proposal's `measure` (a config-string key
  now; K13 formalizes the dimension catalog) to an expert via a schema-validated table
  (`nøkkel→ekspert`, fail-fast) with an optional default; an unmatched measure is UNROUTED.
  `uv run python -m portfolio_optimiser_claude.hitl pending|route`.
- `notify.py` — deliverable notification sinks that never break the no-silent-egress
  invariant (§8): `console` and `file` deliver locally, `webhook` is the one transport that
  leaves the machine and fires ONLY behind an explicit per-run opt-in flag
  (`--allow-webhook-egress`) — mirroring the ingest-spec §8 rule that *the flag is a run
  argument, never a config field, so the config cannot grant itself network access*. The
  webhook transport is injected: the suite passes a canned transport (no socket is ever
  opened), and the real transport lives behind one seam function that the suite never calls (a
  grep-guard proves no network path exists elsewhere in the module). `run.py` (on both
  outcomes — a budget stop notifies too) and `hitl.py` (read-only preserved) share the same
  opt-in-gated CLI seam, refusing a webhook-without-opt-in *before* any spend. The payload
  shape is stack-local (no shared notification spec across the siblings).
- `valuereport.py` — what the loop actually delivered (**offline, read-only**): a
  deterministic projection of the outbox, the inbox and the ledger into three distinct
  columns — *modelled* (what the system claimed), *expert-corrected* (what the §4.2 verdict
  makes of that claim), *realized* (what passed the expert gate into the book) — plus goal
  progress, a quantified learning effect and cost against value. No model call, no clock, no
  new state. The honesty rule (§1) sets its shape: a figure the layers do not carry is
  reported `UNMARKED`, never back-filled from the stage before it. An `approved_with_adjustment`
  verdict changes the amount but carries none in the §4.2 shape, so its corrected value is
  unquantified — and a project nobody has judged has no realized figure at all, rather than a
  zero that reads as a judgment. Learning is measured, not asserted: settled proposals split
  by `run_id` order into an earlier and a later cohort, and a rising approval share is only
  reported alongside the modelled→corrected gap that shrank behind it. Cost (USD, itself a K6
  upper bound) and value (NOK) are printed side by side and never divided — this repo carries
  no sourced exchange rate, and a ratio would invent one. Available standalone
  (`uv run python -m portfolio_optimiser_claude.valuereport --outbox <dir> --inbox <dir>
  [--ledger <file>]`) and as an opt-in side product of a run (`run.py --value-report <file>`,
  which requires `--outbox` and is refused before any spend without one; the report is written
  on both run outcomes and never rewrites the run's own exit code).

## The operator CLI

Everything the framework does is drivable from the command line, and **every flag listed
here exists in the corresponding `--help`** — a load-bearing test
(`test_cli_paritet_loadbearing.py`) reads this README and goes red the moment it documents a
flag no entrance offers (§1).

```text
# one project, reading an expert inbox, filing a run_id-named outbox pair
uv run python -m portfolio_optimiser_claude.run --bundle <dir> --inbox <dir> \
    --out <dir> --outbox <dir> --run-id <id>

# a portfolio: N projects, one shared budget meter, one shared learning store
uv run python -m portfolio_optimiser_claude.run --portfolio <file> --verdict-dir <dir>

# stop before spending when the book already meets the target (exit 4)
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \
    --goals <file> --ledger <file>

# the live-run drill: build everything, capture artifacts, stop before the first call
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \
    --outbox <dir> --run-id <id> --live-dry-run

# what the loop delivered, projected from the outbox + inbox + ledger (no model call)
uv run python -m portfolio_optimiser_claude.valuereport --outbox <dir> --inbox <dir> \
    --ledger <file> --goal-nok <amount> --json <file>

# who still owes a verdict, and who should judge each proposal
uv run python -m portfolio_optimiser_claude.hitl pending --outbox <dir> --inbox <dir>
uv run python -m portfolio_optimiser_claude.hitl route --outbox <dir> --inbox <dir> \
    --routing <file>

# before any spend: what a run would cost, and whether the rig is configured
uv run python -m portfolio_optimiser_claude.costsim --projects <n> --pricing <file>
uv run python -m portfolio_optimiser_claude.preflight --profile <name>
```

Two run-shape rules the entrance enforces rather than papering over: exactly one of
`--bundle`/`--portfolio` is required, and the portfolio path refuses the flags it could not
honour (`--out`, `--outbox`, `--run-id`, `--value-report`, `--inbox`, `--live-dry-run`)
instead of accepting them as no-ops.

## Setting up a knowledge base

The framework is only as good as the OKF bundles it reads, and building those is a **team
job, not a wizard**: technical people and domain experts working together. The honest
expectation is **1–2 weeks of dedicated work** for a good knowledge base — the quality of
that investment decides the quality of the output. The documented process is
[`docs/oppskrift-kunnskapsbase.md`](docs/oppskrift-kunnskapsbase.md). There is deliberately
no onboarding interview and no guided verdict command; the recipe is the deliverable.

### Load-bearing tests (§11)

Every seam is proven by a test that goes **red when the seam is detached** — green-but-dead
is the failure mode the rule exists for. Among them: `test_step1_expel_loadbearing.py`
(the verdict signal reaches the prompt via the fold, and only via the fold),
`test_checker_gate_loadbearing.py` (an explicit checker reject blocks a validated
proposal), `test_step5_refine_loadbearing.py` (the rejection reason verifiably reaches the
retry prompt, and the loop still stops at the cap), `test_step7_async_loop_loadbearing.py`
(a verdict dropped after run A reaches run B's prompt through the file loop, with an
empty-inbox control), `test_step8_promotion_loadbearing.py` (the gate refuses non-approved
verdicts; the promoted signal stays out of the read-context),
`test_portfolio_learning_loadbearing.py` (a verdict available at project k survives into
project k+1's fold via the shared store, with a marker-absent control),
`test_outbox_loadbearing.py` (a completed run's `run_id`-named outbox pair is written on the
entrance path, with a no-outbox control, and the outcome carries the inbox join key),
`test_preflight.py` (a missing credential and a placeholder model id are each refused before
any spend, and the preflight carries no network path of its own),
`test_dry_run_loadbearing.py` (the live-run drill captures its `runconfig` + `preflight`
artifacts and stops before the first model call — a call-counting client proves zero calls,
red the moment the stop seam is detached),
`test_hitl_loadbearing.py` (a proposal with no verdict is listed pending and disappears once
an inbox or promoted verdict shares its id — red the moment the id-join filter is detached —
and hitl never writes any layer, proven by a before/after byte snapshot),
`test_notify_loadbearing.py` and `test_notify_seam_loadbearing.py` (a webhook without the
per-run opt-in flag refuses fail-fast and its transport never fires — red the moment the gate
is detached — the canned transport receives the structured payload, an AST grep-guard proves
no network path lives outside the one injectable seam function, and the run/hitl entrances
emit on their outcomes while hitl stays read-only),
`test_valuereport_loadbearing.py` and `test_valuereport_seam_loadbearing.py` (an unjudged
project's realized value stays unmarked and never mirrors the modelled claim — red the moment
that boundary is detached — a rising approval share is not reported as learning without the
gap arithmetic behind it, the projection writes no byte into the three layers it reads, and
the run entrance produces the report on both outcomes while leaving the run's verdict alone),
`test_cli_paritet_loadbearing.py` (a hard goal already met by the ledger stops the entrance
before a single model call — red the moment the goal check is unwired — the portfolio config's
projects genuinely run through the CLI, and this README's documented flags are checked against
the actual `--help` output),
`test_provenance_sdk_version_loadbearing.py` (a run not produced by the SDK stamps no build
rather than borrowing the installed one — red the moment the stamp reads the environment
instead of the producing client — and the committed S10 record stays un-back-filled),
and `test_sdk_isolation.py` (local config cannot capture the checker).

## The ingest layer — CSV and SQL, in front of the loop

The method spec forbids query-time retrieval against the bundle (§3 Step 1), so **data
reaches the model only via OKF bundles.** The ingest layer is the deterministic step that
satisfies that: a connector reads a real source, and the extract is materialized as an OKF
bundle the existing 8-step loop then consumes **unchanged**. It makes **zero model calls**,
touches no network, and `ingest.py` imports nothing from the SDK — it is pure standard
library. Built from the shared [`ingest-spec.md`](shared/ingest-spec.md) alone.

D7 implements the two conformance-required source types:

- **`file`** — a local CSV catalogue; extraction paths are boundary-checked fail-closed
  against the source `root` (the OKF path rule).
- **`sql`** — a local SQLite database, opened **read-only** (`mode=ro`), one SELECT per
  extraction; the connection location is resolved at run time from a named environment
  variable (`connection_ref`), never stored in the manifest.

Both are frozen by byte-identical golden extractions
([`examples/ingest-golden-file/`](examples/ingest-golden-file/),
[`examples/ingest-golden-sql/`](examples/ingest-golden-sql/)) and by the load-bearing seam
tests (provenance stamping, navigability through the unchanged `okf.py`, the reserved
verdict layer, and re-ingest safety over a promoted verdict).

**Honesty rule (§1):** the `http` source type is an **optional extension point, not built in
D7** — a manifest naming it is rejected fail-fast at validation
(`test_malformed_manifest_is_rejected`), never silently accepted. The HTTP/MCP extension
point is demonstrated only in the MAF sibling (against a local mock, behind an opt-in
network flag); this repo ships no network connector and no live-source integration. How the
layer works and how one would extend it is documented in [`docs/extending.md`](docs/extending.md).

## The live run — S10, executed and validated

The loop's closure is proven offline by a scripted two-run simulation
(`simulation.py`: run A → persona verdict → §6 promotion gate → run B on a fresh store —
the marker crosses runs via the gate, and never without it). On top of that offline proof
sits the programme's single budgeted **real** run (D6: exactly one live API run in the
whole programme), executed 2026-07-03 against the micro bundle
[`shared/examples/bygg-energi-mikro/`](shared/examples/bygg-energi-mikro/):

- exit 0 · validator `validated` · checker `approve` on the first attempt · 2 of 12
  rounds · 36 791 of 150 000 budgeted tokens · **cost $0.127514** (Haiku 4.5, per the
  model map), under a first-class `max_budget_usd` cap.
- The proposal claimed a deliberately conservative 30 000 NOK saving against the
  bundle's p10–p90 band of 68.5k–121k — and validates.
- All four artifacts are committed as fixed reference output in [`runs/s10/`](runs/s10/):
  `proposal.json`, `provenance.json` (with §9 citations), `run_result.json`, `usage.json`.
  They are the record of that run as it happened and are never edited afterwards — the
  provenance stamp's later `sdk_version` field is absent there because the run predates it,
  and back-filling a build id would be a guess presented as provenance. A load-bearing test
  keeps that record frozen.

Honesty rule (§1): everything else in the repo is deterministic and offline; nothing here
claims more live behaviour than that one documented run.

## Stack

Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk/) ≥0.2
(bundles the Claude Code CLI; an API key is needed only at actual `query()` time) ·
`uv` · Pydantic for contract validation.

## Development

```bash
uv sync          # install dependencies
uv run pytest    # the full suite — runs without any API key and without network
uv run ruff check . && uv run ruff format --check .
uv run mypy src  # strict
```

The offline invariant is deliberate: everything below the run layer is pure config/file
logic, so the full suite (including every load-bearing seam proof) runs with no key and
no network.

## Changelog

See [CHANGELOG.md](CHANGELOG.md).