Kjell Tore Guttormsen
a926e4ad46
S2.1-analog: each completed run persists a run_id-named proposal/outcome pair to the outbox — the system's OWN output layer (the role split §3 Step 7 governs the inbox and wiki, not this). The outcome carries outcome type + figures, the two §9 falsifiers mirrored verbatim from the RunResult, the provenance stamp, and verdict_id — minted the SAME way inbox.py mints an expert verdict's id (mint_verdict_id over the proposal's candidate features), so a later inbox verdict about the same candidate joins by id (the K9 key assumption, pinned here in test and reused there). New outbox.py reuses artifacts' deterministic house JSON writer (sorted keys / indent 2 / LF) — same input + same run_id => byte-identical files — and never touches the S10 artifacts.py fasit formats. run.py grows optional --outbox/--run-id; run_id is REQUIRED when the outbox is set (no wall-clock default — a timestamp would break determinism) and is fail-fasted at the CLI BEFORE any client/spend. A budget stop has no proposal, so it writes no pair. New test_outbox_loadbearing.py (14): unit (pair, verdict_id join key on both outcome types, percentiles-vs-reason, verbatim falsifiers, provenance, round-trip, byte-determinism, run_id fail-fast) + wiring (entrance writes the pair; no-outbox control; --outbox-without-run-id fails fast before spend). Detach-proved: drop the persist_outbox call in execute_run -> wiring test RED. 443->457 green, golden byte-exact (13/13), run_s10.py/runs/ untouched, full gate clean (ruff+format+mypy strict). README synced (count + module + seam). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RiTwaKLesgcwXx2mDviqpt
196 lines
12 KiB
Markdown
196 lines
12 KiB
Markdown
# portfolio-optimiser-claude
|
||
|
||
[](LICENSE)
|
||
[](pyproject.toml)
|
||
[](https://github.com/anthropics/claude-agent-sdk-python)
|
||
|
||
Sibling implementation of the portfolio-optimiser method on the **Claude Agent SDK**
|
||
(decision D7). An open, generic Python framework that finds cost savings *inside* each
|
||
project in a portfolio of independent projects: agents generate candidate measures, a
|
||
mandatory deterministic validator gates the numbers, domain experts judge via
|
||
human-in-the-loop, and the system learns from the verdicts.
|
||
|
||
> **Status:** the D7 build (S5–S10) is complete, and the deterministic **ingest layer**
|
||
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
|
||
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
|
||
> seam, each proven by load-bearing tests (457 tests, all running offline without an API
|
||
> key). The programme's single budgeted **live model run has been executed and validated** —
|
||
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
|
||
|
||
> **Disclaimer — technical framework only.** The deployer owns DPIA, risk assessment, and
|
||
> the legal basis for any processing. The framework ships only the technical
|
||
> preconditions: local-only operation, first-class provenance, no silent data egress.
|
||
|
||
## Built from the spec, not the sibling
|
||
|
||
The method itself is framework-neutral and lives in
|
||
[`portfolio-optimiser-commons`](https://git.fromaitochitta.com/ktg/portfolio-optimiser-commons)
|
||
(consumed here as a git subtree under [`shared/`](shared/)): the normative method spec
|
||
(RFC 2119), the OKF bundle-navigation contract, the golden/conformance suite — the *only*
|
||
oracle for the validator — and the shared expert-reviewer persona skill. This repo
|
||
implements that spec on the Claude Agent SDK; it deliberately does **not**
|
||
reverse-engineer the MAF sibling
|
||
([`open/portfolio-optimiser`](https://git.fromaitochitta.com/open/portfolio-optimiser)).
|
||
Two independent implementations of one spec, compared afterwards, is the point of D7.
|
||
|
||
## Architecture — the seams
|
||
|
||
Everything below the run layer is pure config/file logic and runs deterministically,
|
||
offline. Module by module:
|
||
|
||
**Deterministic backbone** (method-spec §3 step 4, §7–§10)
|
||
- `ir.py` — the typed cost-IR of a candidate measure (§7.1).
|
||
- `validator.py` — the deterministic validator; blocking, and frozen by the shared golden
|
||
suite (§7.2), which is the only fasit it answers to.
|
||
- `provenance.py` — the first-class provenance stamp (§9); authoritative data, not
|
||
after-the-fact logging.
|
||
- `contracts.py` — fail-fast startup contracts (§10): stop criteria and budget caps are
|
||
required at startup, and the model map (`data/model_map.json`, role → Claude model id
|
||
per backend profile) is validated before anything runs.
|
||
|
||
**Context seam** (§3 step 1)
|
||
- `okf.py` — read-context built by **navigating** the project's OKF bundle (`index.md` +
|
||
frontmatter + cross-links, progressive disclosure) — never keyword chunk-stuffing. The
|
||
`type: verdict` layer is excluded from the read-context.
|
||
- `experience.py` — the ExpeL-style experience seam: store, structural retrieval, and the
|
||
gated fold. A prior expert verdict reaches the next hypothesis *only* through the fold,
|
||
never by leaking through context.
|
||
|
||
**Agentic loop** (§3 steps 2–5, §8)
|
||
- `budget.py` — the budget meter: no unbounded loop exists anywhere in the framework.
|
||
- `loop.py` — generate, maker–checker debate, gate, and informed refinement: the
|
||
validator's previous rejection reason is fed into the next bounded attempt, so the
|
||
model corrects against the falsification instead of re-answering identically.
|
||
|
||
**Learning loop** (§3 steps 7–8, §4–§6)
|
||
- `inbox.py` — the async verdict-file contract: an expert drops a plain-JSON verdict into
|
||
an inbox folder after a run; a later run ingests it tolerantly and merges it before the
|
||
fold. The shippable entrance for that later run is `run.py`:
|
||
`uv run python -m portfolio_optimiser_claude.run --bundle <dir> --inbox <dir>`.
|
||
- `promotion.py` — the promotion gate, **fail-closed**: only an approved verdict is
|
||
lifted into the OKF context layer; anything else raises and writes nothing.
|
||
- `persona.py` — the expert-reviewer persona sourced from the shared artifact in
|
||
[`shared/skills/expert-reviewer/`](shared/skills/expert-reviewer/) at call time, so the
|
||
shared persona is genuinely consumed and cannot rot silently.
|
||
|
||
**Value layer** (stack-local contract — mirrored from the sibling plan's capability
|
||
description, never from its code)
|
||
- `ledger.py` — the typed savings ledger: realized savings enter the book **only** through
|
||
the fail-closed expert gate (an approved verdict + a named expert + an explicit
|
||
timestamp), and the sum key is dimension-free, so the same realized saving surfaced via
|
||
two dimensions is never double-counted. Persistence is deterministic JSON.
|
||
- `goals.py` — the goal contract: absolute savings target, hard/soft. A hard goal reached
|
||
raises a structured stop event, never a silent stop; the percent-goal baseline is
|
||
D-E-gated and refused explicitly.
|
||
|
||
**Run layer** (the only part that touches the network)
|
||
- `sdk_client.py` — the Claude Agent SDK client, isolated from local configuration
|
||
(`setting_sources=[]`) so no user/project config can leak into a run.
|
||
- `artifacts.py` — §9 citations plus deterministic run-artifact persistence, including on
|
||
structured stops (a budget stop still leaves artifacts behind).
|
||
- `outbox.py` — the outbox output layer (S2.1): each completed run persists a `run_id`-named
|
||
proposal/outcome pair — the system's own output, which it writes freely (the role split
|
||
governs the inbox and wiki, not this). The outcome carries the inbox join key (`verdict_id`,
|
||
minted the same way the inbox mints an expert verdict's id) so outstanding verdicts can be
|
||
tracked and live artifacts captured; bytes reuse the deterministic house JSON writer, and
|
||
the S10 artifact formats are untouched.
|
||
- `run.py` — the generic run entrance: composes merge-inbox → seed → fold (§5) and drives
|
||
the loop under the budget meter, persisting artifacts on both outcomes — a structured
|
||
budget stop included. The model client is injected, so the offline suite proves the
|
||
same orchestration with a scripted client; only the CLI's default constructs the SDK
|
||
client.
|
||
- `portfolio.py` — the sequential multi-project run and learning loop: `run_portfolio` drives
|
||
N projects from a schema-validated reference config, composing each project's context afresh
|
||
(re-entrant, fresh debate state per run) and collecting one result per project in config
|
||
order. Two things are deliberately shared portfolio-wide: the §8 budget meter (the cap) and a
|
||
single learning `VerdictStore` — a verdict available when project k composes survives into
|
||
project k+1's fold (cross-project threading, §5), and an optional `verdict_dir` is the
|
||
portfolio-level expert inbox the system reads before each fold. The default failure policy
|
||
raises (a stack-local choice until D-D flips it to collect-and-continue).
|
||
- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only.
|
||
|
||
### Load-bearing tests (§11)
|
||
|
||
Every seam is proven by a test that goes **red when the seam is detached** — green-but-dead
|
||
is the failure mode the rule exists for. Among them: `test_step1_expel_loadbearing.py`
|
||
(the verdict signal reaches the prompt via the fold, and only via the fold),
|
||
`test_checker_gate_loadbearing.py` (an explicit checker reject blocks a validated
|
||
proposal), `test_step5_refine_loadbearing.py` (the rejection reason verifiably reaches the
|
||
retry prompt, and the loop still stops at the cap), `test_step7_async_loop_loadbearing.py`
|
||
(a verdict dropped after run A reaches run B's prompt through the file loop, with an
|
||
empty-inbox control), `test_step8_promotion_loadbearing.py` (the gate refuses non-approved
|
||
verdicts; the promoted signal stays out of the read-context),
|
||
`test_portfolio_learning_loadbearing.py` (a verdict available at project k survives into
|
||
project k+1's fold via the shared store, with a marker-absent control),
|
||
`test_outbox_loadbearing.py` (a completed run's `run_id`-named outbox pair is written on the
|
||
entrance path, with a no-outbox control, and the outcome carries the inbox join key), and
|
||
`test_sdk_isolation.py` (local config cannot capture the checker).
|
||
|
||
## The ingest layer — CSV and SQL, in front of the loop
|
||
|
||
The method spec forbids query-time retrieval against the bundle (§3 Step 1), so **data
|
||
reaches the model only via OKF bundles.** The ingest layer is the deterministic step that
|
||
satisfies that: a connector reads a real source, and the extract is materialized as an OKF
|
||
bundle the existing 8-step loop then consumes **unchanged**. It makes **zero model calls**,
|
||
touches no network, and `ingest.py` imports nothing from the SDK — it is pure standard
|
||
library. Built from the shared [`ingest-spec.md`](shared/ingest-spec.md) alone.
|
||
|
||
D7 implements the two conformance-required source types:
|
||
|
||
- **`file`** — a local CSV catalogue; extraction paths are boundary-checked fail-closed
|
||
against the source `root` (the OKF path rule).
|
||
- **`sql`** — a local SQLite database, opened **read-only** (`mode=ro`), one SELECT per
|
||
extraction; the connection location is resolved at run time from a named environment
|
||
variable (`connection_ref`), never stored in the manifest.
|
||
|
||
Both are frozen by byte-identical golden extractions
|
||
([`examples/ingest-golden-file/`](examples/ingest-golden-file/),
|
||
[`examples/ingest-golden-sql/`](examples/ingest-golden-sql/)) and by the load-bearing seam
|
||
tests (provenance stamping, navigability through the unchanged `okf.py`, the reserved
|
||
verdict layer, and re-ingest safety over a promoted verdict).
|
||
|
||
**Honesty rule (§1):** the `http` source type is an **optional extension point, not built in
|
||
D7** — a manifest naming it is rejected fail-fast at validation
|
||
(`test_malformed_manifest_is_rejected`), never silently accepted. The HTTP/MCP extension
|
||
point is demonstrated only in the MAF sibling (against a local mock, behind an opt-in
|
||
network flag); this repo ships no network connector and no live-source integration. How the
|
||
layer works and how one would extend it is documented in [`docs/extending.md`](docs/extending.md).
|
||
|
||
## The live run — S10, executed and validated
|
||
|
||
The loop's closure is proven offline by a scripted two-run simulation
|
||
(`simulation.py`: run A → persona verdict → §6 promotion gate → run B on a fresh store —
|
||
the marker crosses runs via the gate, and never without it). On top of that offline proof
|
||
sits the programme's single budgeted **real** run (D6: exactly one live API run in the
|
||
whole programme), executed 2026-07-03 against the micro bundle
|
||
[`shared/examples/bygg-energi-mikro/`](shared/examples/bygg-energi-mikro/):
|
||
|
||
- exit 0 · validator `validated` · checker `approve` on the first attempt · 2 of 12
|
||
rounds · 36 791 of 150 000 budgeted tokens · **cost $0.127514** (Haiku 4.5, per the
|
||
model map), under a first-class `max_budget_usd` cap.
|
||
- The proposal claimed a deliberately conservative 30 000 NOK saving against the
|
||
bundle's p10–p90 band of 68.5k–121k — and validates.
|
||
- All four artifacts are committed as fixed reference output in [`runs/s10/`](runs/s10/):
|
||
`proposal.json`, `provenance.json` (with §9 citations), `run_result.json`, `usage.json`.
|
||
|
||
Honesty rule (§1): everything else in the repo is deterministic and offline; nothing here
|
||
claims more live behaviour than that one documented run.
|
||
|
||
## Stack
|
||
|
||
Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk/) ≥0.2
|
||
(bundles the Claude Code CLI; an API key is needed only at actual `query()` time) ·
|
||
`uv` · Pydantic for contract validation.
|
||
|
||
## Development
|
||
|
||
```bash
|
||
uv sync # install dependencies
|
||
uv run pytest # 457 tests — run without any API key and without network
|
||
uv run ruff check . && uv run ruff format --check .
|
||
uv run mypy src # strict
|
||
```
|
||
|
||
The offline invariant is deliberate: everything below the run layer is pure config/file
|
||
logic, so the full suite (including every load-bearing seam proof) runs with no key and
|
||
no network.
|