From b0b87570db5d2770edd49749eaa1c535fbc398c5 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Fri, 3 Jul 2026 14:49:14 +0200 Subject: [PATCH] docs(readme): bring README up to post-S10 state, structured after the MAF sibling's Status, seam-by-seam architecture, load-bearing test map, the executed S10 live run with committed artifacts, and the offline-suite invariant. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01QdSfQdND84oeq2mbjueLTS --- README.md | 120 ++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 107 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 0682946..55d3e61 100644 --- a/README.md +++ b/README.md @@ -3,29 +3,123 @@ Sibling implementation of the portfolio-optimiser method on the **Claude Agent SDK** (decision D7). An open, generic Python framework that finds cost savings *inside* each project in a portfolio of independent projects: agents generate candidate measures, a -mandatory deterministic validator gates the numbers, domain experts judge via HITL, and -the system learns from the verdicts. +mandatory deterministic validator gates the numbers, domain experts judge via +human-in-the-loop, and the system learns from the verdicts. + +> **Status:** the D7 build (S5–S10) is complete. The deterministic backbone, the agentic +> loop, and the learning loop are wired seam by seam, each proven by load-bearing tests +> (187 tests, all running offline without an API key). The programme's single budgeted +> **live model run has been executed and validated** — its artifacts are committed under +> [`runs/s10/`](runs/s10/) (see below). + +> **Disclaimer — technical framework only.** The deployer owns DPIA, risk assessment, and +> the legal basis for any processing. The framework ships only the technical +> preconditions: local-only operation, first-class provenance, no silent data egress. + +## Built from the spec, not the sibling The method itself is framework-neutral and lives in [`portfolio-optimiser-commons`](https://git.fromaitochitta.com/ktg/portfolio-optimiser-commons) -(consumed here as a git subtree under `shared/`): the method spec, the OKF bundle -navigation contract, the golden/conformance suite (the only oracle for the validator), -and the shared expert-reviewer persona skill. This repo implements that spec on the -Claude Agent SDK — it does not reverse-engineer the MAF sibling. +(consumed here as a git subtree under [`shared/`](shared/)): the normative method spec +(RFC 2119), the OKF bundle-navigation contract, the golden/conformance suite — the *only* +oracle for the validator — and the shared expert-reviewer persona skill. This repo +implements that spec on the Claude Agent SDK; it deliberately does **not** +reverse-engineer the MAF sibling +([`open/portfolio-optimiser`](https://git.fromaitochitta.com/open/portfolio-optimiser)). +Two independent implementations of one spec, compared afterwards, is the point of D7. -> **Disclaimer:** this is a purely technical framework. The deployer owns DPIA, risk -> assessment, and the legal basis for any processing. The framework ships only the -> technical preconditions (local-only operation, provenance, no silent egress). +## Architecture — the seams -## Status +Everything below the run layer is pure config/file logic and runs deterministically, +offline. Module by module: -Scaffold (S5): repo, contracts, and shared core in place — no agent runs yet. +**Deterministic backbone** (method-spec §3 step 4, §7–§10) +- `ir.py` — the typed cost-IR of a candidate measure (§7.1). +- `validator.py` — the deterministic validator; blocking, and frozen by the shared golden + suite (§7.2), which is the only fasit it answers to. +- `provenance.py` — the first-class provenance stamp (§9); authoritative data, not + after-the-fact logging. +- `contracts.py` — fail-fast startup contracts (§10): stop criteria and budget caps are + required at startup, and the model map (`data/model_map.json`, role → Claude model id + per backend profile) is validated before anything runs. + +**Context seam** (§3 step 1) +- `okf.py` — read-context built by **navigating** the project's OKF bundle (`index.md` + + frontmatter + cross-links, progressive disclosure) — never keyword chunk-stuffing. The + `type: verdict` layer is excluded from the read-context. +- `experience.py` — the ExpeL-style experience seam: store, structural retrieval, and the + gated fold. A prior expert verdict reaches the next hypothesis *only* through the fold, + never by leaking through context. + +**Agentic loop** (§3 steps 2–5, §8) +- `budget.py` — the budget meter: no unbounded loop exists anywhere in the framework. +- `loop.py` — generate, maker–checker debate, gate, and informed refinement: the + validator's previous rejection reason is fed into the next bounded attempt, so the + model corrects against the falsification instead of re-answering identically. + +**Learning loop** (§3 steps 7–8, §4–§6) +- `inbox.py` — the async verdict-file contract: an expert drops a plain-JSON verdict into + an inbox folder after a run; a later run ingests it tolerantly and merges it before the + fold. +- `promotion.py` — the promotion gate, **fail-closed**: only an approved verdict is + lifted into the OKF context layer; anything else raises and writes nothing. +- `persona.py` — the expert-reviewer persona sourced from the shared artifact in + [`shared/skills/expert-reviewer/`](shared/skills/expert-reviewer/) at call time, so the + shared persona is genuinely consumed and cannot rot silently. + +**Run layer** (the only part that touches the network) +- `sdk_client.py` — the Claude Agent SDK client, isolated from local configuration + (`setting_sources=[]`) so no user/project config can leak into a run. +- `artifacts.py` — §9 citations plus deterministic run-artifact persistence, including on + structured stops (a budget stop still leaves artifacts behind). +- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only. + +### Load-bearing tests (§11) + +Every seam is proven by a test that goes **red when the seam is detached** — green-but-dead +is the failure mode the rule exists for. Among them: `test_step1_expel_loadbearing.py` +(the verdict signal reaches the prompt via the fold, and only via the fold), +`test_checker_gate_loadbearing.py` (an explicit checker reject blocks a validated +proposal), `test_step5_refine_loadbearing.py` (the rejection reason verifiably reaches the +retry prompt, and the loop still stops at the cap), `test_step7_async_loop_loadbearing.py` +(a verdict dropped after run A reaches run B's prompt through the file loop, with an +empty-inbox control), `test_step8_promotion_loadbearing.py` (the gate refuses non-approved +verdicts; the promoted signal stays out of the read-context), and +`test_sdk_isolation.py` (local config cannot capture the checker). + +## The live run — S10, executed and validated + +Where the MAF sibling proves its loop end-to-end with a scripted offline simulation, this +repo's end-to-end proof is the programme's single budgeted **real** run (D6: exactly one +live API run in the whole programme), executed 2026-07-03 against the micro bundle +[`shared/examples/bygg-energi-mikro/`](shared/examples/bygg-energi-mikro/): + +- exit 0 · validator `validated` · checker `approve` on the first attempt · 2 of 12 + rounds · 36 791 of 150 000 budgeted tokens · **cost $0.127514** (Haiku 4.5, per the + model map), under a first-class `max_budget_usd` cap. +- The proposal claimed a deliberately conservative 30 000 NOK saving against the + bundle's p10–p90 band of 68.5k–121k — and validates. +- All four artifacts are committed as fixed reference output in [`runs/s10/`](runs/s10/): + `proposal.json`, `provenance.json` (with §9 citations), `run_result.json`, `usage.json`. + +Honesty rule (§1): everything else in the repo is deterministic and offline; nothing here +claims more live behaviour than that one documented run. + +## Stack + +Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk/) ≥0.2 +(bundles the Claude Code CLI; an API key is needed only at actual `query()` time) · +`uv` · Pydantic for contract validation. ## Development ```bash uv sync # install dependencies -uv run pytest # test suite (runs without any API key) +uv run pytest # 187 tests — run without any API key and without network uv run ruff check . && uv run ruff format --check . -uv run mypy src +uv run mypy src # strict ``` + +The offline invariant is deliberate: everything below the run layer is pure config/file +logic, so the full suite (including every load-bearing seam proof) runs with no key and +no network.