feat(portfolio): K12 — CLI parity, doc sync, knowledge-base recipe (parity row 24) [skip-docs]

The last ungated build session: the operator now drives the whole build from the
command line, and the documents claim exactly what the code does (§1).

run.py becomes the collecting entrance. Exactly one of --bundle (one project) or
--portfolio (N projects from a schema-validated reference config, with
--verdict-dir as the portfolio-level expert inbox) is required; both and neither
are refused. --goals loads a goal contract and checks it against --ledger's
realized sum BEFORE the first model call: the §8 caps bound spend, the goal bounds
achievement, so a hard target the book already meets stops the run at exit 4
without constructing a client. A soft target reached is a flag and the run
continues; an absent ledger is an empty book, so the goal is still evaluated,
never skipped. The one declared goal also drives --value-report's goal progress —
one contract, never two figures that can disagree.

The portfolio path persists nothing (K3 returns typed results; the outbox names
pairs by run_id, which a portfolio pass has none of). Rather than accept
--out/--outbox/--run-id/--value-report/--inbox/--live-dry-run there and silently
ignore them, the entrance refuses them and says why. run_portfolio is imported
lazily — portfolio.py imports this module, so a module-level import is circular.

Three seams, each detach-proven RED:
- unwire the goal check → the run proceeds and spends → red
- unwire the portfolio branch → the configured projects never run → red
- document a flag no CLI offers → the README honesty grep goes red

That last one is the doc-sync made load-bearing: the test reads README.md,
collects every --flag it documents (excluding third-party dev-tooling lines) and
asserts each exists in the --help of a CLI the README names. The drift it exists
to close was real — README claimed 562 tests, CHANGELOG claimed 265, actual 597.

Docs synced to the code: README gains an operator-CLI section and honest goal/
portfolio descriptions, CHANGELOG is rewritten to what actually shipped, and
docs/oppskrift-kunnskapsbase.md delivers D-H point 1 — the documented team
process for building a knowledge base, with the honest 1–2 week expectation and
every factory-dependent step (verdict translation, demo path) marked NOT BUILT.

597 passed · ruff clean · mypy strict clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 06:42:52 +02:00
commit da93a68ce7
6 changed files with 956 additions and 28 deletions

View file

@ -12,10 +12,11 @@ human-in-the-loop, and the system learns from the verdicts.
> **Status:** the D7 build (S5S10) is complete, and the deterministic **ingest layer**
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
> seam, each proven by load-bearing tests (562 tests, all running offline without an API
> key). The programme's single budgeted **live model run has been executed and validated**
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
> backbone, the agentic loop, the learning loop, the value layer, and the ingest connectors
> are wired seam by seam, each proven by load-bearing tests (597 at the time of writing, all
> running offline without an API key — `uv run pytest` is the source of truth). The
> programme's single budgeted **live model run has been executed and validated** — its
> artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
> **Disclaimer — technical framework only.** The deployer owns DPIA, risk assessment, and
> the legal basis for any processing. The framework ships only the technical
@ -83,7 +84,12 @@ description, never from its code)
two dimensions is never double-counted. Persistence is deterministic JSON.
- `goals.py` — the goal contract: absolute savings target, hard/soft. A hard goal reached
raises a structured stop event, never a silent stop; the percent-goal baseline is
D-E-gated and refused explicitly.
D-E-gated and refused explicitly. On the command line (`run.py --goals <file> --ledger
<file>`) the goal is checked against the ledger's realized sum **before the first model
call**: the budget caps bound spend, the goal bounds achievement, so a target already met
stops the run at exit 4 without buying anything. A soft goal reached is a flag and the run
continues; an absent `--ledger` is an empty book, so the goal is still evaluated — never
skipped.
**Run layer** (the only part that touches the network)
- `sdk_client.py` — the Claude Agent SDK client, isolated from local configuration
@ -96,9 +102,12 @@ description, never from its code)
minted the same way the inbox mints an expert verdict's id) so outstanding verdicts can be
tracked and live artifacts captured; bytes reuse the deterministic house JSON writer, and
the S10 artifact formats are untouched.
- `run.py` — the generic run entrance: composes merge-inbox → seed → fold (§5) and drives
the loop under the budget meter, persisting artifacts on both outcomes — a structured
budget stop included. The model client is injected, so the offline suite proves the
- `run.py` — the generic run entrance and the operator's collecting surface: composes
merge-inbox → seed → fold (§5) and drives the loop under the budget meter, persisting
artifacts on both outcomes — a structured budget stop included. It runs either **one**
project (`--bundle`) or a **portfolio** (`--portfolio`, with `--verdict-dir` as the
portfolio-level expert inbox); exactly one of the two is required, and asking for both is
refused. The model client is injected, so the offline suite proves the
same orchestration with a scripted client; only the CLI's default constructs the SDK
client. `--live-dry-run` is the **live-run drill** (K8): it builds everything a real run
would (contracts → compose → client construction → preflight) and captures a `run_id`-named
@ -114,7 +123,12 @@ description, never from its code)
single learning `VerdictStore` — a verdict available when project k composes survives into
project k+1's fold (cross-project threading, §5), and an optional `verdict_dir` is the
portfolio-level expert inbox the system reads before each fold. The default failure policy
raises (a stack-local choice until D-D flips it to collect-and-continue).
raises (a stack-local choice until D-D flips it to collect-and-continue). Reachable from the
command line as `run.py --portfolio <file> [--verdict-dir <dir>]`. That path **persists
nothing** — it returns typed results and prints one line per project, because the outbox
names its pairs by `run_id` and a portfolio pass has none of its own. Rather than accept
`--outbox`/`--out`/`--value-report` and quietly ignore them, the entrance refuses them
there and points at the per-project `--bundle` runs (§1).
- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only.
- `costsim.py` — pre-run cost simulation (**offline** — the one Run-layer module that never
touches the network): a deterministic UPPER-BOUND USD estimate for a (portfolio-)run
@ -171,6 +185,57 @@ description, never from its code)
which requires `--outbox` and is refused before any spend without one; the report is written
on both run outcomes and never rewrites the run's own exit code).
## The operator CLI
Everything the framework does is drivable from the command line, and **every flag listed
here exists in the corresponding `--help`** — a load-bearing test
(`test_cli_paritet_loadbearing.py`) reads this README and goes red the moment it documents a
flag no entrance offers (§1).
```text
# one project, reading an expert inbox, filing a run_id-named outbox pair
uv run python -m portfolio_optimiser_claude.run --bundle <dir> --inbox <dir> \
--out <dir> --outbox <dir> --run-id <id>
# a portfolio: N projects, one shared budget meter, one shared learning store
uv run python -m portfolio_optimiser_claude.run --portfolio <file> --verdict-dir <dir>
# stop before spending when the book already meets the target (exit 4)
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \
--goals <file> --ledger <file>
# the live-run drill: build everything, capture artifacts, stop before the first call
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \
--outbox <dir> --run-id <id> --live-dry-run
# what the loop delivered, projected from the outbox + inbox + ledger (no model call)
uv run python -m portfolio_optimiser_claude.valuereport --outbox <dir> --inbox <dir> \
--ledger <file> --goal-nok <amount> --json <file>
# who still owes a verdict, and who should judge each proposal
uv run python -m portfolio_optimiser_claude.hitl pending --outbox <dir> --inbox <dir>
uv run python -m portfolio_optimiser_claude.hitl route --outbox <dir> --inbox <dir> \
--routing <file>
# before any spend: what a run would cost, and whether the rig is configured
uv run python -m portfolio_optimiser_claude.costsim --projects <n> --pricing <file>
uv run python -m portfolio_optimiser_claude.preflight --profile <name>
```
Two run-shape rules the entrance enforces rather than papering over: exactly one of
`--bundle`/`--portfolio` is required, and the portfolio path refuses the flags it could not
honour (`--out`, `--outbox`, `--run-id`, `--value-report`, `--inbox`, `--live-dry-run`)
instead of accepting them as no-ops.
## Setting up a knowledge base
The framework is only as good as the OKF bundles it reads, and building those is a **team
job, not a wizard**: technical people and domain experts working together. The honest
expectation is **12 weeks of dedicated work** for a good knowledge base — the quality of
that investment decides the quality of the output. The documented process is
[`docs/oppskrift-kunnskapsbase.md`](docs/oppskrift-kunnskapsbase.md). There is deliberately
no onboarding interview and no guided verdict command; the recipe is the deliverable.
### Load-bearing tests (§11)
Every seam is proven by a test that goes **red when the seam is detached** — green-but-dead
@ -204,6 +269,10 @@ project's realized value stays unmarked and never mirrors the modelled claim —
that boundary is detached — a rising approval share is not reported as learning without the
gap arithmetic behind it, the projection writes no byte into the three layers it reads, and
the run entrance produces the report on both outcomes while leaving the run's verdict alone),
`test_cli_paritet_loadbearing.py` (a hard goal already met by the ledger stops the entrance
before a single model call — red the moment the goal check is unwired — the portfolio config's
projects genuinely run through the CLI, and this README's documented flags are checked against
the actual `--help` output),
and `test_sdk_isolation.py` (local config cannot capture the checker).
## The ingest layer — CSV and SQL, in front of the loop
@ -266,7 +335,7 @@ Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk
```bash
uv sync # install dependencies
uv run pytest # 562 tests — run without any API key and without network
uv run pytest # the full suite — runs without any API key and without network
uv run ruff check . && uv run ruff format --check .
uv run mypy src # strict
```