feat(portfolio): K12 — CLI parity, doc sync, knowledge-base recipe (parity row 24) [skip-docs]
The last ungated build session: the operator now drives the whole build from the command line, and the documents claim exactly what the code does (§1). run.py becomes the collecting entrance. Exactly one of --bundle (one project) or --portfolio (N projects from a schema-validated reference config, with --verdict-dir as the portfolio-level expert inbox) is required; both and neither are refused. --goals loads a goal contract and checks it against --ledger's realized sum BEFORE the first model call: the §8 caps bound spend, the goal bounds achievement, so a hard target the book already meets stops the run at exit 4 without constructing a client. A soft target reached is a flag and the run continues; an absent ledger is an empty book, so the goal is still evaluated, never skipped. The one declared goal also drives --value-report's goal progress — one contract, never two figures that can disagree. The portfolio path persists nothing (K3 returns typed results; the outbox names pairs by run_id, which a portfolio pass has none of). Rather than accept --out/--outbox/--run-id/--value-report/--inbox/--live-dry-run there and silently ignore them, the entrance refuses them and says why. run_portfolio is imported lazily — portfolio.py imports this module, so a module-level import is circular. Three seams, each detach-proven RED: - unwire the goal check → the run proceeds and spends → red - unwire the portfolio branch → the configured projects never run → red - document a flag no CLI offers → the README honesty grep goes red That last one is the doc-sync made load-bearing: the test reads README.md, collects every --flag it documents (excluding third-party dev-tooling lines) and asserts each exists in the --help of a CLI the README names. The drift it exists to close was real — README claimed 562 tests, CHANGELOG claimed 265, actual 597. Docs synced to the code: README gains an operator-CLI section and honest goal/ portfolio descriptions, CHANGELOG is rewritten to what actually shipped, and docs/oppskrift-kunnskapsbase.md delivers D-H point 1 — the documented team process for building a knowledge base, with the honest 1–2 week expectation and every factory-dependent step (verdict translation, demo path) marked NOT BUILT. 597 passed · ruff clean · mypy strict clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
parent
4dcdd8017a
commit
da93a68ce7
6 changed files with 956 additions and 28 deletions
89
README.md
89
README.md
|
|
@ -12,10 +12,11 @@ human-in-the-loop, and the system learns from the verdicts.
|
|||
|
||||
> **Status:** the D7 build (S5–S10) is complete, and the deterministic **ingest layer**
|
||||
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
|
||||
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
|
||||
> seam, each proven by load-bearing tests (562 tests, all running offline without an API
|
||||
> key). The programme's single budgeted **live model run has been executed and validated** —
|
||||
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
|
||||
> backbone, the agentic loop, the learning loop, the value layer, and the ingest connectors
|
||||
> are wired seam by seam, each proven by load-bearing tests (597 at the time of writing, all
|
||||
> running offline without an API key — `uv run pytest` is the source of truth). The
|
||||
> programme's single budgeted **live model run has been executed and validated** — its
|
||||
> artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
|
||||
|
||||
> **Disclaimer — technical framework only.** The deployer owns DPIA, risk assessment, and
|
||||
> the legal basis for any processing. The framework ships only the technical
|
||||
|
|
@ -83,7 +84,12 @@ description, never from its code)
|
|||
two dimensions is never double-counted. Persistence is deterministic JSON.
|
||||
- `goals.py` — the goal contract: absolute savings target, hard/soft. A hard goal reached
|
||||
raises a structured stop event, never a silent stop; the percent-goal baseline is
|
||||
D-E-gated and refused explicitly.
|
||||
D-E-gated and refused explicitly. On the command line (`run.py --goals <file> --ledger
|
||||
<file>`) the goal is checked against the ledger's realized sum **before the first model
|
||||
call**: the budget caps bound spend, the goal bounds achievement, so a target already met
|
||||
stops the run at exit 4 without buying anything. A soft goal reached is a flag and the run
|
||||
continues; an absent `--ledger` is an empty book, so the goal is still evaluated — never
|
||||
skipped.
|
||||
|
||||
**Run layer** (the only part that touches the network)
|
||||
- `sdk_client.py` — the Claude Agent SDK client, isolated from local configuration
|
||||
|
|
@ -96,9 +102,12 @@ description, never from its code)
|
|||
minted the same way the inbox mints an expert verdict's id) so outstanding verdicts can be
|
||||
tracked and live artifacts captured; bytes reuse the deterministic house JSON writer, and
|
||||
the S10 artifact formats are untouched.
|
||||
- `run.py` — the generic run entrance: composes merge-inbox → seed → fold (§5) and drives
|
||||
the loop under the budget meter, persisting artifacts on both outcomes — a structured
|
||||
budget stop included. The model client is injected, so the offline suite proves the
|
||||
- `run.py` — the generic run entrance and the operator's collecting surface: composes
|
||||
merge-inbox → seed → fold (§5) and drives the loop under the budget meter, persisting
|
||||
artifacts on both outcomes — a structured budget stop included. It runs either **one**
|
||||
project (`--bundle`) or a **portfolio** (`--portfolio`, with `--verdict-dir` as the
|
||||
portfolio-level expert inbox); exactly one of the two is required, and asking for both is
|
||||
refused. The model client is injected, so the offline suite proves the
|
||||
same orchestration with a scripted client; only the CLI's default constructs the SDK
|
||||
client. `--live-dry-run` is the **live-run drill** (K8): it builds everything a real run
|
||||
would (contracts → compose → client construction → preflight) and captures a `run_id`-named
|
||||
|
|
@ -114,7 +123,12 @@ description, never from its code)
|
|||
single learning `VerdictStore` — a verdict available when project k composes survives into
|
||||
project k+1's fold (cross-project threading, §5), and an optional `verdict_dir` is the
|
||||
portfolio-level expert inbox the system reads before each fold. The default failure policy
|
||||
raises (a stack-local choice until D-D flips it to collect-and-continue).
|
||||
raises (a stack-local choice until D-D flips it to collect-and-continue). Reachable from the
|
||||
command line as `run.py --portfolio <file> [--verdict-dir <dir>]`. That path **persists
|
||||
nothing** — it returns typed results and prints one line per project, because the outbox
|
||||
names its pairs by `run_id` and a portfolio pass has none of its own. Rather than accept
|
||||
`--outbox`/`--out`/`--value-report` and quietly ignore them, the entrance refuses them
|
||||
there and points at the per-project `--bundle` runs (§1).
|
||||
- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only.
|
||||
- `costsim.py` — pre-run cost simulation (**offline** — the one Run-layer module that never
|
||||
touches the network): a deterministic UPPER-BOUND USD estimate for a (portfolio-)run
|
||||
|
|
@ -171,6 +185,57 @@ description, never from its code)
|
|||
which requires `--outbox` and is refused before any spend without one; the report is written
|
||||
on both run outcomes and never rewrites the run's own exit code).
|
||||
|
||||
## The operator CLI
|
||||
|
||||
Everything the framework does is drivable from the command line, and **every flag listed
|
||||
here exists in the corresponding `--help`** — a load-bearing test
|
||||
(`test_cli_paritet_loadbearing.py`) reads this README and goes red the moment it documents a
|
||||
flag no entrance offers (§1).
|
||||
|
||||
```text
|
||||
# one project, reading an expert inbox, filing a run_id-named outbox pair
|
||||
uv run python -m portfolio_optimiser_claude.run --bundle <dir> --inbox <dir> \
|
||||
--out <dir> --outbox <dir> --run-id <id>
|
||||
|
||||
# a portfolio: N projects, one shared budget meter, one shared learning store
|
||||
uv run python -m portfolio_optimiser_claude.run --portfolio <file> --verdict-dir <dir>
|
||||
|
||||
# stop before spending when the book already meets the target (exit 4)
|
||||
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \
|
||||
--goals <file> --ledger <file>
|
||||
|
||||
# the live-run drill: build everything, capture artifacts, stop before the first call
|
||||
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \
|
||||
--outbox <dir> --run-id <id> --live-dry-run
|
||||
|
||||
# what the loop delivered, projected from the outbox + inbox + ledger (no model call)
|
||||
uv run python -m portfolio_optimiser_claude.valuereport --outbox <dir> --inbox <dir> \
|
||||
--ledger <file> --goal-nok <amount> --json <file>
|
||||
|
||||
# who still owes a verdict, and who should judge each proposal
|
||||
uv run python -m portfolio_optimiser_claude.hitl pending --outbox <dir> --inbox <dir>
|
||||
uv run python -m portfolio_optimiser_claude.hitl route --outbox <dir> --inbox <dir> \
|
||||
--routing <file>
|
||||
|
||||
# before any spend: what a run would cost, and whether the rig is configured
|
||||
uv run python -m portfolio_optimiser_claude.costsim --projects <n> --pricing <file>
|
||||
uv run python -m portfolio_optimiser_claude.preflight --profile <name>
|
||||
```
|
||||
|
||||
Two run-shape rules the entrance enforces rather than papering over: exactly one of
|
||||
`--bundle`/`--portfolio` is required, and the portfolio path refuses the flags it could not
|
||||
honour (`--out`, `--outbox`, `--run-id`, `--value-report`, `--inbox`, `--live-dry-run`)
|
||||
instead of accepting them as no-ops.
|
||||
|
||||
## Setting up a knowledge base
|
||||
|
||||
The framework is only as good as the OKF bundles it reads, and building those is a **team
|
||||
job, not a wizard**: technical people and domain experts working together. The honest
|
||||
expectation is **1–2 weeks of dedicated work** for a good knowledge base — the quality of
|
||||
that investment decides the quality of the output. The documented process is
|
||||
[`docs/oppskrift-kunnskapsbase.md`](docs/oppskrift-kunnskapsbase.md). There is deliberately
|
||||
no onboarding interview and no guided verdict command; the recipe is the deliverable.
|
||||
|
||||
### Load-bearing tests (§11)
|
||||
|
||||
Every seam is proven by a test that goes **red when the seam is detached** — green-but-dead
|
||||
|
|
@ -204,6 +269,10 @@ project's realized value stays unmarked and never mirrors the modelled claim —
|
|||
that boundary is detached — a rising approval share is not reported as learning without the
|
||||
gap arithmetic behind it, the projection writes no byte into the three layers it reads, and
|
||||
the run entrance produces the report on both outcomes while leaving the run's verdict alone),
|
||||
`test_cli_paritet_loadbearing.py` (a hard goal already met by the ledger stops the entrance
|
||||
before a single model call — red the moment the goal check is unwired — the portfolio config's
|
||||
projects genuinely run through the CLI, and this README's documented flags are checked against
|
||||
the actual `--help` output),
|
||||
and `test_sdk_isolation.py` (local config cannot capture the checker).
|
||||
|
||||
## The ingest layer — CSV and SQL, in front of the loop
|
||||
|
|
@ -266,7 +335,7 @@ Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk
|
|||
|
||||
```bash
|
||||
uv sync # install dependencies
|
||||
uv run pytest # 562 tests — run without any API key and without network
|
||||
uv run pytest # the full suite — runs without any API key and without network
|
||||
uv run ruff check . && uv run ruff format --check .
|
||||
uv run mypy src # strict
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue