open/portfolio-optimiser-claude
1
0
Fork 0
Code Issues Releases 1 Activity

feat(portfolio): K12 — CLI parity, doc sync, knowledge-base recipe (parity row 24) [skip-docs]

Browse source 
The last ungated build session: the operator now drives the whole build from the
command line, and the documents claim exactly what the code does (§1).

run.py becomes the collecting entrance. Exactly one of --bundle (one project) or
--portfolio (N projects from a schema-validated reference config, with
--verdict-dir as the portfolio-level expert inbox) is required; both and neither
are refused. --goals loads a goal contract and checks it against --ledger's
realized sum BEFORE the first model call: the §8 caps bound spend, the goal bounds
achievement, so a hard target the book already meets stops the run at exit 4
without constructing a client. A soft target reached is a flag and the run
continues; an absent ledger is an empty book, so the goal is still evaluated,
never skipped. The one declared goal also drives --value-report's goal progress —
one contract, never two figures that can disagree.

The portfolio path persists nothing (K3 returns typed results; the outbox names
pairs by run_id, which a portfolio pass has none of). Rather than accept
--out/--outbox/--run-id/--value-report/--inbox/--live-dry-run there and silently
ignore them, the entrance refuses them and says why. run_portfolio is imported
lazily — portfolio.py imports this module, so a module-level import is circular.

Three seams, each detach-proven RED:
- unwire the goal check → the run proceeds and spends → red
- unwire the portfolio branch → the configured projects never run → red
- document a flag no CLI offers → the README honesty grep goes red

That last one is the doc-sync made load-bearing: the test reads README.md,
collects every --flag it documents (excluding third-party dev-tooling lines) and
asserts each exists in the --help of a CLI the README names. The drift it exists
to close was real — README claimed 562 tests, CHANGELOG claimed 265, actual 597.

Docs synced to the code: README gains an operator-CLI section and honest goal/
portfolio descriptions, CHANGELOG is rewritten to what actually shipped, and
docs/oppskrift-kunnskapsbase.md delivers D-H point 1 — the documented team
process for building a knowledge base, with the honest 1–2 week expectation and
every factory-dependent step (verdict translation, demo path) marked NOT BUILT.

597 passed · ruff clean · mypy strict clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 06:42:52 +02:00
commit da93a68ce7
6 changed files with 956 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

45
CHANGELOG.md
View file

@ -8,12 +8,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- Sibling implementation of the portfolio-optimiser method on the Claude Agent SDK (D7),
built from the shared frozen spec + golden suite alone.
- Deterministic backbone (mandatory blocking validator), the agentic + learning loop
(S5–S10), and the deterministic ingest layer (CSV + SQL source types), each wired seam by
seam and proven by load-bearing tests.
- Test suite: 265 passing tests, all running offline without an API key.
**The method, implemented (D7).** Sibling implementation of the portfolio-optimiser method
on the Claude Agent SDK, built from the shared frozen spec + golden suite alone — never by
reverse-engineering the MAF sibling.
- **Deterministic backbone** — the typed cost-IR, the mandatory blocking validator (frozen by
the shared golden suite, its only oracle), first-class provenance, and fail-fast startup
contracts including the role → model map.
- **Agentic loop** — bounded generation, maker–checker debate, the validator gate, and
informed refinement; the budget meter admits no unbounded loop anywhere, and carries an
optional pre-call run-total USD belt on top of the post-charge token/round caps.
- **Learning loop** — the OKF context seam (navigation, never chunk-stuffing), the ExpeL-style
experience fold, the async expert-verdict inbox, and the fail-closed promotion gate.
- **Ingest layer** — deterministic CSV (`file`) and SQLite (`sql`) connectors in front of the
loop, materializing OKF bundles the unchanged loop consumes; frozen by byte-identical golden
extractions. `http`/MCP is an extension point this repo does **not** build, and a manifest
naming it is rejected fail-fast.
- **Value layer** — the fail-closed savings ledger (dimension-free sum, no double-counting),
the hard/soft goal contract, the outbox output layer, the HITL pending/routing view, the
pre-run cost simulation over schema-validated pricing config, the SDK/API preflight, opt-in
notification sinks (webhook egress only behind an explicit per-run flag), and the per-run
value report (modelled → expert-corrected → realized, goal progress, quantified learning
effect, cost against value).
- **Operator CLI** — one collecting entrance (`run.py`) for a single project (`--bundle`) or a
portfolio (`--portfolio`, `--verdict-dir`), with `--goals` + `--ledger` stopping a run before
any model call when the target is already met, and `--value-report` projecting what the run
delivered. Standalone entrances for `valuereport`, `hitl`, `costsim` and `preflight`. Every
flag the README documents is checked against the actual `--help` output by a load-bearing
test.
- **Knowledge-base recipe** — the documented team process for building the OKF bundles the
framework reads (`docs/oppskrift-kunnskapsbase.md`), with an honest 1–2 week expectation.
- **The programme's one live model run** (S10) — executed and validated at a documented
$0.127514, its four artifacts committed as fixed reference output under `runs/s10/`.
- **Load-bearing tests** — every seam is proven by a test that goes red when the seam is
detached; the whole suite runs offline, with no API key and no network.
### Notes
- Honesty rule (method spec §1): no artifact in this repo claims more than the implementation
does. Scripted stand-ins are labelled as such, unbuilt extension points are named as
unbuilt, and a figure the data does not carry is reported unmarked rather than back-filled.
- Licensed under the MIT License (see `LICENSE`).

89
README.md
View file

@ -12,10 +12,11 @@ human-in-the-loop, and the system learns from the verdicts.
> **Status:** the D7 build (S5–S10) is complete, and the deterministic **ingest layer**
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
> seam, each proven by load-bearing tests (562 tests, all running offline without an API
> key). The programme's single budgeted **live model run has been executed and validated** —
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
> backbone, the agentic loop, the learning loop, the value layer, and the ingest connectors
> are wired seam by seam, each proven by load-bearing tests (597 at the time of writing, all
> running offline without an API key — `uv run pytest` is the source of truth). The
> programme's single budgeted **live model run has been executed and validated** — its
> artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
> **Disclaimer — technical framework only.** The deployer owns DPIA, risk assessment, and
> the legal basis for any processing. The framework ships only the technical
@ -83,7 +84,12 @@ description, never from its code)
two dimensions is never double-counted. Persistence is deterministic JSON.
- `goals.py` — the goal contract: absolute savings target, hard/soft. A hard goal reached
raises a structured stop event, never a silent stop; the percent-goal baseline is
D-E-gated and refused explicitly.
D-E-gated and refused explicitly. On the command line (`run.py --goals <file> --ledger
<file>`) the goal is checked against the ledger's realized sum **before the first model
call**: the budget caps bound spend, the goal bounds achievement, so a target already met
stops the run at exit 4 without buying anything. A soft goal reached is a flag and the run
continues; an absent `--ledger` is an empty book, so the goal is still evaluated — never
skipped.
**Run layer** (the only part that touches the network)
- `sdk_client.py` — the Claude Agent SDK client, isolated from local configuration
@ -96,9 +102,12 @@ description, never from its code)
minted the same way the inbox mints an expert verdict's id) so outstanding verdicts can be
tracked and live artifacts captured; bytes reuse the deterministic house JSON writer, and
the S10 artifact formats are untouched.
- `run.py` — the generic run entrance: composes merge-inbox → seed → fold (§5) and drives
the loop under the budget meter, persisting artifacts on both outcomes — a structured
budget stop included. The model client is injected, so the offline suite proves the
- `run.py` — the generic run entrance and the operator's collecting surface: composes
merge-inbox → seed → fold (§5) and drives the loop under the budget meter, persisting
artifacts on both outcomes — a structured budget stop included. It runs either **one**
project (`--bundle`) or a **portfolio** (`--portfolio`, with `--verdict-dir` as the
portfolio-level expert inbox); exactly one of the two is required, and asking for both is
refused. The model client is injected, so the offline suite proves the
same orchestration with a scripted client; only the CLI's default constructs the SDK
client. `--live-dry-run` is the **live-run drill** (K8): it builds everything a real run
would (contracts → compose → client construction → preflight) and captures a `run_id`-named
@ -114,7 +123,12 @@ description, never from its code)
single learning `VerdictStore` — a verdict available when project k composes survives into
project k+1's fold (cross-project threading, §5), and an optional `verdict_dir` is the
portfolio-level expert inbox the system reads before each fold. The default failure policy
raises (a stack-local choice until D-D flips it to collect-and-continue).
raises (a stack-local choice until D-D flips it to collect-and-continue). Reachable from the
command line as `run.py --portfolio <file> [--verdict-dir <dir>]`. That path **persists
nothing** — it returns typed results and prints one line per project, because the outbox
names its pairs by `run_id` and a portfolio pass has none of its own. Rather than accept
`--outbox`/`--out`/`--value-report` and quietly ignore them, the entrance refuses them
there and points at the per-project `--bundle` runs (§1).
- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only.
- `costsim.py` — pre-run cost simulation (**offline** — the one Run-layer module that never
touches the network): a deterministic UPPER-BOUND USD estimate for a (portfolio-)run
@ -171,6 +185,57 @@ description, never from its code)
which requires `--outbox` and is refused before any spend without one; the report is written
on both run outcomes and never rewrites the run's own exit code).
## The operator CLI
Everything the framework does is drivable from the command line, and **every flag listed
here exists in the corresponding `--help`** — a load-bearing test
(`test_cli_paritet_loadbearing.py`) reads this README and goes red the moment it documents a
flag no entrance offers (§1).
```text
# one project, reading an expert inbox, filing a run_id-named outbox pair
uv run python -m portfolio_optimiser_claude.run --bundle <dir> --inbox <dir> \
--out <dir> --outbox <dir> --run-id <id>
# a portfolio: N projects, one shared budget meter, one shared learning store
uv run python -m portfolio_optimiser_claude.run --portfolio <file> --verdict-dir <dir>
# stop before spending when the book already meets the target (exit 4)
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \
--goals <file> --ledger <file>
# the live-run drill: build everything, capture artifacts, stop before the first call
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \
--outbox <dir> --run-id <id> --live-dry-run
# what the loop delivered, projected from the outbox + inbox + ledger (no model call)
uv run python -m portfolio_optimiser_claude.valuereport --outbox <dir> --inbox <dir> \
--ledger <file> --goal-nok <amount> --json <file>
# who still owes a verdict, and who should judge each proposal
uv run python -m portfolio_optimiser_claude.hitl pending --outbox <dir> --inbox <dir>
uv run python -m portfolio_optimiser_claude.hitl route --outbox <dir> --inbox <dir> \
--routing <file>
# before any spend: what a run would cost, and whether the rig is configured
uv run python -m portfolio_optimiser_claude.costsim --projects <n> --pricing <file>
uv run python -m portfolio_optimiser_claude.preflight --profile <name>
```
Two run-shape rules the entrance enforces rather than papering over: exactly one of
`--bundle`/`--portfolio` is required, and the portfolio path refuses the flags it could not
honour (`--out`, `--outbox`, `--run-id`, `--value-report`, `--inbox`, `--live-dry-run`)
instead of accepting them as no-ops.
## Setting up a knowledge base
The framework is only as good as the OKF bundles it reads, and building those is a **team
job, not a wizard**: technical people and domain experts working together. The honest
expectation is **1–2 weeks of dedicated work** for a good knowledge base — the quality of
that investment decides the quality of the output. The documented process is
[`docs/oppskrift-kunnskapsbase.md`](docs/oppskrift-kunnskapsbase.md). There is deliberately
no onboarding interview and no guided verdict command; the recipe is the deliverable.
### Load-bearing tests (§11)
Every seam is proven by a test that goes **red when the seam is detached** — green-but-dead
@ -204,6 +269,10 @@ project's realized value stays unmarked and never mirrors the modelled claim —
that boundary is detached — a rising approval share is not reported as learning without the
gap arithmetic behind it, the projection writes no byte into the three layers it reads, and
the run entrance produces the report on both outcomes while leaving the run's verdict alone),
`test_cli_paritet_loadbearing.py` (a hard goal already met by the ledger stops the entrance
before a single model call — red the moment the goal check is unwired — the portfolio config's
projects genuinely run through the CLI, and this README's documented flags are checked against
the actual `--help` output),
and `test_sdk_isolation.py` (local config cannot capture the checker).
## The ingest layer — CSV and SQL, in front of the loop
@ -266,7 +335,7 @@ Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk
```bash
uv sync # install dependencies
uv run pytest # 562 tests — run without any API key and without network
uv run pytest # the full suite — runs without any API key and without network
uv run ruff check . && uv run ruff format --check .
uv run mypy src # strict
```

4
docs/extending.md
View file

@ -11,6 +11,10 @@ runs offline: ingest makes **zero model calls**, touches no network, and
[`ingest.py`](../src/portfolio_optimiser_claude/ingest.py) imports nothing from the Agent SDK
— it is pure standard library.
Ingest is the **automated** path to a bundle: it covers the sources that are already tabular.
Everything a domain expert knows that no table holds is built by hand, and that process is a
separate deliverable — see [the knowledge-base recipe](oppskrift-kunnskapsbase.md).
## Where ingest sits
```

163
docs/oppskrift-kunnskapsbase.md Normal file
View file

@ -0,0 +1,163 @@
# Recipe: building a knowledge base
*(Norwegian title kept from the programme plan — «oppskrift» = recipe. Contents in English,
like the rest of this repo's documentation.)*
This is the documented **team process** for building the OKF bundles this framework reads.
It is a recipe, not a wizard: there is deliberately no onboarding interview and no guided
verdict command. Both were evaluated and rejected — a setup this consequential is done by
people talking to each other, and a wizard would hide exactly the decisions that need
deciding.
**Honest expectation: 1–2 weeks of dedicated work** for a knowledge base worth running
against. The quality of that investment decides the quality of everything downstream. A
thin bundle produces confident nonsense; the deterministic validator will catch arithmetic,
but it cannot catch a domain the base never described.
**Honesty rule (method spec §1):** this document separates what this repo *does today* from
what the process *will* look like once the bundle factory exists. Sections marked
**NOT BUILT** describe a target arrangement, not a capability you can use.
## 1. The team
Setup is always done by a small team, never by one person and never by a tool alone:
| Role | Brings | Owns |
|---|---|---|
| Technical | the file formats, the ingest layer, the run entrance, the caps | the bundle's *shape* |
| Domain expert(s) | what is actually true in this domain, and what the numbers hide | the bundle's *content* |
The split matters because the two failure modes are different. A technically perfect bundle
with no domain substance runs cleanly and tells you nothing. A domain-rich pile of documents
that never navigates is invisible to the loop.
## 2. What a knowledge base is here
One **OKF bundle** per project: a directory of plain markdown, plus the IR projection the
deterministic validator consumes. From the shared micro example
([`shared/examples/bygg-energi-mikro/`](../shared/examples/bygg-energi-mikro/)):
```
bygg-energi-mikro/
index.md # REQUIRED entry point; links to everything else
bygg-kontor-nord.md # type: project — the asset and its baseline
tiltak-led-retrofit.md # type: hypothesis — the candidate measure + parameters
metode-ipmvp-a.md # type: methodology — how savings are measured
kilder-realiseringsgap.md # type: reference — sourced literature
verdict-led-fro.md # type: verdict — an expert judgement (see §5)
validator-input.json # the IR projection the validator consumes
```
Four rules the code actually enforces — verify against these, not against intent:
1. **`index.md` is the entry point.** A bundle without one fails fast (`navigate_bundle`).
2. **The index's cross-links define the bundle.** Navigation follows the links in `index.md`;
a file nobody links to is a file the model never sees. Links containing a path separator
are skipped (fail-closed boundary rule), so keep concept files flat and beside the index.
3. **Every non-index file needs a `type` in its frontmatter.** `project`, `hypothesis`,
`methodology`, `reference`, `verdict` are the types in use.
4. **`type: verdict` files are excluded from the read-context.** A verdict reaches the next
hypothesis only through the experience fold, never by leaking through context. This is a
seam, not a preference — see the load-bearing tests.
`validator-input.json` is separate and **required by the run entrance**: it is the candidate
measure projected into the cost-IR (`project_id`, `measure`, `affected_items` with quantity
and unit cost, `claimed_saving_nok`). Writing it is technical work, and it is where the
domain expert's parameters become numbers the validator can refuse.
## 3. The process, week by week
**Day 1–2 — scope and one skeleton.** Pick *one* project and *one* candidate measure. Build
the bundle above end to end, however thin. Run it:
```bash
uv run python -m portfolio_optimiser_claude.preflight
uv run python -m portfolio_optimiser_claude.costsim --projects 1
uv run python -m portfolio_optimiser_claude.run --bundle <dir> --live-dry-run \
--outbox <dir> --run-id skeleton-1
```
The drill builds everything a real run builds and stops before the first model call, so the
rig is proven before anyone spends anything. Do not proceed to content until this is clean.
**Day 3–7 — content, expert-led.** The domain experts fill in what the skeleton only named:
the baseline the asset actually has, the method by which a saving is measured, and — the part
that matters most — the *sourced* reference material describing where modelled numbers and
realized numbers diverge. In the micro example this is `kilder-realiseringsgap.md`, and it is
the reason the learning loop has anything to learn.
**Day 8–10 — seed the judgement layer.** Add at least one `type: verdict` file encoding what
the experts know that the model does not: the correction they would apply and *why*. Without
this the first runs have no experience to fold, and the loop is a validator with extra steps.
**Ongoing — sufficiency review.** Ask the base directly: *is this comprehensive enough to be
useful, and what is missing?* The answer becomes the domain experts' work list. This review
is human-run today; an automated evaluator is part of the factory (**NOT BUILT**, §7).
## 4. The domain expert's interface
The expert is asked exactly three things: **what** to contribute, **how**, and **where**.
They deliver files in their own formats — notes, spreadsheets, reports, memos — and are never
asked to fill in a schema or hand-write JSON.
**Today:** a technical team member transcribes those contributions into bundle files. This is
the interim arrangement, and it is real work — budget for it in the 1–2 weeks. The AI
translator that would do it is **NOT BUILT** (§7).
**Tabular sources are the exception, and they are automated.** If the contribution is a CSV
catalogue or a local SQL database, the ingest layer materializes an OKF bundle from it
deterministically, with provenance frontmatter and a `generated: true` honesty marker. That
path makes zero model calls and touches no network. See
[`docs/extending.md`](extending.md).
## 5. Verdicts — the loop's fuel
After a run, an expert judges the proposal. The verdict is a plain-JSON file dropped into an
inbox folder; a later run merges it before the fold. The shape (§4.2) is deliberately small:
an `id`, a `decision`, a `rationale`, and the proposal features the verdict is about.
**Today:** the same transcription applies — the expert judges in their own words, a technical
team member writes the verdict file. The decision is the human's in both arrangements; only
the typing changes.
**NOT BUILT — the target arrangement.** The expert delivers a free-format file and the
factory AI-translates it into the strict verdict format, because at volume manual translation
is unrealistic. Approval in practice then becomes **spot checks** in the generated bundle,
which only works with one guard in place: *the structured verdict must always point back at
the expert's original file*, so a spot check can compare what the expert wrote against what
the system understood. **The verdict is the human's judgement; the AI is a translator.** Any
implementation of this that drops the provenance pointer breaks the arrangement.
## 6. Reading and editing bundles
Bundles are plain markdown with frontmatter, so **Obsidian or VS Code is the reading solution**
— Obsidian in particular follows the cross-links the way the framework navigates them, which
makes "what will the model actually see?" answerable by clicking. A dedicated reading view is
a later building block in the toolkit repo, not a prerequisite.
## 7. What is not built
Named here so the recipe cannot be mistaken for a description of shipped capability (§1):
- **The bundle factory** — the tooling that would translate free-format expert contributions
(and verdicts) into bundle files. It lives in a separate toolkit repo that **does not exist
yet**. Everything above marked NOT BUILT depends on it.
- **The demo path** — fresh clone → unzip a worked example into a bundle inbox → the factory
builds it → the whole loop runs. Factory-dependent, and honest only once the factory is
real: the point of that demo is that it shows the actual process with finished content.
- **A dedicated reading view** for bundles (see §6).
- **A dimension catalogue** — the formalized scoping of measure types and cost codes. Today
the routing table treats a measure as a config string; the catalogue is gated on a pending
amendment to the shared spec.
## 8. Checklist before the first real run
- [ ] `index.md` exists and links every concept file that should be visible.
- [ ] Every non-index file has a `type` in its frontmatter.
- [ ] `validator-input.json` matches the hypothesis file's parameters.
- [ ] At least one `type: verdict` file encodes real expert correction.
- [ ] Reference material is sourced — a claim without a source is a claim the experts cannot
defend when the model repeats it back.
- [ ] `preflight` is clean and `costsim` gives a cost you have accepted.
- [ ] `--live-dry-run` completes and its captured artifacts look right.
- [ ] Stop criteria and a budget cap are set — the framework refuses to start without them.

299
src/portfolio_optimiser_claude/run.py
View file

@ -13,7 +13,17 @@ constructed only by ``default_client_factory`` on the CLI path (wired, never
executed by the suite — honesty rule §1); the navigated docs dir comes from
the validated startup contract, never straight from the raw argument (§10).
K12 makes the whole build drivable from here: ``--bundle`` runs ONE project,
``--portfolio`` runs N from a schema-validated config (with ``--verdict-dir`` as
the portfolio-level expert inbox), ``--goals`` + ``--ledger`` bound ACHIEVEMENT
where the §8 caps bound spend, and ``--value-report`` projects what the run
delivered. A flag the chosen path cannot honour is REFUSED, never silently
ignored (§1).
Run: uv run python -m portfolio_optimiser_claude.run --bundle <dir> [--inbox <dir>]
# N projects, sequential, one shared meter + one shared learning store:
uv run python -m portfolio_optimiser_claude.run --portfolio <file> \\
[--verdict-dir <dir>]
# K8 live-run drill (builds all, captures artifacts, STOPS before the first call):
uv run python -m portfolio_optimiser_claude.run --bundle <dir> \\
--outbox <dir> --run-id <id> --live-dry-run
@ -22,6 +32,7 @@ Run: uv run python -m portfolio_optimiser_claude.run --bundle <dir> [--inbox <d
from __future__ import annotations
import argparse
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Callable, Sequence
@ -34,7 +45,14 @@ from portfolio_optimiser_claude.artifacts import (
)
from portfolio_optimiser_claude.outbox import persist_outbox
from portfolio_optimiser_claude.budget import BudgetExceeded, BudgetMeter
from portfolio_optimiser_claude.contracts import Contracts, load_contracts, resolve_model
from portfolio_optimiser_claude.contracts import (
Contracts,
ReferenceProjectsContract,
load_contracts,
load_reference_projects,
resolve_model,
)
from portfolio_optimiser_claude.goals import GoalContract, GoalReached
from portfolio_optimiser_claude.preflight import Refusal, run_preflight
from portfolio_optimiser_claude.experience import (
CandidateFeatures,
@ -58,12 +76,17 @@ from portfolio_optimiser_claude.notify import (
notify_config_from_args,
)
from portfolio_optimiser_claude.validator import Rejection
from portfolio_optimiser_claude.valuereport import build_value_report, report_to_json
from portfolio_optimiser_claude.valuereport import build_value_report, load_ledger, report_to_json
_PROPOSER_ROLE = "proposer"
_CHECKER_ROLE = "checker"
# The one backend profile the run resolves against (mirrors SdkModelClient's default).
_DEFAULT_PROFILE = "anthropic"
# Exit codes for the two STRUCTURED stops the entrance can reach. They are
# distinct because the criteria are: 3 bounds SPEND (§8), 4 bounds ACHIEVEMENT
# (K2). Neither is an error — both are outcomes, and both say so in the output.
_BUDGET_STOP_EXIT = 3
_GOAL_STOP_EXIT = 4
# The injected client seam of the entrance: (contracts, max_budget_usd_per_call).
ClientFactory = Callable[[Contracts, float], ModelClient]
@ -194,7 +217,7 @@ def execute_run(
fields={"kind": stop.kind, "observed": stop.observed, "limit": stop.limit},
),
)
return 3
return _BUDGET_STOP_EXIT
provenance = Provenance(
citations=composed.citations,
@ -241,12 +264,145 @@ def execute_run(
return 0
def load_goal(path: Path) -> GoalContract:
"""Load + validate the goal contract fail-fast (§10) — a startup contract like the rest.
A percent goal is D-E-gated and refuses loudly here (``NotImplementedError``
from ``GoalContract``), before a model client exists — never silent semantics.
"""
return GoalContract(**json.loads(path.read_text(encoding="utf-8")))
def check_goal_before_spend(
goal: GoalContract,
ledger_path: Path | None,
*,
notifiers: Sequence[Notifier] = (),
) -> bool:
"""Evaluate the goal against the book BEFORE any model call; True ⇒ stop (K12).
The §8 caps bound SPEND; the goal bounds ACHIEVEMENT — so the honest place
to ask "is the target already met?" is ahead of the first call, not after
paying for one. Observed value is the ledger's dimension-free realized sum
(K1); an absent ``--ledger`` is an EMPTY book (0 realized), so the goal is
still evaluated, simply not reached — never a skipped check.
A HARD goal reached returns True after printing the structured stop and
notifying the configured sinks (a stop is an outcome, exactly as a budget
stop is). A SOFT goal reached is a flag: it prints and the run continues.
"""
realized = load_ledger(ledger_path).total_realized_nok()
try:
reached = goal.check(realized)
except GoalReached as stop:
print(
f"GOAL REACHED (hard): realized {stop.observed_nok} NOK >= target "
f"{stop.target_nok} NOK — stopping BEFORE any model call "
"(the §8 caps bound spend; the goal bounds achievement)."
)
emit(
notifiers,
Notification(
event="run.goal_reached",
summary=(
f"hard goal reached: realized {stop.observed_nok} NOK >= "
f"target {stop.target_nok} NOK — run not started"
),
fields={
"mode": goal.mode,
"target_nok": stop.target_nok,
"observed_nok": stop.observed_nok,
},
),
)
return True
if reached:
print(
f"GOAL REACHED (soft): realized {realized} NOK >= target {goal.target_nok} NOK "
"— flagged, the run continues."
)
else:
print(f"goal ({goal.mode}): realized {realized} of {goal.target_nok} NOK — not reached.")
return False
def execute_portfolio(
client: ModelClient,
projects: ReferenceProjectsContract,
*,
contracts: Contracts,
top_k: int,
max_debate_rounds: int,
max_attempts: int,
verdict_dir: Path | None,
notifiers: Sequence[Notifier] = (),
) -> int:
"""Drive N projects sequentially under ONE §8 meter; print one line per project (K12).
This is the CLI reach to ``run_portfolio`` — the capability itself is K3's
and is not extended here: the pass returns typed results and persists
NOTHING, so nothing is filed and the summary IS the output. A budget stop
propagates out of the shared meter and is reported as the structured stop it
is (exit 3); because this path files no artifacts, none are claimed.
``run_portfolio`` is imported lazily: ``portfolio.py`` imports this module
for ``compose_run_context``, so a module-level import would be circular.
"""
from portfolio_optimiser_claude.portfolio import run_portfolio
meter = BudgetMeter(contracts.termination)
try:
result = run_portfolio(
projects,
client,
meter,
top_k=top_k,
max_debate_rounds=max_debate_rounds,
max_attempts=max_attempts,
verdict_dir=verdict_dir,
)
except BudgetExceeded as stop:
print(f"STOPPED by budget: {stop.kind} observed {stop.observed} > limit {stop.limit}")
print("portfolio: no artifacts written — this path persists nothing (§1).")
emit(
notifiers,
Notification(
event="run.stopped",
summary=f"portfolio stopped by budget: {stop.kind}",
fields={"kind": stop.kind, "observed": stop.observed, "limit": stop.limit},
),
)
return _BUDGET_STOP_EXIT
for project in result.results:
outcome_kind = "rejected" if isinstance(project.run.outcome, Rejection) else "validated"
print(
f"project: {project.project_id} validator={project.run.validator_decision} "
f"checker={project.run.checker_decision} attempts={project.run.attempts} "
f"outcome={outcome_kind}"
)
print(
f"portfolio: {len(result.results)} project(s) completed — results are returned, "
"not filed (this path persists no artifacts)."
)
emit(
notifiers,
Notification(
event="portfolio.completed",
summary=f"portfolio: {len(result.results)} project(s) completed",
fields={"projects": [project.project_id for project in result.results]},
),
)
return 0
def write_value_report(
*,
outbox_dir: Path,
inbox_dir: Path | None,
ledger_path: Path | None,
destination: Path,
goal: GoalContract | None = None,
) -> bool:
"""K11 opt-in: project the persisted layers into a value report AFTER the run.
@ -256,10 +412,14 @@ def write_value_report(
and makes the COMMAND non-zero (the operator asked for a report and did not
get one), but a budget stop stays a budget stop: reporting never rewrites a
run's own verdict. Returns whether the report was written.
A ``goal`` (K12's ``--goals``) is passed straight through, so the operator's
one declared target drives BOTH the pre-spend stop and the report's goal
progress — one contract, never two figures that can disagree.
"""
try:
report = build_value_report(
outbox_dir=outbox_dir, inbox_dir=inbox_dir, ledger_path=ledger_path
outbox_dir=outbox_dir, inbox_dir=inbox_dir, ledger_path=ledger_path, goal=goal
)
except (OSError, TypeError, ValueError) as exc:
print(f"VALUE REPORT FAILED — refusing to project a malformed layer (§10): {exc}")
@ -374,11 +534,33 @@ def main(
canned transport so no socket is opened.
"""
parser = argparse.ArgumentParser(
description="Run one project through the loop (merge inbox → seed → fold → run)."
description=(
"Run one project (--bundle) or a whole portfolio (--portfolio) through the "
"loop (merge inbox → seed → fold → run), under the §8 caps and an optional "
"savings goal."
)
)
parser.add_argument(
"--bundle", type=Path, default=None, help="OKF bundle dir of ONE project to run."
)
parser.add_argument(
"--portfolio",
type=Path,
default=None,
help="reference-projects config (JSON): run N projects sequentially under one "
"shared §8 meter and one shared learning store (K3).",
)
parser.add_argument(
"--verdict-dir",
type=Path,
default=None,
help="portfolio-level expert inbox, READ before each project's fold (requires "
"--portfolio; a single run uses --inbox).",
)
parser.add_argument("--bundle", type=Path, required=True)
parser.add_argument("--inbox", type=Path, default=None)
parser.add_argument("--out", type=Path, default=Path("runs") / "run")
parser.add_argument(
"--out", type=Path, default=None, help="run-artifact dir (default runs/run)."
)
parser.add_argument("--outbox", type=Path, default=None)
parser.add_argument("--run-id", type=str, default=None)
parser.add_argument("--max-rounds", type=int, default=12)
@ -398,7 +580,15 @@ def main(
"--ledger",
type=Path,
default=None,
help="realized-savings ledger read by --value-report; absent = an empty book.",
help="realized-savings ledger (K1) read by --goals and --value-report; "
"absent = an empty book.",
)
parser.add_argument(
"--goals",
type=Path,
default=None,
help="goal contract (JSON: target_nok + hard/soft) checked against --ledger "
"BEFORE any model call; a hard target already reached stops the run (exit 4).",
)
parser.add_argument(
"--live-dry-run",
@ -409,6 +599,42 @@ def main(
add_notify_args(parser)
args = parser.parse_args(argv)
# K12: exactly ONE run shape. Refusing both the empty and the ambiguous call
# keeps the entrance honest — neither flag can silently win over the other.
if (args.bundle is None) == (args.portfolio is None):
parser.error(
"give exactly one of --bundle (one project) or --portfolio (a config of N projects)"
)
if args.verdict_dir is not None and args.portfolio is None:
parser.error(
"--verdict-dir is the PORTFOLIO-level expert inbox and requires --portfolio "
"(a single run reads its inbox with --inbox)"
)
if args.portfolio is not None:
# The portfolio pass returns typed results and persists NOTHING (K3), and
# the outbox names its pairs by run_id, which a portfolio pass has none of.
# Every flag that would therefore do nothing is refused rather than
# silently ignored (§1) — a flag that quietly no-ops is a false claim.
unsupported = [
name
for name, value in (
("--inbox", args.inbox),
("--out", args.out),
("--outbox", args.outbox),
("--run-id", args.run_id),
("--value-report", args.value_report),
)
if value is not None
]
if args.live_dry_run:
unsupported.append("--live-dry-run")
if unsupported:
parser.error(
f"--portfolio does not support {', '.join(sorted(unsupported))}: the "
"portfolio pass persists nothing and has no run_id of its own. Run the "
"projects individually with --bundle to file per-run artifacts."
)
# fail-fast (§10 spirit): a run persisted to the outbox MUST carry an explicit
# run_id — reject BEFORE composing or constructing a client, so no spend rides
# on a run that cannot be filed (no wall-clock default fills the gap). The dry
@ -438,12 +664,61 @@ def main(
except EgressNotPermitted as exc:
parser.error(str(exc))
# §10: ALL startup contracts schema-validated BEFORE any model client exists.
# §10: ALL startup contracts schema-validated BEFORE any model client exists —
# the portfolio config and the goal contract included.
projects: ReferenceProjectsContract | None = None
if args.portfolio is not None:
try:
projects = load_reference_projects(
json.loads(args.portfolio.read_text(encoding="utf-8"))
)
except (OSError, TypeError, ValueError) as exc:
parser.error(f"--portfolio config is not a valid reference-projects file: {exc}")
goal: GoalContract | None = None
if args.goals is not None:
try:
goal = load_goal(args.goals)
except (OSError, TypeError, ValueError) as exc:
parser.error(f"--goals is not a valid goal contract: {exc}")
except NotImplementedError as exc: # the D-E-gated percent goal, refused loudly
parser.error(f"--goals: {exc}")
# The §10 contract carries ONE data source. A portfolio pass carries one per
# project, so the contract records the first and every project still navigates
# its OWN bundle_dir inside run_portfolio — never the contract's.
docs_dir = str(args.bundle) if projects is None else projects.projects[0].bundle_dir
contracts = load_contracts(
data_source={"docs_dir": str(args.bundle), "top_k": args.top_k},
data_source={"docs_dir": docs_dir, "top_k": args.top_k},
termination={"max_rounds": args.max_rounds, "max_tokens": args.max_tokens},
feedback={"decision": "approved", "rationale": "startup shape check (§10)"},
)
# K12: achievement is checked BEFORE spend. A hard target already met by the
# book means the run has nothing left to buy — it stops here, structured,
# with no client ever constructed.
if goal is not None and check_goal_before_spend(goal, args.ledger, notifiers=notifiers):
return _GOAL_STOP_EXIT
factory = default_client_factory if client_factory is None else client_factory
if projects is not None:
print(
f"portfolio: {len(projects.projects)} project(s) caps: "
f"max_rounds={args.max_rounds} max_tokens={args.max_tokens} "
f"max_budget_usd_per_call={args.max_budget_usd_per_call}"
)
return execute_portfolio(
factory(contracts, args.max_budget_usd_per_call),
projects,
contracts=contracts,
top_k=contracts.data_source.top_k,
max_debate_rounds=args.max_debate_rounds,
max_attempts=args.max_attempts,
verdict_dir=args.verdict_dir,
notifiers=notifiers,
)
assert args.bundle is not None # narrowed by the exactly-one check above
# The navigated dir is the CONTRACT's, so the validated config is load-bearing.
composed = compose_run_context(
Path(contracts.data_source.docs_dir), args.inbox, k=contracts.data_source.top_k
@ -454,7 +729,6 @@ def main(
f"max_tokens={args.max_tokens} "
f"max_budget_usd_per_call={args.max_budget_usd_per_call}"
)
factory = default_client_factory if client_factory is None else client_factory
client = factory(contracts, args.max_budget_usd_per_call)
# K8: the live-run drill builds the client (the key-free SDK construction
@ -494,7 +768,7 @@ def main(
client,
composed,
contracts=contracts,
out_dir=args.out,
out_dir=args.out if args.out is not None else Path("runs") / "run",
max_debate_rounds=args.max_debate_rounds,
max_attempts=args.max_attempts,
outbox_dir=args.outbox,
@ -511,6 +785,7 @@ def main(
inbox_dir=args.inbox,
ledger_path=args.ledger,
destination=args.value_report,
goal=goal,
)
if not written and code == 0:
code = 1

384
tests/test_cli_paritet_loadbearing.py Normal file
View file

@ -0,0 +1,384 @@
"""CLI parity + documentation honesty — LOAD-BEARING (K12; method-spec §1, §8, §11).
Two seams this file keeps alive.
**The operator drives the whole build from the command line.** The goal
contract (K2) and the portfolio pass (K3) exist as capabilities; K12 is what
makes them REACHABLE. A hard goal already met by the ledger stops the run
BEFORE any model call — the goal bounds achievement where the §8 budget bounds
spend, and a stop is structured output, never a silent one. Detach the goal
check from the entrance → the run proceeds and spends → red. Detach the
portfolio branch → the config's projects never run → red.
**The README claims exactly what the CLI delivers (§1).** Every ``--flag`` the
README documents must exist in the help of a project CLI the README names.
This is the honesty rule in test form: a documented flag that no entrance
offers is a claim the implementation does not back, and it goes red here the
moment the two drift apart.
"""
from __future__ import annotations
import contextlib
import importlib
import io
import json
import re
from pathlib import Path
from typing import Callable
import pytest
from _scripted import ScriptedClient, reply
from portfolio_optimiser_claude.contracts import Contracts, FeedbackContract
from portfolio_optimiser_claude.ir import load_validator_input
from portfolio_optimiser_claude.ledger import SavingsLedger
from portfolio_optimiser_claude.loop import ModelClient, ModelReply
from portfolio_optimiser_claude.run import main
REPO_ROOT = Path(__file__).resolve().parents[1]
BUNDLE = REPO_ROOT / "shared" / "examples" / "bygg-energi-mikro"
README = REPO_ROOT / "README.md"
ClientFactory = Callable[[Contracts, float], ModelClient]
# --- scripted plumbing (no model, no network — §1) -------------------------------------------
def _validated_replies(runs: int = 1) -> list[ModelReply]:
# The three-turn sequence that drives one project to a VALIDATED outcome,
# repeated once per project the portfolio pass will run.
turns: list[ModelReply] = []
for _ in range(runs):
turns += [
reply("debate reasoning"),
reply("VERDICT: APPROVE"),
reply(json.dumps(load_validator_input(BUNDLE).model_dump())),
]
return turns
def _scripted_factory(runs: int = 1) -> tuple[ClientFactory, list[ScriptedClient]]:
created: list[ScriptedClient] = []
def factory(contracts: Contracts, max_budget_usd_per_call: float) -> ModelClient:
client = ScriptedClient(replies=_validated_replies(runs))
created.append(client)
return client
return factory, created
def _goal_file(tmp_path: Path, *, target_nok: float, mode: str) -> Path:
path = tmp_path / "goal.json"
path.write_text(
json.dumps({"target_nok": target_nok, "mode": mode}), encoding="utf-8", newline="\n"
)
return path
def _ledger_file(tmp_path: Path, *, amount_nok: float) -> Path:
# Realized savings only ever enter the book through the expert gate (K1),
# so the fixture is built the way the ledger itself requires.
ledger = SavingsLedger()
ledger.realize(
project="bygg-kontor-nord",
measure_type="LED-retrofit",
affected_codes=["EL-01"],
amount_nok=amount_nok,
verdict=FeedbackContract(decision="approved", rationale="expert approved (fixture)"),
expert="fixture-expert",
timestamp="2026-07-25T00:00:00Z",
)
path = tmp_path / "ledger.json"
ledger.save(path)
return path
def _portfolio_file(tmp_path: Path, project_ids: list[str]) -> Path:
path = tmp_path / "portfolio.json"
path.write_text(
json.dumps(
{"projects": [{"project_id": pid, "bundle_dir": str(BUNDLE)} for pid in project_ids]}
),
encoding="utf-8",
newline="\n",
)
return path
# --- the goal seam ---------------------------------------------------------------------------
class TestGoalStopOnTheEntrance:
"""LOAD-BEARING (§11): a hard goal already reached stops the run before any spend."""
def test_hard_goal_reached_stops_before_any_model_call(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
# Detach point: drop the goal check from main() → the run proceeds and
# spends → exit 0 and a non-empty call log → RED.
factory, created = _scripted_factory()
code = main(
[
"--bundle",
str(BUNDLE),
"--out",
str(tmp_path / "out"),
"--goals",
str(_goal_file(tmp_path, target_nok=100_000.0, mode="hard")),
"--ledger",
str(_ledger_file(tmp_path, amount_nok=150_000.0)),
],
client_factory=factory,
)
assert code == 4
out = capsys.readouterr().out
assert "GOAL REACHED" in out
assert "100000" in out.replace("_", "") and "150000" in out.replace("_", "")
# The stop is BEFORE any spend: no model call was ever made.
assert all(client.calls == [] for client in created)
# A stopped run leaves no run artifacts — it never ran.
assert not (tmp_path / "out" / "proposal.json").exists()
def test_hard_goal_not_reached_runs_normally(self, tmp_path: Path) -> None:
# Control: the same wiring with a target ABOVE the book runs the project.
factory, created = _scripted_factory()
code = main(
[
"--bundle",
str(BUNDLE),
"--out",
str(tmp_path / "out"),
"--goals",
str(_goal_file(tmp_path, target_nok=500_000.0, mode="hard")),
"--ledger",
str(_ledger_file(tmp_path, amount_nok=150_000.0)),
],
client_factory=factory,
)
assert code == 0
assert created and created[0].calls != []
assert (tmp_path / "out" / "proposal.json").is_file()
def test_soft_goal_reached_flags_without_stopping(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
# A SOFT goal reached is a flag, never a stop (goals.py's own contract,
# preserved across the CLI seam).
factory, created = _scripted_factory()
code = main(
[
"--bundle",
str(BUNDLE),
"--out",
str(tmp_path / "out"),
"--goals",
str(_goal_file(tmp_path, target_nok=100_000.0, mode="soft")),
"--ledger",
str(_ledger_file(tmp_path, amount_nok=150_000.0)),
],
client_factory=factory,
)
assert code == 0
assert "GOAL REACHED (soft)" in capsys.readouterr().out
assert created and created[0].calls != []
def test_goal_without_ledger_reads_an_empty_book(self, tmp_path: Path) -> None:
# An absent ledger is an EMPTY book (0 realized), never a skipped check:
# the goal is evaluated, it is simply not reached.
factory, _ = _scripted_factory()
code = main(
[
"--bundle",
str(BUNDLE),
"--out",
str(tmp_path / "out"),
"--goals",
str(_goal_file(tmp_path, target_nok=1.0, mode="hard")),
],
client_factory=factory,
)
assert code == 0
def test_malformed_goal_is_refused_before_any_spend(self, tmp_path: Path) -> None:
# §10: the goal contract is a startup contract — a percent goal is
# D-E-gated and refuses loudly, before a client is ever constructed.
path = tmp_path / "goal.json"
path.write_text(
json.dumps({"target_nok": 100_000.0, "mode": "hard", "target_percent": 10.0}),
encoding="utf-8",
newline="\n",
)
factory, created = _scripted_factory()
with pytest.raises(SystemExit):
main(
["--bundle", str(BUNDLE), "--out", str(tmp_path / "out"), "--goals", str(path)],
client_factory=factory,
)
assert created == []
# --- the portfolio seam ----------------------------------------------------------------------
class TestPortfolioOnTheEntrance:
"""LOAD-BEARING (§11): the portfolio pass is reachable from the command line."""
def test_portfolio_runs_every_configured_project(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
# Detach point: drop the portfolio branch from main() → the configured
# projects never run → RED.
factory, created = _scripted_factory(runs=2)
code = main(
["--portfolio", str(_portfolio_file(tmp_path, ["prosjekt-a", "prosjekt-b"]))],
client_factory=factory,
)
assert code == 0
out = capsys.readouterr().out
assert "prosjekt-a" in out and "prosjekt-b" in out
# Both projects genuinely drove the loop: 2 projects × 3 scripted turns.
assert created and len(created[0].calls) == 6
def test_portfolio_and_bundle_are_mutually_exclusive(self, tmp_path: Path) -> None:
factory, created = _scripted_factory()
with pytest.raises(SystemExit):
main(
[
"--bundle",
str(BUNDLE),
"--portfolio",
str(_portfolio_file(tmp_path, ["prosjekt-a"])),
],
client_factory=factory,
)
assert created == []
def test_neither_bundle_nor_portfolio_is_refused(self) -> None:
factory, created = _scripted_factory()
with pytest.raises(SystemExit):
main([], client_factory=factory)
assert created == []
def test_verdict_dir_without_portfolio_is_refused(self, tmp_path: Path) -> None:
# --verdict-dir is the PORTFOLIO-level expert inbox; on a single run the
# per-run inbox is --inbox. Accepting it silently would claim a wiring
# that does not exist (§1).
factory, created = _scripted_factory()
with pytest.raises(SystemExit):
main(
[
"--bundle",
str(BUNDLE),
"--verdict-dir",
str(tmp_path / "verdicts"),
],
client_factory=factory,
)
assert created == []
def test_portfolio_refuses_the_run_id_named_flags(self, tmp_path: Path) -> None:
# The portfolio pass persists NOTHING (portfolio.py returns typed results
# and leaves filing to the caller), and the outbox names pairs by run_id.
# Refusing here is the honest alternative to a flag that silently does
# nothing.
factory, created = _scripted_factory()
with pytest.raises(SystemExit):
main(
[
"--portfolio",
str(_portfolio_file(tmp_path, ["prosjekt-a"])),
"--outbox",
str(tmp_path / "outbox"),
"--run-id",
"run-1",
],
client_factory=factory,
)
assert created == []
# --- the documentation-honesty seam (§1) -----------------------------------------------------
# Lines about third-party dev tooling are not claims about this framework's CLI.
_FOREIGN_TOOL_MARKERS = ("ruff", "pytest", "mypy", "uv sync")
_FLAG = re.compile(r"--[a-z][a-z0-9-]*")
_MODULE = re.compile(r"portfolio_optimiser_claude\.([a-z_]+)")
_CHOICES = re.compile(r"\{([a-z0-9_,-]+)\}")
def _capture_help(module_name: str, argv: list[str]) -> str:
module = importlib.import_module(f"portfolio_optimiser_claude.{module_name}")
buffer = io.StringIO()
with contextlib.redirect_stdout(buffer), contextlib.suppress(SystemExit):
module.main(argv)
return buffer.getvalue()
def _full_help(module_name: str) -> str:
"""Top-level help plus every subcommand's help (hitl has ``pending``/``route``)."""
text = _capture_help(module_name, ["--help"])
subcommands: set[str] = set()
for match in _CHOICES.finditer(text):
subcommands.update(match.group(1).split(","))
for sub in sorted(subcommands):
text += _capture_help(module_name, [sub, "--help"])
return text
def _readme_documented_modules() -> list[str]:
return sorted(set(_MODULE.findall(README.read_text(encoding="utf-8"))))
def _readme_documented_flags() -> set[str]:
flags: set[str] = set()
for line in README.read_text(encoding="utf-8").splitlines():
if any(marker in line for marker in _FOREIGN_TOOL_MARKERS):
continue
flags.update(_FLAG.findall(line))
return flags
class TestReadmeClaimsMatchTheCli:
"""LOAD-BEARING (§1, §11): the README never documents a flag the CLI lacks."""
def test_every_documented_module_exposes_a_cli(self) -> None:
modules = _readme_documented_modules()
assert modules, "the README documents no entrance — the honesty grep would be vacuous"
for name in modules:
module = importlib.import_module(f"portfolio_optimiser_claude.{name}")
assert callable(getattr(module, "main", None)), (
f"README documents `python -m portfolio_optimiser_claude.{name}` "
"but the module exposes no CLI entrance"
)
def test_every_documented_flag_exists_in_a_documented_cli(self) -> None:
# RED the moment the README claims a flag the code does not offer —
# the drift K12 exists to close, kept closed from here on.
available = "\n".join(_full_help(name) for name in _readme_documented_modules())
assert "--bundle" in available, "help capture is broken — the grep would be vacuous"
undelivered = sorted(flag for flag in _readme_documented_flags() if flag not in available)
assert undelivered == [], (
f"README documents flags no CLI offers: {undelivered} — "
"either wire them or stop claiming them (§1)"
)
def test_the_operator_surfaces_are_all_documented(self) -> None:
# The other direction, bounded to the flags K12 promises the operator
# can drive from the command line: the run entrance's collecting
# surfaces must actually appear in the README.
documented = _readme_documented_flags()
for flag in (
"--bundle",
"--inbox",
"--outbox",
"--verdict-dir",
"--goals",
"--ledger",
"--portfolio",
"--value-report",
):
assert flag in documented, f"{flag} is an operator surface the README never mentions"