# portfolio-optimiser
Generic, open framework on Microsoft Agent Framework (MAF): multi-agent cost-saving proposals gated by a mandatory deterministic validator, with HITL learning.
[](LICENSE)
[](pyproject.toml)
[](https://github.com/microsoft/agent-framework)
A generic, open framework — built on **Microsoft Agent Framework (MAF)** — that finds cost
savings *inside* each project of a portfolio of independent projects. A swarm of agents
generates candidate measures; a **mandatory deterministic validator** (solver + Monte Carlo)
decides the numbers; domain experts judge the outcomes (human-in-the-loop); and the system
**learns from their verdicts** across runs.
> **Solo-maintained, fork-and-own.** This is a starting point, not a vendor product. One
> maintainer, no SLA, MIT licensed. Issues are welcome as signals; pull requests are not accepted.
> See the [organisation governance](https://git.fromaitochitta.com/open/repo-standard/src/branch/main/GOVERNANCE.md)
> for the full model — including what to adopt instead if you need vendor accountability.
*AI-generated: all code produced by Claude Code through dialog-driven development, with human
review, test and judgement before anything ships.*
A short visual introduction — 12 slides, in Norwegian, for a general audience — ships with the
repo: open [docs/kort-presentasjon.html](docs/kort-presentasjon.html) in any browser.
## Table of Contents
- [Install](#install)
- [Walk the whole chain offline](#walk-the-whole-chain-offline)
- [Non-goals](#non-goals)
- [Built on an LLM wiki: Karpathy's idea, Google's format](#built-on-an-llm-wiki-karpathys-idea-googles-format)
- [AI-first, humans on top](#ai-first-humans-on-top)
- [How it works](#how-it-works)
- [How it is set up](#how-it-is-set-up)
- [What this enables](#what-this-enables)
- [The task API — runnable Python, no wrapper](#the-task-api--runnable-python-no-wrapper)
- [Docs](#docs)
- [Stack & develop](#stack--develop)
## Install
Python ≥3.10, with [`uv`](https://docs.astral.sh/uv/). The package is not published to a package
index — install it from source:
```bash
git clone https://git.fromaitochitta.com/open/portfolio-optimiser.git
cd portfolio-optimiser
uv sync
```
The clone is the documented path because the walkthrough below points at files in the tree. It is
no longer a technical requirement for the DATA: a built wheel carries the shared spec, the persona
skill and the example bundles under [`shared/`](shared/README.md) as packaged data, and the
resolver prefers a working tree when one is present (`PORTFOLIO_SHARED_ROOT` overrides both) — so
an installed distribution finds its knowledge without a checkout.
### Installing a built wheel
A wheel is **not installable on its own**, and the failure is a resolver error rather than a
missing file. Two dependencies are pinned to git tags, and `[tool.uv.sources]` is uv configuration
that does not travel with wheel metadata — so the wheel names `llm-ingestion-okf` and
`llm-ingestion-guard` as bare names that no package index can resolve. Supply the two requirements
alongside the wheel (measured: 65 packages, exit 0):
```bash
uv pip install portfolio_optimiser-1.1.0-py3-none-any.whl \
"llm-ingestion-okf @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.3.2" \
"llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.3.4"
```
Both are tag-pinned deliberately: they are security components, and a version that can move under
an install is a gate that can stop gating without a local diff. `uv sync` from a clone reads the
pins from `pyproject.toml`, which is why the source path above needs none of this.
`uv sync` installs three commands: `portfolio-optimiser` (the CLI), `portfolio-optimiser-demo`
(the offline end-to-end proof) and `portfolio-optimiser-toolbox` (the run-path steps as plain
commands). They are equivalent to the `python -m portfolio_optimiser.run`,
`python -m portfolio_optimiser.simulation` and `python -m portfolio_optimiser.toolbox` forms used
throughout this README, which keep working — the module form is spelled out below so a reader can
see which module answers a given command.
### The toolbox: run-path steps without a model
Every path through the CLI above builds a debate, and therefore a chat client. The steps the
debate is built *on* need no model at all, and `portfolio-optimiser-toolbox` exposes them so an
outside caller — you at a terminal, or an agent that is not this framework — can drive them
directly. No model call, no network, JSON on stdout, and an exit code that says what happened:
`0` the step ran, `2` the call was malformed, `3` the step refused and the refusal is named.
```bash
# Open a knowledge base and report what navigation reached (and every link it could not follow)
uv run portfolio-optimiser-toolbox navigate-bundle --bundle-dir shared/examples/bygg-energi-mikro
# Retrieve citation-ready chunks — the same call the agents' in-process tool makes
uv run portfolio-optimiser-toolbox retrieve-chunks \
--query "LED retrofit" --docs-dir shared/examples/bygg-energi-mikro --top-k 2
# Derive the cost baseline from a priced schedule already in the base
uv run portfolio-optimiser-toolbox cost-baseline --bundle-dir --project-id
# Admit (or refuse, by name) a declared pre-pass cut before it may shape a run
uv run portfolio-optimiser-toolbox prepass-admit --payload --bundle-dir
```
Each subcommand calls the same function the run path calls — not a copy of it. That is what makes
the answers you get here the answers the debate gets.
Verify the install by running the whole suite from the clean clone:
```bash
uv run pytest
```
There is no CI runner in this organization, so nothing runs that suite automatically — the command
above is the verification.
## Walk the whole chain offline
Seven commands, no API key, no network, no cost. They exercise the real loop — context navigation
over the knowledge base, the maker/checker debate, the deterministic validator, the verdict — with
**scripted stand-ins for the agents' answers**. Every scripted invocation prints a banner saying so,
because a scripted run that reads like a model run would be worse than having no offline mode at
all. What this shows is that the loop closes and the gate bites; it does not show how well a given
model would propose or judge.
**1 — Look at the knowledge base.** It is curated markdown, not a black box:
```bash
ls shared/examples/bygg-energi-mikro/
```
**2 — Watch the learning loop close.** Two runs separated by an expert approval, with the second
demonstrably informed by the first:
```bash
uv run python -m portfolio_optimiser.simulation # or: uv run portfolio-optimiser-demo
```
Each run shows the refinement step: the proposer's first claim is falsified by the deterministic
validator, and the corrected claim validates. Between the runs, knowledge travels back on **both
feedback timescales, each carrying its own marker** so neither can stand in for the other: the
persona's approved verdict is promoted into the file-backed wiki (promote → re-seed → fold), and a
second verdict — an operations note an expert drops into an inbox folder *after* the run, the way a
reviewer would days later — is merged from disk into the next run's store before its hypothesis is
formed (write file → merge → fold). Both markers are present in Run B's prompt and absent from
Run A's. Nothing crosses in memory.
The run is **anchored**: the demo copies the example knowledge base and adds a `cost-baseline.json`
— the project's actual cost lines — so the validator's first stage reconciles every cost line a
proposal claims against them, before the solver runs. The declared baseline is printed at the top,
because it is the one property the rest of the output looks identical with or without. Those
numbers are synthetic, derived from the scripted replies themselves; a knowledge base that ships
its own baseline replaces them through the same seam, with no code change.
**3 — Run the loop over a knowledge base, with answers you supply.** Write the stand-in replies,
then point the CLI at the bundle:
```bash
cat > replies.json <<'JSON'
{
"proposer": "{\"measure\":\"LED-retrofit\",\"affected_items\":[{\"code\":\"ENERGI-TOTAL-EL\",\"quantity\":300000,\"unit_cost\":1.0}],\"claimed_saving_nok\":30000}",
"checker": "The numbers are within a feasible range. VERDICT: APPROVE"
}
JSON
uv run python -m portfolio_optimiser.run BYGG-KONTOR-NORD \
--docs-dir shared/examples/bygg-energi-mikro \
--bundle-dir shared/examples/bygg-energi-mikro \
--scripted-replies replies.json
```
Ends in `ValidatedProposal`. Swap `--bundle-dir`/`--docs-dir` for your own bundle to run it over
your own data — that is the point of this door, and the reason it is not the same thing as step 2.
Your bundle needs one file the ingest layer does not write for you: `validator-input.json`, the
candidate the deterministic validator judges (a bundle without it is refused, by design — see
[`docs/extending.md`](docs/extending.md)). Copy the shape from
`shared/examples/bygg-energi-mikro/`.
**4 — Watch it say no.** Raise `claimed_saving_nok` to `250000` in `replies.json` and run the same
command again. The outcome becomes `Rejection`: the deterministic validator refuses a saving the
project's own numbers cannot support, no matter how confidently the proposer asserted it. This is
the part of the method that carries the weight — the agents propose, and something that cannot be
argued with decides.
Read that summary line carefully. Nobody reviewed this run, so it says exactly that:
`Rejection (no expert verdict given; verdict key=…)`. `Rejection` is the **validator's** outcome;
the second half is about the **human**, and there was no human here. The key it quotes is the id
under which an expert verdict on this candidate will arrive later — your join back into the
[expert-answer channel](docs/ekspert-svar.md).
Record one and the line changes:
```bash
uv run python -m portfolio_optimiser.run BYGG-KONTOR-NORD \
--docs-dir shared/examples/bygg-energi-mikro \
--bundle-dir shared/examples/bygg-energi-mikro \
--scripted-replies replies.json \
--decision approved --rationale "the retrofit is within scope"
```
Now it reads `Rejection (verdict id=…, decision=approved)`, which is not a contradiction: a machine
gate that blocks and a human judgement that approves are different questions and are never
collapsed into one field. The two flags go together or not at all — half a verdict is refused by
name, because the missing half is the expert's to write and never ours to default. Until 1.1.0
`--decision` defaulted to `approved`, so every flagless run recorded an approval nobody gave and
carried it into the next project's hypothesis; that default is gone.
**5 — See what it would cost with a real model**, before spending anything:
```bash
uv run python -m portfolio_optimiser.costsim --projects 4 --profile local
```
Modelled upper bounds per role and model, with the source of each price quoted. `--profile local`
prices the free local backend; the estimate is a ceiling, not a bill.
**6 — Run the whole portfolio, and watch the gate anchor to each project separately.** The same
flag works across every bundled reference project at once:
```bash
cat > replies.json <<'JSON'
{
"proposer": "{\"measure\":\"scope_reduction\",\"affected_items\":[{\"code\":\"01.1\",\"quantity\":1,\"unit_cost\":850000}],\"claimed_saving_nok\":40000}",
"checker": "Rigging and site operations can absorb this reduction. VERDICT: APPROVE"
}
JSON
uv run python -m portfolio_optimiser.run --portfolio --scripted-replies replies.json
```
One `ValidatedProposal`, three `Rejection`. All four reference projects carry a cost line `01.1`,
but at four different amounts — so a claim stated against one project's estimate is refused for the
other three. Nothing about the proposal changed between them; what changed is the project's own
numbers, which is the whole point of anchoring the gate to a cost baseline rather than to the
proposal's internal arithmetic.
The four lines quote the same `verdict id=`. That is not a bug: a verdict is keyed on the
*candidate* it judges, not on the project it was judged in, so an identical proposal mints an
identical id by design — that key is how a later run finds the earlier judgement.
A portfolio pass reports what happened to every project. Projects that raised are printed to
stderr with their error, and the command exits non-zero; the projects that completed still print
their outcome, because one dead project must not discard the rest of the pass. A pass stopped
because a savings goal was reached says so too.
The global token cap is reported separately from a goal stop — running out of budget and hitting
your target are not the same event — but note that the cap itself has no command-line flag yet:
only a library caller can install one, so that line is unreachable from the CLI today.
**7 — Report what has actually been realized:**
```bash
uv run python -m portfolio_optimiser.run --report --ledger savings-ledger.json
```
This reads a savings ledger and prints per-project and portfolio totals with each entry's
provenance. It makes no model calls and changes nothing.
**The ledger is an input, and the framework will not write it for you.** It records savings that
were *actually realized* — a contract was changed, an invoice came in lower — which is a fact about
the world, not a conclusion the system is entitled to draw from its own proposals. A validated
proposal is a claim; a ledger entry is a result. Keeping them apart is deliberate, and it is why no
command here produces a ledger as a side effect. Run `--report` before creating one and it says so
plainly (`run report refused: savings ledger not found`).
You write entries when the saving materializes:
```python
from portfolio_optimiser.ledger import LedgerEntry, SavingsLedger, to_ore
ledger = SavingsLedger()
ledger.add_realized(
LedgerEntry(
project_id="FV42-GSV-E1",
dimension="rigg",
candidate_identity="33fba649cade8529",
amount_ore=to_ore(40000),
verdict_id="33fba649cade8529",
provenance="expert Kari Nordmann, 2026-08-05, realized via contract amendment",
)
)
ledger.save("savings-ledger.json")
```
Amounts are held in øre as integers, and `to_ore` is the only conversion — money is quantized once,
per amount, before anything is summed.
> `--live-dry-run` is a different, narrower drill: it builds contracts, clients and budget against
> your own configuration and **stops before the first model call**. It verifies the setup; it does
> not run the loop. `--scripted-replies` runs the whole loop. The two are mutually exclusive and
> passing both is refused rather than one silently winning.
## Non-goals
- **Not a compliance product.** It ships the technical prerequisites — local-only operation,
provenance on every proposal, no silent data egress — and stops there. Processing purpose, DPIA
and risk assessment stay with the deploying organization.
- **Not a portfolio-level reallocator.** It finds savings *inside* each project. Moving budget
between projects, ranking projects against one another and portfolio governance sit above the
method and are out of scope.
- **Not autonomous decision-making.** The deterministic validator can only block; approving a
measure is a domain expert's call (human-in-the-loop), and the framework implements nothing on
the agents' say-so.
- **Not a turnkey vertical solution.** The aim is a generic core with explicit extension points
(data sources, cost models, personas) — not the last 10% of any one domain.
- **Not a model benchmark.** The end-to-end proof runs offline against a scripted stand-in client:
it shows that the loop closes, not how well a given LLM proposes or judges.
> **Status:** the full 8-step agentic loop is wired and proven with load-bearing tests. The
> end-to-end proof is primarily an **offline simulation** with a scripted stand-in client, but one
> **live run** against a real endpoint (`gpt-4.1-mini`, 2026-08-14) has also completed: it ended
> in a correct `rejected` outcome — the deterministic validator caught a cost line the model had
> invented outright (a code absent from the knowledge base), on the tolerance gate rather than the
> stricter existence gate, because the bundle it ran against ships no cost baseline to anchor
> against. No run has yet produced a **validated** proposal against a live model, and every
> human-in-the-loop verdict currently seeded into the knowledge base is a synthetic, AI-authored
> seed marked as such — no genuine expert verdict has entered the tree yet. The **ingest layer**
> (real data sources) is implemented — file/CSV and SQL on both stacks with bit-identical golden
> extractions from the shared spec, plus HTTP as a MAF-only demonstrated extension point against a
> local mock — but exercised only against committed fixtures: no bundle has yet been materialized
> from a live source. Ingested content passes a **content gate** (`materialize_gated`) that scans
> every generated concept with
> [`llm-ingestion-guard`](https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security)
> before any of it reaches the bundle; a refused run writes nothing. A sibling implementation of
> the same method on the **Claude Agents SDK** exists in a separate repository but is currently
> **parked**, not developed in parallel.
> **Disclaimer — technical framework only.** Deploying organizations own their processing
> purposes and assessments (DPIA, risk/ROS, security review). The framework ships the technical
> prerequisites — local-only mode, provenance, no silent data egress — but makes no compliance
> guarantees.
## Built on an LLM wiki: Karpathy's idea, Google's format
The knowledge architecture is the heart of the project, and it is deliberately not ours:
- **The idea** is Andrej Karpathy's **"LLM wiki"**: instead of pointing a model at documents
written for people, you curate a small, versioned body of knowledge written *for the model
to read* — concept files, explicit structure, explicit links.
- **The format** is Google Cloud's **[Open Knowledge Format (OKF)](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)**
(open spec, v0.1), which formalizes that pattern: a knowledge **bundle** is a directory of
markdown files with YAML frontmatter (one required field, `type`), a reserved `index.md`
entry point, and intra-bundle cross-links forming an emergent graph. Custom frontmatter
fields are allowed and must be preserved — which is exactly where this project's own layers
(expert verdicts, ingest provenance) live.
Because OKF is open and vendor-neutral, the *same* bundles are consumed unchanged by both
reference implementations (MAF and the Claude Agents SDK sibling) — the knowledge outlives any
particular agent stack.
**Not RAG.** Agents read a bundle by **navigating** it — `index.md` first, then its
cross-links, with progressive disclosure — never by keyword retrieval or stuffing the whole
bundle into a prompt. Query-time retrieval against the bundle is explicitly forbidden by the
method spec: it would leak the verdict layer around the learning gate.
A cross-link that cannot be followed is **tolerated** (OKF SPEC §4 — navigation never raises) but
no longer **silent**: each one is recorded on `Bundle.skipped` with the file it was written in, the
link text verbatim, and which of the two reasons applied (`missing` — resolves inside the bundle
with no readable file there; `outside-bundle` — resolves outside the bundle root). `--live-dry-run`
and a full run both print the list, and print nothing when every link was followed — so a bundle
that was only half read stops looking like a bundle that was simply smaller.
## AI-first, humans on top
A traditional wiki is built for *people* — optimized for humans finding and reading
information, with machine access bolted on afterwards. This project inverts that order, and is
a concrete example of what that looks like:
- The wiki (the OKF bundle) is written **for the model**: it is the agent's working memory and
the substrate the learning loop reads from and promotes into.
- The **human affordances are layers on top**: experts judge outcomes by dropping a plain JSON
verdict file in an inbox folder; an explicit, fail-closed **promotion gate** is the only path
by which an approved verdict becomes wiki knowledge; reports and reviews are rendered *from*
the machine-readable layers.
Humans stay decisive — nothing enters the wiki without an approval — but the primary reader of
every file is the model, not a person browsing.
## How it works
One run, one project, eight steps — with the learning loop closing across runs:
1. **Understand** — navigate the project's OKF bundle; fold the candidate's *prior expert
verdicts* into the hypothesis prompt (ExpeL-style, retrieved structurally, never by text).
2. **Hypothesise** — one typed candidate measure (strict IR, fail-fast schema).
3. **Debate** — a maker-checker pair argues the reasoning (round-capped).
4. **Validate** — two falsifiers on the same candidate: the **deterministic validator** gates
the numbers (blocking, never optional) and the **checker** gates the reasoning. The validator
is anchored to the project's declared cost baseline, so a proposal cannot invent the cost
lines it claims to save against.
5. **Refine** — a rejected attempt retries *informed* by the rejection reason, under hard
attempt and token caps. Unbounded loops are forbidden everywhere. The falsifications that
informed a later attempt are surfaced on the result (`RunResult.refinements`), so what the
run corrected in response to is inspectable, not just what it ended up with.
6. **Propose or discard** — a validated proposal with risk percentiles, or a typed rejection.
7. **Expert feedback** — days later, an expert drops a verdict file in an inbox folder; a
later run picks it up. Fully resumable; no live session assumed.
8. **Promote** — an *approved* verdict is lifted into the wiki as a `type: verdict` concept
file, navigable by the next run. The gate is fail-closed: raw agent output never
self-promotes.
Every proposal carries provenance (citations into the bundle, model, validator decision, token
usage). Every seam above is protected by a **load-bearing test** — a test designed to *fail*
when the seam is detached, so the loop cannot silently degrade into theater.
## How it is set up
- **One shared, framework-neutral core** ([`shared/`](shared/README.md), a git subtree of
[`portfolio-optimiser-commons`](https://git.fromaitochitta.com/open/portfolio-optimiser-commons)):
the business concept, the normative [method spec](shared/method-spec.md) and
[ingest spec](shared/ingest-spec.md), the expert-reviewer persona as an Agent Skill, and an
example bundle with a golden suite as the only ground truth. Both stacks implement from the
spec alone.
- **Per project: one OKF bundle** — the bundled examples are hand-curated; the ingest layer that
materializes a bundle from a source (file catalogues/CSV + SQL, HTTP as a MAF-only demonstrated
extension point) via a deterministic, schema-validated manifest that runs *before* the loop is
implemented and exercised against committed fixtures — no bundle has yet been materialized from
a live source.
- **Backends:** a run binds to one of two profiles. `local` (the development default) talks to an
OpenAI-compatible endpoint on loopback and makes no egress. `azure` talks to a Foundry project
and resolves its configuration from the environment *at call time*:
- **Endpoint** — `PORTFOLIO_FOUNDRY_PROJECT_ENDPOINT` first, then the `FOUNDRY_PROJECT_ENDPOINT`
that Foundry injects into a hosted agent. Ours wins, so exporting it is always decisive; the
injected name is what lets the same image run hosted with no extra wiring. Neither set is a
fail-fast naming both.
- **Credential** — `AzureCliCredential` on a developer host (constructing it acquires no token;
`az login` stays your manual step), and `ManagedIdentityCredential` when
`FOUNDRY_HOSTING_ENVIRONMENT` is present, because a hosted container has no Azure CLI and the
platform mints it a dedicated Entra identity instead.
Deployment names are never committed: the role→model map ships `REPLACE-WITH-*` placeholders that
fail fast, and `PORTFOLIO_MODEL_MAP` points at an out-of-tree map that wins over the bundled one.
- **Tracing:** off unless `PORTFOLIO_OTEL` holds a value, and then it says where the spans go
before it emits one. `console` writes them to **stderr**, so a traced run's stdout is
byte-identical to an untraced one; `otlp` exports them over the network, and only to an endpoint
you named in one of the standard `OTEL_EXPORTER_OTLP_*_ENDPOINT` variables. Asking for `console`
while such a variable is set is refused rather than quietly honoured — those exporters are built
unconditionally by the framework underneath, so "console" would have been a false statement about
where the run's contents went. With the variable unset, no provider is configured at all: spans
are still made and discarded, and nothing can leave the process. The OTLP exporter packages are
not declared dependencies (they are egress, and heavy in a published wheel); install one yourself
if you use that mode.
- **Run:** the `run.py` CLI has **four modes** — a documented partition, since one invocation
cannot exercise every flag:
- **Single-project** — `PROJECT_ID --docs-dir `, plus optional `--bundle-dir`,
`--verdict-dir`, `--outbox-dir` (which requires `--run-id`), `--dimension-config`,
`--semantic-retrieval`, `--decision`/`--rationale`, `--live-dry-run`, `--proposal-review`
(opt-in: answer the proposal the validator just accepted, at your terminal — see below), and
`--scripted-replies ` (the offline whole-loop door — see
[Walk the whole chain offline](#walk-the-whole-chain-offline); mutually exclusive with
`--live-dry-run`, which stops before the first model call rather than answering it), and
`--explore "" --explore-config ` (opt-in: run an exploration first and let it
shape the mandate this run evaluates — see below), and `--derive-cost-baseline` (opt-in:
anchor the validator on a priced schedule already inside `--bundle-dir` instead of a
hand-written `cost-baseline.json` — see below).
- **Multi-base (P17b)** — `--across-bundle ` repeated once per knowledge base, plus
`--mandate `, `--run-id ` and `--outbox-dir ` (all three required). One
commission, several bases: the mandate is partitioned by each approach's `bundle_id` and the
existing pipeline runs once per base, sequentially, threading ONE verdict store so a verdict
minted against base *k* reaches base *k+1*'s hypothesis. Each base writes its own full
artefact set under `-`, and one `-multibase.json` beside them
records the spend order, the per-base `run_id`, `unreached`, `collisions`, `budget_stop` and
each base's own `stop_reason` — written even when a base is cut short, with a `completed`
field so "nothing was left unreached" cannot be read as "we never found out". `--live-dry-run`
drills every configured base and stops before the first model call. `--bundle-dir` stays ONE
directory and is refused here, as are `--portfolio`, `--explore`, `--prepass-payload` and
`--proposals-from-mandate` — each of those resolves one base, and picking which of N was meant
is not this layer's to decide.
- **Portfolio** — `--portfolio`, plus optional `--goals`, `--ledger`, `--dimension-config`,
`--semantic-retrieval`; it stops early and prints a `goal reached: …` line when the
accumulated ledger meets a goal.
- **Value report (S5.4, read-only)** — `--report --ledger ` rolls up the ledger's realized
savings to stdout: per-project totals, the portfolio total, flagged cross-dimension overlaps
(each counted once), and per-entry provenance. Add `--json` for deterministic JSON instead of
the human table. It makes **no model calls** and is mode-exclusive — only `--ledger`/`--json`
are permitted alongside `--report`; `--report` requires `--ledger`, and a stray `--json`
without `--report` is refused (rc 1, never silently ignored).
```bash
# Single-project, offline drill (builds contracts + clients, stops before the first model call):
uv run python -m portfolio_optimiser.run FV42-GSV-E1 --docs-dir --bundle-dir --live-dry-run
# Portfolio run with a savings goal checked against an accumulated ledger:
uv run python -m portfolio_optimiser.run --portfolio --goals goals.json --ledger ledger.json
# Read-only value report over an accumulated ledger (human table; add --json for JSON):
uv run python -m portfolio_optimiser.run --report --ledger ledger.json
```
`--explore` (U4) is **opt-in** and sits *over* the eight-step loop, never inside it. Given a
prompt and a knowledge base, a Magentic manager decides what to read and which cost-saving
directions are worth testing; what leaves that freedom is a **mandate**, and every number in it
is still gated by the same blocking deterministic validator. The exploration writes nothing —
no outbox artefact, no wiki entry, no verdict.
```bash
uv run python -m portfolio_optimiser.run FV42-GSV-E1 --docs-dir --bundle-dir \
--explore "Find the cheapest saving worth testing here" --explore-config exploration.json
```
`--explore-config` states the bounds, and **every field is required** — `max_rounds`,
`max_tokens`, `max_stall_count`, `max_reset_count`, `max_plan_revisions`, `enable_plan_review`.
None of them has a default, because an omitted cap falls back to an *unbounded* loop rather than
a conservative one.
**A cost code must have a FORM where the input offers forms.** Every `affected_item` code is
classified as `identifier` or `prose` and reported on the proposal's provenance (`code_forms`).
Where the delivered input demonstrably offers identifiers of its own — requirement numbers,
process numbers, coded cost lines — a `prose` code is refused by the deterministic gate, naming
how many identifiers the input offered. Where it offers none, the rule does not fire: a corpus
that does not use identifiers cannot be answered in them. A code the project's cost baseline
carries is never refused for its shape, because stage 0 has already ruled it a real line.
**A clause number is not a price.** A standard numbers its own requirements and a process code
numbers its own settlement posts, and both look exactly like a bare decimal: `10.4`, `1.10.4`,
`12.1`. When a run is **unanchored** — the knowledge base ships no cost baseline, so the
validator's stage 0 never runs — a cost code shaped like a clause number is refused unless it is
one of the reference numbers the base itself declares (`req_number`, `prosessnr`, `seksjon` in a
document's own frontmatter). The refusal names the denominator: *"not one of the 2765 this
knowledge base declares"*. A code the base does declare still validates, so a project priced in
real process codes is untouched; an input that declares no reference numbers at all cannot trip
the rule at all. `code_forms` reports the third value, `requirement`, for a code the base does
declare. Anchored runs are unaffected — stage 0 is the stronger falsifier and keeps the ruling.
**Naming the requirement that binds a direction.** Whoever navigates a knowledge base — the
exploration's hypothesiser and, since the debate started navigating, the proposer — is asked to
name the ONE requirement that binds the direction it commits to, and to declare it with
`declare_requirement(bundle_id, path, ref, approach_id)` after reading it. A declaration naming
a document the run never opened is **refused by name** (`RequirementNotRead`) and comes back as a
turn the model can correct by going and reading it; nothing is recorded until it has.
`approach_id` names the approach the requirement binds — one of the mandate's ids, or
`own-proposal` — and an id the run was not commissioned with is refused with the valid ones named
(`UnknownApproach`). **A proposal whose approach declared nothing cannot be `validated`:** when
the debate held the declaration tool, a candidate whose numbers pass every stage but whose
approach has neither a requirement written into the mandate nor a declaration filed under its
own id is reported `unsupported` — not rejected (the numbers held), not validated (nothing was
named as its ground), never counted or summed. The declaration's *quality* is not judged; a run
can satisfy the rule by declaring any document it read, and that is a known weakness. A marked hypothesis
therefore carries `"requirement": {"path": ..., "ref": ...}`, or an explicit `null` together with
`"why_none"` — a base that holds no requirement for a direction is a finding worth stating, and
the field is never simply omitted. Where an approach carries one, the proposer's prompt names it
and asks for it back verbatim in `measure`, and the declaration is written to
`{run_id}-debate.json` / `{run_id}-exploration.json` under `requirements`. The reply gives back
**the document's own** `title` and `req_number`, read off the base rather than echoed from the
arguments, plus the sentence that says what the declaration binds — so a declaration of the wrong
requirement can be seen to be wrong. The route is named in the instruction as well as the
answer: pass `read_dir` a `filter` word from the approach's own label.
On a commissioned run the reply goes one step further and **compares**: it names the directions
the run was commissioned to pursue and says which of their words appear in the declared
document's own title and number — or that none do. It is a report, never a gate: the declaration
is recorded either way, because a requirement can bind a measure without sharing a word with the
name someone gave it. (Measured over three stress rounds, not one declaration named a concept the
answer key asks for, and making the runs read more did not change that. The rule was replayed
against those traces before it was built: it speaks on 10 of 12 declarations and stays quiet on
2.)
**What the commissioner counts as success reaches the readers.** A mandate's `success_criteria`
used to reach the announcement and nothing else. It is now restated verbatim in the debate's task
message — the prompt where `declare_requirement` is available — through one renderer, and is
omitted entirely when the commission states none.
**A reply that could not be parsed says why, once.** A malformed reply used to be retried with
the byte-identical prompt; measured, one paid run spent eleven of its twelve rounds re-asking a
question it had already answered the same wrong way. The next attempt's prompt now carries the
parse reason — only the reason, never the discarded JSON — as a block of its own, distinct from
a validator rejection (the numbers were refuted) and from expert feedback (a person objected).
The verbatim reply is still captured to `{run_id}-parse-failures.json` as before.
**Answering the plan review (`--plan-review`).** With `enable_plan_review` set, the exploration
stops before the loop is allowed to run and asks you to sign the plan off. `--plan-review`
answers it *at your terminal*: you are shown the plan, and you type `approve` or
`revise `. A revision goes back to the manager, which replans and asks you
again about the **new** plan; `max_plan_revisions` bounds how many revisions are applied. Every
round trip is recorded in `{run_id}-exploration.json` with your words verbatim. Input that ends
without an answer is an error, never a sign-off — an autonomous loop must not run on a plan
nobody approved. The two flags are refused apart: `enable_plan_review` without `--plan-review`
would stop at a review nobody can answer, and `--plan-review` without it would build a reviewer
nobody ever asks.
```bash
uv run python -m portfolio_optimiser.run FV42-GSV-E1 --docs-dir --bundle-dir \
--explore "Find the cheapest saving worth testing here" --explore-config exploration.json \
--plan-review --outbox-dir out --run-id r1
```
The review is **synchronous**: the loop waits on you. That is why the hosted surface refuses it
— blocking an HTTP request on a human would also block the event loop that answers
`/readiness`.
**Declaring the cut instead of navigating for it (`--prepass-payload`).** By default the debate
is handed a pointer and reads the knowledge base itself, opening what it judges relevant. That
works, and it leaves one thing unsaid: how much of the base was *not* read, and why. With
`--prepass-payload` you hand the run a cut that was computed ahead of time and that states its
own denominators — how many concepts were considered, how many delivered, how many withheld, and
under which rule. The debate is given that cut and the navigation tools are withdrawn, because a
debate that can still go looking is not bound by the cut it just declared.
The file is produced outside this framework, by any tool that emits the `okf-consumption/1`
payload shape. Nothing is taken on trust: every delivered excerpt is checked against the document
it names in the knowledge base — the same file, the same bytes, the same text — before a single
model call, so a payload cannot smuggle in text the base does not contain. Expert verdicts and
out-of-scope material are refused here exactly as the navigation tools refuse them.
```bash
uv run python -m portfolio_optimiser.run BYGG-KONTOR-NORD --docs-dir \
--bundle-dir --prepass-payload cut.json --outbox-dir out --run-id r1
```
The cut is printed and written to `{run_id}-prepass.json`. **This buys a declared cut, not a
cheaper run** — the delivered text costs more than a pointer does. When the payload's producer
declares a `title`, `req_number` or `sources` address for an excerpt, that reaches the prompt
alongside the text, so a proposal can cite the requirement a person would — not only the internal
identifier the base assigns it. And note the limit it inherits:
an *empty* delivery is evidence the base does not answer the question, but a *full* one is not
evidence that it does. The excerpts are the closest matches, which is not the same as an answer;
the run says so to the agents in as many words.
Requires `--bundle-dir`. Refused with `--portfolio` (one payload is a cut of one base at one
point in time), with `--report` (which never runs a debate), with `--proposals-from-mandate`
(which returns before the debate), with `--dimension-config` (the cut was computed without
knowing about scopes, so honouring one would drop excerpts its own denominators count as
delivered) and with `--explore` (the exploration reads the whole base with the very tools the
payload withdraws).
**Seeding the exploration with a cut instead of replacing the base (`--prepass-seed`).** The
flag above hands the *debate* a cut and takes its navigation away. This one is the opposite arm,
for the *exploration*: the same kind of pre-computed cut becomes the loop's **starting point**,
and the four navigation tools stay, so it can read past the cut whenever the delivered text does
not carry what it needs. Every excerpt is admitted by exactly the checks the other flag applies
— same document, same bytes, same text, expert verdicts refused — before a single model call.
```bash
uv run python -m portfolio_optimiser.run BYGG-KONTOR-NORD --docs-dir \
--bundle-dir --explore "Find the cheapest saving worth testing here" \
--explore-config exploration.json --prepass-seed cut.json --outbox-dir out --run-id r1
```
The cut is printed before the loop starts and recorded in `{run_id}-exploration.json`, which
carries `rest_reachable` so a reader can tell a *seeded* run from a *bounded* one without having
watched it. **This is not a cheaper run and not a bounded one** — it is a declared place to
begin. Requires `--explore`. Refused with `--prepass-payload` (the two are opposite arms of one
decision, and merging them would mean silently picking one), with `--portfolio`, with `--report`
(which runs no exploration) and with `--checkpoint-dir`, because a resumed run happens in a
process that never saw the cut and its own record would overwrite the declaration with nothing.
**Answering it days later (`--checkpoint-dir` / `--resume`).** A domain expert is rarely at the
terminal when the loop reaches the plan, so the same review can be *parked* to disk instead.
`--checkpoint-dir` writes the suspended workflow there and the open question to
`{run_id}-plan-review.json`, and the process exits. Whenever the expert gets to it — another
day, in a process that never saw the run — they drop `{run_id}-plan-review-answer.json` into a
review inbox, and `--resume` picks it up:
```bash
# day 1 — park the review and exit
uv run python -m portfolio_optimiser.run FV42-GSV-E1 --docs-dir --bundle-dir \
--explore "Find the cheapest saving worth testing here" --explore-config exploration.json \
--checkpoint-dir checkpoints --outbox-dir out --run-id r1
# day N — a fresh process, resuming from what is on disk and nothing else
uv run python -m portfolio_optimiser.run FV42-GSV-E1 --docs-dir --bundle-dir \
--outbox-dir out --checkpoint-dir checkpoints --review-inbox inbox --resume r1
```
A `revise` answered this way does the same thing it does at the terminal: the manager replans
and asks again about the **new** plan. The answer names the `request_id` it answers, and a
mismatch is refused rather than applied — two reviews of one run share a file name, so an answer
left over from the previous round would otherwise sign off a plan the expert never saw. The
vocabulary is the same closed one, anything outside it is refused rather than read as approval,
and `revise` with nothing to revise is refused too. `hitl.pending_plan_reviews(outbox, inbox)`
lists every review still waiting on somebody.
The budget and the revision cap span the suspension — the resumed leg starts from what the
parked one already spent, so a park never hands back a fresh budget. `--plan-review` and
`--checkpoint-dir` are refused together (two doors onto one review), as are `--resume` and
`--explore` (two sources of one exploration).
**Answering the proposal review (`--proposal-review`).** The plan review asks you about the
*plan*, before any candidate exists. This one asks you about the **candidate on the table**:
every time the deterministic validator accepts a proposal, the run stops and shows you the
measure, the cost lines it touches, the claimed saving, the validator's percentiles, the
checker's verdict and how many further attempts are still affordable. You type `approve` to
take it as it stands, or `revise `.
A `revise` is not a note filed for later. It **buys one more generation attempt** under the
caps the run already had — no new loop — and your words go into that attempt's prompt verbatim,
where they stay standing until you next answer. The run then carries whatever the validator
ruled on that later attempt: the expert asks for another try, the machine still decides. Every
round trip is written to `{run_id}-proposal-reviews.json` with your words, which candidate you
were shown, and whether the attempt your `revise` bought actually ran. Input that ends without
an answer stops the run rather than counting as approval.
```bash
uv run python -m portfolio_optimiser.run BYGG-KONTOR-NORD --docs-dir \
--bundle-dir --proposal-review --outbox-dir out --run-id r1
```
`approve` is **not** an expert verdict, and does not become one: it says the candidate may
stand, not that the measure was reviewed and judged. The verdict still arrives the way it
always did — `--decision`/`--rationale` on the run, or a verdict file dropped into
`--verdict-dir` for a later run to read.
The door belongs to single-project mode, and every other combination is refused by name rather
than quietly ignored: `--portfolio` (its waves share one terminal, so several projects'
candidates would arrive at the same prompt with nothing to tell them apart), `--report` (a
read-only roll-up generates nothing to review), `--live-dry-run` (it stops before the first
model call, so no candidate ever reaches you), `--proposals-from-mandate` (that mode settles the
commission deterministically and never generates a candidate), and `--checkpoint-dir` (a parked
exploration returns before any candidate exists — pass `--proposal-review` at `--resume`
instead, which is where the two doors do compose).
Like the plan review it is **synchronous**, so the hosted surface refuses it by name and points
at this CLI. With `--explore --plan-review --proposal-review` you answer two different doors
from the same terminal in sequence: first the plan, then each candidate the validator accepts.
- **Deriving the cost baseline from the knowledge base** — `--derive-cost-baseline` (opt-in,
requires `--bundle-dir`). The deterministic validator anchors a proposal's `affected_items` to
the project's actual cost lines. Normally those come from a hand-written `cost-baseline.json` in
the bundle, or — on the road reference path — from the project's own `cost_items`. Neither is
available when the bundle was *ingested* from tender documents, so this third route reads the
priced schedule that is already in the base: one markdown table whose header names a cost code, a
quantity and a unit price. Both table forms the producer emits are read — the pipe tables its
csv/sql connectors render, and the pandoc simple tables an `.xlsx` becomes.
**It refuses rather than guesses, and that is the point.** A pre-award schedule has its price
column empty, and completing it with a zero or an invented number would hand the gate a baseline
nobody wrote. So the run stops — by name — when no table names all three columns, when more than
one does, when two columns claim the same role, when two rows share a cost code, or when any row
prices nothing. A partly-priced schedule is refused in full: a half-derived baseline would anchor
some codes while the run still reported itself anchored. The header vocabulary and the number
grammar are both closed (plain dot-decimal numbers; a thousands separator or a comma decimal is
refused, not interpreted).
```bash
uv run python -m portfolio_optimiser.run PROSJEKT-1 --docs-dir --bundle-dir \
--derive-cost-baseline
```
- **Capping a run** — `--max-rounds` / `--max-tokens`. Both default to the values a CLI run was
always bound to (3 rounds, 100 000 tokens), so nothing changes unless you state a cap; stating one
is how you raise or lower what a paid run may spend. They apply to `--portfolio` too, and are
refused by name in `--report` mode, where a cap would bind nothing.
- **Judging a finished run against a fasit** — `python -m portfolio_optimiser.stress`. A run's
outbox already carries the evidence (which documents the debate opened, which files the stamp
cites, which approaches validated). This reads it against a context set's own `fasit.json` and
answers three questions per commissioned approach, by machine: was it **grounded** in a document
the fasit says a right answer must reach, does the proposal **name** that requirement, and did it
**hallucinate** a file, a path or a cost code. A fourth, `must_refuse` approach is the
falsification arm: a cost line the base carries no ground for, which must never come back
validated. Every verdict carries its denominators, and an empty outbox — or a base that scans to
no concepts — is refused rather than reported as clean.
One caveat is worth stating, because it decides what the verdict means: when the debate navigates
the base, the provenance stamp cites *every* concept file, so "the fasit path is cited" is true
before any model call. A citation therefore only counts as grounding under a declared pre-pass
cut; otherwise grounding must come from a document the run actually opened. The same correction
now applies to *naming* (P18): a requirement found in a whole-base citation snippet was "named"
before the model said anything, so the snippet half counts only under a narrowed scope too. Both
halves are reported either way.
```bash
uv run python -m portfolio_optimiser.stress contexts/ \
--outbox-dir --run-id
```
- **An identifier that stands everywhere identifies nothing (P18).** The validator's stage 0b
grounds each proposed cost code in the run's non-model-authored input. Plain containment was not
enough: measured on a live run, a proposal put 250 000 NOK on a line coded `R761` — the knowledge
base's own *name*, carried by all 2 756 of its documents — and the whole gate said `validated`.
A code now grounds only if it is at least 3 characters long **and** appears in fewer than 5 % of
the documents the input is made of, with an absolute floor of 10 documents so the share is never
taken over a handful. Both numbers are measured, not chosen: across the four delivered corpora no
code-shaped token of 1 692 distinct ones reaches 5 % (highest 1.35 %), and the shortest real
identifier is 4 characters. The refusal names the denominator — "appears in 2 756 of the 2 756
documents this run was given" — because that reason is fed verbatim into the next attempt's
prompt. A caller that declares no document boundaries is unchanged by construction: one document
can never reach the floor.
- **`--docs-dir` is optional when `--bundle-dir` is given (P18).** On the bundle path `docs_dir` is
never read — retrieval, the chunk tool and the citation check all live in the road branch — so
naming the same directory twice was a requirement for a path that ignores it. Both forms work;
neither turns retrieval into a substitute for ingesting documents into a knowledge base, and the
road path still requires a real `--docs-dir`.
- **Requiring the run to be anchored** — `--require-cost-baseline` (opt-in, requires
`--bundle-dir`). Without a baseline the validator's stage 0 is skipped, and the run says so on
stdout — but it still finishes and still stamps `validator_decision: validated` over cost lines
nothing tied to the project. Measured on a live tender run, every arm invented its codes. This
flag turns that visibility into a refusal: no baseline, no run, and the refusal fires before the
first model call, on `--live-dry-run` as well. It stays opt-in because a bundle that ships no
`cost-baseline.json` is legitimately un-anchored — combine it with `--derive-cost-baseline` when
the base carries a priced schedule.
```bash
uv run python -m portfolio_optimiser.run PROSJEKT-1 --docs-dir --bundle-dir \
--derive-cost-baseline --require-cost-baseline
```
- **The project's own price schedule** — `--cost-baseline FILE` (requires `--bundle-dir` or
`--across-bundle`). A knowledge base carries what is REQUIRED, not what things cost: a road
normal, a standard or a regulation has requirements and no amounts, so a run against one has
nothing for the validator's stage 0 to reconcile against and that stage is skipped. The price
belongs to the project, and this is where you hand it over: FILE is a `cost-baseline.json` — the
same `{"project_id": …, "items": {"": {"quantity": …, "unit_cost": …}}}` shape a bundle may
ship — and it is used INSTEAD of one inside the base. With it, a proposal naming a cost line the
project does not buy is refused as a fabricated line — and the refusal NAMES the codes the
project does have, so the next attempt has something to correct towards. A real line with
invented magnitudes is likewise refused with the real figures named. (Measured on a live tender
round before the codes were named: 26 rejections, every one of them an invented code, and not
one run recovered — a refusal that says only "there are five right answers" carries nothing to
aim at. A long schedule is cut to a fixed number of codes and says so.)
In `--across-bundle` mode the same schedule anchors every base: one project, one price schedule.
Mutually exclusive with `--derive-cost-baseline` (two sources for one baseline), and it satisfies
`--require-cost-baseline`. A missing or malformed file refuses the run before anything starts.
```bash
uv run python -m portfolio_optimiser.run PROSJEKT-1 --bundle-dir \
--cost-baseline prisskjema.json --require-cost-baseline
```
`--explore` is refused together with `--mandate` — they are two sources of one mandate, and
merging would silently overwrite what you wrote. To seed an exploration with a domain expert's
own hypotheses, use `explore(..., seed_approaches=[Approach(...)])`; seeds are always preserved
and always come first, including when the loop stops early. With `--outbox-dir`/`--run-id` the
run also writes `{run_id}-exploration.json`: the per-round ledger, the plan reviews and the
in-loop advisory verdicts, written even when a cap cut the exploration short.
A declaration of the binding requirement is refused when the run has opened fewer than three
distinct documents of the base (capped by the base's own size) — measured, seven of round 4's
thirteen declarations named the base's FIRST requirement after opening exactly one document, and
none of the thirteen named a right one. The refusal is a turn the model can correct, and it
carries both numbers. A path the base does not hold is likewise refused by name, and the refusal
now lists up to five subdirectories of the nearest directory that does hold documents — every one
of them a path `read_dir` will answer for. When that directory has no subdirectories at all, the
refusal names up to five of the DOCUMENTS it holds instead, on the same rule and with the same
property: each name resolves. (Measured over one stress round, three of six guessed document
paths landed on such a directory and got no suggestion at all — including two separate guesses,
in one run, at the name of the single document that level holds.)
Every tool call recorded in `{run_id}-exploration.json` and `{run_id}-debate.json` says HOW the
level was asked for — `filter`, `offset` and `limit` beside the tool name, the base and the path
— so "did the model narrow the level, or page through it" is readable from the artefact rather
than inferred from which documents happened to fall outside a default window.
A run given a mandate also writes `{run_id}-coverage.json`: one row per commissioned approach
with its status and detail, including the ones the run never reached, plus the `stop_reason`
(`tokens` / `rounds`) when a cap cut the commission short. A run without a mandate writes no
such file — coverage is the mandate's report.
The hosted surface takes the same door as `explore_prompt` + `explore_contract` on
`POST /invocations`.
**Several knowledge bases (library API).** An exploration may be given more than one base
(`explore(..., bundle_dirs=[a, b])`). Each approach it shapes records which base it belongs to
(`Approach.bundle_id`), and `run_mandate_across_bundles(mandate, bundle_dirs, ...)` then runs the
pipeline **once per base** — the ordinary `run_project`, with that base's own sub-mandate, and
with each run's project read from that base's own `validator-input.json`. `run_project` itself
still takes one `bundle_dir`, deliberately: it derives the project, the validator's cost
baseline, the agents' read context and the retrieval key from the base it is handed, so a second
directory on that call would mean silently picking one of them. A hypothesis that names no base
is refused when several are configured, rather than routed to a guess. The CLI's `--bundle-dir`
stays single-valued; multi-base is a library door today.
Surveying those bases is deliberately cheap. `list_bundles` costs **O(bases), never O(corpus)**:
each entry carries the base's id, a bounded verbatim opening of its index (with
`index_truncated` beside it when the opening was cut), how many documents and prior expert
verdicts it holds, whether it ships a cost baseline, and how many cross-links could not be
followed — never the whole index. The full index stays one `read_file(id, "index.md")` away, so
the bound is a disclosure level rather than data loss. Measured 2026-08-26 over a real corpus:
112 116 → 362 tokens for three bases, 124 942 → 21 448 for 171
([report](docs/2026-08-26-katalogkostnaden.md)).
**Opening one level is bounded the same way (P18).** `read_dir` answers with a *window*: `total`
says how many entries — subdirectories plus concept documents — the level holds, `offset`/`limit`
say which of them you were given, and a `limit` past the maximum is clamped rather than refused.
`filter` narrows a level instead of paging it: a case-insensitive substring over a document's
title and reference number (`req_number` / `prosessnr`) and over a subdirectory's path, answering
with `total_matches` beside `total`. A filter that matches nothing is an answer, never a refusal.
Measured 2026-09-14 over four delivered bases, default listing of the largest level in each:
`krav/N200` 169 974 → 1 537 characters, `krav/N100` 69 250 → 1 493, R761's root 110 912 → 479
(its cost was 2 728 *subdirectories*, which is why the window covers both kinds), `krav/N500`
39 853 → 1 453. The largest single call any caller can now make is ~7 600 characters.
A path the caller **invented** is refused by name rather than failing opaquely: `read_file` on a
path the base does not hold answers `REFUSED (BundlePathNotFound)` and names the nearest directory
that actually holds documents, so the next call has somewhere to go. Measured over the four paid
runs of 2026-09-14: 10 of 32 `read_file` calls named such a path, and each previously reached the
model as the framework's opaque `Error: Function failed.` while counting toward the three
consecutive tool errors that end a request.
`--semantic-retrieval` (S3.1) is an **opt-in** ranking change, **off by default**. Off, prior
verdicts are ranked exactly as before: a structural score over the affected cost-code set,
measure type and magnitude bucket, with surface text deliberately excluded. On, that score is
blended with a cosine term over the *same* structural triple, which lets a prior verdict on a
*different* cost-code set outrank one that ties structurally.
**What this ships is the seam, not better retrieval.** The bundled `FakeEmbedder` is a
deterministic sha256 projection carrying no semantics, so over a structural tie the resulting
order is deterministic but arbitrary. Retrieval *quality* depends entirely on injecting a real
embedder — `--embedder-config` selects one from a closed registry (never an import path; a
config file can never name arbitrary code to load), and `docs/extending.md` documents the
`Embedder` protocol. The embedding excludes `description`, matching the structural score and the
verdict-id hash, so a flag-on run reads no surface text either.
The flag is accepted in both run modes, but in single-project mode it **requires** `--bundle-dir`
and `--verdict-dir`: without them it cannot take effect, and the run is refused rather than
silently ignoring the flag. Nothing about a flag-off run changes, and no savings claim depends
on it.
A bundle may hold verdicts about **several candidates**, while its `validator-input.json`
describes only one. A `type: verdict` file therefore may declare its own retrieval key in
frontmatter — `affected_codes`, `measure_type`, `claimed_saving_nok` — and is keyed on that;
omit them and it falls back to the bundle's candidate, exactly as before. The three are **all or
nothing**: a partial declaration is refused rather than merged with the bundle candidate, since
the merge would produce a key belonging to neither. `promote_verdict` writes all three, so a
promoted verdict about one candidate never surfaces for another.
A bundle may also ship a **`cost-baseline.json`** — the project's actual cost lines,
`{code: {quantity, unit_cost}}` — and when it does, the deterministic validator reconciles every
affected item of a proposal against it before anything else runs. A cost code the project does not
have is rejected, and so is a real code carrying a quantity or unit cost outside the configured
tolerance (5% by default, relative to the baseline value). Without it, every stage of the gate
reasons only about numbers the proposal supplied itself, so an internally consistent hallucination
passes. The reconciliation validates; it never repairs a proposal into the baseline. A bundle that
ships no baseline is simply un-anchored and runs exactly as before, while a baseline that is
present but malformed is an error rather than a silent fall-back to un-anchored. On the
reference-domain (non-bundle) path the project's own cost items are the baseline, so those runs
are always anchored.
**An un-anchored run says so.** Every run stamps `provenance.cost_baseline_anchored` (a required
boolean, so no stamp can omit it), which reaches the outbox in `{run_id}-proposal.json`; and when
a run is un-anchored the CLI prints one line naming the skipped stage — on `--live-dry-run`, on a
full single run, and per project in portfolio mode. An anchored run prints no such line at all:
a line for something the run does not have is omitted rather than rendered empty. Anchoring stays
optional; this is visibility, not a new refusal.
The **prior-verdict fold — the learning step — happens only on the `--bundle-dir` path**; a
plain `--docs-dir`-only run is single-shot (no fold). `--decision`/`--rationale` apply to the
single-project path only and are **refused** in portfolio and report mode rather than ignored —
a portfolio pass takes each project's verdict from its own row, so a run-level verdict flag has
nowhere to go, and silently dropping a judgement an expert actually typed is the failure this
partition exists to prevent. **`--outbox-dir` must differ from
`--verdict-dir`**: writing the raw outbox into a folder later read as an inbox would re-ingest
raw agent output past the promotion gate (self-contamination) — documented here, deliberately
not CLI-enforced. Stop criteria and budget caps are required at startup. Try the offline
end-to-end proof (no model, no network): `uv run python -m portfolio_optimiser.simulation`.
- **A global token cap across the whole portfolio, enforced before the call.** Per-run caps alone
let N projects cost N times that with no ceiling over the pass. Pass a
`PortfolioMeter(PortfolioBudget(max_total_tokens=…, max_tokens_per_run=…))` to `run_portfolio`
and one ledger bounds the entire pass — and, seeded from `budget.read_spend`, a *series* of
passes. It bites in three places: a remainder that cannot fund one run refuses the pass at
startup (`BudgetRefused`); a project that cannot be funded is **never started**, stopping the
pass structurally (`budget_stop`, completed runs preserved); and a chat call the remainder
cannot pay for is **refused rather than made** (the post-charge check remains, since real usage
is only knowable after the response). Spend persists via `budget.write_spend`, which takes an
explicit stamp and no wall-clock default, so the file is byte-deterministic. Python API only —
not yet exposed on the CLI.
## What this enables
The reference case is portfolio cost review (the example bundle is a building-energy measure),
but the architecture is designed to generalize to any setting with the same shape — candidate
measures inside independent projects, numbers a deterministic tool can check, and judgement
only an expert has:
- **Portfolio reviews** — cost savings, energy efficiency, maintenance and procurement
measures, proposed per project and validated against the project's own data.
- **Compounding organizational memory** — approved expert verdicts become navigable knowledge;
the next run's hypotheses start from what experts actually decided, including realization
gaps no solver can compute.
- **Auditable AI** — an unbroken provenance chain from expert decision back through proposal,
bundle file and text span, and (with ingest) to the source system, query, and timestamp.
- **Vendor-neutral knowledge** — the same bundles drive two different agent stacks; switching
frameworks does not orphan the organization's curated knowledge.
## The task API — runnable Python, no wrapper
`main.py` is the ONE process entry. It serves the hosted-agent runtime contract (port 8088 /
`PORT`, `GET /readiness`, `POST /invocations`, clean SIGTERM exit) from
`portfolio_optimiser.hosting`, wrapping `run_project` — never `Workflow.as_agent()`, which
would bypass the deterministic validator and the rest of the gate chain. No protocol library
is used: the contract is a small stdlib-asyncio server instead (single event loop, no threads —
the same concurrency model as the portfolio waves). Note that the version reason this choice was
originally made for is retired: the prerelease `agent-framework-foundry-hosting` requires
`agent-framework-core>=1.13.0`, and since F15 (2026-09-02) this tree locks 1.16.0. Whether to adopt
that package is an open question this bump deliberately did not reopen — the stdlib server is what
is measured and shipped.
```bash
uv sync --frozen --no-dev # the exact locked resolution every measurement ran against
uv run python main.py # serves the task API
```
`git` must be on PATH for the install: two dependencies are git-tag-pinned direct references,
and wheel metadata alone cannot fetch them.
A `Dockerfile` and an `azure.yaml` shipped here until **14 August 2026** and were removed on an
operator directive after an external trial: what is delivered is runnable Python, and how the
process is containerised, supervised or deployed belongs to whoever runs it. Git history keeps
both files. The raw-text gate that pinned them (`--platform linux/amd64`, one copy of the start
command) was **deleted with them** rather than weakened into a check that could only pass — the
start command now has exactly one copy, in [`DEPLOY.md`](DEPLOY.md), and
`tests/test_handover_package_loadbearing.py` is what keeps it there.
An invocation is a JSON object whitelisted onto `run_project`'s signature — `project_id` and
`docs_dir` required; `verdict_input`, `bundle_dir`, `profile`, `max_rounds`, `max_tokens`
and `top_k` optional. `verdict_input` was required until 1.1.0, which forced an external caller to
invent an expert verdict just to get a run at all; omitting it now means nobody reviewed the run,
and the response's `verdict_id` is the key one would arrive under. Unknown fields are refused by name (400), never silently dropped.
`profile` defaults to `azure` on this surface: the AZURE profile reads its endpoint and
credential from the environment at call time, so the same process runs hosted (managed identity)
and locally (`az login`) without rewiring.
### Handing it to someone else
`scripts/make-handover-package.sh` builds one archive a receiving organisation can deploy without
cloning this repository or having an account here:
```bash
scripts/make-handover-package.sh # → dist/portfolio-optimiser-foundry-.zip
```
The archive is `git archive HEAD` — tracked files only, which is why local-only files cannot enter
it and why nothing curates what a receiver sees. [`DEPLOY.md`](DEPLOY.md) rides
along inside it and answers the receiver's first questions: what the three roles do, what the
process is end to end, why there is no chat interface, and the two environment variables that decide
whether the first deployment works. Gated by `tests/test_handover_package_loadbearing.py`.
## Docs
- [Bestille en kjøring](docs/bestille-en-kjoring.md) *(norsk)* — for the domain expert who
COMMISSIONS a run: naming the approaches the run must evaluate (and/or asking the system for its
own), stating what the run is for, and reading the announcement it prints before spending
anything and the settlement it prints afterwards. The commission directs what is *evaluated*,
never what is *approved*.
- [Ekspert-svar](docs/ekspert-svar.md) *(norsk)* — for the domain expert who has to deliver the
verdict: where a judgement goes, what an approval, an approval-with-correction and a rejection
actually look like, and paste-ready examples of each. Marked throughout as AI-authored and not
verified professional judgement.
- [Building a knowledge base](docs/knowledge-base-recipe.md) — the team recipe (technical +
domain expert) for curating a bundle, with the honest expectation that a good base takes 1–2
weeks of dedicated work.
- [Kunnskapsbase for én kjøring](docs/kunnskapsbase-for-en-kjoring.md) *(norsk)* — how to compose
the base for ONE specific run: which categories of knowledge follow the project, the domain and
the organisation; a content-type table (owner, delivery form, role in the loop, what happens
when it is missing); and a worked road project from the commission to a base that passes the
dry-run check. Every technical claim is marked verified or assumed.
- [OKF consumption contracts](docs/okf-konsum-kontrakter.md) — the three cross-repo facts this
consumer and the producer are both held to: the falsification threshold, the adjudication
state whose absence is `unknown` rather than `absent`, and the `(bundle_id, concept_id)`
identity pair with its three resolution origins. Every number carries its command.
- [Target picture](docs/plan/2026-06-26-maalbilde-agentic-loop.md) — the agentic loop + OKF
knowledge architecture (north star).
- [Prior-art & platform research](docs/research/2026-06-23-prior-art-platform.md) (incl.
implementation register §15).
- [Ingest target picture](docs/plan/2026-07-03-maalbilde-ingest-lag.md) — connectors and the
ingest layer (frozen 2026-07-03).
## Stack & develop
Python ≥3.10 · MAF via the split GA packages (see `pyproject.toml`) · `uv`. Backend profiles:
Azure/Foundry (full) + local (fallback).
```bash
uv sync
uv run pytest
uv run ruff check .
```
### Frozen knowledge bases
The measurements that read a delivered corpus (the v1 gate's rows 6–7, the stress judge, and four
corpus tests) read a **frozen copy** under `~/corpora/po-frosne-bundles/-/`,
outside every repository, pinned by `src/portfolio_optimiser/frozen_bundles.json`. The bundles
themselves are never committed here; only the pin is. A copy that no longer matches its pin fails
loudly (`frosset bundle … avviker fra pin`), and a copy that is absent is `IKKE MÅLT`, never green.
**Renewing a copy is a decision, not maintenance: a new copy and a new pin go in the SAME commit.**
Re-copy the base, recompute with `portfolio_optimiser.frozen_bundles.digest_bundle`, rename the
directory to carry the new short digest, and update `frozen_bundles.json` in that same change.
`--bundle-root` (or `PORTFOLIO_VEGNORMAL_ROOT`) stays as an explicit, **unpinned** live mount — the
way to look at a fresh corpus before deciding to refreeze.