open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(cli): --mandate — a run announces what it will do, and settles for it

Browse source 
Krav 2: a run must be clear about what it shall do and achieve. `--mandate`
makes both ends explicit. BEFORE the first (paid) model call the run prints what
it was commissioned to do — objective, every named approach, whether its own
proposals are allowed, the scope, the caps, and which external services it will
contact. AFTER the run it settles: one row per approach, validated with the
figure, rejected with the validator's reason, or not evaluated with why.

The announcement is placed with the other refusals and above the scripted
banner, for the reason the required-args guard was hoisted there: a refused run
must not first print a banner about work it never did. `--live-dry-run`
announces without contacting anything, so a commission can be inspected before
it costs money.

`settle` renders the goal verdict it is GIVEN and never decides it. `ledger.to_ore`
is the framework's one NOK->øre conversion and the goal comparison already runs on
quantised integers, but `mandate.py` cannot import it without dragging `verdicts`
— and therefore agent_framework — into a deliberately framework-neutral module,
while a private copy of a money conversion is exactly the (p) defect. So the
caller decides and this renders; a goal figure without a decided verdict makes no
claim at all.

The caps the announcement prints come from named constants shared with
`run_project`/`run_portfolio`'s defaults — a second copy could drift and make the
announcement describe a run that never happened.

Load-bearing MEASURED against the whole 717-test suite, four mutations all red:
detach the announcement (4) · detach the settlement print (2) · make the CLI
loader tolerant (2) · announce the commission but never hand it to the run (2).
The last one is the one that matters: without it, a run could print a commission
it had no intention of executing.

DEVIATION from the approved plan, stated rather than quietly dropped: --goals is
still refused outside portfolio mode. Accepting it in single-project mode would
have admitted a flag whose documented function (the goal-stop against the ledger)
still does nothing there — the same accepted-but-inert defect --embedder-config
was just fixed for. The mandate's success_criteria carries "what shall this run
achieve" in the expert's own words instead; the numeric target stays portfolio-level.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULCqjLF61rehj5cZmdUoR3
This commit is contained in:
Kjell Tore Guttormsen 2026-08-05 16:23:53 +02:00
commit b9d795307a
4 changed files with 388 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

43
src/portfolio_optimiser/mandate.py
View file

@ -181,3 +181,46 @@ def announce(
if mandate.success_criteria:
lines.append(f" Success: {mandate.success_criteria}")
return "\n".join(lines)
def settle(
coverage: tuple[ApproachOutcome, ...],
*,
goal_nok: float | None = None,
goal_reached: bool | None = None,
) -> str:
"""Render the settlement: what the run actually did about each commissioned approach.
Empty coverage renders an EMPTY string — a run without a mandate has nothing to settle, and a
header over zero rows would imply a commission that never existed.
The ``Validated total`` is a DISPLAY figure, summed in float NOK exactly like
``PortfolioResult.sum_claimed_saving_nok``, and it decides nothing. The goal verdict is
**passed in**, never computed here: ``ledger.to_ore`` is the framework's one NOK->øre
conversion and the goal comparison already runs on quantised integers, but this module cannot
import it without pulling ``verdicts`` — and with it ``agent_framework`` — into a deliberately
framework-neutral file, while a private copy of a money conversion is precisely the ``(p)``
defect. So the caller decides and this renders. Both ``goal_nok`` and ``goal_reached`` must be
given for the target line to appear; a figure without a decided verdict makes no claim.
"""
if not coverage:
return ""
lines = ["Mandate outcome"]
for row in coverage:
if row.status == "validated":
amount = f"{row.saving_nok:.0f} NOK" if row.saving_nok is not None else "-"
lines.append(f" {row.id:<20} VALIDATED {amount:>14} {row.label}")
elif row.status == "rejected":
lines.append(f" {row.id:<20} REJECTED {row.detail}")
else:
lines.append(f" {row.id:<20} NOT EVALUATED {row.detail}")
total = sum(r.saving_nok or 0.0 for r in coverage if r.status == "validated")
lines.append(f" Validated total: {total:.0f} NOK")
if goal_nok is not None and goal_reached is not None:
lines.append(
f" Target: >= {goal_nok:.0f} NOK — "
f"{'reached' if goal_reached else 'not reached'}"
)
return "\n".join(lines)

78
src/portfolio_optimiser/run.py
View file

@ -60,6 +60,9 @@ from portfolio_optimiser.mandate import (
Approach,
ApproachOutcome,
Mandate,
announce,
load_mandate,
settle,
)
from portfolio_optimiser.provenance import ProvenanceStamp
from portfolio_optimiser.reference_domain import Project, load_reference_projects
@ -91,6 +94,13 @@ from portfolio_optimiser.value_report import (
from portfolio_optimiser.workflow import _MAKER_CHECKER_ROLES, fresh_workflow
#: The caps a CLI run actually uses. Named constants rather than repeated literals because the
#: run announcement (Trekk A2) PRINTS them: a second copy could drift and make the announcement
#: describe a run that never happened.
_DEFAULT_MAX_ROUNDS = 3
_DEFAULT_MAX_TOKENS = 100_000
@dataclass(frozen=True)
class RunResult:
"""The outcome of one project run: the validated/rejected proposal, its first-class
@ -392,8 +402,8 @@ async def run_project(
outbox_dir: str | None = None,
run_id: str | None = None,
client_factory: Callable[[str], BaseChatClient] | None = None,
max_rounds: int = 3,
max_tokens: int = 100_000,
max_rounds: int = _DEFAULT_MAX_ROUNDS,
max_tokens: int = _DEFAULT_MAX_TOKENS,
top_k: int = 3,
enable_layer1_hitl: bool = False,
notify: Callable[[Verdict], None] | None = None,
@ -834,14 +844,15 @@ async def run_portfolio(
goals: GoalConfig | None = None,
store: VerdictStore | None = None,
client_factory: Callable[[str], BaseChatClient] | None = None,
max_rounds: int = 3,
max_tokens: int = 100_000,
max_rounds: int = _DEFAULT_MAX_ROUNDS,
max_tokens: int = _DEFAULT_MAX_TOKENS,
top_k: int = 3,
concurrency: int = 1,
meter_factory: Callable[[], TokenMeter] | None = None,
portfolio_meter: PortfolioMeter | None = None,
semantic_retrieval: bool = False,
embedder: Embedder | None = None,
mandate: Mandate | None = None,
) -> PortfolioResult:
"""Fan out over a portfolio of independent projects SEQUENTIALLY, composing ``run_project``
as-is (every project's execution state — meter, debate, retrieval context — is built fresh
@ -1038,6 +1049,7 @@ async def run_portfolio(
top_k=top_k,
semantic_retrieval=semantic_retrieval,
embedder=embedder,
mandate=mandate,
meter=_run_meter(meter_factory, portfolio_meter, max_rounds),
)
for pid, snapshot in snapshots
@ -1160,6 +1172,16 @@ def main(argv: list[str] | None = None) -> int:
help="fail-fast dimension scope config (JSON): scopes the run to one cost axis; a "
"missing or malformed file refuses the run (authoritative startup config, not a RAW inbox)",
)
parser.add_argument(
"--mandate",
default=None,
metavar="FILE",
help="run mandate (JSON, fail-fast): what a domain expert commissions this run to "
"evaluate — named approaches and/or the system's own — plus the objective and success "
"criteria. The run ANNOUNCES it before the first model call and SETTLES against it "
"afterwards, one row per approach. Valid in both modes; in portfolio mode it applies to "
"every project in the pass",
)
parser.add_argument(
"--embedder-config",
default=None,
@ -1397,6 +1419,19 @@ def main(argv: list[str] | None = None) -> int:
)
return 1
# The commission, loaded fail-fast BEFORE anything runs: a missing or malformed mandate is
# REFUSED rather than degraded to "no mandate", because the settlement would then describe work
# nobody ordered. Placed with the other refusals and ABOVE the scripted banner, for the same
# reason the required-args guard was hoisted there — a refused run must not first print a
# banner claiming a scripted loop closed.
mandate: Mandate | None = None
if args.mandate is not None:
try:
mandate = load_mandate(args.mandate)
except (FileNotFoundError, ValidationError, ValueError) as exc:
print(f"run refused: {exc}", file=sys.stderr)
return 1
# The scripted door (offline WHOLE-loop run over the caller's own data). Resolved BEFORE the
# dry-run branch so the two offline modes cannot both be honoured — and BEFORE the portfolio
# dispatch, because the door serves BOTH modes. It originally sat below that dispatch, which
@ -1432,6 +1467,29 @@ def main(argv: list[str] | None = None) -> int:
scripted_client_factory = scripted_factory(replies, [])
print(_SCRIPTED_BANNER)
if mandate is not None:
# The scope line reads the dimension config only to NAME it. A config that fails to load is
# left unnamed here and refused a moment later by the dispatch below, which stays the single
# owner of that refusal (and of its mode-specific wording) — announcing must never change
# which error an operator sees.
dimension_label: str | None = None
if args.dimension_config is not None:
try:
_dim = load_dimension(args.dimension_config)
except (FileNotFoundError, ValidationError, ValueError):
dimension_label = None
else:
dimension_label = f"{_dim.id} ({_dim.label})"
print(
announce(
mandate,
project_id=args.project_id or "the portfolio",
max_rounds=_DEFAULT_MAX_ROUNDS,
max_tokens=_DEFAULT_MAX_TOKENS,
dimension_label=dimension_label,
)
)
if args.portfolio:
# Portfolio mode (Step 3): dispatch to the EXISTING run_portfolio via the fail-fast loaders
# (run_portfolio itself is unchanged). Loader/ValueError failures surface through the same
@ -1456,6 +1514,7 @@ def main(argv: list[str] | None = None) -> int:
goals=goals,
semantic_retrieval=args.semantic_retrieval,
client_factory=scripted_client_factory,
mandate=mandate,
)
)
except (ValueError, FileNotFoundError, ValidationError) as exc:
@ -1463,6 +1522,11 @@ def main(argv: list[str] | None = None) -> int:
return 1
for r in portfolio_result.runs:
print(f"{type(r.outcome).__name__}: verdict id={r.verdict.id}")
# One settlement per project: the mandate applies to each project in the pass, so
# each project answers for it separately. Empty without a mandate.
project_settlement = settle(r.coverage)
if project_settlement:
print(project_settlement)
if portfolio_result.stop_reason is not None:
sr = portfolio_result.stop_reason
print(
@ -1574,6 +1638,7 @@ def main(argv: list[str] | None = None) -> int:
verdict_input={"decision": args.decision, "rationale": args.rationale},
semantic_retrieval=args.semantic_retrieval,
client_factory=scripted_client_factory,
mandate=mandate,
)
),
)
@ -1584,6 +1649,11 @@ def main(argv: list[str] | None = None) -> int:
return 1
kind = type(result.outcome).__name__
print(f"{args.project_id}: {kind} (verdict id={result.verdict.id}, decision={args.decision})")
# The settlement against the commission (Trekk A4). Empty without a mandate, so an
# un-commissioned run prints exactly what it printed before.
settlement = settle(result.coverage)
if settlement:
print(settlement)
return 0

85
tests/test_mandate.py
View file

@ -24,9 +24,11 @@ from pydantic import ValidationError
from portfolio_optimiser.mandate import (
OWN_PROPOSAL_ID,
Approach,
ApproachOutcome,
Mandate,
announce,
load_mandate,
settle,
)
@ -222,3 +224,86 @@ def test_announce_states_the_caps() -> None:
def test_announce_is_deterministic() -> None:
"""Byte-stable: no wall clock, no set-iteration ordering — so it can be golden-tested."""
assert _announce() == _announce()
# --- the settlement (Trekk A4): what the run actually did about each approach -------------------
_ROWS = (
ApproachOutcome(id="led-retrofit", label="LED", status="validated", saving_nok=30_000.0),
ApproachOutcome(
id="service-contract",
label="Contract",
status="rejected",
detail="claimed 200000 exceeds feasible 90000",
),
ApproachOutcome(
id="night-setback",
label="Setback",
status="not_evaluated",
detail="budget exhausted before this approach was evaluated",
),
)
def test_settle_reports_every_row_including_the_unevaluated_one() -> None:
"""All three statuses reach the report. The unevaluated row is the point: an approach the run
never got to must be visibly unreached, not absent."""
text = settle(_ROWS)
assert "led-retrofit" in text
assert "service-contract" in text
assert "night-setback" in text
assert "NOT EVALUATED" in text
def test_settle_carries_the_rejection_reason() -> None:
"""A rejected approach without its reason tells the expert nothing they can act on."""
assert "claimed 200000 exceeds feasible 90000" in settle(_ROWS)
def test_settle_totals_only_the_validated_savings() -> None:
"""The total is what passed the validator — never the claimed sum of everything attempted."""
text = settle(_ROWS)
assert "30000" in text
assert "200000 NOK" not in text # the rejected claim is never folded into a total
def test_settle_states_whether_the_target_was_reached() -> None:
"""Krav 2's second half: the run answers, in its own output, whether it achieved what it was
commissioned to achieve."""
missed = settle(_ROWS, goal_nok=150_000.0, goal_reached=False)
hit = settle(_ROWS, goal_nok=10_000.0, goal_reached=True)
assert "not reached" in missed
assert "reached" in hit and "not reached" not in hit
def test_settle_renders_the_goal_verdict_it_is_GIVEN_and_never_decides_it() -> None:
"""The renderer must not form a SECOND opinion on a money question.
``ledger.to_ore`` is the framework's one NOK->øre conversion and the goal comparison already
runs on quantised integers; this module cannot import it without dragging ``verdicts`` — and
therefore ``agent_framework`` — into a deliberately framework-neutral file, and a private copy
of a money conversion is exactly the ``(p)`` defect. So the caller decides and this renders.
Told the opposite of what its own float sum suggests, it prints what it was told.
"""
text = settle(_ROWS, goal_nok=150_000.0, goal_reached=True)
assert "not reached" not in text
def test_settle_omits_the_target_line_when_no_goal_was_configured() -> None:
"""No goal configured -> no verdict on a goal. An absent target must not read as a missed one."""
assert "Target" not in settle(_ROWS)
# ...and a goal figure without a decided verdict renders no claim either.
assert "Target" not in settle(_ROWS, goal_nok=150_000.0)
def test_settle_is_empty_without_coverage() -> None:
"""No mandate -> nothing to settle. An empty block beats a header over zero rows, which would
imply a commission that never existed."""
assert settle(()) == ""
def test_settle_is_deterministic() -> None:
"""Byte-stable, like the announcement."""
assert settle(_ROWS, goal_nok=150_000.0, goal_reached=False) == settle(
_ROWS, goal_nok=150_000.0, goal_reached=False
)

186
tests/test_mandate_cli.py Normal file
View file

@ -0,0 +1,186 @@
"""The CLI door onto the mandate (Trekk A6): ``--mandate <file.json>``.
Krav 1 and 2 are only real if a domain expert can reach them from a command line. This pins the
three visible halves:
1. the commission is loaded FAIL-FAST — a missing or malformed mandate refuses the run (rc 1),
because a silently degraded commission would make the settlement describe work nobody ordered;
2. the run ANNOUNCES what it will do BEFORE it does it, and the announcement precedes the outcome
line in the output (an after-the-fact statement of intent is not a statement of intent);
3. the run SETTLES against that commission afterwards, naming every approach.
The control asserts both blocks are absent without ``--mandate`` — so no assertion above can pass
on a constant the CLI prints anyway.
"""
from __future__ import annotations
import json
import shutil
from pathlib import Path
import pytest
from portfolio_optimiser import run
BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
_VALID_PROPOSAL = (
'{"measure":"LED-retrofit av kontorbelysning","affected_items":'
'[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":30000}'
)
_CHECKER_APPROVE = "Tallene er innenfor feasibelt område og resonnementet holder. VERDICT: APPROVE"
_MANDATE = {
"objective": "Kutt energikostnad uten ombygging i 2026.",
"approaches": [
{
"id": "led-retrofit",
"label": "LED-retrofit av kontorbelysning",
"description": "Drift mener armaturene er originale.",
}
],
"allow_own_proposals": False,
"success_criteria": "Minst ett tiltak som passerer validatoren.",
}
@pytest.fixture(autouse=True)
def _isolate_model_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("PORTFOLIO_MODEL_MAP", raising=False)
monkeypatch.delenv("PORTFOLIO_FOUNDRY_PROJECT_ENDPOINT", raising=False)
@pytest.fixture()
def bundle(tmp_path: Path) -> Path:
"""A throwaway COPY — the commons-owned fixture is never mutated by a test."""
dst = tmp_path / "bundle"
shutil.copytree(BUNDLE_DIR, dst)
return dst
@pytest.fixture()
def replies_file(tmp_path: Path) -> Path:
path = tmp_path / "replies.json"
path.write_text(
json.dumps({"proposer": _VALID_PROPOSAL, "checker": _CHECKER_APPROVE}), encoding="utf-8"
)
return path
@pytest.fixture()
def mandate_file(tmp_path: Path) -> Path:
path = tmp_path / "mandate.json"
path.write_text(json.dumps(_MANDATE), encoding="utf-8")
return path
def _argv(bundle: Path, replies: Path, *extra: str) -> list[str]:
return [
"BYGG-KONTOR-NORD",
"--docs-dir",
str(bundle),
"--bundle-dir",
str(bundle),
"--scripted-replies",
str(replies),
*extra,
]
def test_missing_mandate_refuses_the_run(bundle, replies_file, capsys) -> None:
"""A mandate that is not there refuses — never treated as 'no mandate given'."""
rc = run.main(_argv(bundle, replies_file, "--mandate", str(bundle / "nope.json")))
assert rc == 1
assert "refused" in capsys.readouterr().err.lower()
def test_malformed_mandate_refuses_the_run(bundle, replies_file, tmp_path, capsys) -> None:
"""A malformed commission refuses too: authoritative startup config, not a tolerant inbox."""
bad = tmp_path / "bad.json"
bad.write_text('{"approaches": []}', encoding="utf-8") # no objective
rc = run.main(_argv(bundle, replies_file, "--mandate", str(bad)))
assert rc == 1
assert "refused" in capsys.readouterr().err.lower()
def test_run_announces_the_mandate_before_it_runs(
bundle, replies_file, mandate_file, capsys
) -> None:
"""The announcement names the objective and every commissioned approach, and it appears BEFORE
the run's outcome line — intent stated up front, not reported after the fact."""
rc = run.main(_argv(bundle, replies_file, "--mandate", str(mandate_file)))
out = capsys.readouterr().out
assert rc == 0, out
assert "Run mandate for BYGG-KONTOR-NORD" in out
assert "Kutt energikostnad uten ombygging i 2026." in out
assert "led-retrofit" in out
assert out.index("Run mandate for") < out.index("BYGG-KONTOR-NORD: ")
def test_run_settles_against_the_mandate_afterwards(
bundle, replies_file, mandate_file, capsys
) -> None:
"""The settlement names every commissioned approach and its status, AFTER the run."""
rc = run.main(_argv(bundle, replies_file, "--mandate", str(mandate_file)))
out = capsys.readouterr().out
assert rc == 0, out
assert "Mandate outcome" in out
assert "led-retrofit" in out.split("Mandate outcome")[1]
assert out.index("Run mandate for") < out.index("Mandate outcome")
def test_dry_run_announces_without_calling_a_model(bundle, mandate_file, capsys) -> None:
"""``--live-dry-run`` shows what the run WOULD do — the announcement is offline by
construction, so an operator can inspect a commission before paying for it."""
rc = run.main(
[
"BYGG-KONTOR-NORD",
"--docs-dir",
str(bundle),
"--bundle-dir",
str(bundle),
"--mandate",
str(mandate_file),
"--live-dry-run",
]
)
out = capsys.readouterr().out
assert rc == 0, out
assert "Run mandate for BYGG-KONTOR-NORD" in out
assert "LIVE-DRY-RUN OK" in out
# A dry run stops before the first model call, so there is nothing to settle.
assert "Mandate outcome" not in out
def test_portfolio_mode_applies_the_mandate_to_every_project(
replies_file, mandate_file, capsys
) -> None:
"""The mandate applies to every project in a portfolio pass, and each project settles for it
separately — a threading line with no test of its own is the green-but-dead trap this repo
was bitten by in Fase 2."""
rc = run.main(
[
"--portfolio",
"--scripted-replies",
str(replies_file),
"--mandate",
str(mandate_file),
]
)
out = capsys.readouterr().out
assert rc == 0, out
assert "Run mandate for the portfolio" in out
settlements = out.count("Mandate outcome")
assert settlements >= 2, f"expected one settlement per project, saw {settlements}"
assert "led-retrofit" in out.split("Mandate outcome")[1]
def test_no_mandate_prints_neither_block(bundle, replies_file, capsys) -> None:
"""CONTROL: without ``--mandate`` the CLI output is the pre-Trekk-A one — no announcement, no
settlement. Every assertion above would otherwise be able to pass on constant output."""
rc = run.main(_argv(bundle, replies_file))
out = capsys.readouterr().out
assert rc == 0, out
assert "Run mandate for" not in out
assert "Mandate outcome" not in out