open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(p16): --max-rounds/--max-tokens -- the cap on a PAID run had no operator door

Browse source 
B2's free drill earned its keep on the first command. STATE.md, docs/2026-09-12-p14-kontekstsett.md
and the order all publish the same stress command ending "--max-rounds 8 --max-tokens 120000".
Measured: run.py accepts neither, all four --live-dry-run drills refused with "unrecognized
arguments", and main() never passed max_rounds/max_tokens to run_project at all -- so every CLI run
ever made was silently bound to _DEFAULT_MAX_ROUNDS=3 / _DEFAULT_MAX_TOKENS=100_000, with no way to
raise or lower the cap on a run being paid for. Three surfaces described a door that did not exist.

Widening, never breaking: both flags default to exactly those values, so every existing invocation
is byte-identical. Wired to BOTH dispatches -- run_portfolio takes the same two parameters and
main() dropped them there too -- and refused by name in report mode, which returns above every
dispatch (the F4 silent-drop gap).

Load-bearing MEASURED (6 arms, ALL RED before the fix), four mutations all red, green control
1669/5 (from 1663/5, superset, 0 removed), golden BYTE-UNCHANGED (ea8c534...).

MEASURED, REPORTED, NOT FIXED: single-project mode still requires --docs-dir even when
--bundle-dir is given and docs_dir is unused on the bundle path, so the documented command would
have refused for that reason too. The README's own form works; loosening the guard is its own call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-14 12:10:59 +02:00
commit 5f94c92476
4 changed files with 198 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

22
CLAUDE.md
View file

@ -2444,6 +2444,28 @@ Python ≥3.10. MAF (`agent-framework-core` 1.16.0, `-orchestrations` 1.1.1 —
den, ikke at modellen sa hvorfor (det leses for hand og rapporteres merket manuelt); og en
helbase-snippet som baerer en `ref` er svakere bevis enn modellens eget `measure` — derfor er de
to halvdelene skilt i utfallet.
- **Taket pa en BETALT kjoring hadde ingen operatorflate, og tre flater beskrev en dor som ikke
fantes (P16 B2, 14.09):** `STATE.md`, `docs/2026-09-12-p14-kontekstsett.md § 4.1` og ordre
`20260914T091846Z` publiserer alle den samme stresskommandoen, som ender `--max-rounds 8
--max-tokens 120000`. MALT: `run.py` tok ingen av dem, alle fire gratis `--live-dry-run` nektet
med `unrecognized arguments`, og `main()` sendte **aldri** `max_rounds`/`max_tokens` videre — sa
HVER CLI-kjoring noensinne var stille bundet til `_DEFAULT_MAX_ROUNDS=3` /
`_DEFAULT_MAX_TOKENS=100_000`, uten mate for en operator a heve eller senke taket pa noe de
betalte for. Fase-3-klassen (en pastand flaten gjor om seg selv), spredt over operatorens egne
instruksjoner. **UTVIDELSE, aldri bryting:** begge flagg DEFAULTER til nettopp de verdiene, sa
hver eksisterende invokasjon er byte-identisk; det som endrer seg er at taket kan UTTALES.
**Wiret til BEGGE dispatcher** (`run_project` OG `run_portfolio` — sistnevnte tar de samme to
parameterne og `main()` droppet dem der ogsaa, sa et portefoljepass kunne heller ikke kappes), og
**nektet i report-modus VED NAVN**: report-modus returnerer OVER hver dispatch, sa en utelatelse
er et stille DROPP, ikke en nekt (F4-gapet) — og de er skillbare kun fordi argparse-defaulten ER
den historiske verdien. Load-bearing MALT (`tests/test_budget_flags_loadbearing.py`, 6 armer, ALLE
RODE for fiksen), fire mutasjoner alle rode + gronn kontroll **1669/5** og golden BYTE-UENDRET:
M12 flaggene nar aldri `run_project` (2) · M13 nar aldri `run_portfolio` (1) · M14 droppet fra
`report_forbidden` (2) · M15 defaulten drifter fra det kjoringene brukte (3). **Aerlighets-grense,
MALT og IKKE fikset:** enkeltprosjekt-modus krever fortsatt `--docs-dir` (`run.py:2953`) selv nar
`--bundle-dir` er oppgitt og `docs_dir` er ubrukt pa bundle-stien — den dokumenterte kommandoen
ville nektet av DEN grunnen ogsaa; READMEs egen form (`--docs-dir <bundle> --bundle-dir <bundle>`)
er det som virker, og a lose opp guarden er en egen operatorbeslutning.
- **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet.
- Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase.

5
README.md
View file

@ -629,6 +629,11 @@ when the seam is detached, so the loop cannot silently degrade into theater.
--derive-cost-baseline
```
- **Capping a run** — `--max-rounds` / `--max-tokens`. Both default to the values a CLI run was
always bound to (3 rounds, 100 000 tokens), so nothing changes unless you state a cap; stating one
is how you raise or lower what a paid run may spend. They apply to `--portfolio` too, and are
refused by name in `--report` mode, where a cap would bind nothing.
- **Judging a finished run against a fasit** — `python -m portfolio_optimiser.stress`. A run's
outbox already carries the evidence (which documents the debate opened, which files the stamp
cites, which approaches validated). This reads it against a context set's own `fasit.json` and

26
src/portfolio_optimiser/run.py
View file

@ -2656,6 +2656,21 @@ def main(argv: list[str] | None = None) -> int:
help="the expert's reasoning behind --decision (required with it; an expert verdict is a "
"decision AND its reasoning)",
)
parser.add_argument(
"--max-rounds",
type=int,
default=_DEFAULT_MAX_ROUNDS,
help=(
"per-run round cap (P16 B2). DEFAULTS to what every CLI run was implicitly bound to "
"before this flag existed, so nothing changes unless it is stated"
),
)
parser.add_argument(
"--max-tokens",
type=int,
default=_DEFAULT_MAX_TOKENS,
help="per-run token cap (P16 B2). Same default, same reason as --max-rounds",
)
parser.add_argument(
"--live-dry-run",
action="store_true",
@ -2819,6 +2834,11 @@ def main(argv: list[str] | None = None) -> int:
# F4 class exactly: report mode returns above every dispatch, so an omission here is a
# silent drop and not a refusal.
"--mandate": args.mandate is not None,
# P16 B2, same rung: a cap stated in report mode binds nothing, and report mode returns
# above the dispatch that would have honoured it. Distinguishable only because the
# argparse default IS the historic value, so "stated" and "omitted" differ.
"--max-rounds": args.max_rounds != _DEFAULT_MAX_ROUNDS,
"--max-tokens": args.max_tokens != _DEFAULT_MAX_TOKENS,
}
if any(report_forbidden.values()):
print(
@ -3787,6 +3807,8 @@ def main(argv: list[str] | None = None) -> int:
client_factory=scripted_client_factory,
mandate=mandate,
mcp_servers=mcp_servers,
max_rounds=args.max_rounds,
max_tokens=args.max_tokens,
)
)
except (ValueError, FileNotFoundError, ValidationError) as exc:
@ -3867,6 +3889,8 @@ def main(argv: list[str] | None = None) -> int:
if args.embedder_config
else None
),
max_rounds=args.max_rounds,
max_tokens=args.max_tokens,
outbox_dir=args.outbox_dir,
run_id=args.run_id,
prepass_payload=prepass_payload,
@ -3943,6 +3967,8 @@ def main(argv: list[str] | None = None) -> int:
if args.embedder_config
else None
),
max_rounds=args.max_rounds,
max_tokens=args.max_tokens,
outbox_dir=args.outbox_dir,
run_id=args.run_id,
prepass_payload=prepass_payload,

145
tests/test_budget_flags_loadbearing.py Normal file
View file

@ -0,0 +1,145 @@
"""P16 B2 - the CLI had NO door onto a paid run's cap, and the documented command proved it.
**The measured silence.** ``STATE.md``, ``docs/2026-09-12-p14-kontekstsett.md § 4.1`` and order
``20260914T091846Z`` all publish the same stress command, ending ``--max-rounds 8 --max-tokens
120000``. Measured 14.09: ``run.py`` accepts neither flag, all four free ``--live-dry-run`` drills
refused with ``unrecognized arguments``, and ``main()`` never passed ``max_rounds``/``max_tokens``
to ``run_project`` at all - so **every CLI run ever made was silently bound to the defaults**
(``max_rounds=3``, ``max_tokens=100_000``) with no way for an operator to raise or lower the cap on
a run they were paying for. Three surfaces described a door that did not exist: the Fase-3 class
(a claim the surface makes about itself), spread across the operator's own instructions.
**Widening, never breaking.** Both flags DEFAULT to the values ``run_project`` already used, so
every invocation that exists is byte-identical; what changes is that the cap can now be stated.
**Wired to BOTH dispatches, and refused in report mode.** ``run_portfolio`` takes the same two
parameters and ``main()`` dropped them there too, so a portfolio pass could not be capped either.
Report mode returns ABOVE every dispatch, so a flag left out of ``report_forbidden`` is a SILENT
DROP rather than a refusal - the gap F4 measured on ``--plan-review``.
Arms: (a) the flags parse and reach ``run_project`` * (b) they reach ``run_portfolio`` *
(c) the defaults are unchanged when the flags are absent * (d) report mode refuses each by name *
(e) the free dry-run drill accepts the documented command form.
"""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
from typing import Any
import pytest
from portfolio_optimiser import run as run_module
_REPO = Path(__file__).resolve().parents[1]
def _base(root: Path) -> Path:
base = root / "b"
(base / "krav").mkdir(parents=True)
(base / "index.md").write_text("---\nbundle_id: b\n---\n\n- [c](krav/c.md)\n", encoding="utf-8")
(base / "krav" / "c.md").write_text(
'---\ntype: concept\ntitle: "C"\n---\n\nbody\n', encoding="utf-8"
)
return base
def _capture(monkeypatch: pytest.MonkeyPatch, target: str) -> dict[str, Any]:
"""Record the kwargs main() hands the named coroutine, then stop the run."""
seen: dict[str, Any] = {}
async def _fake(*args: Any, **kwargs: Any) -> Any:
seen.update(kwargs)
raise SystemExit(0)
monkeypatch.setattr(run_module, target, _fake)
return seen
def test_a_the_two_flags_reach_run_project(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
seen = _capture(monkeypatch, "run_project")
with pytest.raises(SystemExit):
b = str(_base(tmp_path))
run_module.main(
[
"P1",
"--profile",
"local",
"--docs-dir",
b,
"--bundle-dir",
b,
"--max-rounds",
"8",
"--max-tokens",
"120000",
]
)
assert seen["max_rounds"] == 8
assert seen["max_tokens"] == 120000
def test_b_the_two_flags_reach_run_portfolio(monkeypatch: pytest.MonkeyPatch) -> None:
seen = _capture(monkeypatch, "run_portfolio")
with pytest.raises(SystemExit):
run_module.main(
["--portfolio", "--profile", "local", "--max-rounds", "9", "--max-tokens", "77000"]
)
assert seen["max_rounds"] == 9
assert seen["max_tokens"] == 77000
def test_c_absent_flags_keep_the_values_every_run_so_far_used(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""Widening, never breaking: the defaults ARE what main() bound implicitly before."""
seen = _capture(monkeypatch, "run_project")
with pytest.raises(SystemExit):
b = str(_base(tmp_path))
run_module.main(["P1", "--profile", "local", "--docs-dir", b, "--bundle-dir", b])
assert seen["max_rounds"] == run_module._DEFAULT_MAX_ROUNDS == 3
assert seen["max_tokens"] == run_module._DEFAULT_MAX_TOKENS == 100_000
@pytest.mark.parametrize("flag,value", [("--max-rounds", "8"), ("--max-tokens", "120000")])
def test_d_report_mode_refuses_each_flag_by_name(
flag: str, value: str, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
ledger = tmp_path / "l.json"
ledger.write_text("[]", encoding="utf-8")
# The control: the SAME argv without the flag is ACCEPTED, so rc 1 below is the mutant's
# opposite outcome and not the fixture refusing for a reason of its own.
assert run_module.main(["--report", "--ledger", str(ledger)]) == 0
assert run_module.main(["--report", "--ledger", str(ledger), flag, value]) == 1
assert "mode-exclusive" in capsys.readouterr().err
def test_e_the_documented_dry_run_command_form_is_accepted(tmp_path: Path) -> None:
"""The B2 drill itself: the command STATE.md and the order publish must at least parse."""
base = _base(tmp_path)
proc = subprocess.run(
[
sys.executable,
"-m",
"portfolio_optimiser.run",
"P1",
"--profile",
"local",
"--docs-dir",
str(base),
"--bundle-dir",
str(base),
"--max-rounds",
"8",
"--max-tokens",
"120000",
"--live-dry-run",
],
capture_output=True,
text=True,
cwd=_REPO,
)
assert "unrecognized arguments" not in proc.stderr, proc.stderr
assert proc.returncode == 0, proc.stderr