test(portfolio): cover the CLI→run_portfolio wire for --verdict-dir (§11 gap)

The gap, found by the mutation sweep of 2026-07-25: verdict_dir=args.verdict_dir
→ None in run.py's execute_portfolio call left the suite 603/603 GREEN. The
flag was wired but not guarded — the inner merge (test_portfolio_learning_
loadbearing.py), the argparse refusal (--verdict-dir without --portfolio) and
the README↔--help sync all stay green under that mutation, so none of them
covered the forwarding itself.

One load-bearing test, no production code. It authors an expert verdict into a
tmp portfolio inbox — keyed on the bundle's own codes + measure type so it ranks
into the fold, with a distinct saving so its id cannot collide with the bundle's
seed — drives main(["--portfolio", …, "--verdict-dir", X]) with the scripted
client, and asserts the verdict's id AND a marker token (present nowhere in the
bundle) reach the proposer prompt.

Detach proof (mutation restored from a COPY, never git checkout): the wire
mutated to None → 1 failed, 603 passed, and the failure is this test alone.
Restored → 604 passed, ruff format left 71 files unchanged, ruff check + mypy
--strict clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 12:23:45 +02:00
commit 25d9bc6fe8

View file

@ -8,7 +8,10 @@ makes them REACHABLE. A hard goal already met by the ledger stops the run
BEFORE any model call the goal bounds achievement where the §8 budget bounds
spend, and a stop is structured output, never a silent one. Detach the goal
check from the entrance the run proceeds and spends red. Detach the
portfolio branch the config's projects never run → red.
portfolio branch the config's projects never run → red. Stop forwarding
``--verdict-dir`` the expert inbox the operator named reaches no fold red:
the flag's refusal path and the README↔``--help`` sync leave that wire uncovered,
so it is asserted here on the fold itself.
**The README claims exactly what the CLI delivers (§1).** Every ``--flag`` the
README documents must exist in the help of a project CLI the README names.
@ -32,6 +35,8 @@ import pytest
from _scripted import ScriptedClient, reply
from portfolio_optimiser_claude.contracts import Contracts, FeedbackContract
from portfolio_optimiser_claude.experience import CandidateFeatures
from portfolio_optimiser_claude.inbox import VerdictDocument, write_verdict
from portfolio_optimiser_claude.ir import load_validator_input
from portfolio_optimiser_claude.ledger import SavingsLedger
from portfolio_optimiser_claude.loop import ModelClient, ModelReply
@ -97,6 +102,35 @@ def _ledger_file(tmp_path: Path, *, amount_nok: float) -> Path:
return path
# A token that lives NOWHERE in the bundle — its presence in a prompt can only
# have come through the inbox the operator named with --verdict-dir, never from
# the bundle context or from seeding.
_VERDICT_DIR_MARKER = "K12-CLI-VERDICT-DIR-MARKER"
def _portfolio_inbox(tmp_path: Path) -> tuple[Path, VerdictDocument]:
"""An EXPERT verdict in a portfolio inbox, keyed near the bundle's candidate.
Same codes + measure type as the bundle's proposal (so it ranks into the
fold) with a distinct saving (so its id cannot collide with the bundle's own
seed). The marker rides in the rationale the fold's learning signal.
"""
candidate = CandidateFeatures.from_proposal(load_validator_input(BUNDLE))
verdict = VerdictDocument.from_candidate(
CandidateFeatures(
affected_codes=candidate.affected_codes,
measure_type=candidate.measure_type,
claimed_saving_nok=candidate.claimed_saving_nok + 3000.0,
),
decision="approved",
rationale=f"prior portfolio verdict — realiseringskorreksjon [{_VERDICT_DIR_MARKER}]",
description="K12 CLI portfolio-inbox fixture (surface text, excluded from ranking)",
)
verdict_dir = tmp_path / "portfolio-inbox"
write_verdict(verdict_dir, verdict)
return verdict_dir, verdict
def _portfolio_file(tmp_path: Path, project_ids: list[str]) -> Path:
path = tmp_path / "portfolio.json"
path.write_text(
@ -263,6 +297,32 @@ class TestPortfolioOnTheEntrance:
main([], client_factory=factory)
assert created == []
def test_verdict_dir_reaches_every_projects_fold(self, tmp_path: Path) -> None:
# Detach point: stop forwarding args.verdict_dir out of main() (pass None
# to execute_portfolio) → the inbox the operator named never reaches a
# fold → RED. The merge itself is proven inside run_portfolio
# (test_portfolio_learning_loadbearing.py); what this keeps alive is the
# CLI→run_portfolio WIRE — the refusal test below and the README↔--help
# sync both stay green under that mutation, so neither covers it.
verdict_dir, inbox_verdict = _portfolio_inbox(tmp_path)
factory, created = _scripted_factory(runs=1)
code = main(
[
"--portfolio",
str(_portfolio_file(tmp_path, ["prosjekt-a"])),
"--verdict-dir",
str(verdict_dir),
],
client_factory=factory,
)
assert code == 0
assert created
prompts = created[0].prompts("proposer")
assert prompts, "the project must have driven the loop at all"
assert any(_VERDICT_DIR_MARKER in p and inbox_verdict.id in p for p in prompts), (
"the verdict from --verdict-dir (id + marker) must reach the project's fold"
)
def test_verdict_dir_without_portfolio_is_refused(self, tmp_path: Path) -> None:
# --verdict-dir is the PORTFOLIO-level expert inbox; on a single run the
# per-run inbox is --inbox. Accepting it silently would claim a wiring