feat(portfolio): K2 — sequential multi-project run (parity row 4)

New portfolio.py: run_portfolio drives N projects sequentially from a
schema-validated reference config, composing each project's §5 context
(merge inbox -> seed -> fold) and running the loop core UNCHANGED per
project, collecting one typed result per project IN CONFIG ORDER. This is
the run path MAF got in its Fase 1 and D7 never had — the prior entrances
(run.py, run_s10.py) drive a single bundle. PortfolioResult holds
per-project results tagged with the config project_id.

Re-entrancy (§3 Step 3): each project composes its OWN context inside the
loop, never a hoisted shared one, so nothing survives one project into the
next except the explicitly shared mutable state — the §8 budget meter, a
portfolio-wide cap. Failure policy is a STACK-LOCAL choice until D-D: the
default RAISES (today everything is thrown); K18 flips it to
collect-and-continue when the D-D wave model lands.

New config contract in contracts.py: ReferenceProjectContract (project_id +
required non-empty bundle_dir + optional inbox_dir) + ReferenceProjectsContract,
loaded fail-fast by load_reference_projects (§10) — a project without a
bundle path is refused before any run. New data/reference_projects.json
example (shape-validated, never executed by the suite). New repo-local
mini-bundle fixture under tests/data/ (a distinct second project, VFD-retrofit
— ALDRI in shared/).

Two detach proofs delivered: drop the bundle_dir Field requirement -> a run
starts on the invalid config and only crashes mid-run -> the fail-fast test
goes red; hoist the per-project composition out of the loop -> project 2 runs
on project 1's context and the VFD marker never reaches its prompt -> the
re-entrancy test goes red. 11 new tests (test_portfolio.py 5 +
test_contracts.py TestReferenceProjects 6). 426 -> 437 tests, golden
byte-exact, full gate clean (ruff + format + mypy strict). README synced
(test count + a Run layer module block).

[skip-docs] — README documents the new module; CLAUDE.md holds invariants
(rules/commands) only, and K2 adds no new invariant, command, or convention.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RiTwaKLesgcwXx2mDviqpt
This commit is contained in:
Kjell Tore Guttormsen 2026-07-23 21:39:39 +02:00
commit f2c64da9ee
10 changed files with 406 additions and 3 deletions

View file

@ -13,7 +13,7 @@ human-in-the-loop, and the system learns from the verdicts.
> **Status:** the D7 build (S5S10) is complete, and the deterministic **ingest layer** > **Status:** the D7 build (S5S10) is complete, and the deterministic **ingest layer**
> (CSV and SQL source types) has since been added in front of the loop. The deterministic > (CSV and SQL source types) has since been added in front of the loop. The deterministic
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by > backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
> seam, each proven by load-bearing tests (426 tests, all running offline without an API > seam, each proven by load-bearing tests (437 tests, all running offline without an API
> key). The programme's single budgeted **live model run has been executed and validated** > key). The programme's single budgeted **live model run has been executed and validated**
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below). > its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
@ -93,6 +93,11 @@ description, never from its code)
budget stop included. The model client is injected, so the offline suite proves the budget stop included. The model client is injected, so the offline suite proves the
same orchestration with a scripted client; only the CLI's default constructs the SDK same orchestration with a scripted client; only the CLI's default constructs the SDK
client. client.
- `portfolio.py` — the sequential multi-project run: `run_portfolio` drives N projects from
a schema-validated reference config, composing each project's context afresh (re-entrant,
fresh debate state per run) and collecting one result per project in config order. The
§8 budget meter is the one explicitly shared, portfolio-wide cap; the default failure
policy raises (a stack-local choice until D-D flips it to collect-and-continue).
- `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only. - `run_s10.py` — the programme's ONE live run (cost discipline D6); run-path only.
### Load-bearing tests (§11) ### Load-bearing tests (§11)
@ -168,7 +173,7 @@ Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk
```bash ```bash
uv sync # install dependencies uv sync # install dependencies
uv run pytest # 426 tests — run without any API key and without network uv run pytest # 437 tests — run without any API key and without network
uv run ruff check . && uv run ruff format --check . uv run ruff check . && uv run ruff format --check .
uv run mypy src # strict uv run mypy src # strict
``` ```

View file

@ -28,6 +28,7 @@ from typing import Annotated, Any, Literal
from pydantic import BaseModel, Field, model_validator from pydantic import BaseModel, Field, model_validator
_MODEL_MAP_RESOURCE = "data/model_map.json" _MODEL_MAP_RESOURCE = "data/model_map.json"
_REFERENCE_PROJECTS_RESOURCE = "data/reference_projects.json"
# C2.6: an empty-string model id is a startup schema error, never a client-layer one. # C2.6: an empty-string model id is a startup schema error, never a client-layer one.
_ModelId = Annotated[str, Field(min_length=1)] _ModelId = Annotated[str, Field(min_length=1)]
@ -67,6 +68,25 @@ class FeedbackContract(BaseModel):
rationale: str = Field(min_length=1) rationale: str = Field(min_length=1)
class ReferenceProjectContract(BaseModel):
"""One portfolio entry: a project id + its OKF bundle dir (+ an optional inbox).
The ``bundle_dir`` is REQUIRED and non-empty an entry without a bundle path
is a startup schema error (§10), never a run that starts and then crashes for
lack of a data source. ``inbox_dir`` is optional (the tolerant §5 inbox).
"""
project_id: str = Field(min_length=1)
bundle_dir: str = Field(min_length=1)
inbox_dir: str | None = None
class ReferenceProjectsContract(BaseModel):
"""The schema-validated portfolio config (§10: fail-fast before any run)."""
projects: list[ReferenceProjectContract] = Field(min_length=1)
class Contracts(BaseModel): class Contracts(BaseModel):
"""The validated bundle of all startup contracts.""" """The validated bundle of all startup contracts."""
@ -98,6 +118,27 @@ def _bundled_model_map() -> dict[str, Any]:
return raw return raw
def _bundled_reference_projects() -> dict[str, Any]:
raw: dict[str, Any] = json.loads(
files("portfolio_optimiser_claude")
.joinpath(_REFERENCE_PROJECTS_RESOURCE)
.read_text(encoding="utf-8")
)
return raw
def load_reference_projects(raw: dict[str, Any] | None = None) -> ReferenceProjectsContract:
"""Validate the portfolio config at startup (§10, fail-fast before any run).
Raises ``pydantic.ValidationError`` on the first malformed entry a project
without a bundle path never reaches ``run_portfolio``. ``raw`` defaults to the
bundled ``data/reference_projects.json`` EXAMPLE, whose shape (not its paths)
is validated; the suite never executes that example (offline invariant).
"""
data = _bundled_reference_projects() if raw is None else raw
return ReferenceProjectsContract(**data)
def load_contracts( def load_contracts(
data_source: dict[str, Any], data_source: dict[str, Any],
termination: dict[str, Any], termination: dict[str, Any],

View file

@ -0,0 +1,9 @@
{
"_note": "EXAMPLE portfolio config (contracts.ReferenceProjectsContract). Schema-validated fail-fast at startup (§10); the suite validates only its SHAPE and never executes it (offline invariant). bundle_dir/inbox_dir are illustrative paths relative to the repo root — operators point them at their own OKF bundles. The one shipped example bundle is shared/examples/bygg-energi-mikro; a real portfolio lists several.",
"projects": [
{
"project_id": "BYGG-KONTOR-NORD",
"bundle_dir": "shared/examples/bygg-energi-mikro"
}
]
}

View file

@ -0,0 +1,77 @@
"""Sequential multi-project portfolio run (method-spec §3, §5, §8; paritetsrad 4).
``run_portfolio`` drives N projects SEQUENTIALLY from a schema-validated reference
config. For each project it composes the §5 read-context (merge inbox seed
fold, via ``compose_run_context``) and runs the loop core UNCHANGED
(``run_project``), collecting one typed result per project IN CONFIG ORDER the
run path MAF got in its Fase 1 and D7 never had (the prior entrances,
``run_s10.py`` and ``run.py``, drive a single bundle).
Re-entrancy (§3 Step 3): the loop core keeps all debate state local to a run, so
nothing survives one project into the next EXCEPT the explicitly shared mutable
state the §8 budget ``meter``, a PORTFOLIO-WIDE cap threaded through every run.
Each project composes its OWN context inside the loop, never a hoisted, shared one.
Failure policy is a STACK-LOCAL choice until D-D: the default RAISES (today
everything is thrown a budget stop or any run error propagates and the portfolio
stops). K18 flips this to a collect-and-continue wave model when the D-D fasit
lands. Goals/ledger are out of scope here beyond being accepted as optional
arguments in a later session (K3/K11); this module only wires the run path.
"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from portfolio_optimiser_claude.budget import BudgetMeter
from portfolio_optimiser_claude.contracts import ReferenceProjectsContract
from portfolio_optimiser_claude.loop import ModelClient, RunResult, run_project
from portfolio_optimiser_claude.run import compose_run_context
@dataclass(frozen=True)
class ProjectRunResult:
"""One project's outcome in the portfolio, tagged with its config ``project_id``."""
project_id: str
run: RunResult
@dataclass(frozen=True)
class PortfolioResult:
"""The portfolio's per-project results, in config order."""
results: list[ProjectRunResult]
def run_portfolio(
projects: ReferenceProjectsContract,
client: ModelClient,
meter: BudgetMeter,
*,
top_k: int,
max_debate_rounds: int,
max_attempts: int,
) -> PortfolioResult:
"""Run each configured project through the loop, collecting results in config order.
The ``projects`` config is ALREADY schema-validated (``load_reference_projects``,
§10) an invalid entry never reaches here. The shared ``meter`` is the
portfolio-wide §8 cap; the default failure policy RAISES on the first run error.
"""
results: list[ProjectRunResult] = []
for project in projects.projects:
inbox_dir = Path(project.inbox_dir) if project.inbox_dir is not None else None
# Fresh composition per project (re-entrancy §3 Step 3) — never hoisted.
composed = compose_run_context(Path(project.bundle_dir), inbox_dir, k=top_k)
run = run_project(
client,
composed.context,
meter=meter,
max_debate_rounds=max_debate_rounds,
max_attempts=max_attempts,
default_project_id=composed.ir_projection.project_id,
)
results.append(ProjectRunResult(project_id=project.project_id, run=run))
return PortfolioResult(results=results)

View file

@ -0,0 +1,21 @@
---
type: index
title: "Mini-bundle (K2-fixture) — pumpe-VFD, ett tiltak"
description: "Repo-lokal mini OKF-bundle for portefølje-testing (K2): ett bygg, ett VFD-retrofit-tiltak. ALDRI i shared/ — dette er en test-fixture, ikke en delt fasit."
tags: [fixture, portefølje, K2]
timestamp: 2026-07-16
---
# Mini-bundle (K2-fixture) — K2-VFD-CONTEXT-MARKER
En repo-lokal **mini OKF-bundle** brukt KUN av porteføljetesten (`test_portfolio.py`): ett
bygg, ett kandidat-tiltak (VFD-retrofit på sirkulasjonspumper). Den ligger under `tests/data/`
med vilje — den er en test-fixture og deles ALDRI som fasit i `shared/`.
Formålet er et andre, distinkt prosjekt ved siden av `shared/examples/bygg-energi-mikro`, så
`run_portfolio` kan kjøre N prosjekter sekvensielt med per-prosjekt kontekst.
## Innhold
- [tiltak-vfd.md](tiltak-vfd.md) — `type: hypothesis` — VFD-retrofit-kandidaten.
- [verdict-vfd-fro.md](verdict-vfd-fro.md) — `type: verdict` — frøsatt ekspert-dom (ExpeL-frø).

View file

@ -0,0 +1,21 @@
---
type: hypothesis
title: "VFD-retrofit av sirkulasjonspumper"
description: "Turtallsregulering (variable frequency drive) på 12 sirkulasjonspumper som i dag går på konstant hastighet."
measure_id: VFD-RETROFIT-01
resource: PUMPE-SOR
tags: [pumper, VFD, energieffektivisering]
timestamp: 2026-07-16
---
# VFD-retrofit av sirkulasjonspumper
Kandidat-tiltaket: montere frekvensomformere (VFD) på 12 sirkulasjonspumper i et
sørvendt næringsbygg. Pumpene går i dag på konstant turtall; med turtallsregulering
følger de faktisk lastbehov, og pumpeeffekten faller tilnærmet kubisk med turtallet.
## Mapping til validatoren
`validator-input.json` projiserer tiltaket inn i kost-IR-en: `affected_items` er byggets
årlige pumpe-energikostnad (`ENERGI-PUMPE-EL`), `claimed_saving_nok` er den modellerte
VFD-besparelsen, og `assumptions` bærer energipris-båndet for Monte Carlo-en.

View file

@ -0,0 +1,16 @@
{
"_note": "IR-projeksjon (ir.SavingsProposal) for K2-fixturen. Distinkt fra bygg-energi-mikro: annet prosjekt, annet tiltak, annen kost-kode. claimed_saving_nok ligger godt under feasibel-grensen (0.30 x 120000 = 36000) slik at validate_proposal deterministisk gir ValidatedProposal uansett MC-band.",
"project_id": "PUMPE-SOR",
"measure": "VFD-retrofit av 12 sirkulasjonspumper (konstant -> turtallsregulert drift)",
"affected_items": [
{
"code": "ENERGI-PUMPE-EL",
"quantity": 120000,
"unit_cost": 1.0
}
],
"claimed_saving_nok": 12000,
"assumptions": {
"ENERGI-PUMPE-EL": [0.70, 1.40]
}
}

View file

@ -0,0 +1,26 @@
---
type: verdict
title: "Ekspert-dom (frø): VFD-retrofit — godkjent med realiseringskorreksjon"
description: "Frøsatt ekspert-dom for VFD-tiltaket. Koder et realiseringsgap distinkt fra LED-frøet (0.75, ikke 0.82) så porteføljetesten kan skille prosjektenes kontekst."
resource: PUMPE-SOR
measure_id: VFD-RETROFIT-01
decision: approved_with_adjustment
realization_rate: 0.75
modelled_saving_nok: 12000
expected_actual_saving_nok: 9000
gap_source: pump-affinity-oversimplification
context_key: "naeringsbygg; pumpe-last=stipulert"
provenance: "frø — AI-forfattet test-fixture; erstattes av ekte HITL i produksjon"
tags: [verdict, realization-rate, ExpeL-seed, fixture]
timestamp: 2026-07-16
---
# Ekspert-dom (frø): VFD-retrofit
> **Dette er et frø**, ikke en ekte dom — en test-fixture for porteføljekjøringen.
## Dommen
**Beslutning:** godkjent — med realiseringskorreksjon. Den modellerte besparelsen er
teknisk plausibel, men i drift realiseres erfaringsvis ~75 % fordi den antatte
affinitets-forenklingen overvurderer hvor mye tid pumpene faktisk kjører på redusert turtall.

View file

@ -12,7 +12,7 @@ from typing import Any
import pytest import pytest
from pydantic import ValidationError from pydantic import ValidationError
from portfolio_optimiser_claude.contracts import load_contracts from portfolio_optimiser_claude.contracts import load_contracts, load_reference_projects
VALID_DATA_SOURCE: dict[str, Any] = {"docs_dir": "docs", "top_k": 3} VALID_DATA_SOURCE: dict[str, Any] = {"docs_dir": "docs", "top_k": 3}
VALID_TERMINATION: dict[str, Any] = {"max_rounds": 4, "max_tokens": 20_000} VALID_TERMINATION: dict[str, Any] = {"max_rounds": 4, "max_tokens": 20_000}
@ -118,6 +118,42 @@ class TestModelMap:
assert "default" in contracts.model_map.profiles["anthropic"] assert "default" in contracts.model_map.profiles["anthropic"]
class TestReferenceProjects:
"""§10 (K2): the portfolio config — each entry needs a non-empty bundle path."""
def test_valid_config_accepted(self) -> None:
config = load_reference_projects(
{"projects": [{"project_id": "P1", "bundle_dir": "bundles/p1"}]}
)
assert config.projects[0].project_id == "P1"
assert config.projects[0].inbox_dir is None
def test_missing_bundle_dir_rejected(self) -> None:
# The fail-fast seam K2's detach proof removes: no bundle path is a
# startup schema error, never a run that starts then crashes for lack of data.
with pytest.raises(ValidationError):
load_reference_projects({"projects": [{"project_id": "P1"}]})
def test_empty_bundle_dir_rejected(self) -> None:
with pytest.raises(ValidationError):
load_reference_projects({"projects": [{"project_id": "P1", "bundle_dir": ""}]})
def test_empty_project_id_rejected(self) -> None:
with pytest.raises(ValidationError):
load_reference_projects({"projects": [{"project_id": "", "bundle_dir": "b"}]})
def test_empty_projects_list_rejected(self) -> None:
with pytest.raises(ValidationError):
load_reference_projects({"projects": []})
def test_bundled_reference_projects_is_valid(self) -> None:
# raw=None falls back to the bundled data/reference_projects.json EXAMPLE,
# whose shape must itself satisfy the contract (fail-fast on shipped config).
config = load_reference_projects()
assert config.projects
assert all(p.bundle_dir for p in config.projects)
def test_happy_path_validates_all_four() -> None: def test_happy_path_validates_all_four() -> None:
contracts = load() contracts = load()
assert contracts.data_source.top_k == 3 assert contracts.data_source.top_k == 3

151
tests/test_portfolio.py Normal file
View file

@ -0,0 +1,151 @@
"""Portfolio run — LOAD-BEARING (K2; method-spec §3, §5, §8, §10, §11; paritetsrad 4).
The seam this file keeps alive: ``run_portfolio`` drives N projects SEQUENTIALLY
from a schema-validated reference config, composing each project's §5 context
(merge inbox seed fold) and running the loop core UNCHANGED per project,
collecting per-project results IN CONFIG ORDER. This is the run path MAF got in
its Fase 1 and D7 never had the only prior run entrance (``run_s10.py``,
``run.py``) drives a single bundle.
Detach proofs (each restored from a copy, never ``git checkout``):
* Fail-fast (§10): drop the ``bundle_dir`` Field requirement in
``ReferenceProjectContract`` (e.g. ``bundle_dir: str = ""``) a config missing
its bundle path no longer raises at load ``run_portfolio`` STARTS on the
invalid config and only crashes mid-run ``test_malformed_config_never_reaches
_a_client`` goes red (``FileNotFoundError``, not ``ValidationError``).
* Re-entrancy (§3 Step 3, the key assumption): hoist the per-project
``compose_run_context`` call OUT of the loop (compose project 1 once, reuse it
for every project) project 2 runs on project 1's context → the VFD marker
never reaches any prompt ``test_each_project_composes_its_own_context`` goes red.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from pydantic import ValidationError
from _scripted import ScriptedClient, reply
from portfolio_optimiser_claude.budget import BudgetExceeded, BudgetMeter
from portfolio_optimiser_claude.contracts import (
ReferenceProjectsContract,
TerminationContract,
load_reference_projects,
)
from portfolio_optimiser_claude.ir import load_validator_input
from portfolio_optimiser_claude.portfolio import PortfolioResult, run_portfolio
from portfolio_optimiser_claude.validator import ValidatedProposal
_REPO = Path(__file__).resolve().parents[1]
LED_BUNDLE = _REPO / "shared" / "examples" / "bygg-energi-mikro"
VFD_BUNDLE = _REPO / "tests" / "data" / "mini-bundle"
# Distinctive tokens that reach the composed context of ONE bundle only.
LED_MARKER = "LED-retrofit" # in the shared bundle's rendered context
VFD_MARKER = "K2-VFD-CONTEXT-MARKER" # embedded in the repo-local fixture's index
def _meter(*, max_rounds: int = 100, max_tokens: int = 10_000) -> BudgetMeter:
return BudgetMeter(TerminationContract(max_rounds=max_rounds, max_tokens=max_tokens))
def _happy_replies(bundles: list[Path]) -> list[object]:
"""Three scripted replies per project: debate reasoning → APPROVE → candidate JSON.
The candidate JSON is each bundle's own validated IR projection, so the
deterministic validator returns ``ValidatedProposal`` first try (§3 Steps 24).
"""
replies: list[object] = []
for bundle in bundles:
replies += [
reply("debate reasoning"),
reply("VERDICT: APPROVE"),
reply(json.dumps(load_validator_input(bundle).model_dump())),
]
return replies
def _config(entries: list[tuple[str, Path]]) -> ReferenceProjectsContract:
return load_reference_projects(
{"projects": [{"project_id": pid, "bundle_dir": str(path)} for pid, path in entries]}
)
class TestSequentialConfigOrder:
"""Two-project config → both run, results collected in config order (§3, paritetsrad 4)."""
def test_two_project_config_runs_both_in_config_order(self) -> None:
projects = _config([("BYGG-KONTOR-NORD", LED_BUNDLE), ("PUMPE-SOR", VFD_BUNDLE)])
client = ScriptedClient(replies=_happy_replies([LED_BUNDLE, VFD_BUNDLE]))
result = run_portfolio(
projects, client, _meter(), top_k=3, max_debate_rounds=3, max_attempts=3
)
assert isinstance(result, PortfolioResult)
assert [r.project_id for r in result.results] == ["BYGG-KONTOR-NORD", "PUMPE-SOR"]
assert all(isinstance(r.run.outcome, ValidatedProposal) for r in result.results)
def test_result_order_follows_config_order_not_incidentally(self) -> None:
# Reverse the config → the results reverse with it: order is the config's.
projects = _config([("PUMPE-SOR", VFD_BUNDLE), ("BYGG-KONTOR-NORD", LED_BUNDLE)])
client = ScriptedClient(replies=_happy_replies([VFD_BUNDLE, LED_BUNDLE]))
result = run_portfolio(
projects, client, _meter(), top_k=3, max_debate_rounds=3, max_attempts=3
)
assert [r.project_id for r in result.results] == ["PUMPE-SOR", "BYGG-KONTOR-NORD"]
class TestReEntrancy:
"""Key assumption (§3 Step 3): the loop core is re-entrant — fresh state per project.
Two sequential runs share no mutable state beyond the explicitly shared meter:
each project composes its OWN context, so the two projects' distinctive markers
never appear together in a single prompt.
"""
def test_each_project_composes_its_own_context(self) -> None:
projects = _config([("BYGG-KONTOR-NORD", LED_BUNDLE), ("PUMPE-SOR", VFD_BUNDLE)])
client = ScriptedClient(replies=_happy_replies([LED_BUNDLE, VFD_BUNDLE]))
run_portfolio(projects, client, _meter(), top_k=3, max_debate_rounds=3, max_attempts=3)
proposer_prompts = client.prompts("proposer")
led = [p for p in proposer_prompts if LED_MARKER in p]
vfd = [p for p in proposer_prompts if VFD_MARKER in p]
assert led, "project 1's own context must reach its prompts"
assert vfd, "project 2's own context must reach its prompts (no reuse of project 1's)"
# No single prompt ever mixes the two projects' contexts.
assert not any(LED_MARKER in p and VFD_MARKER in p for p in proposer_prompts)
class TestFailurePolicyRaise:
"""Failure policy is a stack-local choice until D-D: the default RAISES (§8).
Today everything is thrown; K18 flips this to collect-and-continue when the
D-D wave model lands. A budget stop in project 1 propagates as the typed
``BudgetExceeded`` and the portfolio stops project 2 is never touched.
"""
def test_project_error_propagates_and_stops_the_portfolio(self) -> None:
projects = _config([("BYGG-KONTOR-NORD", LED_BUNDLE), ("PUMPE-SOR", VFD_BUNDLE)])
client = ScriptedClient(replies=_happy_replies([LED_BUNDLE, VFD_BUNDLE]))
# max_tokens=5: project 1's first proposer reply (10 tokens) breaches the cap.
meter = _meter(max_tokens=5)
with pytest.raises(BudgetExceeded):
run_portfolio(projects, client, meter, top_k=3, max_debate_rounds=3, max_attempts=3)
# The stop fired inside project 1's first call; project 2 never ran.
assert len(client.calls) == 1
assert not any(VFD_MARKER in prompt for _role, prompt in client.calls)
class TestFailFastSchemaValidation:
"""§10: a malformed config is refused at load, BEFORE any model client is touched."""
def test_malformed_config_never_reaches_a_client(self) -> None:
spy = ScriptedClient(replies=[reply("must never be used")])
with pytest.raises(ValidationError):
projects = load_reference_projects({"projects": [{"project_id": "NO-BUNDLE"}]})
run_portfolio(projects, spy, _meter(), top_k=3, max_debate_rounds=3, max_attempts=3)
assert spy.calls == []