feat(portfolio): K2 — sequential multi-project run (parity row 4)
New portfolio.py: run_portfolio drives N projects sequentially from a schema-validated reference config, composing each project's §5 context (merge inbox -> seed -> fold) and running the loop core UNCHANGED per project, collecting one typed result per project IN CONFIG ORDER. This is the run path MAF got in its Fase 1 and D7 never had — the prior entrances (run.py, run_s10.py) drive a single bundle. PortfolioResult holds per-project results tagged with the config project_id. Re-entrancy (§3 Step 3): each project composes its OWN context inside the loop, never a hoisted shared one, so nothing survives one project into the next except the explicitly shared mutable state — the §8 budget meter, a portfolio-wide cap. Failure policy is a STACK-LOCAL choice until D-D: the default RAISES (today everything is thrown); K18 flips it to collect-and-continue when the D-D wave model lands. New config contract in contracts.py: ReferenceProjectContract (project_id + required non-empty bundle_dir + optional inbox_dir) + ReferenceProjectsContract, loaded fail-fast by load_reference_projects (§10) — a project without a bundle path is refused before any run. New data/reference_projects.json example (shape-validated, never executed by the suite). New repo-local mini-bundle fixture under tests/data/ (a distinct second project, VFD-retrofit — ALDRI in shared/). Two detach proofs delivered: drop the bundle_dir Field requirement -> a run starts on the invalid config and only crashes mid-run -> the fail-fast test goes red; hoist the per-project composition out of the loop -> project 2 runs on project 1's context and the VFD marker never reaches its prompt -> the re-entrancy test goes red. 11 new tests (test_portfolio.py 5 + test_contracts.py TestReferenceProjects 6). 426 -> 437 tests, golden byte-exact, full gate clean (ruff + format + mypy strict). README synced (test count + a Run layer module block). [skip-docs] — README documents the new module; CLAUDE.md holds invariants (rules/commands) only, and K2 adds no new invariant, command, or convention. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RiTwaKLesgcwXx2mDviqpt
This commit is contained in:
parent
a7e8ffecb8
commit
f2c64da9ee
10 changed files with 406 additions and 3 deletions
21
tests/data/mini-bundle/index.md
Normal file
21
tests/data/mini-bundle/index.md
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
---
|
||||
type: index
|
||||
title: "Mini-bundle (K2-fixture) — pumpe-VFD, ett tiltak"
|
||||
description: "Repo-lokal mini OKF-bundle for portefølje-testing (K2): ett bygg, ett VFD-retrofit-tiltak. ALDRI i shared/ — dette er en test-fixture, ikke en delt fasit."
|
||||
tags: [fixture, portefølje, K2]
|
||||
timestamp: 2026-07-16
|
||||
---
|
||||
|
||||
# Mini-bundle (K2-fixture) — K2-VFD-CONTEXT-MARKER
|
||||
|
||||
En repo-lokal **mini OKF-bundle** brukt KUN av porteføljetesten (`test_portfolio.py`): ett
|
||||
bygg, ett kandidat-tiltak (VFD-retrofit på sirkulasjonspumper). Den ligger under `tests/data/`
|
||||
med vilje — den er en test-fixture og deles ALDRI som fasit i `shared/`.
|
||||
|
||||
Formålet er et andre, distinkt prosjekt ved siden av `shared/examples/bygg-energi-mikro`, så
|
||||
`run_portfolio` kan kjøre N prosjekter sekvensielt med per-prosjekt kontekst.
|
||||
|
||||
## Innhold
|
||||
|
||||
- [tiltak-vfd.md](tiltak-vfd.md) — `type: hypothesis` — VFD-retrofit-kandidaten.
|
||||
- [verdict-vfd-fro.md](verdict-vfd-fro.md) — `type: verdict` — frøsatt ekspert-dom (ExpeL-frø).
|
||||
21
tests/data/mini-bundle/tiltak-vfd.md
Normal file
21
tests/data/mini-bundle/tiltak-vfd.md
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
---
|
||||
type: hypothesis
|
||||
title: "VFD-retrofit av sirkulasjonspumper"
|
||||
description: "Turtallsregulering (variable frequency drive) på 12 sirkulasjonspumper som i dag går på konstant hastighet."
|
||||
measure_id: VFD-RETROFIT-01
|
||||
resource: PUMPE-SOR
|
||||
tags: [pumper, VFD, energieffektivisering]
|
||||
timestamp: 2026-07-16
|
||||
---
|
||||
|
||||
# VFD-retrofit av sirkulasjonspumper
|
||||
|
||||
Kandidat-tiltaket: montere frekvensomformere (VFD) på 12 sirkulasjonspumper i et
|
||||
sørvendt næringsbygg. Pumpene går i dag på konstant turtall; med turtallsregulering
|
||||
følger de faktisk lastbehov, og pumpeeffekten faller tilnærmet kubisk med turtallet.
|
||||
|
||||
## Mapping til validatoren
|
||||
|
||||
`validator-input.json` projiserer tiltaket inn i kost-IR-en: `affected_items` er byggets
|
||||
årlige pumpe-energikostnad (`ENERGI-PUMPE-EL`), `claimed_saving_nok` er den modellerte
|
||||
VFD-besparelsen, og `assumptions` bærer energipris-båndet for Monte Carlo-en.
|
||||
16
tests/data/mini-bundle/validator-input.json
Normal file
16
tests/data/mini-bundle/validator-input.json
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
{
|
||||
"_note": "IR-projeksjon (ir.SavingsProposal) for K2-fixturen. Distinkt fra bygg-energi-mikro: annet prosjekt, annet tiltak, annen kost-kode. claimed_saving_nok ligger godt under feasibel-grensen (0.30 x 120000 = 36000) slik at validate_proposal deterministisk gir ValidatedProposal uansett MC-band.",
|
||||
"project_id": "PUMPE-SOR",
|
||||
"measure": "VFD-retrofit av 12 sirkulasjonspumper (konstant -> turtallsregulert drift)",
|
||||
"affected_items": [
|
||||
{
|
||||
"code": "ENERGI-PUMPE-EL",
|
||||
"quantity": 120000,
|
||||
"unit_cost": 1.0
|
||||
}
|
||||
],
|
||||
"claimed_saving_nok": 12000,
|
||||
"assumptions": {
|
||||
"ENERGI-PUMPE-EL": [0.70, 1.40]
|
||||
}
|
||||
}
|
||||
26
tests/data/mini-bundle/verdict-vfd-fro.md
Normal file
26
tests/data/mini-bundle/verdict-vfd-fro.md
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
---
|
||||
type: verdict
|
||||
title: "Ekspert-dom (frø): VFD-retrofit — godkjent med realiseringskorreksjon"
|
||||
description: "Frøsatt ekspert-dom for VFD-tiltaket. Koder et realiseringsgap distinkt fra LED-frøet (0.75, ikke 0.82) så porteføljetesten kan skille prosjektenes kontekst."
|
||||
resource: PUMPE-SOR
|
||||
measure_id: VFD-RETROFIT-01
|
||||
decision: approved_with_adjustment
|
||||
realization_rate: 0.75
|
||||
modelled_saving_nok: 12000
|
||||
expected_actual_saving_nok: 9000
|
||||
gap_source: pump-affinity-oversimplification
|
||||
context_key: "naeringsbygg; pumpe-last=stipulert"
|
||||
provenance: "frø — AI-forfattet test-fixture; erstattes av ekte HITL i produksjon"
|
||||
tags: [verdict, realization-rate, ExpeL-seed, fixture]
|
||||
timestamp: 2026-07-16
|
||||
---
|
||||
|
||||
# Ekspert-dom (frø): VFD-retrofit
|
||||
|
||||
> **Dette er et frø**, ikke en ekte dom — en test-fixture for porteføljekjøringen.
|
||||
|
||||
## Dommen
|
||||
|
||||
**Beslutning:** godkjent — med realiseringskorreksjon. Den modellerte besparelsen er
|
||||
teknisk plausibel, men i drift realiseres erfaringsvis ~75 % fordi den antatte
|
||||
affinitets-forenklingen overvurderer hvor mye tid pumpene faktisk kjører på redusert turtall.
|
||||
|
|
@ -12,7 +12,7 @@ from typing import Any
|
|||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from portfolio_optimiser_claude.contracts import load_contracts
|
||||
from portfolio_optimiser_claude.contracts import load_contracts, load_reference_projects
|
||||
|
||||
VALID_DATA_SOURCE: dict[str, Any] = {"docs_dir": "docs", "top_k": 3}
|
||||
VALID_TERMINATION: dict[str, Any] = {"max_rounds": 4, "max_tokens": 20_000}
|
||||
|
|
@ -118,6 +118,42 @@ class TestModelMap:
|
|||
assert "default" in contracts.model_map.profiles["anthropic"]
|
||||
|
||||
|
||||
class TestReferenceProjects:
|
||||
"""§10 (K2): the portfolio config — each entry needs a non-empty bundle path."""
|
||||
|
||||
def test_valid_config_accepted(self) -> None:
|
||||
config = load_reference_projects(
|
||||
{"projects": [{"project_id": "P1", "bundle_dir": "bundles/p1"}]}
|
||||
)
|
||||
assert config.projects[0].project_id == "P1"
|
||||
assert config.projects[0].inbox_dir is None
|
||||
|
||||
def test_missing_bundle_dir_rejected(self) -> None:
|
||||
# The fail-fast seam K2's detach proof removes: no bundle path is a
|
||||
# startup schema error, never a run that starts then crashes for lack of data.
|
||||
with pytest.raises(ValidationError):
|
||||
load_reference_projects({"projects": [{"project_id": "P1"}]})
|
||||
|
||||
def test_empty_bundle_dir_rejected(self) -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
load_reference_projects({"projects": [{"project_id": "P1", "bundle_dir": ""}]})
|
||||
|
||||
def test_empty_project_id_rejected(self) -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
load_reference_projects({"projects": [{"project_id": "", "bundle_dir": "b"}]})
|
||||
|
||||
def test_empty_projects_list_rejected(self) -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
load_reference_projects({"projects": []})
|
||||
|
||||
def test_bundled_reference_projects_is_valid(self) -> None:
|
||||
# raw=None falls back to the bundled data/reference_projects.json EXAMPLE,
|
||||
# whose shape must itself satisfy the contract (fail-fast on shipped config).
|
||||
config = load_reference_projects()
|
||||
assert config.projects
|
||||
assert all(p.bundle_dir for p in config.projects)
|
||||
|
||||
|
||||
def test_happy_path_validates_all_four() -> None:
|
||||
contracts = load()
|
||||
assert contracts.data_source.top_k == 3
|
||||
|
|
|
|||
151
tests/test_portfolio.py
Normal file
151
tests/test_portfolio.py
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
"""Portfolio run — LOAD-BEARING (K2; method-spec §3, §5, §8, §10, §11; paritetsrad 4).
|
||||
|
||||
The seam this file keeps alive: ``run_portfolio`` drives N projects SEQUENTIALLY
|
||||
from a schema-validated reference config, composing each project's §5 context
|
||||
(merge inbox → seed → fold) and running the loop core UNCHANGED per project,
|
||||
collecting per-project results IN CONFIG ORDER. This is the run path MAF got in
|
||||
its Fase 1 and D7 never had — the only prior run entrance (``run_s10.py``,
|
||||
``run.py``) drives a single bundle.
|
||||
|
||||
Detach proofs (each restored from a copy, never ``git checkout``):
|
||||
|
||||
* Fail-fast (§10): drop the ``bundle_dir`` Field requirement in
|
||||
``ReferenceProjectContract`` (e.g. ``bundle_dir: str = ""``) → a config missing
|
||||
its bundle path no longer raises at load → ``run_portfolio`` STARTS on the
|
||||
invalid config and only crashes mid-run → ``test_malformed_config_never_reaches
|
||||
_a_client`` goes red (``FileNotFoundError``, not ``ValidationError``).
|
||||
* Re-entrancy (§3 Step 3, the key assumption): hoist the per-project
|
||||
``compose_run_context`` call OUT of the loop (compose project 1 once, reuse it
|
||||
for every project) → project 2 runs on project 1's context → the VFD marker
|
||||
never reaches any prompt → ``test_each_project_composes_its_own_context`` goes red.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from _scripted import ScriptedClient, reply
|
||||
|
||||
from portfolio_optimiser_claude.budget import BudgetExceeded, BudgetMeter
|
||||
from portfolio_optimiser_claude.contracts import (
|
||||
ReferenceProjectsContract,
|
||||
TerminationContract,
|
||||
load_reference_projects,
|
||||
)
|
||||
from portfolio_optimiser_claude.ir import load_validator_input
|
||||
from portfolio_optimiser_claude.portfolio import PortfolioResult, run_portfolio
|
||||
from portfolio_optimiser_claude.validator import ValidatedProposal
|
||||
|
||||
_REPO = Path(__file__).resolve().parents[1]
|
||||
LED_BUNDLE = _REPO / "shared" / "examples" / "bygg-energi-mikro"
|
||||
VFD_BUNDLE = _REPO / "tests" / "data" / "mini-bundle"
|
||||
|
||||
# Distinctive tokens that reach the composed context of ONE bundle only.
|
||||
LED_MARKER = "LED-retrofit" # in the shared bundle's rendered context
|
||||
VFD_MARKER = "K2-VFD-CONTEXT-MARKER" # embedded in the repo-local fixture's index
|
||||
|
||||
|
||||
def _meter(*, max_rounds: int = 100, max_tokens: int = 10_000) -> BudgetMeter:
|
||||
return BudgetMeter(TerminationContract(max_rounds=max_rounds, max_tokens=max_tokens))
|
||||
|
||||
|
||||
def _happy_replies(bundles: list[Path]) -> list[object]:
|
||||
"""Three scripted replies per project: debate reasoning → APPROVE → candidate JSON.
|
||||
|
||||
The candidate JSON is each bundle's own validated IR projection, so the
|
||||
deterministic validator returns ``ValidatedProposal`` first try (§3 Steps 2–4).
|
||||
"""
|
||||
replies: list[object] = []
|
||||
for bundle in bundles:
|
||||
replies += [
|
||||
reply("debate reasoning"),
|
||||
reply("VERDICT: APPROVE"),
|
||||
reply(json.dumps(load_validator_input(bundle).model_dump())),
|
||||
]
|
||||
return replies
|
||||
|
||||
|
||||
def _config(entries: list[tuple[str, Path]]) -> ReferenceProjectsContract:
|
||||
return load_reference_projects(
|
||||
{"projects": [{"project_id": pid, "bundle_dir": str(path)} for pid, path in entries]}
|
||||
)
|
||||
|
||||
|
||||
class TestSequentialConfigOrder:
|
||||
"""Two-project config → both run, results collected in config order (§3, paritetsrad 4)."""
|
||||
|
||||
def test_two_project_config_runs_both_in_config_order(self) -> None:
|
||||
projects = _config([("BYGG-KONTOR-NORD", LED_BUNDLE), ("PUMPE-SOR", VFD_BUNDLE)])
|
||||
client = ScriptedClient(replies=_happy_replies([LED_BUNDLE, VFD_BUNDLE]))
|
||||
result = run_portfolio(
|
||||
projects, client, _meter(), top_k=3, max_debate_rounds=3, max_attempts=3
|
||||
)
|
||||
assert isinstance(result, PortfolioResult)
|
||||
assert [r.project_id for r in result.results] == ["BYGG-KONTOR-NORD", "PUMPE-SOR"]
|
||||
assert all(isinstance(r.run.outcome, ValidatedProposal) for r in result.results)
|
||||
|
||||
def test_result_order_follows_config_order_not_incidentally(self) -> None:
|
||||
# Reverse the config → the results reverse with it: order is the config's.
|
||||
projects = _config([("PUMPE-SOR", VFD_BUNDLE), ("BYGG-KONTOR-NORD", LED_BUNDLE)])
|
||||
client = ScriptedClient(replies=_happy_replies([VFD_BUNDLE, LED_BUNDLE]))
|
||||
result = run_portfolio(
|
||||
projects, client, _meter(), top_k=3, max_debate_rounds=3, max_attempts=3
|
||||
)
|
||||
assert [r.project_id for r in result.results] == ["PUMPE-SOR", "BYGG-KONTOR-NORD"]
|
||||
|
||||
|
||||
class TestReEntrancy:
|
||||
"""Key assumption (§3 Step 3): the loop core is re-entrant — fresh state per project.
|
||||
|
||||
Two sequential runs share no mutable state beyond the explicitly shared meter:
|
||||
each project composes its OWN context, so the two projects' distinctive markers
|
||||
never appear together in a single prompt.
|
||||
"""
|
||||
|
||||
def test_each_project_composes_its_own_context(self) -> None:
|
||||
projects = _config([("BYGG-KONTOR-NORD", LED_BUNDLE), ("PUMPE-SOR", VFD_BUNDLE)])
|
||||
client = ScriptedClient(replies=_happy_replies([LED_BUNDLE, VFD_BUNDLE]))
|
||||
run_portfolio(projects, client, _meter(), top_k=3, max_debate_rounds=3, max_attempts=3)
|
||||
|
||||
proposer_prompts = client.prompts("proposer")
|
||||
led = [p for p in proposer_prompts if LED_MARKER in p]
|
||||
vfd = [p for p in proposer_prompts if VFD_MARKER in p]
|
||||
assert led, "project 1's own context must reach its prompts"
|
||||
assert vfd, "project 2's own context must reach its prompts (no reuse of project 1's)"
|
||||
# No single prompt ever mixes the two projects' contexts.
|
||||
assert not any(LED_MARKER in p and VFD_MARKER in p for p in proposer_prompts)
|
||||
|
||||
|
||||
class TestFailurePolicyRaise:
|
||||
"""Failure policy is a stack-local choice until D-D: the default RAISES (§8).
|
||||
|
||||
Today everything is thrown; K18 flips this to collect-and-continue when the
|
||||
D-D wave model lands. A budget stop in project 1 propagates as the typed
|
||||
``BudgetExceeded`` and the portfolio stops — project 2 is never touched.
|
||||
"""
|
||||
|
||||
def test_project_error_propagates_and_stops_the_portfolio(self) -> None:
|
||||
projects = _config([("BYGG-KONTOR-NORD", LED_BUNDLE), ("PUMPE-SOR", VFD_BUNDLE)])
|
||||
client = ScriptedClient(replies=_happy_replies([LED_BUNDLE, VFD_BUNDLE]))
|
||||
# max_tokens=5: project 1's first proposer reply (10 tokens) breaches the cap.
|
||||
meter = _meter(max_tokens=5)
|
||||
with pytest.raises(BudgetExceeded):
|
||||
run_portfolio(projects, client, meter, top_k=3, max_debate_rounds=3, max_attempts=3)
|
||||
# The stop fired inside project 1's first call; project 2 never ran.
|
||||
assert len(client.calls) == 1
|
||||
assert not any(VFD_MARKER in prompt for _role, prompt in client.calls)
|
||||
|
||||
|
||||
class TestFailFastSchemaValidation:
|
||||
"""§10: a malformed config is refused at load, BEFORE any model client is touched."""
|
||||
|
||||
def test_malformed_config_never_reaches_a_client(self) -> None:
|
||||
spy = ScriptedClient(replies=[reply("must never be used")])
|
||||
with pytest.raises(ValidationError):
|
||||
projects = load_reference_projects({"projects": [{"project_id": "NO-BUNDLE"}]})
|
||||
run_portfolio(projects, spy, _meter(), top_k=3, max_debate_rounds=3, max_attempts=3)
|
||||
assert spy.calls == []
|
||||
Loading…
Add table
Add a link
Reference in a new issue