open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(validator): anchor the deterministic gate to the project's real cost baseline (S4.0)

Browse source 
Every stage of validate_proposal reasoned only about numbers the proposal itself
supplied, so an internally-consistent hallucination cleared the whole gate (F3).
A new stage 0 reconciles each affected_item against the project's CostBaseline
before the CBC solve: an unknown cost code is rejected, and a real code carrying
a quantity/unit_cost outside the configured tolerance (5% default, relative to
the baseline value) is rejected. Validation, never repair.

The baseline argument is OPTIONAL (None = pre-S4.0 behaviour), but both run
paths set it: the road path projects project.cost_items, the bundle path loads
cost-baseline.json when the bundle ships one. Bundles written before the
amendment stay un-anchored, so the commons-owned goldens run byte-identically;
a baseline that exists but is malformed still raises on both loaders.

F8: the method-specific cap now comes from the METHOD_CAPS registry (measure
type -> fraction, injectable) instead of an energy_efficiency string comparison.

The baseline format and tolerance semantics were decided locally — the commons
amendment (D-A pt. 2) never arrived, exactly as in S3.2. D7 mirroring stays open.

Three portfolio fixtures quoted cost codes belonging to OTHER projects; the new
gate caught them. They now quote each project's own lines, and the two copied
REPLIES tables import the single source instead of drifting from it.

Load-bearing measured (tests/test_s40_cost_baseline_loadbearing.py), six
mutations all red: detach the reconciliation stage; detach the magnitude
tolerance; detach the road wiring; detach the bundle wiring; ignore the injected
cap registry; make the optional loader tolerant of malformed content. Control:
with the road wiring detached the repaired portfolio fixtures still pass, so
they are not masking the seam. 597 -> 612 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JdwK7bQ4BZkWH4t8MRDKb4
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 17:19:31 +02:00
commit 126807aee7
16 changed files with 645 additions and 69 deletions
Download patch file Download diff file Expand all files Collapse all files

110
src/portfolio_optimiser/validator.py
View file

@ -24,13 +24,13 @@ from __future__ import annotations
import random
import statistics
import warnings
from collections.abc import Callable
from collections.abc import Callable, Mapping
from contextlib import contextmanager
from dataclasses import dataclass
import pulp
from portfolio_optimiser.ir import AffectedItem, SavingsProposal
from portfolio_optimiser.ir import AffectedItem, CostBaseline, CostBaselineLine, SavingsProposal
from portfolio_optimiser.reference_domain import Project
MAX_SAVING_FRACTION = 0.30
@ -46,6 +46,21 @@ INDEPENDENT gate: it can reject a proposal the generic P90 stage passes (not red
fraction is calibrated against the reference domain; the CONDITION (a method-scoped stricter cap) is
the encoded rule. Returns the same ``Rejection`` type — a validator stage, not a new gate."""
METHOD_CAPS: dict[str, float] = {_ENERGY_METHOD_MEASURE: _ENERGY_METHOD_MAX_FRACTION}
"""S4.0 (F8): the method-cap REGISTRY — measure type -> method-scoped max saving fraction. The
rule used to be an ``if proposal.measure == "energy_efficiency"`` branch, so encoding a second
assessment method meant editing the validator. It is now data: a caller passes its own registry
(``validate_proposal(..., method_caps=...)``), keyed by the measure type a dimension admits
(``dimension.allowed_measure_types``), and the built-in entry stays the default so the Step-9
behaviour is unchanged. Deliberately NOT a config file yet — the deliverable is the key-by-config
seam (90%-prinsippet), not a settings format."""
BASELINE_TOLERANCE_DEFAULT = 0.05
"""S4.0: the relative deviation a reconciled ``AffectedItem`` may show against its cost-baseline
line (5%). A tolerance is needed at all because a proposer restates magnitudes in prose-derived,
rounded form; it is small because its whole purpose is to leave no room for a FABRICATED magnitude.
Config, not policy: every caller can tighten or loosen it per run (``tolerance=``)."""
_MC_SAMPLES = 512
_MC_SEED = 20260624
@ -122,9 +137,83 @@ def _monte_carlo(
return deciles[0], deciles[4], deciles[8] # P10, P50, P90
def validate_proposal(proposal: SavingsProposal) -> ValidatedProposal | Rejection:
def baseline_from_project(project: Project) -> CostBaseline:
"""Project a road reference-domain ``Project``'s ``cost_items`` into the ``CostBaseline``
contract — the road-path counterpart of ``okf.load_cost_baseline`` (S4.0). The road path always
HAS its baseline (the estimate is the project), so this projection is total: no optional
variant, and a run on this path is always anchored."""
return CostBaseline(
project_id=project.id,
items={
ci.code: CostBaselineLine(quantity=ci.quantity, unit_cost=ci.unit_cost)
for ci in project.cost_items
},
)
def _reconcile_against_baseline(
proposal: SavingsProposal, baseline: CostBaseline, tolerance: float
) -> Rejection | None:
"""S4.0 (F3): every affected item must correspond to a REAL line of the project's cost baseline.
Two independent failures, both fail-closed:
* the cost code is absent from the baseline — a fabricated line;
* the code is real but its ``quantity``/``unit_cost`` deviates from the baseline line by more
than ``tolerance`` (relative to the BASELINE value, which is the ground truth) — a real code
carrying a fabricated magnitude.
Returns the first ``Rejection`` (validator's own type — never a new gate), or ``None`` when the
proposal reconciles. Items are checked in their stated order so the reason is deterministic.
A validation, never a repair: the proposal is rejected, not silently corrected to the baseline."""
for item in proposal.affected_items:
line = baseline.items.get(item.code)
if line is None:
return Rejection(
proposal=proposal,
reason=(
f"unknown cost code {item.code!r}: not in project {baseline.project_id}'s "
f"cost baseline ({len(baseline.items)} known codes)"
),
)
for field, claimed, actual in (
("quantity", item.quantity, line.quantity),
("unit_cost", item.unit_cost, line.unit_cost),
):
if abs(claimed - actual) > tolerance * actual:
return Rejection(
proposal=proposal,
reason=(
f"{field} {claimed:g} for cost code {item.code!r} is outside the "
f"{tolerance:.1%} tolerance around the baseline {field} {actual:g}"
),
)
return None
def validate_proposal(
proposal: SavingsProposal,
*,
baseline: CostBaseline | None = None,
tolerance: float = BASELINE_TOLERANCE_DEFAULT,
method_caps: Mapping[str, float] | None = None,
) -> ValidatedProposal | Rejection:
"""Deterministic blocking validation. Returns a ``ValidatedProposal`` only when the
claim is feasible; otherwise a ``Rejection`` that cannot be consumed as validated."""
claim is feasible; otherwise a ``Rejection`` that cannot be consumed as validated.
``baseline`` (S4.0, F3) anchors the gate to the project's ACTUAL cost lines: without it every
stage reasons only about numbers the proposal supplied itself, so an internally-consistent
hallucination clears the gate. It is OPTIONAL — ``None`` is exactly the pre-S4.0 behaviour, so a
caller with no baseline (a bundle authored before the amendment) is unchanged — but both run
paths SET it. ``tolerance`` is the reconciliation's config knob; ``method_caps`` overrides the
built-in method-cap registry (F8)."""
# Stage 0 (S4.0): reconcile against the cost baseline BEFORE the solver. It is the cheapest
# stage and the only one that can tell a fabricated line from a real one — spending a CBC solve
# on numbers that do not belong to the project is work on a claim that cannot be validated.
if baseline is not None:
blocked = _reconcile_against_baseline(proposal, baseline, tolerance)
if blocked is not None:
return blocked
# Stage 1 (Pydantic) already ran at construction. Stage 2: real CBC solve.
nominal = _solve_max_feasible(proposal.affected_items, MAX_SAVING_FRACTION)
# Stage 3: Monte Carlo percentiles of the feasible saving.
@ -152,15 +241,18 @@ def validate_proposal(proposal: SavingsProposal) -> ValidatedProposal | Rejectio
# Stage 5 (Step 9, SC7-B): a method-specific rule STRICTER than the generic cap. A proposal in
# the energy method (IPMVP Option A) must clear a lower, method-scoped feasible — an INDEPENDENT
# gate that can reject a proposal the P90 stage passed. Same ``Rejection`` type, not a new gate.
if proposal.measure == _ENERGY_METHOD_MEASURE:
method_feasible = _ENERGY_METHOD_MAX_FRACTION * sum(
it.total for it in proposal.affected_items
)
# F8 (S4.0): the cap is looked up in a REGISTRY keyed by measure type (config), not compared
# against the ``energy_efficiency`` literal — a second assessment method is now data, not an
# edit to this function. The built-in registry keeps the Step-9 behaviour identical.
caps = METHOD_CAPS if method_caps is None else method_caps
method_fraction = caps.get(proposal.measure)
if method_fraction is not None:
method_feasible = method_fraction * sum(it.total for it in proposal.affected_items)
if proposal.claimed_saving_nok > method_feasible:
return Rejection(
proposal=proposal,
reason=(
f"claimed {proposal.claimed_saving_nok:.0f} exceeds the {_ENERGY_METHOD_MEASURE} "
f"claimed {proposal.claimed_saving_nok:.0f} exceeds the {proposal.measure} "
f"method cap {method_feasible:.0f} (stricter than the generic P90)"
),
)