feat(validator): anchor the deterministic gate to the project's real cost baseline (S4.0)

Every stage of validate_proposal reasoned only about numbers the proposal itself
supplied, so an internally-consistent hallucination cleared the whole gate (F3).
A new stage 0 reconciles each affected_item against the project's CostBaseline
before the CBC solve: an unknown cost code is rejected, and a real code carrying
a quantity/unit_cost outside the configured tolerance (5% default, relative to
the baseline value) is rejected. Validation, never repair.

The baseline argument is OPTIONAL (None = pre-S4.0 behaviour), but both run
paths set it: the road path projects project.cost_items, the bundle path loads
cost-baseline.json when the bundle ships one. Bundles written before the
amendment stay un-anchored, so the commons-owned goldens run byte-identically;
a baseline that exists but is malformed still raises on both loaders.

F8: the method-specific cap now comes from the METHOD_CAPS registry (measure
type -> fraction, injectable) instead of an energy_efficiency string comparison.

The baseline format and tolerance semantics were decided locally — the commons
amendment (D-A pt. 2) never arrived, exactly as in S3.2. D7 mirroring stays open.

Three portfolio fixtures quoted cost codes belonging to OTHER projects; the new
gate caught them. They now quote each project's own lines, and the two copied
REPLIES tables import the single source instead of drifting from it.

Load-bearing measured (tests/test_s40_cost_baseline_loadbearing.py), six
mutations all red: detach the reconciliation stage; detach the magnitude
tolerance; detach the road wiring; detach the bundle wiring; ignore the injected
cap registry; make the optional loader tolerant of malformed content. Control:
with the road wiring detached the repaired portfolio fixtures still pass, so
they are not masking the seam. 597 -> 612 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JdwK7bQ4BZkWH4t8MRDKb4
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 17:19:31 +02:00
commit 126807aee7
16 changed files with 645 additions and 69 deletions

View file

@ -8,11 +8,13 @@ the gated live arm (Step 14).
from __future__ import annotations
import json
from collections.abc import Callable, Sequence
import pytest
from agent_framework import BaseChatClient
from portfolio_optimiser.reference_domain import load_reference_projects
from portfolio_optimiser.simulation import ScriptedChatClient
from portfolio_optimiser.verdicts import VerdictStore, seed_store
@ -57,29 +59,63 @@ def make_client_factory() -> Callable[..., Callable[[str], BaseChatClient]]:
return _make
# A generic VALID SavingsProposal reply for any project not present in a portfolio reply map:
# affected total = 1 x 100_000 = 100_000, P90 = 0.30 x 100_000 = 30_000, claimed 20_000 <= both
# (Pydantic affected-total invariant and the validator P90 gate) -> always validates.
_DEFAULT_CLAIM = 20_000
# Last-resort reply for a prompt naming NO known reference project (the anchored per-project
# fallback below cannot be built then). Kept for that case only.
_PORTFOLIO_DEFAULT_REPLY = (
'{"measure":"Reduce scope","affected_items":'
'[{"code":"01.1","quantity":1,"unit_cost":100000}],"claimed_saving_nok":20000}'
)
def _anchored_default_replies() -> dict[str, str]:
"""A VALID default proposal PER reference project, quoting that project's OWN first cost line
verbatim (S4.0): since the road path anchors the validator to ``project.cost_items``, a generic
reply carrying an invented magnitude for code ``01.1`` is now correctly rejected as a
fabricated cost line. Anchoring the fixture is the fix; weakening the gate is not.
``claimed_saving_nok`` stays ``20_000`` for every project, exactly as the single generic reply
claimed before, so every ledger/goal/budget assertion built on that figure is unchanged. Each
project's first line is ``01.1 Rigg og drift`` at >= 480 000 NOK, so P90 (>= 144 000) clears the
claim on every project."""
replies: dict[str, str] = {}
for project in load_reference_projects():
line = project.cost_items[0]
replies[project.id] = json.dumps(
{
"measure": "Reduce scope",
"affected_items": [
{"code": line.code, "quantity": line.quantity, "unit_cost": line.unit_cost}
],
"claimed_saving_nok": _DEFAULT_CLAIM,
}
)
return replies
class _ProjectAwareUsageChatClient(ScriptedChatClient):
"""Selects its reply by scanning the incoming prompt for a known ``project_id`` substring (the
prompt embeds ``project.id`` at run.py:162 and generate.py:48), falling back to a default valid
proposal so ``run_portfolio``'s single ``client_factory`` stays production-shaped while tests
vary the proposal per project. A THIN subclass: the prompt-scan lives in its selector, the shared
``_inner_get_response`` body in the canonical."""
``_inner_get_response`` body in the canonical.
The fallback is itself project-aware (S4.0): a prompt naming a reference project gets that
project's baseline-anchored default reply, so an un-mapped project still produces a proposal the
anchored validator admits. Only a prompt naming NO known project falls through to
``default_reply``."""
def __init__(
self, replies: dict[str, str], *, default_reply: str, tokens_per_reply: int = 8
) -> None:
table = dict(replies)
anchored = _anchored_default_replies()
def _select(blob: str, _role: str) -> str:
return next((r for pid, r in table.items() if pid in blob), default_reply)
explicit = next((r for pid, r in table.items() if pid in blob), None)
if explicit is not None:
return explicit
return next((r for pid, r in anchored.items() if pid in blob), default_reply)
super().__init__(
reply_selector=_select, default_reply=default_reply, tokens_per_reply=tokens_per_reply