feat(validator): anchor the deterministic gate to the project's real cost baseline (S4.0)
Every stage of validate_proposal reasoned only about numbers the proposal itself supplied, so an internally-consistent hallucination cleared the whole gate (F3). A new stage 0 reconciles each affected_item against the project's CostBaseline before the CBC solve: an unknown cost code is rejected, and a real code carrying a quantity/unit_cost outside the configured tolerance (5% default, relative to the baseline value) is rejected. Validation, never repair. The baseline argument is OPTIONAL (None = pre-S4.0 behaviour), but both run paths set it: the road path projects project.cost_items, the bundle path loads cost-baseline.json when the bundle ships one. Bundles written before the amendment stay un-anchored, so the commons-owned goldens run byte-identically; a baseline that exists but is malformed still raises on both loaders. F8: the method-specific cap now comes from the METHOD_CAPS registry (measure type -> fraction, injectable) instead of an energy_efficiency string comparison. The baseline format and tolerance semantics were decided locally — the commons amendment (D-A pt. 2) never arrived, exactly as in S3.2. D7 mirroring stays open. Three portfolio fixtures quoted cost codes belonging to OTHER projects; the new gate caught them. They now quote each project's own lines, and the two copied REPLIES tables import the single source instead of drifting from it. Load-bearing measured (tests/test_s40_cost_baseline_loadbearing.py), six mutations all red: detach the reconciliation stage; detach the magnitude tolerance; detach the road wiring; detach the bundle wiring; ignore the injected cap registry; make the optional loader tolerant of malformed content. Control: with the road wiring detached the repaired portfolio fixtures still pass, so they are not masking the seam. 597 -> 612 tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JdwK7bQ4BZkWH4t8MRDKb4
This commit is contained in:
parent
012adc0a3c
commit
126807aee7
16 changed files with 645 additions and 69 deletions
|
|
@ -8,11 +8,13 @@ the gated live arm (Step 14).
|
|||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Callable, Sequence
|
||||
|
||||
import pytest
|
||||
from agent_framework import BaseChatClient
|
||||
|
||||
from portfolio_optimiser.reference_domain import load_reference_projects
|
||||
from portfolio_optimiser.simulation import ScriptedChatClient
|
||||
from portfolio_optimiser.verdicts import VerdictStore, seed_store
|
||||
|
||||
|
|
@ -57,29 +59,63 @@ def make_client_factory() -> Callable[..., Callable[[str], BaseChatClient]]:
|
|||
return _make
|
||||
|
||||
|
||||
# A generic VALID SavingsProposal reply for any project not present in a portfolio reply map:
|
||||
# affected total = 1 x 100_000 = 100_000, P90 = 0.30 x 100_000 = 30_000, claimed 20_000 <= both
|
||||
# (Pydantic affected-total invariant and the validator P90 gate) -> always validates.
|
||||
_DEFAULT_CLAIM = 20_000
|
||||
# Last-resort reply for a prompt naming NO known reference project (the anchored per-project
|
||||
# fallback below cannot be built then). Kept for that case only.
|
||||
_PORTFOLIO_DEFAULT_REPLY = (
|
||||
'{"measure":"Reduce scope","affected_items":'
|
||||
'[{"code":"01.1","quantity":1,"unit_cost":100000}],"claimed_saving_nok":20000}'
|
||||
)
|
||||
|
||||
|
||||
def _anchored_default_replies() -> dict[str, str]:
|
||||
"""A VALID default proposal PER reference project, quoting that project's OWN first cost line
|
||||
verbatim (S4.0): since the road path anchors the validator to ``project.cost_items``, a generic
|
||||
reply carrying an invented magnitude for code ``01.1`` is now — correctly — rejected as a
|
||||
fabricated cost line. Anchoring the fixture is the fix; weakening the gate is not.
|
||||
|
||||
``claimed_saving_nok`` stays ``20_000`` for every project, exactly as the single generic reply
|
||||
claimed before, so every ledger/goal/budget assertion built on that figure is unchanged. Each
|
||||
project's first line is ``01.1 Rigg og drift`` at >= 480 000 NOK, so P90 (>= 144 000) clears the
|
||||
claim on every project."""
|
||||
replies: dict[str, str] = {}
|
||||
for project in load_reference_projects():
|
||||
line = project.cost_items[0]
|
||||
replies[project.id] = json.dumps(
|
||||
{
|
||||
"measure": "Reduce scope",
|
||||
"affected_items": [
|
||||
{"code": line.code, "quantity": line.quantity, "unit_cost": line.unit_cost}
|
||||
],
|
||||
"claimed_saving_nok": _DEFAULT_CLAIM,
|
||||
}
|
||||
)
|
||||
return replies
|
||||
|
||||
|
||||
class _ProjectAwareUsageChatClient(ScriptedChatClient):
|
||||
"""Selects its reply by scanning the incoming prompt for a known ``project_id`` substring (the
|
||||
prompt embeds ``project.id`` at run.py:162 and generate.py:48), falling back to a default valid
|
||||
proposal — so ``run_portfolio``'s single ``client_factory`` stays production-shaped while tests
|
||||
vary the proposal per project. A THIN subclass: the prompt-scan lives in its selector, the shared
|
||||
``_inner_get_response`` body in the canonical."""
|
||||
``_inner_get_response`` body in the canonical.
|
||||
|
||||
The fallback is itself project-aware (S4.0): a prompt naming a reference project gets that
|
||||
project's baseline-anchored default reply, so an un-mapped project still produces a proposal the
|
||||
anchored validator admits. Only a prompt naming NO known project falls through to
|
||||
``default_reply``."""
|
||||
|
||||
def __init__(
|
||||
self, replies: dict[str, str], *, default_reply: str, tokens_per_reply: int = 8
|
||||
) -> None:
|
||||
table = dict(replies)
|
||||
anchored = _anchored_default_replies()
|
||||
|
||||
def _select(blob: str, _role: str) -> str:
|
||||
return next((r for pid, r in table.items() if pid in blob), default_reply)
|
||||
explicit = next((r for pid, r in table.items() if pid in blob), None)
|
||||
if explicit is not None:
|
||||
return explicit
|
||||
return next((r for pid, r in anchored.items() if pid in blob), default_reply)
|
||||
|
||||
super().__init__(
|
||||
reply_selector=_select, default_reply=default_reply, tokens_per_reply=tokens_per_reply
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue