feat(visibility): en uforankret kjoering sier det - felt + linje (ORDRE 20260821T092039Z)
Maalt oekt 48: en bundle-kjoering uten cost-baseline.json gikk rc 0 uten et ord, og `grep baseline provenance.py outbox.py` ga null treff - hverken stdout, stempelet eller utboksen bar at validatorens steg 0 ble hoppet over. To tenner, begge smaa, og begge fra kjoeringens ENE oppslag av baselinen (koe-(p)): 1. ProvenanceStamp.cost_baseline_anchored - PAAKREVD bool uten default. Begge defaults lyver: True lar en glemsom konstruktoer paastaa en ankring som ikke skjedde, False underrapporterer en ekte. Naar utboksen gratis (write_proposal dumper hele stempelet). DryRunReport baerer det samme - en dry-run stopper foer noe stempel finnes. 2. run.cost_baseline_notice(anchored) - ENESTE renderer, tar den alt opploeste booleanen, returnerer None naar kjoeringen ER forankret (omisjon, aldri en tom rad). Printes paa tre flater: --live-dry-run, full enkeltkjoering, og per prosjekt i portefoeljemodus. IKKE foldet inn i mandate.announce, og det er en MAALING: den fyrer kun med --mandate, saa nettopp de bare bundle-dry-runsene defekten ble maalt paa ville fortsatt sagt ingenting - og den renderes foer run_project, altsaa foer noen har opploest baselinen. Ankeringen forblir VALGFRI (en pre-amendment-base kjoerer uendret) - dette er synlighet, ikke en ny nekt. Golden-transkriptet er byte-uendret: demoen kjoerer en base som HAR fila. Load-bearing MAALT (tests/test_baseline_visibility_loadbearing.py, 11 tester), seks mutasjoner alle roede mot HELE suiten + groenn kontroll 885/5: konstant stamp-wiring (3 roede) - konstant dry-run-wiring (1) - detach dry-run-printen (1) - renderer returnerer alltid linja (2, inkl. den forankrede kontrollen) - detach full-run-printen (1) - detach portefoelje-printen (1). Portefoelje-armen er DEFENSIV og uttalt (ingen referanse-prosjekt setter bundle_dir; budget_stop-presedensen, crafted PortfolioResult). Det paakrevde feltet tvang fem eksisterende test-konstruktoerer til aa ta stilling. Dokumentene som beskrev den gamle stillheten er rettet: kunnskapsbase-for-en-kjoring.md S4.1 (tabellraden re-maalt live), S6 og S7; README «How it is set up»; CLAUDE.md S4.0-raden. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DA6HAF8HFQxGYC2h6ypRQe
This commit is contained in:
parent
0bc8dae5c0
commit
156312c32e
11 changed files with 481 additions and 17 deletions
327
tests/test_baseline_visibility_loadbearing.py
Normal file
327
tests/test_baseline_visibility_loadbearing.py
Normal file
|
|
@ -0,0 +1,327 @@
|
|||
"""An UN-ANCHORED run says so — in a machine-readable field AND in one line on stdout.
|
||||
|
||||
S4.0 made the deterministic gate anchorable: when a bundle ships ``cost-baseline.json`` the
|
||||
validator's stage 0 reconciles every ``affected_item`` against the project's own cost lines BEFORE
|
||||
the solver, and when it does not, that stage is simply skipped (``None`` = pre-amendment behaviour,
|
||||
which is what keeps every commons-owned golden bundle running). The anchoring stayed OPTIONAL on
|
||||
purpose — and that is not what this file changes.
|
||||
|
||||
What it changes is that the skip was INVISIBLE. Measured (session 48, ``9d149b3``): four
|
||||
``--live-dry-run``s over copies of the veglys bundle — intact rc 0 · without ``validator-input.json``
|
||||
rc 1 · **without ``cost-baseline.json`` rc 0 with no message at all** · corrupt baseline rc 1. And
|
||||
``grep baseline provenance.py outbox.py`` returned 0 hits, so neither the stamp nor the outbox
|
||||
artefacts carried it either. An operator could therefore run the whole gate un-anchored, read a
|
||||
clean rc 0, and have nothing anywhere to tell them the fabrication stage never ran.
|
||||
|
||||
Two teeth, both small:
|
||||
|
||||
1. ``ProvenanceStamp.cost_baseline_anchored`` — a REQUIRED bool, no default. "Was the gate
|
||||
anchored" is a binary fact about a falsifier, in the same class as ``BudgetExceeded``'s
|
||||
``kind``/``limit``/``observed`` (kø-(y)): it must be readable by machine, not inferred from
|
||||
prose. It carries no default because BOTH defaults lie — ``True`` would let a forgetful
|
||||
constructor claim an anchoring that never happened, ``False`` would under-claim a real one — and
|
||||
a binary fact with no honest default is exactly what a required field is for. It reaches the
|
||||
outbox for free: ``outbox.write_proposal`` dumps the whole stamp.
|
||||
2. ``run.cost_baseline_notice`` — ONE renderer, rendering ONE line when the run is un-anchored and
|
||||
``None`` when it is anchored. Omission, not an empty row, mirrors ``mandate.announce``'s rule
|
||||
that a line for something the run does not have is left out rather than rendered blank.
|
||||
|
||||
**The line is rendered from the run's OWN resolution, never from a second read of the bundle**
|
||||
(kø-(p)). ``run_project`` is the single place that calls ``okf.load_optional_cost_baseline`` on the
|
||||
run path; the fact leaves the run as a typed field on ``DryRunReport`` and on ``ProvenanceStamp``,
|
||||
and ``main`` prints from that. Rendering it inside ``mandate.announce`` was MEASURED and rejected:
|
||||
``announce`` fires only when ``--mandate`` is given, so the very runs this file exists for — the
|
||||
four bare dry-runs above, none of which had a mandate — would still have printed nothing.
|
||||
|
||||
Arms:
|
||||
(a) the provenance field is ``False`` on an un-anchored bundle run and ``True`` on an anchored one,
|
||||
end-to-end through ``run_project`` (+ the road path, which is anchored by construction);
|
||||
(b) the notice EXISTS un-anchored and is ABSENT anchored — asserted on a sentinel that the anchored
|
||||
branch cannot contain, because it prints no line at all (never a substring both branches share:
|
||||
the 08-09 class);
|
||||
(c) both CLI surfaces carry it — ``--live-dry-run`` and the full run;
|
||||
(d) the outbox artefact carries the field.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from conftest import SyntheticUsageChatClient
|
||||
|
||||
from portfolio_optimiser import run
|
||||
from portfolio_optimiser.ir import AffectedItem, SavingsProposal
|
||||
from portfolio_optimiser.provenance import Citation, ProvenanceStamp
|
||||
from portfolio_optimiser.retrieval import TextSpan
|
||||
from portfolio_optimiser.run import (
|
||||
DryRunReport,
|
||||
PortfolioResult,
|
||||
RunResult,
|
||||
cost_baseline_notice,
|
||||
run_project,
|
||||
)
|
||||
from portfolio_optimiser.validator import Rejection
|
||||
from portfolio_optimiser.verdicts import ProposalFeatures, Verdict, VerdictStore
|
||||
|
||||
_DATA = Path(__file__).resolve().parents[1] / "src" / "portfolio_optimiser" / "data" / "bundles"
|
||||
#: The ONLY repo-local bundle shipping a ``cost-baseline.json`` (S4.0 fixture) -> anchored.
|
||||
BASELINE_BUNDLE = _DATA / "bygg-energi-baseline-mikro"
|
||||
#: A bundle written before the amendment -> legitimately un-anchored (this is the case under test).
|
||||
PRE_AMENDMENT_BUNDLE = _DATA / "bygg-energi-mikro-a"
|
||||
|
||||
_VERDICT_INPUT = {"decision": "approved", "rationale": "expert reviewed (sim)"}
|
||||
|
||||
#: A reply that reconciles against the S4.0 fixture's own line, so the ANCHORED control run reaches
|
||||
#: a proposal rather than being rejected by stage 0 — the arms below are about visibility, and a
|
||||
#: control that died in the gate would not exercise the stamp.
|
||||
_REPLY = json.dumps(
|
||||
{
|
||||
"measure": "LED-retrofit",
|
||||
"affected_items": [{"code": "ENERGI-TOTAL-EL", "quantity": 180000, "unit_cost": 1.0}],
|
||||
"claimed_saving_nok": 30000,
|
||||
}
|
||||
)
|
||||
|
||||
#: The word the un-anchored line carries and the anchored branch cannot: it prints NO line at all.
|
||||
_SENTINEL = "un-anchored"
|
||||
|
||||
|
||||
def _factory(reply: str = _REPLY):
|
||||
def factory(role: str):
|
||||
return SyntheticUsageChatClient(default_reply=reply)
|
||||
|
||||
return factory
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _isolate_model_env(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Hermetic env (mirrors ``test_scripted_cli_door_loadbearing``): the operator's Foundry
|
||||
overrides must not reach the CLI arms."""
|
||||
monkeypatch.delenv("PORTFOLIO_MODEL_MAP", raising=False)
|
||||
monkeypatch.delenv("PORTFOLIO_FOUNDRY_PROJECT_ENDPOINT", raising=False)
|
||||
|
||||
|
||||
# --- Arm (a): the structured field ----------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_provenance_records_an_unanchored_bundle_run(fresh_store) -> None:
|
||||
"""RED: a bundle with no ``cost-baseline.json`` stamps ``cost_baseline_anchored=False``. Detach
|
||||
the wiring (stamp a constant, or drop the field) and the run again records nothing about the
|
||||
skipped stage."""
|
||||
result = await run_project(
|
||||
"BYGG-ENERGI-MIKRO-A",
|
||||
"local",
|
||||
docs_dir=str(PRE_AMENDMENT_BUNDLE),
|
||||
bundle_dir=str(PRE_AMENDMENT_BUNDLE),
|
||||
verdict_input=_VERDICT_INPUT,
|
||||
client_factory=_factory(),
|
||||
store=fresh_store,
|
||||
)
|
||||
assert result.provenance.cost_baseline_anchored is False
|
||||
|
||||
|
||||
async def test_provenance_records_an_anchored_bundle_run(fresh_store) -> None:
|
||||
"""Causality control: the SAME code path over a bundle that DOES ship a baseline stamps
|
||||
``True``. Without this the arm above would pass on a constant ``False``."""
|
||||
result = await run_project(
|
||||
"BYGG-ENERGI-BASELINE-MIKRO",
|
||||
"local",
|
||||
docs_dir=str(BASELINE_BUNDLE),
|
||||
bundle_dir=str(BASELINE_BUNDLE),
|
||||
verdict_input=_VERDICT_INPUT,
|
||||
client_factory=_factory(),
|
||||
store=fresh_store,
|
||||
)
|
||||
assert result.provenance.cost_baseline_anchored is True
|
||||
|
||||
|
||||
async def test_road_path_is_anchored_by_construction(docs_dir, fresh_store) -> None:
|
||||
"""The road path derives its baseline from the reference project's own ``cost_items``, so it is
|
||||
ALWAYS anchored — the stamp says so rather than leaving the reader to know it."""
|
||||
result = await run_project(
|
||||
"FV42-GSV-E1",
|
||||
"local",
|
||||
docs_dir=docs_dir,
|
||||
verdict_input=_VERDICT_INPUT,
|
||||
client_factory=_factory(
|
||||
json.dumps(
|
||||
{
|
||||
"measure": "Reduce scope",
|
||||
"affected_items": [{"code": "05.2", "quantity": 4300.0, "unit_cost": 215.0}],
|
||||
"claimed_saving_nok": 200000.0,
|
||||
}
|
||||
)
|
||||
),
|
||||
store=fresh_store,
|
||||
)
|
||||
assert result.provenance.cost_baseline_anchored is True
|
||||
|
||||
|
||||
def test_the_field_has_no_default() -> None:
|
||||
"""A binary fact about a falsifier gets no default: both defaults lie (see the module docstring),
|
||||
so a stamp that forgot to say must not construct at all."""
|
||||
with pytest.raises(Exception):
|
||||
ProvenanceStamp( # type: ignore[call-arg]
|
||||
citations=[],
|
||||
model="m",
|
||||
role="proposer",
|
||||
validator_decision="validated",
|
||||
token_usage=0,
|
||||
)
|
||||
|
||||
|
||||
# --- Arm (b): the renderer ------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_notice_is_rendered_only_when_unanchored() -> None:
|
||||
"""One renderer, two branches that share NO wording: un-anchored returns a line carrying the
|
||||
sentinel, anchored returns ``None`` (omitted, never an empty row — ``announce``'s rule)."""
|
||||
unanchored = cost_baseline_notice(False)
|
||||
assert unanchored is not None
|
||||
assert _SENTINEL in unanchored
|
||||
assert cost_baseline_notice(True) is None
|
||||
|
||||
|
||||
def test_dry_run_report_carries_the_anchoring(fresh_store) -> None:
|
||||
"""The dry-run type is the carrier for the surface the order measured: a run that stops before
|
||||
the first model call still knows whether the gate would have been anchored."""
|
||||
assert "cost_baseline_anchored" in DryRunReport.__dataclass_fields__
|
||||
|
||||
|
||||
# --- Arm (c): both CLI surfaces -------------------------------------------------------------------
|
||||
|
||||
|
||||
def _dry_run(bundle: Path, project_id: str) -> list[str]:
|
||||
return [
|
||||
project_id,
|
||||
"--docs-dir",
|
||||
str(bundle),
|
||||
"--bundle-dir",
|
||||
str(bundle),
|
||||
"--live-dry-run",
|
||||
]
|
||||
|
||||
|
||||
def test_cli_dry_run_announces_an_unanchored_bundle(capsys) -> None:
|
||||
"""RED (the measured defect, verbatim): ``--live-dry-run`` over a bundle without
|
||||
``cost-baseline.json`` exits 0 — and now SAYS the gate is un-anchored instead of exiting
|
||||
silently."""
|
||||
rc = run.main(_dry_run(PRE_AMENDMENT_BUNDLE, "BYGG-ENERGI-MIKRO-A"))
|
||||
assert rc == 0
|
||||
assert _SENTINEL in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_cli_dry_run_says_nothing_when_the_bundle_is_anchored(capsys) -> None:
|
||||
"""Control: the anchored bundle prints NO baseline line at all. Lines for what a run does not
|
||||
have are omitted (``announce``); a run that IS anchored has nothing to warn about."""
|
||||
rc = run.main(_dry_run(BASELINE_BUNDLE, "BYGG-ENERGI-BASELINE-MIKRO"))
|
||||
assert rc == 0
|
||||
out = capsys.readouterr().out
|
||||
assert _SENTINEL not in out
|
||||
assert "Cost baseline" not in out
|
||||
|
||||
|
||||
def test_cli_full_run_announces_an_unanchored_bundle(tmp_path, capsys) -> None:
|
||||
"""The full-run surface too, through the offline scripted door — so the notice is a property of
|
||||
a RUN, not of the dry-run branch alone."""
|
||||
replies = tmp_path / "replies.json"
|
||||
replies.write_text(
|
||||
json.dumps({"proposer": _REPLY, "checker": "Holder. VERDICT: APPROVE"}), encoding="utf-8"
|
||||
)
|
||||
rc = run.main(
|
||||
[
|
||||
"BYGG-ENERGI-MIKRO-A",
|
||||
"--docs-dir",
|
||||
str(PRE_AMENDMENT_BUNDLE),
|
||||
"--bundle-dir",
|
||||
str(PRE_AMENDMENT_BUNDLE),
|
||||
"--scripted-replies",
|
||||
str(replies),
|
||||
]
|
||||
)
|
||||
assert rc == 0
|
||||
assert _SENTINEL in capsys.readouterr().out
|
||||
|
||||
|
||||
# --- Arm (d): the outbox artefact -----------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_outbox_proposal_carries_the_anchoring(tmp_path, fresh_store) -> None:
|
||||
"""The outbox needed no change of its own: the artefact dumps the whole stamp, so the field
|
||||
lands in ``{run_id}-proposal.json`` the moment it exists on the stamp."""
|
||||
await run_project(
|
||||
"BYGG-ENERGI-MIKRO-A",
|
||||
"local",
|
||||
docs_dir=str(PRE_AMENDMENT_BUNDLE),
|
||||
bundle_dir=str(PRE_AMENDMENT_BUNDLE),
|
||||
verdict_input=_VERDICT_INPUT,
|
||||
client_factory=_factory(),
|
||||
store=fresh_store,
|
||||
outbox_dir=str(tmp_path),
|
||||
run_id="vis-1",
|
||||
)
|
||||
payload = json.loads((tmp_path / "vis-1-proposal.json").read_text(encoding="utf-8"))
|
||||
assert payload["provenance"]["cost_baseline_anchored"] is False
|
||||
|
||||
|
||||
# --- Arm (e): the portfolio surface (DEFENSIVE, and said out loud) --------------------------------
|
||||
|
||||
|
||||
def _unanchored_run() -> RunResult:
|
||||
"""One ``RunResult`` whose stamp says the gate was un-anchored."""
|
||||
proposal = SavingsProposal(
|
||||
project_id="P",
|
||||
measure="m",
|
||||
affected_items=[AffectedItem(code="05.2", quantity=1.0, unit_cost=1.0)],
|
||||
claimed_saving_nok=1.0,
|
||||
assumptions={},
|
||||
)
|
||||
return RunResult(
|
||||
outcome=Rejection(proposal=proposal, reason="r"),
|
||||
provenance=ProvenanceStamp(
|
||||
citations=[Citation(file="f.md", locator=TextSpan(0, 1), snippet="x")],
|
||||
model="synthetic",
|
||||
role="proposer",
|
||||
validator_decision="rejected",
|
||||
token_usage=1,
|
||||
cost_baseline_anchored=False,
|
||||
),
|
||||
verdict=Verdict(
|
||||
id="v1",
|
||||
proposal_features=ProposalFeatures(
|
||||
affected_codes=frozenset({"05.2"}), measure_type="m", claimed_saving_nok=1.0
|
||||
),
|
||||
decision="rejected",
|
||||
rationale="r",
|
||||
),
|
||||
retrieved=[],
|
||||
store=VerdictStore([]),
|
||||
debate_output="",
|
||||
)
|
||||
|
||||
|
||||
def test_portfolio_surface_announces_an_unanchored_run(monkeypatch, capsys) -> None:
|
||||
"""The portfolio branch reports per project, because anchoring is a per-project fact.
|
||||
|
||||
Driven by a CRAFTED ``PortfolioResult`` (the ``budget_stop`` precedent in
|
||||
``test_portfolio_cli_offline_loadbearing``), and for the same measured reason: no reference
|
||||
project sets ``bundle_dir``, so every portfolio run today takes the road path and is anchored by
|
||||
construction. This arm is therefore DEFENSIVE — it guards the surface for the day a bundle-backed
|
||||
project is wired into a pass, rather than covering a path reachable now."""
|
||||
|
||||
async def _fake(*_args, **_kwargs) -> PortfolioResult:
|
||||
return PortfolioResult(
|
||||
runs=(_unanchored_run(),),
|
||||
store=VerdictStore([]),
|
||||
validated_count=0,
|
||||
rejected_count=1,
|
||||
sum_claimed_saving_nok=0.0,
|
||||
sum_token_usage=1,
|
||||
)
|
||||
|
||||
monkeypatch.setattr(run, "run_portfolio", _fake)
|
||||
rc = run.main(["--portfolio"])
|
||||
assert rc == 0
|
||||
assert _SENTINEL in capsys.readouterr().out
|
||||
|
|
@ -33,6 +33,9 @@ _PROVENANCE = ProvenanceStamp(
|
|||
role="proposer",
|
||||
validator_decision="validated",
|
||||
token_usage=8,
|
||||
# These fixtures stand in for an ordinary complete run; the road path is anchored by
|
||||
# construction, so ``True`` is the honest value here. The un-anchored case has its own file.
|
||||
cost_baseline_anchored=True,
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -31,6 +31,9 @@ _PROVENANCE = ProvenanceStamp(
|
|||
role="proposer",
|
||||
validator_decision="validated",
|
||||
token_usage=8,
|
||||
# These fixtures stand in for an ordinary complete run; the road path is anchored by
|
||||
# construction, so ``True`` is the honest value here. The un-anchored case has its own file.
|
||||
cost_baseline_anchored=True,
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -98,6 +98,9 @@ _PROVENANCE = ProvenanceStamp(
|
|||
role="proposer",
|
||||
validator_decision="validated",
|
||||
token_usage=8,
|
||||
# These fixtures stand in for an ordinary complete run; the road path is anchored by
|
||||
# construction, so ``True`` is the honest value here. The un-anchored case has its own file.
|
||||
cost_baseline_anchored=True,
|
||||
)
|
||||
_VALIDATED = ValidatedProposal(
|
||||
proposal=_PROPOSAL, p10=100.0, p50=150.0, p90=200.0, nominal_feasible=180.0
|
||||
|
|
|
|||
|
|
@ -62,6 +62,9 @@ _PROVENANCE = ProvenanceStamp(
|
|||
role="proposer",
|
||||
validator_decision="validated",
|
||||
token_usage=8,
|
||||
# These fixtures stand in for an ordinary complete run; the road path is anchored by
|
||||
# construction, so ``True`` is the honest value here. The un-anchored case has its own file.
|
||||
cost_baseline_anchored=True,
|
||||
)
|
||||
_VALIDATED = ValidatedProposal(
|
||||
proposal=_PROPOSAL, p10=100.0, p50=150.0, p90=200.0, nominal_feasible=180.0
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ def _stamp() -> ProvenanceStamp:
|
|||
role="proposer",
|
||||
validator_decision="validated",
|
||||
token_usage=42,
|
||||
cost_baseline_anchored=True,
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -30,6 +31,7 @@ def test_zero_citations_is_rejected() -> None:
|
|||
role="proposer",
|
||||
validator_decision="validated",
|
||||
token_usage=1,
|
||||
cost_baseline_anchored=True,
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue