portfolio-optimiser-claude/tests/test_valuereport_loadbearing.py
Kjell Tore Guttormsen 8a141370f3 test(valuereport): bind _SHARE_DIGITS to its MEASURED band, not to itself
The constant was detach-proof but value-unproven: mutating 6 -> 4 left all 628
tests green, so nothing held the figure to the claim it carries. Measuring what
actually constrains it gave a narrower answer than the premise assumed — 6 -> 4
CANNOT be made red without inventing a resolution requirement no layer states,
and §1 forbids asserting more than the implementation carries.

Measured band, both ends now load-bearing:
  * d >= 17 -> the 1-ULP float tail of a cohort subtraction reaches the JSON
    bytes (0.1 - 0.3 publishes as -0.19999999999999998, not -0.2).
  * d <= 2  -> the rendered percent moves (2/7 renders 29.0%, not 28.6%).
  * d in [3, 16] -> identical to every consumer this system has.

Both proofs are stated WITHOUT reference to the constant's own value — the
exact decimal difference of the two PUBLISHED shares, and a percent computed
from the RAW NOK figures — so they bind the claim rather than the number. A
literal like 0.142857 would only have bound 6 to itself.

The :61 comment justified only the upper end; it now records the measurement
and says plainly that 6 is convention inside the band, not a derived figure.

Mutation-verified: d=2 RED, d=3/4/5/16 GREEN, d=17 RED. Suite 628 -> 631.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
2026-07-26 15:28:20 +02:00

507 lines
20 KiB
Python

"""Per-run value report — LOAD-BEARING (S5.4-analog; §1, §11; paritetsrad 25; K11).
The seam this file keeps alive: every run/portfolio pass can produce a
deterministic value report — modelled → expert-corrected → realized, goal
progress, a QUANTIFIED learning effect, and cost-against-value — as a PURE
PROJECTION over state that already exists (the outbox pairs from K5, the §4.2
inbox verdicts, the K1 ledger) with not one model call and not one byte written.
The honesty rule (§1) is the spine of this module, so it is the spine of this
file: a figure the layers do not carry is reported as UNMARKED, never filled in
from the stage before it. An expert who has not judged is not an expert who
approved; an ``approved_with_adjustment`` verdict carries no adjusted amount in
the §4.2 shape, so the corrected value is unquantified — NOT the modelled claim.
A report that silently promoted modelled figures into the realized column would
be the single most flattering lie this system could tell about itself.
Three detached seams proven RED here:
* Detach proof (the honesty boundary): let the corrected/realized figure fall
back to the modelled claim for a project with no verdict →
``test_project_without_verdict_has_unmarked_realized`` and
``test_unquantified_adjustment_is_never_the_claim`` go red.
* Detach proof (the gap computation): drop the gap arithmetic from the cohort
roll-up so ``gap_share`` stays flat/None → ``test_learning_effect_rises_with_
approval_share`` goes red (a rising approval share alone must NOT be reported
as learning without the shrinking gap that backs it).
* Detach proof (the projection purity): let any build path write a byte into the
three read layers → ``test_report_never_writes_the_read_layers`` goes red.
EVERY test projects from a per-test COPY of the committed fixtures, never the
committed tree in place. That is not tidiness — the purity test was green-but-
dead when tests read the tree directly: an earlier test's stray write landed in
the committed tree, so it was already present in the "before" snapshot and the
detached write reproduced it byte-for-byte. Reading only copies removes the
channel through which one test can pre-seed another's evidence.
Key assumption (tested, not asserted in prose): the outbox + inbox + ledger carry
everything the report needs — it is a pure projection with NO new state. Proven
by building the whole report from the COMMITTED fixtures under
``tests/data/valuereport/`` without executing a single run.
"""
from __future__ import annotations
import json
import shutil
from dataclasses import dataclass
from decimal import Decimal
from pathlib import Path
from typing import Sequence
import pytest
from portfolio_optimiser_claude.experience import CandidateFeatures
from portfolio_optimiser_claude.goals import GoalContract
from portfolio_optimiser_claude.inbox import VerdictDocument, write_verdict
from portfolio_optimiser_claude.ir import AffectedItem, SavingsProposal
from portfolio_optimiser_claude.valuereport import (
_SHARE_DIGITS,
ProjectValue,
ValueReport,
build_value_report,
main,
render_report,
report_to_json,
)
FIXTURES = Path(__file__).parent / "data" / "valuereport"
# The committed fixture scenario, stated once so every expectation below is
# readable against it (generated with the REAL primitives, so the verdict_id
# join is genuine — never hand-typed ids):
#
# run-001 FV42-P1 LED-retrofit 100 000 -> verdict: rejected
# run-002 FV42-P1 VFD-pumpe 80 000 -> verdict: approved -> ledger
# run-003 FV42-P2 SD-anlegg 50 000 -> NO verdict (pending)
# run-004 FV42-P3 Etterisolering 60 000 -> verdict: approved_with_adjustment
#
# Ledger total: 80 000 (only the plain approval passed the expert gate).
MODELLED_TOTAL = 290_000.0
REALIZED_TOTAL = 80_000.0
# The committed tree, pinned: a stray file written by a run under test would
# otherwise be committed as if it belonged here.
EXPECTED_FIXTURE_FILES = {
"inbox/82d6e4e60ae87e14.json",
"inbox/94a98883515d1267.json",
"inbox/fe354d69ef8a47b1.json",
"ledger.json",
"outbox/run-001-outcome.json",
"outbox/run-001-proposal.json",
"outbox/run-002-outcome.json",
"outbox/run-002-proposal.json",
"outbox/run-003-outcome.json",
"outbox/run-003-proposal.json",
"outbox/run-004-outcome.json",
"outbox/run-004-proposal.json",
}
@dataclass(frozen=True)
class Layers:
"""The three read layers, as a private copy of the committed fixtures."""
root: Path
outbox: Path
inbox: Path
ledger: Path
@pytest.fixture
def layers(tmp_path: Path) -> Layers:
"""A fresh copy of the committed fixtures — the tree under test is never the real one."""
root = tmp_path / "layers"
shutil.copytree(FIXTURES, root)
return Layers(
root=root, outbox=root / "outbox", inbox=root / "inbox", ledger=root / "ledger.json"
)
def _build(layers: Layers, **kwargs: object) -> ValueReport:
return build_value_report(
outbox_dir=layers.outbox,
inbox_dir=layers.inbox,
ledger_path=layers.ledger,
**kwargs, # type: ignore[arg-type]
)
def _project(report: ValueReport, project_id: str) -> ProjectValue:
return next(project for project in report.projects if project.project_id == project_id)
# --- the three stages: modelled -> expert-corrected -> realized ------------------------------
def test_committed_fixture_tree_is_exactly_what_it_claims() -> None:
"""No stray file has crept into the committed layers (a run under test writing home)."""
present = {
str(path.relative_to(FIXTURES))
for path in FIXTURES.rglob("*")
if path.is_file() and "__pycache__" not in path.parts
}
assert present == EXPECTED_FIXTURE_FILES
def test_report_is_a_pure_projection_of_the_committed_fixtures(layers: Layers) -> None:
"""The key assumption: outbox + inbox + ledger carry everything (no run needed)."""
report = _build(layers)
assert report.modelled_nok == MODELLED_TOTAL
assert report.realized_nok == REALIZED_TOTAL
assert report.n_proposals == 4
assert report.n_settled == 3
assert report.n_pending == 1
def test_three_stages_are_distinct_columns_per_project(layers: Layers) -> None:
"""Each stage is measured from its OWN layer — never copied from the one before."""
p1 = _project(_build(layers), "FV42-P1") # one rejected (->0), one approved (->80k)
assert p1.modelled_nok == 180_000.0
assert p1.expert_corrected_nok == 80_000.0
assert p1.realized_nok == 80_000.0
def test_project_without_verdict_has_unmarked_realized(layers: Layers) -> None:
"""§1 honesty: no verdict → realized is UNMARKED, and NEVER equal to modelled.
Detach the honesty boundary (the figure falls back to the modelled claim) and
this test goes red — which is the whole point of it existing.
"""
p2 = _project(_build(layers), "FV42-P2") # run-003, no verdict at all
assert p2.realized_nok is None
assert p2.expert_corrected_nok is None
assert p2.modelled_nok == 50_000.0
assert p2.realized_nok != p2.modelled_nok
def test_unquantified_adjustment_is_never_the_claim(layers: Layers) -> None:
"""``approved_with_adjustment`` carries NO amount in §4.2 → unquantified, not the claim."""
p3 = _project(_build(layers), "FV42-P3") # run-004, adjusted without a figure
assert p3.expert_corrected_nok is None
assert p3.realized_nok is None
assert p3.modelled_nok == 60_000.0
def test_partial_quantification_is_counted_not_hidden(layers: Layers) -> None:
"""A partly-quantified portfolio says so in numbers — never a silent full-looking sum."""
report = _build(layers)
assert report.expert_corrected_nok == 80_000.0
assert report.quantified_proposals == 2
assert report.unquantified_proposals == 2
# --- quantified learning effect (the plan's first RED) ---------------------------------------
def test_learning_effect_rises_with_approval_share(layers: Layers) -> None:
"""Two cohorts of runs with a rising approval share → the metric rises AND the gap shrinks.
Detach the gap computation (cohort gap_share left flat/None) and this test
goes red: a rising approval share on its own is NOT evidence of learning, so
the report must never claim it without the gap that backs it.
"""
learning = _build(layers).learning
assert learning.comparable is True
# earlier cohort = run-001 (rejected), later = run-002 + run-004 (both accepted)
assert learning.earlier_approval_share == 0.0
assert learning.later_approval_share == 1.0
assert learning.approval_share_delta == 1.0
# ...and the modelled→corrected gap shrank from "the whole claim" to nothing.
assert learning.earlier_gap_share == 1.0
assert learning.later_gap_share == 0.0
assert learning.gap_share_delta == -1.0
assert learning.gap_shrinking is True
def test_learning_needs_two_cohorts_before_it_claims_anything(layers: Layers) -> None:
"""One settled run cannot evidence a trend — the report says so instead of inventing one."""
for verdict_file in layers.inbox.glob("*.json"):
verdict_file.unlink() # no verdicts at all → nothing settled to compare
learning = _build(layers).learning
assert learning.comparable is False
assert learning.approval_share_delta is None
assert learning.gap_share_delta is None
assert learning.gap_shrinking is None
# --- what _SHARE_DIGITS is worth: the MEASURED band, not a detach proof -----------------------
#
# The constant was detach-proof but value-unproven: mutating 6 → 4 left all 628
# tests green, so nothing bound the figure to the claim it carries. These two
# tests bind the band that was actually MEASURED, and it is narrower than the
# comment alone suggested:
#
# * d >= 17 → the 1-ULP float tail of a cohort subtraction survives into the
# JSON bytes (0.1 - 0.3 publishes as -0.19999999999999998, not -0.2).
# * d <= 2 → the rendered percent moves (2/7 renders 29.0%, not 28.6%).
# * d in [3, 16] → every consumer this system has behaves identically.
#
# So 6 → 4 CANNOT be made red without inventing a resolution requirement no
# layer states — and §1 forbids asserting more than the implementation carries.
# The honest binding is the band; the choice of 6 inside it is convention.
_COHORT_PROJECT = "FV42-P1"
@dataclass(frozen=True)
class _Settled:
"""One judged proposal for a synthetic two-cohort history."""
run_id: str
claimed_nok: float
decision: str
# Cohort split is the midpoint of the settled proposals, so the first two runs
# are the earlier cohort and the last two the later one. A rejected verdict
# corrects to 0 and an approved one leaves the claim standing, which is what
# makes each cohort's gap_share the ratio of rejected NOK to quantified NOK.
#
# gap_share 30000/100000 = 0.3 → 10000/100000 = 0.1. The float subtraction
# 0.1 - 0.3 is -0.19999999999999998: a tail the rounding exists to absorb.
_NOISE_HISTORY = (
_Settled("run-101", 30_000.0, "rejected"),
_Settled("run-102", 70_000.0, "approved"),
_Settled("run-103", 10_000.0, "rejected"),
_Settled("run-104", 90_000.0, "approved"),
)
# gap_share 20000/70000 = 2/7 → 30000/70000 = 3/7: repeating decimals, so the
# rendered percent is sensitive to how much of them survives the rounding.
_REPEATING_HISTORY = (
_Settled("run-201", 20_000.0, "rejected"),
_Settled("run-202", 50_000.0, "approved"),
_Settled("run-203", 30_000.0, "rejected"),
_Settled("run-204", 40_000.0, "approved"),
)
def _history_layers(root: Path, history: Sequence[_Settled]) -> tuple[Path, Path]:
"""Write an outbox pair + inbox verdict per proposal, joined by a REAL minted id.
Ids are minted through ``VerdictDocument.from_candidate`` rather than typed
by hand, so the join under test is the production one (K5/§4.2).
"""
outbox, inbox = root / "outbox", root / "inbox"
outbox.mkdir(parents=True)
for item in history:
proposal = SavingsProposal(
project_id=_COHORT_PROJECT,
measure=f"measure-{item.run_id}",
affected_items=[AffectedItem(code="VT-07", quantity=1.0, unit_cost=item.claimed_nok)],
claimed_saving_nok=item.claimed_nok,
)
verdict = VerdictDocument.from_candidate(
CandidateFeatures.from_proposal(proposal),
decision=item.decision,
rationale=f"cohort fixture for {item.run_id}",
description=f"{proposal.measure} in {_COHORT_PROJECT}",
)
write_verdict(inbox, verdict)
(outbox / f"{item.run_id}-proposal.json").write_text(
json.dumps(proposal.model_dump(), sort_keys=True, indent=2), encoding="utf-8"
)
(outbox / f"{item.run_id}-outcome.json").write_text(
json.dumps({"run_id": item.run_id, "verdict_id": verdict.id}, sort_keys=True, indent=2),
encoding="utf-8",
)
return outbox, inbox
def test_share_rounding_keeps_the_float_tail_out_of_the_json_bytes(tmp_path: Path) -> None:
"""VALUE proof (upper bound): raise _SHARE_DIGITS to 17 and this goes RED.
The claim on the constant is that the JSON bytes never carry float noise.
Stated without reference to the constant's own value: the PUBLISHED delta is
the exact decimal difference of the two PUBLISHED cohort shares. Comparing
against the constant-derived literal would only bind the number to itself.
"""
outbox, inbox = _history_layers(tmp_path / "noise", _NOISE_HISTORY)
learning = json.loads(report_to_json(build_value_report(outbox_dir=outbox, inbox_dir=inbox)))[
"learning"
]
earlier, later = learning["earlier_gap_share"], learning["later_gap_share"]
# Guard the fixture itself: this history is only evidence while the raw
# subtraction actually has a tail to absorb.
assert repr(later - earlier) == "-0.19999999999999998"
assert Decimal(str(learning["gap_share_delta"])) == Decimal(str(later)) - Decimal(str(earlier))
def test_share_rounding_survives_into_the_rendered_percent(tmp_path: Path) -> None:
"""VALUE proof (lower bound): drop _SHARE_DIGITS to 2 and this goes RED.
``_pct`` renders a share as a percent with one decimal. The expectation is
computed from the RAW NOK figures, so it holds the rounding to the ratio the
layers carry rather than to whatever the constant currently happens to be.
"""
outbox, inbox = _history_layers(tmp_path / "repeating", _REPEATING_HISTORY)
rendered = render_report(build_value_report(outbox_dir=outbox, inbox_dir=inbox))
assert f"{20_000 / 70_000 * 100:.1f}%" in rendered # earlier cohort gap, 28.6%
assert f"{30_000 / 70_000 * 100:.1f}%" in rendered # later cohort gap, 42.9%
def test_share_digits_sits_inside_the_band_its_two_value_proofs_bound() -> None:
"""The band the two proofs above measure, asserted on the constant itself."""
assert 3 <= _SHARE_DIGITS <= 16
# --- goal progress + cost against value ------------------------------------------------------
def test_goal_progress_measures_realized_against_the_target(layers: Layers) -> None:
"""Goal progress is measured on REALIZED value — the only figure that survived the gate."""
report = _build(layers, goal=GoalContract(target_nok=200_000.0, mode="soft"))
assert report.goal is not None
assert report.goal.target_nok == 200_000.0
assert report.goal.realized_nok == REALIZED_TOTAL
assert report.goal.share == 0.4
assert report.goal.reached is False
def test_cost_against_value_never_invents_an_exchange_rate(layers: Layers) -> None:
"""USD cost and NOK value sit side by side — no ratio, because no sourced FX rate exists."""
report = _build(layers, estimated_cost_usd=0.127514)
assert report.cost is not None
assert report.cost.estimated_cost_usd == 0.127514
assert report.cost.realized_value_nok == REALIZED_TOTAL
assert "ESTIMAT" in report.cost.note
# The honest omission: the two currencies are NOT divided into a fake ratio.
assert not hasattr(report.cost, "ratio")
# --- byte determinism (the plan's third RED) --------------------------------------------------
def test_report_bytes_are_deterministic(layers: Layers, tmp_path: Path) -> None:
"""Same ledger + outbox + inbox → byte-identical report (the ``diff`` the plan asks for)."""
first = tmp_path / "first.json"
second = tmp_path / "second.json"
first.write_text(report_to_json(_build(layers)), encoding="utf-8", newline="\n")
second.write_text(report_to_json(_build(layers)), encoding="utf-8", newline="\n")
assert first.read_bytes() == second.read_bytes()
def test_report_json_is_house_style_bytes(layers: Layers) -> None:
"""Sorted keys, 2-space indent, LF only, trailing newline — the house JSON contract."""
payload = report_to_json(_build(layers))
assert payload.endswith("\n")
assert "\r" not in payload
parsed = json.loads(payload)
assert payload == json.dumps(parsed, sort_keys=True, indent=2, ensure_ascii=False) + "\n"
def test_markdown_render_marks_every_unmarked_figure(layers: Layers) -> None:
"""The rendered report never shows a blank where a number is missing — it says so."""
rendered = render_report(_build(layers))
assert "FV42-P2" in rendered
assert "UNMARKED" in rendered # the pending/unquantified projects are labelled
assert "ESTIMAT" not in rendered # no cost estimate was passed in → no cost section
# --- projection purity: the report writes NOTHING ---------------------------------------------
def _snapshot(root: Path) -> dict[str, bytes]:
return {
str(p.relative_to(root)): p.read_bytes() for p in sorted(root.rglob("*")) if p.is_file()
}
def test_report_never_writes_the_read_layers(layers: Layers) -> None:
"""Read-only over all three layers (§3 Step 7) — byte snapshot before/after.
Honest limit: a write that reproduces a file's exact existing bytes is
invisible to a content snapshot. The failure this guards is state leaking
into the read layers, which changes bytes or adds files.
"""
before = _snapshot(layers.root)
_build(
layers,
goal=GoalContract(target_nok=200_000.0, mode="soft"),
estimated_cost_usd=0.5,
)
assert _snapshot(layers.root) == before
# --- the CLI seam -----------------------------------------------------------------------------
def test_cli_writes_the_report_and_exits_zero(
layers: Layers, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""The thin CLI: project the three layers, print the render, write the JSON."""
out = tmp_path / "value-report.json"
code = main(
[
"--outbox",
str(layers.outbox),
"--inbox",
str(layers.inbox),
"--ledger",
str(layers.ledger),
"--json",
str(out),
]
)
assert code == 0
assert out.read_text(encoding="utf-8") == report_to_json(_build(layers))
assert "VERDIRAPPORT" in capsys.readouterr().out
def test_cli_refuses_a_malformed_ledger_fail_fast(layers: Layers) -> None:
"""§10 (SC5): a valid-JSON-but-wrong-shape ledger is refused, never read as an empty book."""
layers.ledger.write_text('{"entries": {}}', encoding="utf-8")
code = main(
[
"--outbox",
str(layers.outbox),
"--inbox",
str(layers.inbox),
"--ledger",
str(layers.ledger),
]
)
assert code == 1
def test_cli_refuses_a_non_object_ledger_fail_fast(layers: Layers) -> None:
"""A JSON array is not a book either — the `**` unpack must not surface as a crash."""
layers.ledger.write_text("[]", encoding="utf-8")
code = main(
[
"--outbox",
str(layers.outbox),
"--inbox",
str(layers.inbox),
"--ledger",
str(layers.ledger),
]
)
assert code == 1