test(validator): the gate was pinned in its arithmetic, not in its decision [mutation-ok]

[mutation-ok] rationale: no mutation is pinned in code. `src/` is byte-identical
to HEAD (git diff HEAD -- src/ is empty; `if False` occurs 0 times under src/).
The three matches the guard found are PROSE in the docs note and the test file's
docstring, quoting the detach mutation that was run and restored. Every harness
run in this session verified its restore by sha256 from disk.

S2.7 is the D7 mirroring queue's topmost unmeasured candidate. The MAF sibling
tightened its validator in two halves -- (a) a structural block on
`claimed > nominal_feasible`, (b) an IR invariant `low <= unit_cost <= high`.
Both halves are GATED here on D-A pkt. 1 + a commons pull, and the defect they
answer is confirmed on our side as C-F2. So neither is built. The question this
answers is the one that is answerable offline: is today's boundary -- "the ONE
numeric gate is p90" -- load-bearing?

One rule, structurally: a claim is numerically bounded in exactly two places in
src, each with its own spec role (ir.py:50 §7.1, validator.py:68 §3 Step 4).
The sibling's drift shape does not exist here.

But the coverage splits cleanly across the rule. Measured with
scripts/mutation_harness.py, denominator tests/ (all 955), each run
sha256-restored: everything the gate COMPUTES is red, because the golden
fixture freezes it -- policy cap, band branch, band endpoint order, MC seed,
p90 cut point, nominal_feasible. Everything the gate DECIDES WITH is
green-but-dead -- bound to p10, bound to nominal_feasible, and loosening the
comparison each left all 955 green. The golden freezes what the validator
produces, so it cannot help with the one thing it does not observe: which bound
the gate reads. Swapping p90 for nominal_feasible IS the gated S2.7 half (a),
and it would have landed with the suite green, before D-A was decided.

Closed by tests/test_validator_gate_loadbearing.py (8 tests, 955 -> 963). Each
clause is green before and red after exactly its own mutation, with the golden
figures green in BOTH runs -- which shows mechanically that the mutation moved
the decision, not the arithmetic. The two IR tests are pinned with --red-at
against the invariant's own message, since they die in a helper. The AST
population control was proved against a BEHAVIOUR-PRESERVING mutation (the gate
widened to a logically equivalent conjunction) with all three behavioural
controls green: a new gate site is invisible to any behavioural test, which is
why it is there.

Two things the measurement gave in addition. Under the containment mutation the
golden test stayed green, confirming mechanically that half (b) is
golden-compatible when D-A lands. And the IR carries no ORDER on band endpoints
either -- strictly more than C-F2 names: (1.40, 0.70) is accepted, and while
random.uniform still draws from [0.70, 1.40], it walks the seeded stream
backwards, which is a different p90 (120456.91 against 121057.09).

Honest limit: pinning that a claim above nominal_feasible validates today is not
an endorsement of it. C-F2 calls that a MAJOR spec-level defect and the fix is
gated, not declined. These tests make the gated work arrive as a visible red
test and a decision, never as a silent swap. No src change, no spec text
touched, the fasit untouched.

Dated under the D7 frame: work AFTER 2026-08-09, not independent convergence.

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-07 00:07:19 +02:00
commit 544655b4c8
2 changed files with 287 additions and 2 deletions

View file

@ -35,9 +35,9 @@ spørringen som ble kjørt** — ikke fila den ble kjørt mot.
## D7-speilingskøen
Åtte kandidater for speiling mellom D7-søsknene. **Ingen er besluttet** — de står som
kandidater, ikke som planlagt arbeid:
kandidater, ikke som planlagt arbeid. **Tre er målt, 5 står igjen:**
- S2.7
- ~~S2.7~~ — **MÅLT 2026-09-07, se under**
- S3.2
- S4.0 (`126807a`)
- (p) `to_ore` — TO kallsteder
@ -126,6 +126,71 @@ siden.
**Datering (D7-rammen):** arbeid ETTER 2026-08-09 — skal **ikke** leses som uavhengig konvergens.
### S2.7 — gaten er ÉN regel, og den var pinnet i ARITMETIKKEN, ikke i BESLUTNINGEN (målt 2026-09-07)
Kandidaten står i køen fordi søskenet strammet validatoren i to halvdeler: (a) en strukturell
blokk på `claimed > nominal_feasible`, og (b) en IR-invariant `low ≤ unit_cost ≤ high`. **Begge
halvdeler er GATET her** på D-A pkt. 1 + commons-pull (paritetsplanens rad 12), og defekten de
svarer på er bekreftet på vår side som C-F2 (`docs/review-2026-07.md`). Speilings-spørsmålet som
KAN besvares offline i dag er derfor et annet: **er dagens grense — «den ENE numeriske gaten er
p90» — load-bearing?**
**Populasjonen først.** En claim er numerisk avgrenset i nøyaktig TO steder i `src/`, med hver
sin spec-rolle: skjema-invarianten (claim ≤ items-total, §7.1, `ir.py:50`) og validator-gaten
(claim ≤ p90, §3 Steg 4, `validator.py:68`). `_FEASIBLE_FRACTION` forekommer kun i `validator.py`,
og det finnes ingen annen Monte Carlo eller `quantiles`-beregning i pakken (positiv kontroll:
samme spørring finner `validate_proposal` i `loop.py:284`). Søskenets drift-form — to gater som
har glidd fra hverandre — finnes altså ikke her.
**Men dekningen deler seg rent på tvers av regelen.** Med `scripts/mutation_harness.py`, nevner
`tests/` (hele suiten, 955 tester), hver kjøring sha256-restaurert:
- Å detache gaten helt (`if … > p90:``if False:`) er **RØD**`test_validator.py` fanger den.
- Alt gaten **REGNER UT** er RØDT, og goldenen er grunnen: policy-taket (`0.30``0.31`),
band-grenen (ignorér bands), band-endepunktenes rekkefølge, MC-seeden, p90-kuttpunkt-indeksen
og `nominal_feasible`-formelen reddet alle suiten.
- Alt gaten **BESLUTTER MED** er **grønn-men-dødt**: å bytte grensen til `p10`, å bytte den til
`nominal_feasible`, og å løsne `>` til `>=` lot alle 955 testene stå grønne.
Det skillet ER funnet, og det er skarpere enn «sømmen er tynn»: goldenen fryser hvert tall
validatoren PRODUSERER, og kan derfor ikke hjelpe med det ene den ikke observerer — hvilken
grense gaten LESER. Konsekvensen er konkret: **å bytte `p90` mot `nominal_feasible` ER den gatede
S2.7-halvdel (a), og den ville landet med suiten grønn**, før D-A er besluttet.
**Pinnet av** `tests/test_validator_gate_loadbearing.py` (8 tester, 955 → 963). Value-beviset er
kjørt, ikke påstått: hver klausul-test er grønn før og rød etter nøyaktig sin egen mutasjon, med
golden-tallene grønne i BEGGE kjøringer — som mekanisk viser at mutasjonen flyttet BESLUTNINGEN,
ikke aritmetikken. De to IR-testene er pinnet med `--red-at` mot invariantens egen feilmelding,
fordi de dør i en hjelpefunksjon og ikke i testkroppen.
**Mutasjonene ble vist å endre oppførsel FØR deres grønne ble lest som hull** (fellen fra økt 39).
Alle tre grensetilfellene er nåbare og ble kjørt: en claim nøyaktig PÅ p90 finnes (degenerert
band, `0.30 × 1000 == 300.0`), og med goldenens eget band valideres en claim på 100 000 mens
`nominal_feasible` er 90 000 — C-F2s første moteksempel, reprodusert live på p90 = 121 057.09,
sammen med det andre (claim 55 000 mot nominal 30 000, p90 = 65 058.49). Begge tallene er
identiske med review-ens, som bekrefter at defekten er spec-båren.
**To ting målingen ga i tillegg.** (1) Populasjonskontrollen (AST) ble bevist mot en
**oppførselsbevarende** mutasjon — `> p90` utvidet til `> p10 and > p90`, som er logisk identisk
når `p10 ≤ p90` — og alle tre oppførselskontrollene forble grønne. En ny gate-plassering er
usynlig for enhver oppførselstest; det er nettopp derfor AST-kontrollen står der. (2) Under
containment-mutasjonen forble golden-testen **grønn**, hvilket mekanisk bekrefter review-ens
påstand om at S2.7 halvdel (b) er golden-kompatibel når D-A lander.
**Nytt utover C-F2:** IR-en har heller ingen ORDNING på band-endepunktene. `(1.40, 0.70)`
aksepteres; `random.uniform(1.40, 0.70)` trekker fortsatt fra [0.70, 1.40], så området korrumperes
ikke — men den seedede strømmen vandres baklengs, hvilket er en ANNEN p90 (målt: 120 456.91 mot
121 057.09). Et band hvis betydning avhenger av argument-rekkefølgen er ennå ikke et band.
**Ærlig grense — hva dette IKKE sier.** Å pinne at en claim over `nominal_feasible` validerer i
dag er ingen godkjenning av oppførselen; C-F2 kaller den en MAJOR spec-nivå-defekt, og fiksen er
GATET, ikke avvist. Testene pinner grensen slik at det gatede arbeidet MÅ ankomme som en synlig
rød test og en beslutning, aldri som et stille bytte. Ingen `src/`-endring er gjort, ingen
spec-tekst rørt, og fasiten er ikke berørt.
**Datering (D7-rammen):** arbeid ETTER 2026-08-09 — skal **ikke** leses som uavhengig konvergens.
Rammen rundt køen: å lese søskenets kode er tillatt (`3bdf7f0`), men kopiering skal kun skje
der det tjener løsningen, aldri som snarvei. **Uavhengighets-beviset er DATERT** t.o.m.
2026-08-09; arbeid etter den datoen kan ikke leses som uavhengig konvergens.

View file

@ -0,0 +1,220 @@
"""S2.7: the validator's numeric gate — is the DECISION pinned, or only the ARITHMETIC?
The D7 mirroring queue (``docs/2026-08-18-funn-koeer-og-gater.md § D7-speilingskøen``)
carries S2.7 because the MAF sibling tightened its validator in two halves: (a) a
structural block on ``claimed > nominal_feasible``, and (b) an IR invariant
``low <= unit_cost <= high``. Both halves are GATED here on D-A pkt. 1 + a commons
pull (parity plan row 12), and the defect they answer is confirmed on our side as
C-F2 (``docs/review-2026-07.md``). So this file does NOT build either half. The
mirroring question it answers is the one that is answerable offline today: **is
today's boundary — "the ONE numeric gate is p90" — load-bearing?**
MEASURED 2026-09-07 with ``scripts/mutation_harness.py``, denominator ``tests/``
(the whole suite, 955 tests), every run restored sha256-verified:
- Detaching the gate outright (``if ... > p90:`` -> ``if False:``) is RED.
``test_validator.py`` catches it. The seam has an edge guard.
- Every clause of WHAT THE GATE COMPUTES is RED, and the golden fixture is why:
the policy cap (``0.30`` -> ``0.31``), the band branch (ignore bands), the band
endpoint order, the Monte Carlo seed, the p90 cut point index, and the
``nominal_feasible`` formula each reddened the suite.
- Every clause of WHAT THE GATE DECIDES WITH is GREEN-BUT-DEAD. Swapping the bound
to ``p10``, swapping it to ``nominal_feasible``, and loosening the comparison from
``>`` to ``>=`` each left all 955 tests green. Nothing noticed.
That split is the finding, and it is sharper than "the seam is thin": the golden
fixture freezes every number the validator PRODUCES, so it cannot help with the one
thing it does not observe which bound the gate reads. The consequence is concrete.
Swapping ``p90`` for ``nominal_feasible`` IS the gated S2.7 half (a). It would land
with the suite green, silently, before D-A has been decided and a gate change that
arrives without a red test is exactly what this repo's §11 rule exists to prevent.
The mutations were shown to change behaviour before their greens were read as holes
(the økt 39 trap: the harness reports a no-op and an undetected seam identically).
All three boundary inputs are reachable and were run: a claim exactly at p90 exists
(degenerate band, 0.30 x 1000 == 300.0), and with the golden band a claim of 100 000
validates while ``nominal_feasible`` is 90 000 C-F2's first counterexample,
reproduced live at p90 = 121 057.09, together with its second (claim 55 000 against
nominal 30 000, p90 = 65 058.49).
HONEST LIMIT what this does NOT say. Pinning that a claim above ``nominal_feasible``
validates TODAY is not an endorsement of that behaviour; C-F2 calls it a MAJOR
spec-level defect and the fix is gated, not declined. These tests pin the boundary so
that the gated work must arrive as a visible red test and a decision, never as a
silent swap. Nor do they touch the sibling's code or the frozen spec. And the band
tests below assert only that today's IR ACCEPTS a band that excludes ``unit_cost``;
the golden bundle's own band ([0.70, 1.40]) encloses it, so half (b) remains
golden-compatible when D-A lands.
Dated under the D7 frame: this is work AFTER 2026-08-09 and must NOT be read as
independent convergence with the sibling.
"""
from __future__ import annotations
import ast
from pathlib import Path
import pytest
from pydantic import ValidationError
from portfolio_optimiser_claude.ir import SavingsProposal
from portfolio_optimiser_claude.validator import (
Rejection,
ValidatedProposal,
validate_proposal,
)
SRC_PKG = Path(__file__).resolve().parents[1] / "src" / "portfolio_optimiser_claude"
# The golden bundle's own shape (shared/examples/bygg-energi-mikro/validator-input.json):
# one item, a real spread band, so p10 < nominal_feasible < p90 are three DISTINCT
# bounds and a test can tell which one the gate reads. A degenerate band collapses
# all three onto the same number, which is why test_validator.py's fixture — correct
# for what it pins — cannot answer this question.
_QUANTITY = 300_000.0
_UNIT_COST = 1.0
_BAND = (0.70, 1.40)
def _proposal(claimed: float, band: tuple[float, float] = _BAND) -> SavingsProposal:
"""Build through the CONSTRUCTOR, never ``model_copy``.
``model_copy(update=...)`` does not re-run pydantic's model validators, so a
proposal assembled that way would sail past the very invariant the band tests
below are pinned against a layer you do not run is not a seam (økt 37).
"""
return SavingsProposal(
project_id="BYGG-KONTOR-NORD",
measure="S2.7 gate probe",
affected_items=[{"code": "EL", "quantity": _QUANTITY, "unit_cost": _UNIT_COST}],
claimed_saving_nok=claimed,
assumptions={"EL": band},
)
def _bounds() -> ValidatedProposal:
"""Learn today's bounds from the validator itself, then pin the DECISION to them.
Deliberately not hardcoded: the numbers are the golden suite's job (§7), and
reading them from a run keeps every test below attributable to which bound the
gate READS, never to what it computes. The percentiles do not depend on the
claim, so probing with a token claim is sound.
"""
outcome = validate_proposal(_proposal(1.0))
assert isinstance(outcome, ValidatedProposal), "the probe must validate to expose bounds"
return outcome
class TestThereIsExactlyOneNumericGateOnAClaim:
"""The population control, and it runs first: every clause below is worth only
as much as the claim that there is exactly ONE place where a gate can be moved."""
def test_a_claim_is_bounded_in_exactly_two_places_with_distinct_roles(self) -> None:
# The sibling's S2.7 shape made measurable here. Two numeric bounds on a
# claim exist BY DESIGN and answer different specs: the schema invariant
# (claim <= the items' own total, §7.1, ir.py) and the validator gate
# (claim <= the optimistic feasible bound, §3 Step 4, validator.py). RED the
# day a third appears somewhere else in src — a claim gated in a place no
# one is looking is how a weak gate hides. Read from the AST, not prose.
sites: dict[str, int] = {}
for module in sorted(SRC_PKG.glob("*.py")):
tree = ast.parse(module.read_text(encoding="utf-8"))
found = [
node
for node in ast.walk(tree)
if isinstance(node, ast.Compare)
and isinstance(node.left, ast.Attribute)
and node.left.attr == "claimed_saving_nok"
]
if found:
sites[module.name] = len(found)
assert sites == {"ir.py": 1, "validator.py": 1}, (
f"a claim is now numerically bounded in {sites} — the two-rule design "
"(schema invariant + validator gate) has drifted"
)
def test_the_gate_rejects_a_claim_above_the_optimistic_bound(self) -> None:
# POSITIVE CONTROL FOR EVERY NEGATIVE BELOW. Each test that follows asserts
# that some claim VALIDATES; that is vacuous unless rejection demonstrably
# happens at all on this path and on this band. RED under detaching the gate
# (`if ... > p90:` -> `if False:`).
outcome = validate_proposal(_proposal(_bounds().p90 * 1.05))
assert isinstance(outcome, Rejection)
class TestWhichBoundTheGateReadsIsLoadBearing:
"""One test per clause of ``if proposal.claimed_saving_nok > p90:`` — each proved
RED under exactly the mutation it names, with the golden figures green in both
runs (the mutation moved the DECISION, not the arithmetic)."""
def test_the_bound_is_the_optimistic_p90_not_the_nominal_feasible(self) -> None:
# RED under `> p90` -> `> nominal_feasible`, which IS the gated S2.7 half (a).
# C-F2's counterexample, pinned as today's boundary: a self-authored band can
# carry a claim above the closed-form nominal bound and it validates. This
# test does not bless that; it makes the gated fix cost a visible red test
# instead of arriving silently.
bounds = _bounds()
assert bounds.nominal_feasible < bounds.p90, "non-vacuity: the two bounds must differ"
claimed = (bounds.nominal_feasible + bounds.p90) / 2
outcome = validate_proposal(_proposal(claimed))
assert isinstance(outcome, ValidatedProposal)
assert outcome.claimed_saving_nok > outcome.nominal_feasible
def test_the_bound_is_the_optimistic_p90_not_a_central_percentile(self) -> None:
# RED under `> p90` -> `> p50` (and under `> p10`). §3 Step 4 makes the gate
# the OPTIMISTIC edge of the simulation: it blocks what is infeasible even at
# the favourable end, and deliberately does not arbitrate the middle. Sliding
# the bound down to a central percentile is a different policy, not a fix.
bounds = _bounds()
assert bounds.p50 < bounds.p90, "non-vacuity: the two bounds must differ"
outcome = validate_proposal(_proposal((bounds.p50 + bounds.p90) / 2))
assert isinstance(outcome, ValidatedProposal)
def test_the_comparison_is_strict_so_a_claim_exactly_at_p90_validates(self) -> None:
# RED under `>` -> `>=`. The bound is INCLUSIVE: p90 is a feasible value, not
# the first infeasible one, and `test_bygg_energi_mikro.py` states the same
# boundary from the other side (`claimed_saving_nok <= p90`). Reachable, not
# theoretical: a degenerate band collapses p90 onto 0.30 x the fixed total.
bounds = _bounds()
outcome = validate_proposal(_proposal(bounds.p90))
assert isinstance(outcome, ValidatedProposal)
assert outcome.claimed_saving_nok == outcome.p90
class TestTodaysIRAcceptsABandItDoesNotCheck:
"""S2.7 half (b) — the invariant ``low <= unit_cost <= high`` does NOT exist here.
Pinned as today's boundary so the gated work cannot land silently, each proved RED
under the ir.py mutation that adds exactly the invariant it names."""
def test_a_band_that_excludes_the_unit_cost_is_accepted_today(self) -> None:
# RED under adding `low <= unit_cost <= high` to the IR's model validator.
# C-F2's second counterexample: a band entirely ABOVE unit_cost inflates every
# sample, so the optimistic bound is drawn from costs the item never had.
excluding_band = (1.8, 2.2)
assert not (excluding_band[0] <= _UNIT_COST <= excluding_band[1]), "non-vacuity"
outcome = validate_proposal(_proposal(1.0, band=excluding_band))
assert isinstance(outcome, ValidatedProposal)
assert outcome.p90 > _bounds().p90
def test_a_reversed_band_is_accepted_today(self) -> None:
# RED under adding `low <= high` to the IR's model validator. Strictly
# stronger than C-F2, which only names containment: the endpoints carry no
# ORDER either. `random.uniform(1.40, 0.70)` still draws from [0.70, 1.40],
# so this does not corrupt the range — it walks the seeded stream backwards,
# which is a DIFFERENT p90 (measured: 120 456.91 against 121 057.09). A band
# whose meaning depends on argument order is not yet a band.
reversed_band = (_BAND[1], _BAND[0])
assert reversed_band[0] > reversed_band[1], "non-vacuity"
outcome = validate_proposal(_proposal(1.0, band=reversed_band))
assert isinstance(outcome, ValidatedProposal)
assert outcome.p90 != pytest.approx(_bounds().p90)
def test_the_ir_does_still_refuse_a_malformed_band(self) -> None:
# POSITIVE CONTROL for the two negatives above: "the IR accepts it" is only
# a finding if the IR refuses anything at all about bands. It refuses
# non-finite and negative endpoints (R-2 hardening, ir.py `_BandEndpoint`).
with pytest.raises(ValidationError):
_proposal(1.0, band=(-1.0, 1.4))
with pytest.raises(ValidationError):
_proposal(1.0, band=(0.7, float("inf")))