portfolio-optimiser/tests/test_step7_demo_inbox_loadbearing.py
Kjell Tore Guttormsen 1e11dcb96c feat(simulation): the demo now RUNS the Step-7 file inbox it narrates (P1/S1.a)
The Step-7 trace line said "lang fil-løkke" while the verdict arrived as a
function argument (`verdict_input`) — the short, in-run capture. The long loop
was tested but never exercised by the thing on stage.

An expert now drops a real verdict FILE (`write_verdict`) into an inbox between
the runs, and Run B is given `verdict_dir=`, so `run_project` merges it into the
store before the Step-1 fold.

Not done as the plan point was worded, and the difference is load-bearing:
routing the PERSONA verdict through the inbox would have put ONE marker on two
paths — Step 7 (inbox) and Step 8 (promotion) both end in Run B's prompt, so
either could carry it alone and `test_simulation_loadbearing.py`'s promotion
assertion would have stayed green with promotion detached. A second verdict with
its own marker keeps both seams independently red-able; `simulate_learning_loop`
raises when the two markers are equal. The inbox sits beside the bundle copy,
never inside it, and the id is an explicit sentinel (a minted id would collide
with the promoted verdict's, and `VerdictStore.add` is first-write-wins).

766 -> 769 passed (773 collected). Criterion 6 re-measured: stdout byte-identical
across two runs; stderr unchanged at 6 lines. Mutations measured against the full
suite, four red + a green control: detach `verdict_dir=` · point Run B at an empty
folder while the file is still written · marker set to `realization_rate: 0.82`
(measured present in the verdict seed) · marker set to `energy performance gap`
(measured present in a navigated concept file) · benign rename of the inbox dir.

Honesty limit found while measuring: the last two mutations fell on the causality
assertion, not the Run A control — generation prompts carry the debate output, not
the bundle context. The pair holds, but each assert defends a different property.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FVYDeJ9evZicgU5r3roZVW
2026-08-09 13:13:21 +02:00

100 lines
5.4 KiB
Python

"""Step-7 in the DEMO: the simulation must actually USE the async file inbox it claims to show.
The gap (egnethetsreview Funn 2, measured): ``simulation._run_trace_lines`` labels Step 7 "lang
fil-løkke", but ``simulate_learning_loop`` called ``run_project`` WITHOUT ``verdict_dir`` — the
persona verdict reached the run as a function argument (``verdict_input``, the SHORT in-run
capture). The long loop was tested (``test_step7_async_loop_loadbearing.py``) but never exercised
by the thing on stage. Same class as Step 5 before 2026-08-07: a step that is *narrated* is not
thereby *shown*.
The fix routes a SECOND verdict through a real folder: after Run A an expert drops a verdict FILE
into an inbox (``write_verdict``), and Run B is given ``verdict_dir=`` so ``run_project`` merges it
into the store BEFORE the Step-1 fold.
Why a second, separately-keyed marker instead of routing the persona verdict through the inbox:
Step 7 (file inbox) and Step 8 (wiki promotion) are two DIFFERENT mechanisms that both end in Run
B's hypothesis prompt. Had one marker travelled both paths, either path could carry it alone —
``test_simulation_loadbearing.py``'s promotion assertion would then stay green with promotion
detached, i.e. it would become vacuous. Two markers keep each seam independently red-able, which is
the whole point of a load-bearing test.
"""
from __future__ import annotations
import json
from pathlib import Path
from portfolio_optimiser.simulation import simulate_learning_loop
from portfolio_optimiser.validator import ValidatedProposal
BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
async def test_dropped_inbox_verdict_reaches_run_b_in_the_simulation(tmp_path) -> None:
"""LOAD-BEARING: a verdict file dropped into the inbox AFTER Run A reaches Run B's hypothesis
prompt in the SIMULATION — the demo's Step-7 claim, executed rather than narrated. Goes RED the
moment ``verdict_dir=`` is detached from the simulation's Run B (the marker never arrives).
The Run A control is causality, not decoration: the file does not exist during Run A, so its
marker must be absent there. If it were present in both, the positive assertion would be
measuring the bundle, not the loop."""
result = await simulate_learning_loop(str(BUNDLE_DIR), str(tmp_path))
assert isinstance(result.run_a.outcome, ValidatedProposal)
assert isinstance(result.run_b.outcome, ValidatedProposal)
assert not result.inbox_marker_in_run_a_prompt, (
"Run A carried the inbox marker before the file existed — the positive result below would "
"not be caused by the file loop"
)
assert result.inbox_marker_in_run_b_prompt, (
"the verdict dropped into the inbox after Run A did not reach Run B's hypothesis prompt — "
"the demo narrates a long file loop it does not run"
)
async def test_the_inbox_verdict_is_a_real_file_outside_the_wiki(tmp_path) -> None:
"""The medium is the point: the payload must be a FILE an expert could have written by hand,
and it must live OUTSIDE the bundle copy. The role split (målbilde §3) is that the system READS
the inbox — the expert/persona writes it — and the wiki is the separate, gated Step-8 layer.
An inbox nested inside the bundle would be navigable context, which is a different mechanism."""
result = await simulate_learning_loop(str(BUNDLE_DIR), str(tmp_path))
assert result.inbox_path.is_file(), "the dropped verdict is not a file on disk"
payload = json.loads(result.inbox_path.read_text(encoding="utf-8"))
assert result.inbox_marker in payload["rationale"], (
"the marker is not carried by the file's own rationale — it did not travel through the file"
)
assert payload["decision"] == "approved"
bundle_copy = Path(tmp_path) / "bundle"
assert bundle_copy.is_dir()
assert bundle_copy not in result.inbox_path.parents, (
"the inbox lies inside the bundle copy — a verdict there would reach the next run as "
"navigable context, not through the Step-7 file loop"
)
async def test_the_two_learning_paths_stay_separately_observable(tmp_path) -> None:
"""CONTROL for the pair: Step 7 (file inbox) and Step 8 (wiki promotion) must carry DISTINCT
markers and both must cross into Run B. With one shared marker, detaching either seam would
still leave the other carrying it — and both load-bearing tests would pass while a mechanism
was gone. This test is what makes the other two, and
``test_simulation_loadbearing.py``, mean what they say."""
result = await simulate_learning_loop(str(BUNDLE_DIR), str(tmp_path))
assert result.marker != result.inbox_marker, (
"the promotion and inbox paths share a marker — neither seam can then be detached "
"observably"
)
assert result.marker_in_run_b_prompt, "the Step-8 promotion path stopped crossing"
assert result.inbox_marker_in_run_b_prompt, "the Step-7 file path stopped crossing"
# The marker can only come from the file: it appears nowhere in the source bundle. (The
# promotion marker's own absence is asserted by test_simulation_loadbearing.py.)
for path in BUNDLE_DIR.rglob("*"):
if path.is_file():
assert result.inbox_marker not in path.read_text(encoding="utf-8"), (
f"the inbox marker occurs in the bundle ({path.name}) — the positive assertion "
f"could pass for the wrong reason"
)