test(round-builder): the determinism arm builds twice in two processes, not twice in one

Found while killing mutants: an ordering made to depend on hash() survived
test_two_builds_of_the_same_outbox_are_byte_identical untouched. One interpreter has one hash
seed, so the two builds agreed with each other and with nothing else -- the arm could not tell a
deterministic report from a report that is merely consistent within a process. The ordering arm
caught that mutant, so the seam was witnessed; this one was not witnessing it.

The second build now runs as a subprocess with PYTHONHASHSEED=1. Re-measured with the same
mutant: both arms red, where before only one was.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 07:35:44 +02:00
commit 86c61a343b
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q

View file

@ -14,6 +14,7 @@ it cannot parse.
from __future__ import annotations from __future__ import annotations
import json import json
import os
import shutil import shutil
import subprocess import subprocess
import sys import sys
@ -450,10 +451,37 @@ def test_two_builds_of_the_same_outbox_are_byte_identical(tmp_path: Path) -> Non
Row 4 counts content lines IN ORDER, so a report that reshuffles between builds would read as Row 4 counts content lines IN ORDER, so a report that reshuffles between builds would read as
an expert's edit. Both the whole-file bytes and the heading sequence are checked, because two an expert's edit. Both the whole-file bytes and the heading sequence are checked, because two
files can differ in bytes for a reason that is not order.""" files can differ in bytes for a reason that is not order.
The second build runs in a SUBPROCESS with a different ``PYTHONHASHSEED``, and that is not
decoration: measured 19.09, an ordering made to depend on ``hash()`` survived a second
in-process build untouched one interpreter has one hash seed, so the two builds agreed with
each other and with nothing else. Two processes is the cheapest way to be a witness rather
than a coincidence."""
source = _outbox(tmp_path) source = _outbox(tmp_path)
first = _build(tmp_path, outbox=source, rounds_dir=tmp_path / "a").round_dir first = _build(tmp_path, outbox=source, rounds_dir=tmp_path / "a").round_dir
second = _build(tmp_path, outbox=source, rounds_dir=tmp_path / "b").round_dir second = tmp_path / "b" / "0"
env = {**os.environ, "PYTHONHASHSEED": "1"}
other = subprocess.run(
[
sys.executable,
"-m",
"portfolio_optimiser.evals.round_builder",
"--outbox",
str(source),
"--round",
"0",
"--rounds-dir",
str(tmp_path / "b"),
"--ran-at",
_RAN_AT,
],
capture_output=True,
text=True,
cwd=Path(rb.__file__).resolve().parents[3],
env=env,
)
assert other.returncode == 0, other.stderr
for name in ("report.md", "outcome.json"): for name in ("report.md", "outcome.json"):
assert (first / name).read_bytes() == (second / name).read_bytes(), name assert (first / name).read_bytes() == (second / name).read_bytes(), name
headings = [ headings = [