Third writing of the same sentence, third time untrue. Measured in this working tree, by the arm
committed red before this one: FOUR of the five types outside the measured seven do not sit in an
outbox that also holds a coverage -- and for `plan-review` that says nothing at all, because the
type has 0 artefacts in the repo. `multibase` DOES sit there, in four outboxes
(`p17b-multibase/`, `p20-stress/`, `p21-stress/`, `p22-stress/`, all `lindaas`), and each of those
four holds TWO coverage files. That is why they fall outside the "exactly one coverage" rule the
union of seven is counted over -- and the same fact is the proof that seven is a FLOOR and not a
ceiling, which is the argument the false sentence was trying to make.
The comment above `_RUN_LEVEL_TYPES` carried the same claim in English and is corrected with it.
The arm now counts the outboxes and holds the ledger's row to the count, so a fourth writing is a
failing test rather than a review finding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED FIRST, and only one of these can be red in the ordinary way: the ledger's own sentence about
the five artefact types outside the measured seven is UNTRUE, so the new arm that counts the
outboxes fails on ASSERT (1 of 44 arms red, 0 on ImportError/AttributeError/collection). The
other changes fix ARMS, not code -- src/ is untouched -- so their discrimination is shown by the
mutant campaign in the next commit's message: surviving before, felled after.
- `_STAGE_SENTENCES`: stage -> (the clause the refusal heading opens with, a word only that
stage's explanation carries), written out here the way `_STATUS_WORDS` is. The arm that names
where an approach fell built its expected heading from `rb.STAGE_PROSE` and then agreed with
it: PM's mutant P14 (the sentences for `stage4-p90` and `stage0b-grounding` swapped) passed all
43 arms, and the report then said an approach fell on the grounding when it fell on the
uncertainty calculation. Both halves are compared against this file's table, on the ONE heading
line located by prefix, and the whole table is covered -- not only the two stages the fixture
exercises, because a fixture is not a denominator.
- That each mark belongs to exactly ONE stage is COUNTED in the test, so swapping explanations
alone is a failure too.
- `_section(report, heading)`: every `split("## ...")[1]` in the file goes through it. Renaming a
heading failed four arms on 19.09, three of them on IndexError -- a traceback that says a list
was too short, not that the report lost the section the expert reads first.
- New arm: `multibase` DOES lie in outboxes that also hold coverage (four of them, two coverage
files each). The arm counts the outboxes and requires the ledger's sentence to say what the
count says; the false clause is pinned verbatim so its fourth writing is a failing test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED FIRST, HONESTLY: 0 of the 3 new arms can be red at HEAD, because neither finding is a code
defect. The binder already printed every one of these things correctly; what was missing was an
arm that looked. There is no fix to be red before. The proof that the arms discriminate is the
mutant run, not this commit — and that run is reported here in full.
REPRODUCED AS SURVIVORS FIRST, against the 40 arms as they stood: the checkpoint's N3
(`_status_word`, "rejected" -> "validert") and N2 (the heading's amount truncated to whole
kroner) both passed 40 of 40. Eight more of my own against the same first screen: the verdict
word dropped entirely, the label paired with the NEXT row's word, `not_evaluated` read as
"avvist", `unsupported` read as "validert", the commissioned count off by one, and the heading's
"kroner" changed to "kr" — 8 of 10 survived. Two were caught (the sum off by one øre, the
overview sorted).
AFTER: 12 of 12 felled, every one on an AssertionError about behaviour, 0 on
ImportError/AttributeError/ModuleNotFoundError. Control unmutated: 43 of 43. Mutants lived only
in a scratch clone of `git archive HEAD`; the work tree was never mutated.
THE CAUSE WAS MEASURED, NOT GUESSED. `"60 000,01" in text` also matched the "Berørte
kostnadslinjer" line, which is built from `unit_cost` and which the mutant never touched. Counted
over the fixture's own report, 4 of this file's 11 positive substring-assert sites have their
needle on MORE THAN ONE line, and 3 of those 4 are asserts about one particular line. Those three
now compare the WHOLE line: `- **{label}** — {word}`, `### {label} — {amount} kroner`,
`### {label} — falt på {stage}`. The fourth claims only presence, and presence is what it
measures, so it still says `in`. The docstring's "cannot pass by accident" now says which half it
covers.
The word table is written out HERE rather than imported from `_status_word`: an arm that asks the
binder which word it chose and then agrees with it cannot tell "avvist" from "validert". It is
pinned to the MANDATE's own status list (`ApproachOutcome.status`, four values) the way the stage
vocabulary is pinned to the validator's — `_status_word` falls back on `str(row["status"])`, so a
status added there without a word here reaches the expert as a bare English identifier.
"validert" is a PREFIX of "validert, men uten erklært krav", which is exactly why no substring
assert can separate those two.
Also, two prose claims that promised more than the measurement:
- "Every artefact TYPE a real run leaves in an outbox" -> the types the runs MEASURED left.
`run.py` calls TEN of outbox.py's ten writers; `exploration`, `prepass`, `multibase`,
`plan-review` and `proposal-reviews` are five more types. Re-counted over every outbox in this
repo holding exactly one coverage — 15 directories, out of 25 coverage files in 20 directories
— the union is still 7, so 7 is a measured floor and not a ceiling. The arm is renamed to say
so.
- "Four of the seven are RUN-level" -> FIVE. `_RUN_LEVEL_TYPES` has five members.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A green mutation is a finding. Of the twelve planted against this file, eleven fell and
one survived: dropping `row["validated"]` from validated_ore's filter changed nothing.
It is equivalent — but only for as long as the OTHER guard holds. derive_outcome already
refuses to put a refused row's figure into (d), so on every outcome this builder writes,
validated_nok is None wherever validated is False and the two guards are
indistinguishable from the outside. Deleting either one alone is free today; deleting
both is the leak, and nothing witnessed that.
validated_ore is public and takes any outcome mapping, so the arm that separates them
hands it the outcome a future coverage writer could produce: a refused row whose amount
already sits in validated_nok. 137 000 002 øre, counted from the table, against the
227 000 002 a leak would give.
12 of 12 mutants now fall; control in the clone is 40 of 40.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 19.09 checkpoint planted 20 mutants against this file and SEVEN survived the whole
suite: the report's round number, an artefact type outside proposal/outcome/coverage
dropped in silence, a never-evaluated approach left off its own list, the source quote
removed entirely, the quote's COUNT removed, the affected cost lines removed, and a
refused row's amount counted into the validated total. None of them survived because the
builder was wrong — it does all seven things correctly. They survived because no arm in
709 lines ever looked (grep for citation/cost-line words gave 2 hits, both in the
fixture, none in an assert). An unwitnessed seam is a seam the next edit may delete for
free.
Two fixture defects made three of those blind spots unreachable rather than merely
unmeasured:
* THE DENOMINATOR. A real run leaves SEVEN artefact types; the fixture wrote three.
Counted here, not remembered: over the four archived runs the checkpoint read
(tunnel-hauglia-2027-04/-06/-07/-08), each <run_id>-<rest>.json is typed proposal /
outcome when <rest> ends there and <rest> itself otherwise. -06, -07 and -08 hold all
seven (15 files each); -04 holds six (12 files, no parse-failures — that file is
written only when something failed to parse, so its absence is the signal). Union =
7, written by two commands: run.py writes six, stress.py writes -verdict.json. Six
are now written through the product's own writers; -verdict.json is bytes, because
the builder never reads it and that difference is stated rather than hidden.
* THE CITATIONS. Every proposal carried an identical stamp, so "1 av 1 siterte steder"
could not tell a dropped COUNT from a kept one. Counts are now distinct per approach,
and a shared-list mode reproduces what the real runs actually do.
The refused row now carries an amount, and no archived run does: measured on the same
four outboxes, every rejected row has saving_nok = None. The guard that only a validated
row's figure becomes validated_nok therefore aims at a coverage writer that does not
exist yet — and a fixture that cannot produce the number cannot witness the guard at all.
4 of the 11 new arms are RED on ASSERT, all four on behaviour that does not exist yet:
a dangling symlink as the round directory (exists() follows the link, so the build dies
on the filesystem's own FileExistsError instead of a sentence), the identical citation
list stated once instead of five times, the same cost line named on both sides of the
verdict, and a removed approach shown by the label the expert saw instead of a raw id.
The other 7 pin behaviour that is already correct; they are not red here, and the proof
that they discriminate is the mutant run, not this commit.
The cause of the identical quote was MEASURED before anything was written, because
"the builder reads the wrong field" and "the outbox says the same thing five times" want
opposite fixes: all five proposals in tunnel-hauglia-2027-08 carry byte-identical
270-citation lists (one sha256 across all five), and so do -04, -06 and -07. It is the
run's whole retrieved context stamped once per proposal. The report cannot make that
quote informative; it can stop repeating it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found while killing mutants: an ordering made to depend on hash() survived
test_two_builds_of_the_same_outbox_are_byte_identical untouched. One interpreter has one hash
seed, so the two builds agreed with each other and with nothing else -- the arm could not tell a
deterministic report from a report that is merely consistent within a process. The ordering arm
caught that mutant, so the seam was witnessed; this one was not witnessing it.
The second build now runs as a subprocess with PYTHONHASHSEED=1. Re-measured with the same
mutant: both arms red, where before only one was.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
python -m portfolio_optimiser.evals.round_builder --outbox <dir> --round <n> --ran-at <ISO>
writes <rounds-dir>/<n>/ with the run's artefacts COPIED in, outcome.json derived from that
copy, and report.md -- the one artefact in a round a domain expert reads and corrects. Round 0
of the v1 criterion can now be made; it counted 0 of 3 because it could not be, which is a
different failure from a round nobody had held.
What it derives it derives with the gate's own functions rather than a second copy: verify_run
decides whether the run stands up to itself (an artefact contradicting its coverage row, a
half-missing family and a stray artefact are all refused AT THE SOURCE, before a byte is
written), stage_of gives column (c), row_changed gives the report's "changed since the previous
round", parse_time refuses a stamp without a zone, safe_rounds_dir refuses a round directory the
repo would commit. The validated total is ledger.to_ore per amount, summed as integers.
Two things it never does, and both are the point. It never writes the operator's attestation --
the gate stops at FORM OK without one, and that is correct, because no arrangement of files can
witness that a run happened. And it never invents: --ran-at is required because no outbox
artefact carries a clock, and feedback_ids stays empty because no run records which feedback
item produced which row. The report says "ingen tilbakemelding forklarer dette" on every changed
row rather than hiding that model noise and an answered objection look alike.
Chosen and why: --ran-at as a required argument rather than the coverage file's mtime, because
an mtime is a filesystem attribute one call sets and reading it as evidence made row 2 green on
a tree nothing had run in (18.09). The report carries no raw stage identifier -- every stage
sentence is "<short name>: <explanation>" so the one-line diff of what changed has words a
reader can act on. A citation shows its COUNT, because a run that cited 446 places and one that
cited one must not look the same.
[skip-docs]: the ledger row is in docs/invarianter.md, which is where this repo's rules live.
README is the product's front door and this is an operator tool behind `python -m`, the same
class as costsim/hitl/preflight, which README deliberately does not carry; v1-rounds/ is
gitignored internal machinery and the gate itself is not in README either. CLAUDE.md was emptied
of exactly this kind of row in session 130 and is not the place to put one back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 0 of the v1 criterion cannot be made today: nothing binds a finished run's outbox to
<rounds-dir>/<n>/, and nothing in src writes markdown a domain expert could read. These tests
state what a builder has to do before one exists, and every number they assert is counted a
second time from the fixture's own table rather than read back from the builder.
Red on assertions, not on import: round_builder.py lands as a contract -- dataclass, signatures,
neutral returns -- so each test fails in its own body.
Two gate helpers become public rather than being copied: row_changed (the report's "changed since
the previous round" section must not disagree with the gate about what changed) and
safe_rounds_dir (the builder CREATES the directory the gate only reads, and the writer is where a
leak of the expert's feedback has to be stopped).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
