B-gate row 1's premise, made callable. Every path through the framework CLI constructs a chat
client, so an outside caller -- a human at a terminal, or an agent that is NOT po -- could not
reach a single run-path step without paying for a model. These four steps need no model at all.
One CLI, four subcommands, one core call each:
navigate-bundle --bundle-dir -> okf.navigate_bundle
cost-baseline --bundle-dir --project-id -> okf.derive_cost_baseline
retrieve-chunks --query --docs-dir [--top-k] -> datasource.retrieve_chunks
prepass-admit --payload --bundle-dir [--dimension] -> prepass.admit_payload
Each handler is a thin adapter: strings in, the SAME function the run path calls, JSON on stdout,
and an exit code that says what happened (0 ran, 2 malformed call, 3 the step refused, named).
A handler that computed anything of its own would be a second implementation of a run-path step,
and the outside caller would stop getting what the debate gets.
Dispatch is an explicit branch per command, not argparse's `set_defaults(handler=...)`: the table
hides the one thing a reader wants to see, and B-gate row 1 asks the same question of the source
(it walks the call graph from `main` down to the step's symbol), where a callable in a Namespace
is a hop neither can follow.
Probes (`tests/test_toolbox_doors.py`, 10 arms): each starts the door as a SUBPROCESS with the
subcommand in argv and asserts on what it wrote -- never by importing the core function, which is
the whole difference the gate exists to measure. The yardstick is outside the door in every arm:
the filesystem (navigate-bundle, including the one deliberate outside-bundle link), a table
transcribed from the priced fixture (cost-baseline), the in-process seam it must equal byte for
byte (retrieve-chunks), and the producer's own checked-in payload (prepass-admit). Every refusal
arm has an rc-0 control beside it.
`portfolio-optimiser-toolbox` is the THIRD console script, and the pin test now says why: it is
the door the other two cannot be used for. README and CLAUDE.md updated with the command and the
reason it exists; every documented invocation was run.
Row 1: 1 -> 5 of 17 (four subcommands + `gate`, which the class fix in e47be68 stopped rejecting
on a name technicality). No other row moved; exit 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Third repair of the same failure, and the first that does not read a NAME. The judge measured
row 1 to 17 of 17 with 17 one-line probes and a door module without a single import: "the probe
touches the door" was still a text check, one notch further in.
What is measured now, in the probe's own source: an EXECUTION SITE (a process starter carrying
the door's name in its arguments, or the entry imported from the door's module and called), and
an assert that is DATA-DEPENDENT on that call. Dead code is pruned first -- a call in a branch
that never runs, or after a return, is no execution. The honest forms the suite already uses
keep counting: the command built in a variable first, and the subprocess started in a helper
that returns it.
All seven forms the judge measured (a1, a2, b1, b2, c1, c2 + the one-liner recipe) now give
0 of 1, each with an rc-0 control in the same setup.
[mutation-ok] -- no guard is switched off. The two `if False:` lines in this diff are the
judge's b1 probe form planted as test DATA (a Python string written into a tmp_path file the
gate only PARSES, never runs) and the docstring naming why dead code is pruned. The commit adds
the rule that makes `if False:` around a door call stop counting.
Naming the step becomes a DISCRIMINATOR, as a class: it is required only when more than one step
stands behind the same door. Driving a door with a sole step IS driving that step -- which is why
`gate` (a real end-to-end door probe against v1_gate) was rejected on a name technicality. Row 1:
0 -> 1 of 17. For a subcommand door the command name must stand in what was actually executed,
and the CLI itself must be registered or `-m`-runnable: an add_parser in a module nobody can
start is no door.
held_out: a reason must be named prose. "-", "todo", "x", "." were all accepted as a reason by
`.strip()`; measured against the contract's own 28 reasons (shortest: 30 chars, three words).
Row 3: the five forms the checkpoint measured as open AND cheap to close are closed -- importlib
for both SDK names, deno/npm exec/yarn dlx, and the official TypeScript SDK. 12 of 12 -> 15 of 15,
still 0 hits over 512 published files. The limit sentence stops enumerating what remains ("EXACTLY
TWO", then "THREE" -- both falsified by the first new measurement) and says what the guard IS.
N7 stays open in the gate and is closed in the suite: a deleted off-path step still shrinks the
denominator silently there, so the four are written out in an arm that goes red when one goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured, not asserted: PM's P14 reproduced in BOTH directions (survived 43 of 43 pre-round arms,
felled by three arms after), 7 of 7 own mutants felled on AssertionError about behaviour with 0
IndexError, and the class the order named -- "an assert built from the binder's own table" --
counted at 4 of 12 rb.<table> assert sites and closed, with the independent side named for the
other eight.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Third writing of the same sentence, third time untrue. Measured in this working tree, by the arm
committed red before this one: FOUR of the five types outside the measured seven do not sit in an
outbox that also holds a coverage -- and for `plan-review` that says nothing at all, because the
type has 0 artefacts in the repo. `multibase` DOES sit there, in four outboxes
(`p17b-multibase/`, `p20-stress/`, `p21-stress/`, `p22-stress/`, all `lindaas`), and each of those
four holds TWO coverage files. That is why they fall outside the "exactly one coverage" rule the
union of seven is counted over -- and the same fact is the proof that seven is a FLOOR and not a
ceiling, which is the argument the false sentence was trying to make.
The comment above `_RUN_LEVEL_TYPES` carried the same claim in English and is corrected with it.
The arm now counts the outboxes and holds the ledger's row to the count, so a fourth writing is a
failing test rather than a review finding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED FIRST, and only one of these can be red in the ordinary way: the ledger's own sentence about
the five artefact types outside the measured seven is UNTRUE, so the new arm that counts the
outboxes fails on ASSERT (1 of 44 arms red, 0 on ImportError/AttributeError/collection). The
other changes fix ARMS, not code -- src/ is untouched -- so their discrimination is shown by the
mutant campaign in the next commit's message: surviving before, felled after.
- `_STAGE_SENTENCES`: stage -> (the clause the refusal heading opens with, a word only that
stage's explanation carries), written out here the way `_STATUS_WORDS` is. The arm that names
where an approach fell built its expected heading from `rb.STAGE_PROSE` and then agreed with
it: PM's mutant P14 (the sentences for `stage4-p90` and `stage0b-grounding` swapped) passed all
43 arms, and the report then said an approach fell on the grounding when it fell on the
uncertainty calculation. Both halves are compared against this file's table, on the ONE heading
line located by prefix, and the whole table is covered -- not only the two stages the fixture
exercises, because a fixture is not a denominator.
- That each mark belongs to exactly ONE stage is COUNTED in the test, so swapping explanations
alone is a failure too.
- `_section(report, heading)`: every `split("## ...")[1]` in the file goes through it. Renaming a
heading failed four arms on 19.09, three of them on IndexError -- a traceback that says a list
was too short, not that the report lost the section the expert reads first.
- New arm: `multibase` DOES lie in outboxes that also hold coverage (four of them, two coverage
files each). The arm counts the outboxes and requires the ledger's sentence to say what the
count says; the false clause is pinned verbatim so its fourth writing is a failing test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Row updated, not appended to: the B-gate invariant now carries both repairs of
19.09 and the numbers that moved in each. Two precision errors the checkpoint
named are corrected in place -- "no row became greener" is true of colour and not
of numbers (row 5 went 1 of 2 -> 4 of 5 when the unit was split), and ENTRY_KINDS
has three arts, not four.
What the new paragraph records: the probe binding and the two traps measured while
building it (substring, and a local module alias), the price in row 1 (3 of 17 ->
0 of 17, each of the three named with its reason), the limit statement that claimed
two remaining ways where six were measured -- four closed, three named and left
open with their reason -- the held_out door out of the denominator, the five small
holes closed and the two limits stated rather than closed, and the thirteenth
mutant: it survived twice, and the first arm written to fell it proved something
already true.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First attempt at M12 still passed 86 of 86 -- the arm planted a docstring that
MENTIONED the module in prose, and the gate compares whole strings, so prose never
reached the check either way. The arm proved something already true and left the
mutant alive.
The cheapest cheat is the bare name: a string that IS the dotted module, standing
alone. That is what the arm plants now, and the same for the step's symbol.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mutant M12 against the delivered tree: let _meaningful return the whole body, so a
string standing alone counts. 85 of 85 GREEN. That is a finding, not a pass: the
repair's own rule -- "the gate never reads a string that stands alone" -- had no arm
watching it, and without one the fix would have drifted back to "mentioning the door
is touching it", which is the same defect one floor in.
The arm carries its rc-0 control first (a real probe WITH a docstring still counts),
then breaks each half on its own: a probe whose docstring names the real door while
its call drives another module, and one whose docstring names the step while its call
passes another word. Both must be refused, each with its own reason.
12 of 12 mutants now felled, control 86 of 86.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Row 1 goes 3 of 17 -> 0 of 17. Nothing was removed from the product and no row
changed colour; the three that counted stopped counting because the contract they
satisfied was satisfiable without capability.
BEARING 1 -- the probe is bound to the step, and the binding is MEASURED in the
probe's own source. Chosen: read the probe (ast) rather than demand it live in a
contract-named file, because a file name is a convention a stub meets as easily as a
real probe. Three traits, each measured: it drives the DOOR (the dotted module or the
registered command name appears as a string it uses -- anywhere but a docstring,
because the honest form assembles argv in a variable first), it names the STEP (the
symbol, id or subcommand as a whole WORD in what it passes INTO a call or calls), and
it asserts at all. A probe claimed by two steps proves at most one and the gate cannot
tell which -- so neither.
Two traps found while measuring, both closed:
- substring: "gate" is not named by portfolio_optimiser.evals.v1_gate
- local alias: the first cut accepted step `gate` because the probe file imports the
module AS `gate`. Names are therefore read only where they are sent or called.
What that costs, measured against the contract that stands:
rundebinding drives the door, names no step (was green)
rapport never goes through the door at all (was green)
gate drives the door, names no step (was green)
BEARING 2 -- the limit statement said exactly TWO ways remained; the checkpoint
planted 21 call forms and measured SIX. Four are closed with a guard each (the
official Python SDK in both spellings, the node and uv runners, a dynamic import);
three remain and are now named: a runtime-composed name, a name from an environment
variable, a base64-decoded name. Left open deliberately -- the encodings are not
enumerable and our own contract stores base64 by design. Row 3: 9 of 9 -> 12 of 12,
still GREEN, 0 hits over 512 files. One of the three caught a command written in this
round's own test docstring; it was rewritten, not exempted.
BEARING 3 -- held_out accepted an EMPTY reason and shrank the denominator, while the
summary said "held out with a reason" either way. A blank reason is no reason: the
symbol stays in the denominator as a call without a door, the summary counts reasons,
and the four steps declared OUTSIDE the run path are now named one by one as having no
derived source instead of being counted in silence.
Five small rests, closed: a pruned manifest (451 of 512 was still GREEN) is now NOT
MEASURED, one sentinel per area the old handlist missed; a non-UTF-8 file is read as
byte text instead of counted and skipped; a symlink out of the tree is named and
fails the row; a runbook whose whole content is "x" no longer passes, the contract
names its sections; and the row states that its ratio is not a coverage measure.
Two stated, not closed, each with its reason in the row's own attestation: a po call
moved one floor down into a helper leaves the denominator (following helpers would
pull private ones in and make the denominator the curated list this row exists to
avoid), and row 3's k/n can still be padded by a guard with no measured escape behind
it. The ledger's two precision errors are corrected: "no row became greener" is true
of colour, not of numbers, and ENTRY_KINDS has three arts, not four.
Suite: 2172 passed, 5 skipped, 5 xfailed in 645 s. ruff and mypy clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED FIRST, second repair. PM's checkpoint on c1bdb37 measured N2b: row 1 goes from
3 of 17 to 17 of 17 GREEN with fourteen stub doors and ONE unrelated green test used
as the behaviour probe for all of them. The gate only looked up whether a nodeid was
`passed`; nothing bound the probe to its step. EXTERNAL_DOOR promised the probe
"calls the door and reads the artefact it writes" and there was no code for it --
the same class of defect as entry["kind"] one round ago.
20 new arms, 19 RED, every one on an assert about behaviour (43 assert lines, 0
ImportError, 0 AttributeError, 0 collection errors). The twentieth is green by
construction: every held_out entry in the contract does carry a reason today, and
the arm exists so that stops being true loudly.
Each refusal arm carries its own rc-0 control FIRST -- the same probe, the same
surface, accepted -- then breaks exactly one trait. The constructed probe is now
WRITTEN (`_write_probe`), not merely named: the repaired gate reads the probe's own
source, so a nodeid that does not exist cannot be a behaviour.
Red on the three bearing rests:
probe not bound to its step door untouched, step unnamed, nothing asserted,
test missing, one probe shared by two steps, and
N2b against the contract that stands (17 -> k=0)
limit statement untrue the six escapes PM measured as unseen, planted one
at a time, and the "TWO ways remain" sentence
held_out accepts empty a blank reason shrinks the denominator; the summary
counts entries, not reasons; the four steps outside
the run path are counted without being named
Red on five of the nine small rests: pruned manifest (451 of 512 still GREEN), a
non-UTF-8 file counted but never scanned, a symlink out of the tree, a runbook whose
whole content is "x", and the row's ratio read as a coverage measure.
One test-hygiene change, not a product fix: the git-dependent arm now skips itself
without .git instead of failing, so "red in a clean extract" stops growing with each
new arm of that class.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The round binder's row gets its two corrected denominators, and the first screen gets a row of
its own: what survived, why none of it was a code defect, and why that makes "red first"
impossible to satisfy with a commit rather than with a mutant run.
The 7-type sentence said "en ekte kjøring" and measured four archived runs. Re-counted: `run.py`
calls TEN of outbox.py's ten writers, so five more types exist; and the union over every outbox
in this repo with exactly one coverage (15 directories, of 25 coverage files in 20 directories)
is also 7. Seven is the floor the measurement gives, not a ceiling — the binder copies on a glob,
not on that list.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED FIRST, HONESTLY: 0 of the 3 new arms can be red at HEAD, because neither finding is a code
defect. The binder already printed every one of these things correctly; what was missing was an
arm that looked. There is no fix to be red before. The proof that the arms discriminate is the
mutant run, not this commit — and that run is reported here in full.
REPRODUCED AS SURVIVORS FIRST, against the 40 arms as they stood: the checkpoint's N3
(`_status_word`, "rejected" -> "validert") and N2 (the heading's amount truncated to whole
kroner) both passed 40 of 40. Eight more of my own against the same first screen: the verdict
word dropped entirely, the label paired with the NEXT row's word, `not_evaluated` read as
"avvist", `unsupported` read as "validert", the commissioned count off by one, and the heading's
"kroner" changed to "kr" — 8 of 10 survived. Two were caught (the sum off by one øre, the
overview sorted).
AFTER: 12 of 12 felled, every one on an AssertionError about behaviour, 0 on
ImportError/AttributeError/ModuleNotFoundError. Control unmutated: 43 of 43. Mutants lived only
in a scratch clone of `git archive HEAD`; the work tree was never mutated.
THE CAUSE WAS MEASURED, NOT GUESSED. `"60 000,01" in text` also matched the "Berørte
kostnadslinjer" line, which is built from `unit_cost` and which the mutant never touched. Counted
over the fixture's own report, 4 of this file's 11 positive substring-assert sites have their
needle on MORE THAN ONE line, and 3 of those 4 are asserts about one particular line. Those three
now compare the WHOLE line: `- **{label}** — {word}`, `### {label} — {amount} kroner`,
`### {label} — falt på {stage}`. The fourth claims only presence, and presence is what it
measures, so it still says `in`. The docstring's "cannot pass by accident" now says which half it
covers.
The word table is written out HERE rather than imported from `_status_word`: an arm that asks the
binder which word it chose and then agrees with it cannot tell "avvist" from "validert". It is
pinned to the MANDATE's own status list (`ApproachOutcome.status`, four values) the way the stage
vocabulary is pinned to the validator's — `_status_word` falls back on `str(row["status"])`, so a
status added there without a word here reaches the expert as a bare English identifier.
"validert" is a PREFIX of "validert, men uten erklært krav", which is exactly why no substring
assert can separate those two.
Also, two prose claims that promised more than the measurement:
- "Every artefact TYPE a real run leaves in an outbox" -> the types the runs MEASURED left.
`run.py` calls TEN of outbox.py's ten writers; `exploration`, `prepass`, `multibase`,
`plan-review` and `proposal-reviews` are five more types. Re-counted over every outbox in this
repo holding exactly one coverage — 15 directories, out of 25 coverage files in 20 directories
— the union is still 7, so 7 is a measured floor and not a ceiling. The arm is renamed to say
so.
- "Four of the seven are RUN-level" -> FIVE. `_RUN_LEVEL_TYPES` has five members.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The row claimed three things the checkpoint measured as false: that every denominator comes from
the source (row 1's M=13 was a curated list in the gate's own b_gate.json), that the surface is
what is published (it was a hand list of eleven roots seeing 433 of 512 files), and that the gate
counts an MCP-registered door as callable (there was no code for it — "kind" appeared 0 times).
The 435-file figure was the working tree, two gitignored .local.md files included, so it was not
reproducible from the commit either.
Rewritten to what is true after the repair, with the old numbers kept as history so the row reads
as a correction and not as a clean slate: 3 of 17 · 0 of 2 · 9 of 9 · 0 of 3 · 4 of 5 · IKKE MAALT.
The markdown-fence rule's own measurement is refreshed too: on the bigger surface it now carries
TWO prose lines, not one — the second is a plan document quoting a command the operator ran in his
own session.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three of the new row-3 arms pinned 512 files and the literal source label, which only a tree WITH
git metadata can answer; a clean extract (git archive | tar -x) would have turned all three red for
the same reason the three already-named extraction artefacts go red. The property they test holds
in both worlds, so they now assert it there: main.py, examples/, spikes/ and contexts/ are inside
the guard whichever manifest was read, and the row prints whatever denominator it measured.
The exact number and the label belong to ONE arm — the one whose subject IS the git manifest. In an
extract it is an extraction artefact of the same class as the others, and says so in its docstring.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Row 1's M was a curated list of 13 in the gate's OWN b_gate.json. It is now DERIVED: run_path_calls
reads what run.py::run_project actually calls (41 po-functions, re-counted here against PM's 39 —
the two that differ are generate_via_llm and fresh_workflow, held out because they need a chat
client). A call that is neither a declared step nor named-with-a-reason counts in the denominator
WITHOUT a door, so the number cannot shrink in silence: removing a step from the contract now
leaves N unchanged and names the orphan. The four outbox writers the run path uses and nobody had
declared (write_prepass, write_parse_failures, write_proposal_reviews, write_debate_tools) are
steps now; 28 calls are held out, each with its reason printed under the row.
A door must be REGISTERED and PROVEN. entry["kind"] is read (it was read 0 times before) and must
be one of three kinds the gate has code to verify: console-script in pyproject, module-main with
its own __main__ guard, subcommand registered in the module's argparse. On top of that every step
needs a named probe that calls the door and reads the artefact. The MCP sentence is struck from
EXTERNAL_DOOR: it claimed a capability with no code behind it.
Row 3 now scans the repo manifest (git ls-files, or the tree itself in an extract), not a hand
list of 11 roots: 512 published files instead of 433, so main.py, examples/, spikes/, contexts/,
CLAUDE.md and llms.txt are inside the guard for the first time. Three new patterns catch the
indirect invocations that walked past the old six — absolute path, list in a variable, constant,
shell string — 5 of 5 of PM's variants are refused now, with 0 false positives measured over the
whole surface. An empty surface is IKKE MAALT, not GREEN: the row demands a sentinel file and
prints the file count and the manifest it read.
Rows 4, 5 and 6 get sources for their denominators. Row 4 counts only checks whose named source
symbol resolves in the code. Row 5 counts each structural requirement on its own (2 profile
members + factory + seam + probe = 5) instead of collapsing three into one unit. Row 6's N comes
from the artefacts the contract names, and the attestation is VALIDATED: it must name the
contract's runbook, carry its sha256, say who ran it, and bear a real ISO date that is not in the
future — v1_gate's own date rule, reused, BOM tolerated as there.
Measured in a scratch clone (/tmp/claude-po/bgate-mut): 12 of 12 mutants felled, control 65 of 65.
All six of PM's broken attacks reproduced as refused, with the rc-0 control green.
No row got greener: 3 of 17 (was 3 of 13), 0 of 2, GREEN, 0 of 3, 4 of 5 (was 1 of 2), IKKE MAALT.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The PM checkpoint on 207337c judged the gate DELVIS: row 1's M=13 is a curated list in the
gate's OWN b_gate.json (the run path has 41 po-calls, 7 of 10 outbox writers), rows 4, 5 and 6
have denominators with no source at all, 6 of 10 cheat-attacks got through, and the
never-Claude guard sees 433 of 512 published files.
This commit is the red half. Every arm fails on an ASSERT about behaviour, never at collection:
the four names that do not exist yet (ENTRY_KINDS, run_path_calls, registered_entry,
published_files) are stubbed here with DELIBERATELY wrong values — everything is a door, the
run path calls nothing, the surface is empty — so each arm measures the defect rather than the
absence of a symbol.
30 av 31 red on assert. The one that is green is the rc-0 control
(test_a_valid_attestation_is_the_only_thing_that_turns_row6_green): a valid attestation must
turn row 6 green both before and after, or the row refuses everything, which proves as little
as refusing nothing. All 34 pre-existing arms stay green — measured, not assumed.
The planted claude-invocations are base64 in the test file for the same reason the contract's
patterns are: tests/ is itself part of the surface row 3 scans, and a cleartext variant here
would register as its own finding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The round builder's row claimed "et sitat bærer ANTALLET siterte steder. Load-bearing
MÅLT" for a half that was not measured: the mutant dropping the count survived the whole
suite. The claim is true from today, and the row now says from WHEN — a ledger that
back-dates a measurement is worth less than one that admits the gap.
The new row records what the checkpoint found and what it cost to close: seven mutants
survived, none of them because the code was wrong, all of them because no arm looked.
Three were unreachable rather than merely unmeasured — the fixture wrote three of seven
artefact types, and every proposal carried the same citation stamp, so "1 av 1 siterte
steder" could not tell a dropped count from a kept one.
The denominator is written with its method, not as a number to be remembered: over the
four archived runs, -06/-07/-08 hold all seven types and -04 holds six (parse-failures is
written only when something failed to parse, so its absence is the signal). Union = 7,
from two commands.
And the citation cause is recorded as a measurement rather than a diagnosis, because
"the builder reads the wrong field" and "the outbox says the same thing five times"
want opposite fixes: one sha256 across all five proposals, in all four runs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A green mutation is a finding. Of the twelve planted against this file, eleven fell and
one survived: dropping `row["validated"]` from validated_ore's filter changed nothing.
It is equivalent — but only for as long as the OTHER guard holds. derive_outcome already
refuses to put a refused row's figure into (d), so on every outcome this builder writes,
validated_nok is None wherever validated is False and the two guards are
indistinguishable from the outside. Deleting either one alone is free today; deleting
both is the leak, and nothing witnessed that.
validated_ore is public and takes any outcome mapping, so the arm that separates them
hands it the outcome a future coverage writer could produce: a refused row whose amount
already sits in validated_nok. 137 000 002 øre, counted from the table, against the
227 000 002 a leak would give.
12 of 12 mutants now fall; control in the clone is 40 of 40.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The four arms that were red on assert, in the order the reader meets them.
A DANGLING SYMLINK as the round directory now refuses instead of tracebacking.
exists() FOLLOWS a link, so a dangling one answers False and slipped straight past
"a round is never overwritten"; the build then died on the filesystem's own
FileExistsError. is_symlink() is checked FIRST, the message says what a link would cost
(the round's content would sit somewhere the gate does not measure), and the link is
left exactly as it was found — nothing is written, exit code 1 like every other refusal.
ONE CITATION LIST SHARED BY EVERY PROPOSAL is now stated once. The cause was measured
before anything was written, because "the builder reads the wrong field" and "the outbox
says the same thing five times" want opposite fixes: in all four archived runs every
proposal carries a byte-identical 270-citation list — the run's whole retrieved context,
stamped once per proposal. No report can make that quote say something about the
individual measure. So when every proposal carries the same list, the report says so
once, says what the list actually is ("hva kjøringen leste, ikke hva det enkelte tiltaket
bygger på"), and drops the five copies. When the lists differ, nothing changes: the quote
and its COUNT stay under each proposal, which is where they mean something.
THE SAME COST LINE ON BOTH SIDES OF THE VERDICT is named where it happens. The 19.09
report refused TUN-LYS-01 under one label and validated the same line under another and
said nothing, so a reader met two figures for one budget line with no way to see they
collided. Both sides now carry the sentence, in the run's own row order.
A REMOVED APPROACH is shown by the label the expert saw, with the id in parentheses. It
is the one row whose human name is not in this run's coverage, so the label is read from
the coverage inside the PREVIOUS round's own outbox — derived from what the round already
carries, not a new column in outcome.json. An unreadable coverage falls back on the bare
id; a missing label is not a reason to refuse a round.
39 of 39 arms green. Nothing here touches what the gate reads: outcome.json keeps its
four columns and the builder still never writes the attestation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 19.09 checkpoint planted 20 mutants against this file and SEVEN survived the whole
suite: the report's round number, an artefact type outside proposal/outcome/coverage
dropped in silence, a never-evaluated approach left off its own list, the source quote
removed entirely, the quote's COUNT removed, the affected cost lines removed, and a
refused row's amount counted into the validated total. None of them survived because the
builder was wrong — it does all seven things correctly. They survived because no arm in
709 lines ever looked (grep for citation/cost-line words gave 2 hits, both in the
fixture, none in an assert). An unwitnessed seam is a seam the next edit may delete for
free.
Two fixture defects made three of those blind spots unreachable rather than merely
unmeasured:
* THE DENOMINATOR. A real run leaves SEVEN artefact types; the fixture wrote three.
Counted here, not remembered: over the four archived runs the checkpoint read
(tunnel-hauglia-2027-04/-06/-07/-08), each <run_id>-<rest>.json is typed proposal /
outcome when <rest> ends there and <rest> itself otherwise. -06, -07 and -08 hold all
seven (15 files each); -04 holds six (12 files, no parse-failures — that file is
written only when something failed to parse, so its absence is the signal). Union =
7, written by two commands: run.py writes six, stress.py writes -verdict.json. Six
are now written through the product's own writers; -verdict.json is bytes, because
the builder never reads it and that difference is stated rather than hidden.
* THE CITATIONS. Every proposal carried an identical stamp, so "1 av 1 siterte steder"
could not tell a dropped COUNT from a kept one. Counts are now distinct per approach,
and a shared-list mode reproduces what the real runs actually do.
The refused row now carries an amount, and no archived run does: measured on the same
four outboxes, every rejected row has saving_nok = None. The guard that only a validated
row's figure becomes validated_nok therefore aims at a coverage writer that does not
exist yet — and a fixture that cannot produce the number cannot witness the guard at all.
4 of the 11 new arms are RED on ASSERT, all four on behaviour that does not exist yet:
a dangling symlink as the round directory (exists() follows the link, so the build dies
on the filesystem's own FileExistsError instead of a sentence), the identical citation
list stated once instead of five times, the same cost line named on both sides of the
verdict, and a removed approach shown by the label the expert saw instead of a raw id.
The other 7 pin behaviour that is already correct; they are not red here, and the proof
that they discriminate is the mutant run, not this commit.
The cause of the identical quote was MEASURED before anything was written, because
"the builder reads the wrong field" and "the outbox says the same thing five times" want
opposite fixes: all five proposals in tunnel-hauglia-2027-08 carry byte-identical
270-citation lists (one sha256 across all five), and so do -04, -06 and -07. It is the
run's whole retrieved context stamped once per proposal. The report cannot make that
quote informative; it can stop repeating it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Repo convention: every measured decision gets a row in the ledger with the load-bearing test that
turns red when the decision is undone. The row records the three choices the gate states in its
own output (CLI door over MCP, no budget ceiling in toolbox mode, row 6 never green without the
operator), the base64 reason, and the eight mutants that fell in the scratch clone — including
M-6, which shows row 3's green is a measurement and not a vacuous zero: without the
fenced-block rule one prose line in a research doc turns the row red.
v1 gate re-measured after the work: 0/3 · 0/3 · 3/8 · ingen rapport · 3/8 · IKKE MÅLT · 1/20 —
unchanged. PLAN.md § Ferdig-kriteriet untouched, as the order required.
Suite after git add: 2106 passed / 5 skipped / 5 xfailed, rc=0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
python -m portfolio_optimiser.evals.b_gate — one command, offline, no model call, exit 1 today:
1 steg i kjørestien kallbare utenfra 3 av 13 RØD
2 roller som kan leveres utenfra 0 av 2 RØD
3 vakter mot en vei fra po til Claude 6 av 6 GRØNN (435 published files)
4 løpet drevet uten et eneste modellkall 0 av 3 RØD
5 Foundry-veien urørt og samme artefaktfamilie 1 av 2 RØD
6 kjøreboka finnes og er kjørt 0 av 2 IKKE MÅLT
Every denominator is read off the source, never off a list in the gate. Row 1 counts the steps
of the run path that resolve to a symbol AND have a call site; a step is externally callable only
when a CLI (or MCP-registered) entry reaches it without any chat-client construct on the way —
which is why the ten run.py steps are red and round_builder's two plus the v1 gate are green. Row
2 reads the roles off workflow._MAKER_CHECKER_ROLES. Row 3's patterns each carry a known-positive
AND a known-negative fixture, so a guard that cannot hit is not counted as a zero.
Three decisions the operator cannot answer without reading code, made here and stated in the
gate's own output:
* the external door is a CLI subcommand, not MCP — po already has five main() and two console
commands, and MCP would need a server the run path does not have. The gate still counts an
MCP-registered door, so the choice does not bind the next order.
* the budget guard in B is NOT po's: BudgetMiddleware is fail-closed on missing usage and is
never constructed without a chat client, so keeping it here would turn fail-closed into
fail-open. The ceiling in B is the Claude Code session's own spend, which po neither sees nor
steers. The Foundry path keeps its ceiling unchanged.
* row 6 is IKKE MÅLT, never green, until the operator attests that the runbook actually drove an
analysis — a file the gate never writes, the same rule as the v1 gate's attestation.
Row 3's pattern text is base64 in the config so the contract cannot register as its own finding;
that is what lets the row run without an exclusion list, and a row without exclusions is a row
nobody can switch off by adding a filename.
Suite after: 2106 passed / 5 skipped / 5 xfailed (was 2072/5/5; +34 new, none changed).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Operator decision 19.09.2026: in development and test Claude Code LEADS and portfolio-optimiser
is the toolbox. po never calls Claude; production stays on Foundry. This commit writes the
measurement RED — the contract first, the capability later.
Six rows, each with a denominator taken from the SOURCE and counted independently here:
1. toolbox complete — every deterministic step of the run path, anchored to (module, symbol) and
to the scope that calls it; k = steps reachable from a CLI entry WITHOUT a chat client.
2. what the model delivered can be delivered from outside — denominator read off
workflow._MAKER_CHECKER_ROLES, k = roles with passing named probes.
3. po has no path to Claude — six patterns, each carrying its own known-positive AND
known-negative fixture. Pattern text is base64 in the config so the gate's own contract cannot
register as a hit against the surface it scans.
4. no model calls in toolbox mode — three named probes.
5. the Foundry path untouched — profiles, factory and injection seam read from source, plus a
schema-comparison probe.
6. the runbook exists — NOT MEASURED until the operator attests, and the gate never writes that
attestation itself.
Red now for the reason that matters: the tests fail to import a module that does not exist yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found while killing mutants: an ordering made to depend on hash() survived
test_two_builds_of_the_same_outbox_are_byte_identical untouched. One interpreter has one hash
seed, so the two builds agreed with each other and with nothing else -- the arm could not tell a
deterministic report from a report that is merely consistent within a process. The ordering arm
caught that mutant, so the seam was witnessed; this one was not witnessing it.
The second build now runs as a subprocess with PYTHONHASHSEED=1. Re-measured with the same
mutant: both arms red, where before only one was.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
python -m portfolio_optimiser.evals.round_builder --outbox <dir> --round <n> --ran-at <ISO>
writes <rounds-dir>/<n>/ with the run's artefacts COPIED in, outcome.json derived from that
copy, and report.md -- the one artefact in a round a domain expert reads and corrects. Round 0
of the v1 criterion can now be made; it counted 0 of 3 because it could not be, which is a
different failure from a round nobody had held.
What it derives it derives with the gate's own functions rather than a second copy: verify_run
decides whether the run stands up to itself (an artefact contradicting its coverage row, a
half-missing family and a stray artefact are all refused AT THE SOURCE, before a byte is
written), stage_of gives column (c), row_changed gives the report's "changed since the previous
round", parse_time refuses a stamp without a zone, safe_rounds_dir refuses a round directory the
repo would commit. The validated total is ledger.to_ore per amount, summed as integers.
Two things it never does, and both are the point. It never writes the operator's attestation --
the gate stops at FORM OK without one, and that is correct, because no arrangement of files can
witness that a run happened. And it never invents: --ran-at is required because no outbox
artefact carries a clock, and feedback_ids stays empty because no run records which feedback
item produced which row. The report says "ingen tilbakemelding forklarer dette" on every changed
row rather than hiding that model noise and an answered objection look alike.
Chosen and why: --ran-at as a required argument rather than the coverage file's mtime, because
an mtime is a filesystem attribute one call sets and reading it as evidence made row 2 green on
a tree nothing had run in (18.09). The report carries no raw stage identifier -- every stage
sentence is "<short name>: <explanation>" so the one-line diff of what changed has words a
reader can act on. A citation shows its COUNT, because a run that cited 446 places and one that
cited one must not look the same.
[skip-docs]: the ledger row is in docs/invarianter.md, which is where this repo's rules live.
README is the product's front door and this is an operator tool behind `python -m`, the same
class as costsim/hitl/preflight, which README deliberately does not carry; v1-rounds/ is
gitignored internal machinery and the gate itself is not in README either. CLAUDE.md was emptied
of exactly this kind of row in session 130 and is not the place to put one back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round 0 of the v1 criterion cannot be made today: nothing binds a finished run's outbox to
<rounds-dir>/<n>/, and nothing in src writes markdown a domain expert could read. These tests
state what a builder has to do before one exists, and every number they assert is counted a
second time from the fixture's own table rather than read back from the builder.
Red on assertions, not on import: round_builder.py lands as a contract -- dataclass, signatures,
neutral returns -- so each test fails in its own body.
Two gate helpers become public rather than being copied: row_changed (the report's "changed since
the previous round" section must not disagree with the gate about what changed) and
safe_rounds_dir (the builder CREATES the directory the gate only reads, and the writer is where a
leak of the expert's feedback has to be stopped).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
K5 survived a full run (a mutant ignoring an extra or removed file left the suite green) and K4
(sort removed) was caught only by corpus data. Now pinned with no corpus: the add arm, the remove
arm, and a shuffled directory listing digesting the same. The gate says "pin-drift" for a drifted
copy instead of "artefakter mangler"; the unread "store" key is gone from frozen_bundles.json; the
forbidden-path scan covers md/yaml/yml/toml and proves a known positive per suffix.
Five mutants killed in a scratch clone (drop-last-entry 8 failed, no-sort, no-names, drift-as-missing,
scan-suffixes-narrowed 1 failed each).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Chosen: a key written twice is RED (PM recommendation: two answers to one question are not an answer). A time without an offset is RED as ambiguous. A hard link is NOT refused but declared a limit in the output (content binds round, run and date either way). A BOM is tolerated. The clock is a parameter (now=), read by ONE check.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The PM checkpoint of 18.09 paid M-6's price three times and row 2 still read 3 of 3 GREEN:
a whole artefact family written by hand and made internally consistent (verdict_id minted
with the product's own rule, about sixty lines of script), four REAL runs' artefacts under
a handwritten feedback file, and `<n>/outbox` symlinked out of the round to a real run
elsewhere. `RUN_ATTESTATION` stood in the output the whole time, and the row was green
anyway — which is the finding: a row that PRINTS its limit and reports GREEN has not
stated a limit, it has decorated one.
One of the three is a hole and is closed; two are not holes at all.
1. The outbox must lie in the round ITSELF (`outbox_escape`). Derived is only derived if
the filesystem cannot redirect it: `<n>/outbox` as a symlink, or any symlinked artefact
inside it, is refused by name.
2. Rows 1 and 2 can no longer be green from files. What the computation gives is
`FORM_OK` ("FORM OK, IKKE BEVIST"): every check the gate knows how to make passed, the
round counts 0 against the criterion, and the exit code still fails — it is un-green in
every way RED is. What separates the two is what the reader must do next, not what the
gate concludes.
3. Green is `<n>/attestering.txt`, per round: the operator's own statement, naming the
round, the run and the date. Plain text with three keyed lines, deliberately not JSON —
it is the one artefact in this contract a PERSON types, and a missing brace must never
be able to read as "not confirmed". Nothing in the product writes one, and a test in
`src/` says so: a product that can produce a witness to its own run has produced
exactly the thing these rows cannot.
A MISSING attestation is FORM OK: nobody has confirmed anything yet, and the gate is
not entitled to call that a lie. A PRESENT one that names another round or run, carries
no date, an unparsable date, or a date before the run, is RED: it is a statement about
something this is not. Row 2 needs round 0 attested as well, since round 1 is measured
against it — an unattested baseline is an unattested comparison.
The attestation is read raw off `outcome.json` (`_declared_run`), not through
`read_outcome`, so row 1 — which reads no run — does not inherit row 2's artefact checks
through the back door.
Nothing moved on the criterion: 0/3 · 0/3 · 3/8 · no report · 3/8 · IKKE MÅLT · 1/20,
exit 1. What changed is that rows 1-2 now name what they are waiting for, and it is not
another file.
Suite after `git add` on a clean tree: `uv run pytest -q` = 2014 passed, 5 skipped,
5 xfailed (508 s). The attack of b769537/ab00016 is the red half of this commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
M-6 moved the price of a forgery from `touch` to reproducing the product's own artefact
set. The PM checkpoint of 18.09 then paid that price three ways, and row 2 read 3 of 3
GREEN each time:
F1 the whole artefact family written by hand and made internally consistent — coverage,
one proposal/outcome pair per approach, `verdict_id` minted with the product's own
rule. About sixty lines of script. `RUN_ATTESTATION` stood in the output; the row was
green anyway.
F2 four REAL runs' artefacts under a handwritten feedback file.
F4 `<n>/outbox` made a symlink to a real run's directory somewhere else, so a DERIVED
path was redirected by the filesystem.
F4 is a hole. F1 and F2 are not: no arrangement of files can be told apart from a round
that happened, because what is missing is not a check but a WITNESS. These tests say so.
Red here, 16 of them:
- rows 1 and 2 must stop at `gate.FORM_OK` — 0 against the criterion, exit still 1 — on a
tree where every computed check passes but nobody has attested the rounds (F1, F2);
- F2 is built through `outbox.write_outbox` and `write_coverage`, the exact bytes a real
run leaves behind, so it doubles as the control that `verify_run` reads the PRODUCT's
output and not merely the shape this test file writes;
- an outbox that is a symlink out of the round, and an artefact symlinked into one, are
refused by name (F4);
- the step from FORM OK to GREEN is `<n>/attestering.txt`, per round, round 0 included
because row 2 measures round 1 against it: present and matching -> GREEN (the rc-0
control, asserted first in every arm), missing -> FORM OK, present but naming another
round, another run, no date, an unparsable date or a date before the run -> RED;
- nothing in `src/` may write that file: a product that can produce a witness to its own
run has produced exactly the thing rows 1-2 cannot.
One more arm, green on arrival, and that is the finding: the guard at `v1_gate.py:415`
(`keyed != {(run_id, aid)}`), pinned to a constant-false branch, left the suite at 68
passed, identical to baseline. The guard bit, nothing read it — so
`_mut_the_family_is_labelled_for_another_run` gives the pair the right FILENAME and
another run's labels inside.
The fixture now writes the operator's attestation on every round it builds, and the
file's name is pinned in the test file as well (`_ATTEST_FILE`), the way M-5 pins the
rest of the contract: renaming it in the gate alone must fail a test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 18.09 re-measurement took row 2 to 3 of 3 GREEN on a tree this product had never
run in: four handwritten outcome.json, four handwritten <run_id>-coverage.json in an
outbox the forger named in those same files, and os.utime for the ordering. "Round 0
must be a named real run" was implemented as "a file with that name exists" — which
touch satisfies. The attack is committed as a red test in b769537.
Three bindings, chosen because each removes one of the forger's three moves:
1. The outbox is DERIVED, never declared. It is <rounds-dir>/<n>/outbox/, and an
outcome.json that names one is refused by name. A path a round file chooses is a
path it can point at a directory the same hand just wrote.
2. "The run exists" now means the run's own artefacts agree WITH EACH OTHER on content
the gate recomputes (verify_run). Every evaluated approach has the proposal/outcome
pair write_outbox actually persists; outcome_type IS the coverage status, reason IS
its detail, a validated row's figure IS the proposal's own claimed_saving_nok, the
provenance stamp follows the same branch, and verdict_id is RE-MINTED here from the
proposal's own IR with the product's one minting rule (A5) rather than read. A
not_evaluated approach wrote neither file, and an artefact naming an approach the
coverage omits belongs to another run. verdicts._features_from_ir is made public for
this: a second private copy in the gate could drift from the rule the run stamped
with, which would turn the binding into a coincidence.
3. mtime decides nothing. The run's time is the round's declared ran_at (ISO-8601 with
zone, required). An mtime is not evidence — it is a filesystem attribute one utime
call sets.
What this does NOT do is prove a run happened. Nothing in a directory can: the outbox
writers are byte-deterministic and carry no clock by contract. So row 2 states its own
limit on every run instead of leaving GREEN to imply it (Row.attests / RUN_ATTESTATION):
that a run was actually made, and when, is the operator's to confirm. The cost of a
forgery moves from touch to reproducing the product's own artefact set, minting rule
included.
Measured, in a scratch copy, never in the work tree — four new mutants in this class,
each one line, each felled by the whole suite:
M7 if "outbox" in data: -> if False: 1 failed (declares its own outbox)
M8 if verdict_id != minted: -> if False: 1 failed (key is not the IR's)
M9 if strays: -> if False: 1 failed (artefact of another run)
M10 ran_at -> coverage mtime 18 failed (incl. the utime test)
Control, same scratch copy, unmutated: 1993 passed, 10 skipped, 5 xfailed.
Work tree, re-run after git add: uv run pytest -q -> 1998 passed, 5 skipped, 5 xfailed.
Gate: uv run python -m portfolio_optimiser.evals.v1_gate -> exit 1, row 2 RED (0 of 3).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 18.09 re-measurement of 9825b26 made row 2 read 3 of 3 GREEN without a run existing
anywhere: four handwritten outcome.json, four handwritten <run_id>-coverage.json in an outbox
named by those same files, and os.utime for the ordering. read_outcome does cross-check the
outcome against the coverage — but `outbox` is a free path from the round file and the coverage
file is written by the same hand, so "round 0 must be a named real run" is implemented as "a file
with that name exists", which touch satisfies.
The attack is rebuilt in the test rather than driven through the _outcome fixture, so the
fixtures can be raised to a whole run family without the attack drifting with them. Row 1 is
asserted GREEN on the same tree: the feedback IS well formed there, which is what makes this an
attack on row 2 and not a broken fixture.
RED as committed (3, 'GRØNN') != (0, 'RØD') — the fix is the next commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The MAF-free rule is a family of guards, not one list: _MAF_FREE_MODULES
holds nine modules (tests/test_okf.py:22-32), ingest/ingest_mcp carry
their own, hitl/notify add a transitive import-graph guard. The repo map
says so instead of claiming one AST guard over eight modules.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One-hour walkthrough in seven parts: purpose, architecture, the elements,
the workflow and its feedback loops by duration, the MAF harness (what it
is, its building blocks, what this repo chose and refused), the quality
method with measured status, and the work after v1.1. 48 inline SVG
figures, white background, every content slide carries a source line into
the tree. Numbers carry their denominators; the two-denominator caveat on
the stress rounds and the two attempts of the 14.08 live run are stated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CLAUDE.md had grown to 310 919 bytes against Claude Code's 150 000-character
injection limit, so every row past the cut reached no session. The 93 measured
rows move to docs/invarianter.md in their original order; CLAUDE.md keeps the
eight short standing rules and a pointer, and says new rows are written there.
Verified as a partition: every moved line appears in the original section in
order, the eight kept rows likewise, and head/tail of CLAUDE.md are byte-
identical apart from the visitor note. One code span is reworded and the ledger
head says so: the funn 99 row cited MAF's DEFAULT_MAX_CONSECUTIVE_ERRORS_PER_
REQUEST with its value inside the span, which the doc-constant-sync gate reads
as a citation of a constant in this package (fail-closed on an unknown name).
The ledger is registered in _LIVE_DOCS, so that gate now also guards it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Measured 2026-09-17 17:43: vegnormal-okf rebuilt build/ferdig/r761-2025 while this
repository's v1 gate, the stress judge and four corpus tests pointed straight at it.
Rows 6-7 went IKKE MAALT and five tests fell, for a change no one here made. The failure
mode was never falsehood - the gate says IKKE MAALT and exits non-zero, never green - it
was instability: two projects shared a directory neither owns, so what this repository
MEASURES could move without a commit here.
A copy alone would push that directory one move away, so the copy comes with a pin.
frozen_bundles.json (tracked) carries path + sha256 + file count per base; the bundles
themselves are NEVER committed here. Three states, separated by construction: match ->
resolves; gone -> FrozenBundleMissing (an OSError, so the gate's existing except OSError
gives IKKE MAALT + exit 1 unchanged and the corpus tests SKIP, MAJOR-3's ceiling); drift
-> FrozenBundleDrift (a ValueError), loud, named, and never a skip. The two classes are
deliberately unrelated: a caller that catches "missing" to skip must not swallow "drift".
The NAME is hashed alongside the bytes, and the directory name carries the first 12 chars
of the digest so a stale copy is visible in ls. Renewal is a decision: new copy + new pin
in the SAME commit (README). --bundle-root / PORTFOLIO_VEGNORMAL_ROOT stays as the
operator's explicit, UNPINNED live mount.
Iron Law: the tests were written and run RED first (collection error, then two arms of my
own making). Load-bearing MEASURED, eight mutations all red against the WHOLE suite with a
green control of 1984 passed / 5 skipped / 5 xfailed and a strict node-id superset
(1977 -> 1994, 0 removed): M1 the pin is never verified (7) - M2 drift collapsed into
missing (5) - M3 the name is not hashed (40) - M4 the gate seam reverted to root/name (1) -
M5 the corpus helpers skip on drift too (4, one per file) - M6a the slash spelling back in
src (1) - M6b the quoted path segment back in a test (1) - M7 the directory name drops the
short digest (1, and 45 skipped, which proves absence is a SKIP and not a false green) -
M8 the explicit override ignored (3, two of them in test_stress_judge_loadbearing.py,
independent witnesses older than this work).
M2 FALSIFIED THE TEST FIRST: the four parametrised arms did not go red, they went to SKIP
(5 -> 9 skipped) and stayed green - pytest.skip inside a pytest.raises is not a failure.
The arm now catches pytest.skip.Exception explicitly and turns it into an AssertionError.
grep -rnE 'vegnormal-okf/build|["'"'"']vegnormal-okf["'"'"']' src tests contexts -> 0
(3 + 4 hits before; the three remaining prose mentions document history and are allowed).
Gate re-run against the frozen copy: identical to the live mount (rows 0/3 - 0/3 - 3/8 -
no report - 3/8 - IKKE MAALT - 1/20, exit 1).
Order 20260917T223645Z-1296211942-from-.claude.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An independent review made rows 1, 2 and 4 green from a handwritten
directory in a minute, and 10 of 20 mutants survived the gate's tests.
Rounds now need a new point and their own ids, a timezone-aware given_at
in order, and a report the feedback was given on; every outcome must name
a run whose own coverage confirms (a)-(d), the feedback must fall between
the two runs, and a NOK change under 1 % is noise. Row 4 counts content
lines kept unchanged and in order, shows the expert's additions, and calls
a byte-identical copy untouched unless round 3 acknowledges it. Row 6
counts the runs' own proposals. Types 3 and 7 are proven through the real
flags with the action in the result (still 3 of 8). The contract numbers
and the evidence register are pinned to their source. Every run prints
that rows 1-2 cannot prove who wrote the feedback. A rounds directory
inside the repo that git would commit, and a missing stress or bundle
root, are usage errors.
The review's 20 mutants, re-run: 20 of 20 killed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stress round 6 validated three falsification arms, and every validated
approach rested only on run-level declarations nobody can attribute to one
approach. declare_requirement now takes a required approach_id (a mandate
id or own-proposal; an unknown id is refused naming the valid ones), and a
ValidatedProposal whose approach has neither a mandate requirement nor a
declaration under its own id becomes validator.Unsupported - a Rejection
subclass carrying the validator's own ruling, reported as `unsupported` in
coverage, the outcome artefact, the settlement and the judge, and never
counted or summed. The rule is active whenever the debate held the
declaration tool, the micro base included; the road and pre-pass paths are
untouched. Declaration quality is not judged, so the rule can be satisfied
by declaring any document the run read.
The v1 gate's row 6 probes pass; its artefact half reads IKKE MÅLT because
stress round 6 predates approach-addressed declarations, and IKKE MÅLT is
never green - it fails the exit code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The eight U-IDs and their type pointers were checked against the approved
list (no deviation) and the data file now says approved, with the date and
the source. Row 5 no longer reports "not approved" but the measured count:
3 of 8 (U12, U4, U6), because a point counts only when every type it points
at is green and only types 1, 3 and 7 are. No gate logic changed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One command says how far the repo is from v1, row by row, with an exit
code: rounds with a real domain expert 0/3, traced measurable change 0/3,
feedback types with a way in and an action 3/8 (1, 3, 7), round 3 report
kept - none, MAF points with a green type pointer 0/8 (list not approved),
validated without the approach's own declaration 10/10 in stress round 6,
and `named` 1/20 as a diagnosis that never moves the exit code.
The gate defines the contract (a fixed rounds directory, gitignored by
default), not the generator. Rows 3 and 6 run named tests with
--runxfail; the red probes are xfail(strict=True) so the suite stays green
while the gap is real. No product code changed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The persistent plan (global fourth continuity layer, 2026-09-15) follows
STATE's remote rule: this repo publishes to open/, so PLAN.md must never
be tracked. git check-ignore -v PLAN.md returned rc=1 before this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five paid runs, rc 0 on all five, 913 320 tokens, no round cap hit, anchored 6 of 6.
DEL A hit hard: naming the project's own cost codes in stage 0's refusal took `priced` from 0 of 20
to 16 of 20 and validated approaches from 0 of 20 to 10 of 20, with invented cost codes down from 26
to 12. The largest single movement any part has produced in six rounds.
DEL B moved a number three rounds had not: declarations pointing at an answer-key concept went
0 of 13 -> 0 of 12 -> 5 of 16, and requirement_hit per approach 0 of 20 -> 3 of 20.
DEL C halved the guessing: read_file against a document the base does not hold went 7 of 52 to
3 of 51, read_dir against a level it does not hold 8 of 128 to 2 of 97. Two of the three remaining
read_file misses got the NEW document clause, so DEL C fired live.
AND THE CLOSING EXPOSED SOMETHING LARGER. must_refuse -- the falsification arm, the one approach per
set the base has no ground for -- went 5 of 5 to 2 of 5. Three were VALIDATED. P21's 5 of 5 was not
sharp: it was achieved because the model invented a cost code, not because the base lacks ground.
The order predicted exactly this. Now the arm discriminates, and it says nothing in the gate speaks
to whether the base supports the direction. That is finding 1 and it stands first, with a named
solution and an operator decision attached.
Also recorded: the order's own cited figures re-measured with denominators before anything was built
on them. "requirement_hit 0 of 12" mixes a per-approach field (0 of 20) with the declaration count
(0 of 12); "0 of 20 validated" and "26 rejections" are two different populations (20 approach rows
vs 20 + 6 own-proposals); and the read-miss figures hold under the context_files definition while
the judge's filesystem one gives 6 of 52, not 7. Rounds 4 and 5 are re-judged with the same
instrument; round 3 is deliberately NOT, because filling in anchored/priced/stage from prose would
be inventing a measurement.
Honesty limits: five paid runs on one question are not a sample; every amount in the price schedules
is invented and the FORM is what is not; the cost figure is an ASSUMPTION at list price with no
invoice read; and must_refuse 2 of 5 is not evidence the system got worse, it is evidence the
previous measurement could not discriminate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
P21/C2 made a refusal for an absent path name the ancestor's SUBDIRECTORIES, and it bought what it
was built for: read_dir against a level the base does not hold went from 16 of 104 to 8 of 128. It
did nothing for documents -- read_file against a document the base does not hold went 2 of 38 to
7 of 52 -- and the reason is structural: the nearest listable ancestor of a guessed DOCUMENT path
often holds documents and no subdirectories, and then the neighbour clause was omitted, deliberately,
because an empty list is a sentence with nothing in it.
Measured over round 5's six read_file misses, THREE land on such an ancestor: krav/N100 with 445
documents, and R761/1 with exactly ONE -- which two separate guesses in one run were both reaching
for. The other three have subdirectories and were already answered.
okf.nearest_documents is the sibling of nearest_subdirectories, never a widening of it: never both
clauses, and the subdirectory branch stays FIRST, which is what keeps every C2 refusal byte-identical.
Built from context_files and through the same in_dimension predicate the listing uses, so a refusal
can never advertise the type: verdict layer by path, and every name it hands back resolves -- measured
by feeding each one back into read_file, not by asserting the list is non-empty.
A MUTATION FOUND THE RANKING UNWITNESSED, and that is recorded rather than dropped: replacing
_shared_prefix with a plain reverse sort left the whole suite green. The bound, the source and the
resolve property were all gated; the ORDER was not. For R761/1 that costs nothing, but a level of a
delivered corpus can hold 445, and then which five it names is the whole value of the clause. The new
arm builds a level where the closest name is also the LONGEST, so a length rule puts it last and an
alphabetical one puts another first -- only the prefix rule puts it first.
Load-bearing MEASURED (tests/test_document_neighbours_loadbearing.py, 10 arms), seven mutations all
red against the WHOLE suite + green control 1891/5 (from 1881/5, superset, 0 removed) and golden
demo-transcript.stdout BYTE-UNCHANGED (shasum -a 1 of the CONTENT =
ea8c534773acdbe41ae68f2c55724d69aaf8be4f): C1 detach the document branch in read_file (5 red) -
C2 detach it in read_dir (1) - C3 build from files (1) - C4 ignore the dimension (1) - C5 no bound
(1) - C6 both clauses at once (1) - C7 a second ranking rule (1, after the test was fixed; green
before, which is the finding).
Honesty limits, stated: the foreign-dimension arm was VACUOUSLY green before this change (nothing
was named, so nothing could leak) and is gated only now -- C4 is what makes it real; no LIVE model
has read the new clause (DEL D is the measurement); and the clause is help text, not a gate -- it
cannot make a guessed path right, only cheaper to correct.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
P19 DEL A made a direction name the requirement that binds it; P20/A1 made the reply carry the
DOCUMENT's own title and number instead of echoing the caller's arguments. Re-measured at the head
of this session against the six round-5 debate traces: requirement_hit is 0 of 20 approach rows and
0 of 12 declarations -- the third round in a row at zero. P21/C1 made the runs LOOK first and it
worked on its own terms (distinct documents before a declaration went 1,1,1,2,5,13 -> 3,3,5,7,11,12)
and the hit did not move. The runs were made to read MORE, not righter.
The reply now compares: it names the directions the run was commissioned to pursue and says which
of their words appear in the declared document's own title and number, or that none do.
A REPORT, never a gate -- the declaration is recorded either way, because a requirement can bind a
measure without sharing a word with the name someone gave it, which is exactly how the alternative
rule the C1 measurement rejected failed one rung over. The words compared are the DOCUMENT's, never
`ref`: a comparison against the caller's own argument can only ever agree. Matching is generous in
both directions, and that failure direction is chosen -- a false "no overlap" pushes a model away
from a declaration that was right, a false "overlap" merely keeps the report quiet.
MEASURED BEFORE IT WAS BUILT, offline against the six traces as the order required (no paid calls
in DEL B): the rule speaks on 10 of 12 declarations and stays quiet on 2. A rule that spoke on 12
of 12, or on 0 of 12, could not tell the two classes apart.
`labels` defaults to empty, so every call site written before today is byte-identical and the three
keys are ABSENT rather than empty -- "there was nothing to compare against" and "we compared and
found nothing" are different facts. RUN-level, as the declaration itself is (P19 A4).
Also re-measured: the order cited requirement_hit as "0 of 12". The field is per APPROACH (0 of 20);
12 is the number of DECLARATIONS (7 distinct, 0 hits). Both zero, so the conclusion stands, but they
are two populations.
Load-bearing MEASURED (tests/test_requirement_comparison_loadbearing.py, 8 arms), eight mutations
all red against the WHOLE suite + green control 1881/5 (from 1873/5, superset, 0 removed) and
golden demo-transcript.stdout BYTE-UNCHANGED (shasum -a 1 of the CONTENT =
ea8c534773acdbe41ae68f2c55724d69aaf8be4f): B1 detach the run.py wiring (1 red, that arm alone) -
B2 always report an overlap (5) - B3 never report one (2) - B4 compare against the caller's ref
(1, that arm alone) - B5 make it a gate (5) - B6 emit the keys with no directions (2, one an OLDER
independent witness) - B7 exact token equality instead of substring (1) - B8 drop the minimum word
length (3).
Honesty limits, stated: no LIVE model has read the comparison yet (DEL D is the measurement); the
report cannot say a requirement IS right, only that it shares no word with the direction; and
finding 4 (`named` 1/20) is this same matter from the other side, so DEL D measures it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
P21 was the first ANCHORED stress round and it bought something real -- must_refuse 5/5, all
five caught on stage0-baseline. It cost something measured just as clearly: 0 of 20 approaches
validated, and 26 of 26 rejections (20 approach rows plus 6 own-proposals -- a LARGER population
than the 20) read `unknown cost code '<invention>': not in project P's cost baseline (5 known
codes)`. The model invented signalregulering_konstruksjon, VENTIL_IMP, RIGG01, baerelag_asfalt
and 22 more, and it could not have done otherwise: the price schedule reaches the VALIDATOR and
never the proposer, and the refusal stated the COUNT of known codes, not one name. Step 5 feeds
that sentence verbatim into the next attempt -- and "you guessed wrong, there are five right
answers" carries nothing to correct towards.
The contrast already lived in the same stage: the MAGNITUDE half NAMES the baseline value, and
that is the half that let the loop converge in session 94. This gives that property to the other
half, in one place, and Step 5 carries it forward for free.
The window is a FIXED COUNT of whole codes (20), never a share, and it counts codes rather than
characters because a character cut can sever a code mid-name and hand the proposer an identifier
that exists nowhere. The cut is announced; a schedule that fits is not marked truncated; the
order is the schedule's own.
Measured with denominators: every cost baseline in this repo or its measured corpora is at most
six codes, and the largest real delivered price schedule measured is K2's prissammenstilling at
14 priced rows. Nothing measured reaches the window; it exists for the R761-style mengdebeskrivelse.
Load-bearing MEASURED (tests/test_named_known_codes_loadbearing.py, 10 arms), eight mutations all
red against the WHOLE suite + green control 1873/5 (from 1863/5, superset, 0 removed) and golden
demo-transcript.stdout BYTE-UNCHANGED (shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f):
A1 revert to the bare count (7 red) - A2 a share instead of a fixed window (6) - A3 sorted (1) -
A4 silent cut (2) - A5 character slice (3) - A6 no bound at all (3) - A7 break the rejection_stage
marker (2, one an OLDER independent witness) - A8 grow the magnitude half with a code list
(2, one an OLDER independent witness).
Honesty limits, stated: no paid run yet confirms this changes the outcome live (that is DEL D);
the truncation branch is exercised only synthetically because nothing measured reaches the window;
and the schedule still does not reach the prompt, so the first attempt guesses as before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five paid runs, rc 0 in all, 1 130 145 tokens (~NOK 7,6 under P20's stated list-price
assumption; no invoice read).
What the anchoring bought, measured off the artefacts rather than inferred from the
flag: cost_baseline_anchored true 6/6, and the falsification arm a4 passed 5 of 5 with
ALL FIVE caught by stage0-baseline -- the one stage that knows what the project buys.
Round 4, re-judged with the same instrument: 4 of 5, every one of them on stage 0b, and
the fifth VALIDATED.
C1 fired live and changed behaviour: distinct documents opened before a declaration went
1/1/1/2/5/13 -> 3/3/5/7/11/12, and THREE declarations were refused mid-run (15 calls, 12
recorded) after which the model read more and declared again. C2: read_dir against a
level the base does not hold fell from 16 of 104 to 8 of 128.
And what it cost, measured just as plainly: 0 of 20 approaches validated (round 4: 4).
All 26 rejections read "unknown cost code" -- because the schedule reaches the VALIDATOR
and never the prompt, and stage 0's refusal names HOW MANY codes the project has and not
WHICH. Step 5 feeds that refusal back verbatim, and "you guessed wrong, there are five
right ones" cannot be corrected. Contrast the magnitude arm, which names the baseline
value and therefore converges.
Four remaining findings, each with a NAMED solution and an estimate -- the first is one
sentence in _reconcile_against_baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
C1. Round 4 produced 13 declarations over six runs and NOT ONE named a fasit concept.
The distinct documents opened before each were 1,1,1,1,1,1,1,2,5,5,6,13,13: seven
declared the base's FIRST requirement after opening exactly ONE document.
The order offered two rules and asked which discriminates. Replayed against the real
listings: "the declared document must have come back from a read_dir filtered on a word
from the approach's label" refuses 13 of 13 -- including Soraasen's 12.11, the closest
any run came -- because ZERO of the 13 were reached through a filtered listing at all.
A gate that refuses every measured case, right and wrong alike, cannot discriminate.
"fewer than k distinct documents opened" at k=3 refuses 8 of 13 and keeps the five that
navigated. k=3, 4 and 5 refuse the SAME eight -- the distribution has a gap between 2
and 5 -- so the threshold is not on a cliff, and 3 is the lowest of that plateau.
DISTINCT paths, not calls, and capped by the base's own size so a small base stays
declarable.
C2. Over the same traces 18 of 143 path-bearing calls named a path the base does not
hold, ELEVEN of them one run walking R761/4-3, 4.3, 4-2, 4-1, 4-0, 4-5, 4-6 while the
real names are R761/4, R761/41, R761/42. The refusal already named the nearest listable
ancestor; now it also names up to five of that rung's own subdirectories, ranked by
longest common prefix with the segment that failed. ONE copy shared by both refusal
sites, built from context_files through in_dimension, so every name handed back resolves
and the verdict layer can never be advertised in an apology. MEASURED after: 16 of 18.
Load-bearing MEASURED, four mutations all red against the WHOLE suite, green control
1863/5, golden byte-unchanged: C3(i) the declaration gate detached (2 red) . C3(ii) the
neighbour list empty (6) . C3(iii) built from files (1, the verdict arm alone) . C3(iv)
count CALLS instead of distinct documents (1, the repetition arm alone).
Three existing arms REWRITTEN, not weakened: all three read one document and declared,
which is the measured failure class exactly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four paid stress rounds ran entirely UN-ANCHORED, all of them, because the one file
loader reads cost-baseline.json out of the BUNDLE and no vegnormal ships one: N100,
N200, N500 and R761 are knowledge, and knowledge carries requirements, never amounts.
The validator's stage 0 -- the one stage that tells an invented cost line from a line
this project actually buys -- was skipped in every single run, so "validated" could not
mean what it says. P20 G1/G2 measured real R761 process numbers (12.11 three times on
Soraasen, 1.1.1 on Lindaas) validating with amounts nobody had anywhere.
--cost-baseline FILE is PM decision (e), taken over the three alternatives P20 wrote
down. A LOADED object, never a path (prepass_payload's rule): the CLI owns the file and
loads it ONCE, so the notice, the stamp and every base of an --across-bundle pass all
descend from one read. ONE parse, two doors -- load_cost_baseline delegates to
load_cost_baseline_file -- while safe_resolve stays on the bundle door alone, because a
project's own schedule is legitimately outside every base. No tolerant twin: this path
exists only because an operator NAMED a file.
DEL B: five anchored context sets, a1-a3 with their line and a4 with none, so stage 0 is
what catches the falsification arm. THE ORDER'S OWN ARM (h) WAS FELLED BY MEASUREMENT:
"no baseline code is a requirement number the base declares" is measured 0 of 4 on the
project-coded sets and 5 of 5 on kontrakt-sorasen -- which is what R761 Prosesskoden IS,
a bill of quantities priced BY process code. The complement keeps both, and the order's
own mutation still bites.
DEL B3: the judge reports anchored (off the run's own stamp), priced per row, and WHICH
falsifier caught the falsification arm.
Load-bearing MEASURED, five mutations all red against the WHOLE suite, green control
1850/5 (from 1809/5, superset, 0 removed), golden byte-unchanged:
A3(i) the flag is read but the baseline is unused (3 red) . A3(ii) only the first base
gets it (1) . A3(iii) report_forbidden drops it (1) . B2(i) a4 gets a line (1, arm (g)
alone) . B2(ii) a code swapped to 12.11 (2, arms (f) and (h)).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
