Replace the combined concept total, the distinct-token total and the
per-level document count of earlier example bases with general wording
in prose, comments and docstrings. No constant, assertion or test data
changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the exact document counts of earlier example bases (and the
per-base counts in the sources-format note) with general wording or
N-of-N in prose, comments and docstrings. Percentages and numerators
stay; no constant, assertion or test data changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Say "referansesti" for the reference-project path in the P7 note, and
replace the declared-number counts of earlier example material with the
shipped synthetic catalogue's own (300 prosessnr, 306 declared numbers)
or a general wording. Comments, docstrings and prose only; no constant
or assertion changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rename the reference-project helpers and test names (_reference_k,
test_reference_path_*) and the cost-baseline helper (_kontor_it_baseline,
which already returned KONTOR-IT-E1). Names only; no assertion changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One entry on the architecture presentation's not-to-build callout named
a code from the retired example material. It is removed; the other
items and the callout's count are unchanged.
The local term list now catches that code form too (it is not tracked,
so there is no red commit): measured red on exactly this line, 1 file of
the tracked set, then green after the edit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shared/ now carries portfolio-optimiser-commons 8aa19ec (squash merge in the
parent commits), and no file under it matches the local term list any more
(22 of 67 files before, 0 of 50 after). The shared/ exception and the test
that kept it alive are removed, so the repository-wide check reads every
tracked file. The surface-count pin follows the manifest: 1427 -> 1410.
CHANGELOG 1.3.0 gains one line for the sync; the version stays 1.3.0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
8aa19ec refactor(examples)!: remove two example bundles; 0.2.0 in CHANGELOG
6c4e863 test(scripts): term-list hygiene check, term list kept outside the tree
git-subtree-dir: shared
git-subtree-split: 8aa19ec774036d337f8668532d16e6fec26d65e5
Version sync in every file that carries it: pyproject.toml, __init__.py,
tests/test_smoke.py, README (status heading, its anchor, the wheel command),
uv.lock and a new CHANGELOG section. No tag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The context sets, the packaged knowledge bases and the example bundles are
replaced by one fictitious example set about IT operations in an invented
organisation: three context sets (serverrom-2027, driftsavtale-2027 and the
two-base drift-og-avtale-2027), two synthetic knowledge bases under
src/portfolio_optimiser/data/kunnskapsbaser and two example bundles under
src/portfolio_optimiser/data/bundles. Numbers, codes and structural values in
tests and fixtures are kept; names, ids and wording change. Dated measurement
documents that only recorded runs on the replaced material are deleted.
Gate figures measured on the new set are not comparable with earlier ones.
The exclusion gate from the previous commit is green: 0 tracked files hit
outside the shared/ subtree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reads every file git ls-files reports and fails on any line matching a
pattern from the gitignored tests/excluded-terms.local.md. The list itself
stays local; where it is absent the gate skips and says why.
Red on this tree: 185 tracked files hit (1 failed, 4 passed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The example config now has the placeholder `/path/to/okf` for `command`, and the README explains
how to find the real path (`command -v okf` outside this project's environment). It also explains
why the bare name fails under `uv run`: it resolves to this framework's own okf 0.8.5, which has
no `mcp` command. A new pin asserts that the example's command is an absolute path; it was red at
daccdbe. The xfail smoke test stays as the measurement behind the rule.
Chose this over lifting the okf dependency to 1.1: it fixes the startup failure without a version
bump and without touching --prepass-payload.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A new test builds a small invented collection with `okf build` and connects through
load_mcp_config -> build_mcp_tools -> async with. No model call, no network. It measures:
- the four tools are offered, and a narrower allowed_tools narrows them;
- okf_list names the collection;
- okf_ask returns the excerpt.
That holds only when the config gives the full path to okf 1.1. With the README's bare `okf`,
a run under `uv run` or an activated venv finds this project's own dependency
llm-ingestion-okf 0.8.5 first. That version has no `mcp` subcommand, and the server fails at
startup ("invalid choice: 'mcp'"). This is pinned as xfail(strict=True, raises=ToolException)
and not repaired, per the order.
The tests skip when no okf >= 1.1.0 is found on PATH outside the venv. The version is measured
from the server's own initialize answer. README limits now state what was measured. The
published-surface pin moves from 518 to 519 for the new file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README section with a before/after command and the limits stated plainly (link not yet tried in a
real run, agent adherence not measured, --prepass-payload rejects okf-consumption/2). Adds
examples/okf-server.mcp.json, pinned by a test that loads it through load_mcp_config; the
published-surface pin moves 516 -> 518 for the two new files.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Session 146 fixed `--run-id` across the seven doors and called it a class. It was not: the class
is every caller-supplied value that reaches a file name, and `write_outbox` composes
`{run_id}-{approach_id}`. Measured 20.09 -- `--approach-id a/../../../ESCAPE` answered 0 and put
the artefacts three levels above the directory the caller named. Counted rather than assumed: 11
path compositions in `outbox.py`, 2 such values, both now through one `_checked_name`.
The ledger sentence that said containment was UNREACHABLE after the string rule was untrue, and
the approach-id escape is the disproof -- the removed check would have caught it. It is back, but
in `outbox._artefact_path`, where the composition is, not in the door. That is the difference
that makes it reachable: the string rule lives in the door, while `run.py` hands its own
`--run-id` straight to the writers and goes past it. Checked before the directory is created, so
a refusal leaves nothing behind, and it covers the next flag someone interpolates into a name.
The judge's exact call now answers 3 with 0 files outside. Suite 2291/0/5/5 (746 s), ruff clean,
mypy 0. Both gates re-run after `git add`: v1 exit 1 (0/3, 0/3, 3/8, no report, 3/8, NOT
MEASURED, 1/20), B exit 1 (15/17, 0/2, 15/15 over 516 files, 0/3, 4/5, NOT MEASURED) -- no row
moved, and row 3's denominator held because the probe grew in place rather than as a new file.
Also: the presentation deck said 1.1.0 was the current version in two places. 1.2.0 now stands in
every tracked place that claims the repo's version. No bump, no tag, no new capability.
[skip-docs]
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`--run-id` was guarded as a class; `--approach-id` was not. `write_outbox` composes
`{run_id}-{approach_id}`, so the same escape ran through the unguarded half: measured 20.09,
`--approach-id a/../../../ESCAPE` answered 0 and put the artefacts three levels above the
directory the caller named.
Counted rather than assumed: 11 path compositions in `outbox.py`, two caller-supplied values in
them. Four forms x the one door approach-id reaches, each with the intermediate directory
created first -- without it two of the four were green at HEAD on a missing directory, which is
a reason that is not the guard.
The containment check is asserted where it IS reachable: the writers themselves, called
directly. That is the run path's own way in -- `run.py` hands its `--run-id` straight to
`outbox.write_*`, past the string rule that lives in the door -- so the arms are not vacuous.
10 arms red on ASSERT about behaviour: 4 approach-id (exit 0), 6 writer-level (DID NOT RAISE).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pin is doing its job: one new tracked file, and the arm that counts the git manifest
independently went red rather than letting the denominator drift. Row 3's fifteen guards
still measure zero hits, now over 516 files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two rows and the matching CHANGELOG entries. The first records the measurement rather than
the decision: `--outbox-dir <d>/inni --run-id ../../ESCAPE` answered 0 and wrote two levels
above the directory the caller named, on a door whose directory argument was already
guarded. It also records the check that did NOT survive -- the containment half was
unreachable after the string rule and is written down as dead code removed, not as a second
layer of defence.
The second row is the smaller and older failure: a rule called load-bearing in prose, with
no arm that would notice if it were deleted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GREEN. `_checked_run_id` rejects a run id that carries a path -- a separator (`/`, `\`,
`os.sep`, `os.altsep`), a bare `.` or `..`, an absolute prefix, an embedded NUL, or the
empty string -- before any door touches the filesystem. A refusal, exit 3 with the reason
named, and no directory left behind: the call parsed, so 2 would be the wrong code.
Applied as a class. The composition is identical in all seven writers, so fixing the door
the escape was measured on would have left six the same shape.
One check, not two, and that is a measurement. The first draft also required the composed
path to resolve inside the resolved output directory; mutating that check away left all 36
arms green, because after the string rule there is no composition that can leave the
directory. An unreachable check is not defence in depth, it is dead code that reads like
defence, so it is gone -- and the property it claimed is asserted where it IS reachable,
in the probe's accepting arm.
Mutants, each run against the door probes with the tree restored from scratch in between:
guard dropped from one door -> 4 arms fall; whole guard disabled -> all 28 escape arms
fall; `--stop-reason` required -> default="" -> the new stop-reason arm falls, where it
previously survived the entire suite. The one that survived (the containment half) became
the finding above rather than a green tick.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED. Seven outbox doors compose `<out_dir>/<run_id>-<artefact>.json` and never look at the
run id, so `--outbox-dir <d>/inni --run-id ../../ESCAPE` writes two levels ABOVE the
directory the caller named and answers 0. The guard stood on one half of the path and was
missing on the other. Reproduced on write-prepass before this file existed.
The class is measured, not one case: all seven doors against four path-carrying forms --
relative-up, separator, bare `..`, absolute. Each arm asserts a non-zero exit AND that the
file tree under tmp_path is byte-unchanged afterwards; the first alone would be satisfied
by a door that refused after writing. An rc-0 control per door runs the same argv with a
clean run id, so the refusals measure the run id rather than a malformed call.
The file also carries the guard the `--stop-reason` rule never had. `required=True` is
called load-bearing in the ledger -- "the run finished" and "we never found out" must not
be the same value -- but swapping it for `default=""` passed the whole suite. The arm
fells that mutant: no flag is a usage error, and the empty string stays a value the caller
can supply on purpose.
Measured at this commit: 28 failed, 8 passed (the seven controls and the stop-reason arm).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The version moves 1.1.0 -> 1.2.0 in every place it is written: pyproject, the package
__init__, the smoke test that pins it, the README's wheel-install command (gate-pinned
against the build) and uv.lock -- five sites, plus the new CHANGELOG section. A new,
backward-compatible surface, no removals: fourteen run-path steps that had only ever been
reachable through a path that builds a chat client now each have a command of their own.
The README gains a status block at the top, and half of it is what v1 is NOT: no domain
expert has validated an output, three planned rounds of expert feedback were set aside
rather than passed, five of the eight kinds of feedback the design names have no typed way
back into a run, and the multi-agent debate is exercised in the suite only against scripted
clients. That half is the point -- a reader who installs this should learn the limits from
the first screen, not from a gate they have to run themselves.
Measured on this tree at this commit: suite 2238 passed / 0 failed / 5 skipped / 5 xfailed
(769 s), ruff check and mypy src clean, and both gates re-run after staging and unchanged
(v1 gate exit 1: 0/3 - 0/3 - 3/8 - no report - 3/8 - NOT MEASURED - 1/20; B gate exit 1:
15/17 - 0/2 - 15/15 - 0/3 - 4/5 - NOT MEASURED).
No tag is created here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The row the seam needs: why the outcome is derived rather than declared (the writer
branches on the TYPE, so a door taking it as an argument would let anyone author an
outbox of claims), why the outbox directory is always the caller's to name, why the
shape guard is a 3 and not a 2, and where the ground truth for each probe comes from.
It also writes down what did NOT move and why, so the next session does not rediscover
it: `rundebinding` and `rapport` share `round_builder`'s `main` with nothing in argv to
tell them apart, and `build_report` is a sub-step of `build_round` — a `report`
subcommand would be new capability, not a door over an existing step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`_EXTERNAL_TODAY` and `_WITH_A_DOOR_TODAY` are the arm's own count, not the gate's, so
the seven new doors have to be written down here before the suite agrees they exist. The
two assertions that pinned `kjørekonfig`/`utboks`/`prepass-artefakt`/`parse-feil` as
door-LESS are inverted rather than deleted: they were the arm's discriminator, and a
door that silently went away must still turn this red.
The manifest count moves 514 -> 515 for the one new probe file.
Also a probe for the shape guard: the writers take `Mapping`/`Sequence` and would
serialize whatever they were handed, so a list where an object belongs would become a
valid file with the wrong shape and the run that reads it later would be the one to
fail. The door refuses it as a 3 (the call parsed, so 2 is the wrong code) and leaves no
directory behind. Carries its rc-0 control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every one of the run path's seven `outbox.write_*` steps was reachable only through
`run.main`, and every path through that builds a chat client. The steps need no model:
they take already-rendered data and put it on disk in a byte-deterministic form.
Seven subcommands, seven thin adapters. The outbox directory is always the caller's to
name -- never a default, never the repository's own, because a step that wrote into a
folder the framework also reads as an inbox would bypass the Step-8 promotion gate.
`write-outbox` is the one that is not purely mechanical: `outbox.write_outbox` branches
on the outcome TYPE, so a door that took the outcome as an argument would let anyone
author an outbox of claims and hand it to Step 8 as results. The door DERIVES it through
`validate_proposal` -- the run path's own composition -- and a blocked proposal exits 3
with the artefacts still written, since that is where the rejection is recorded.
`verdict_id` stays an argument: `verdict-key` already owns that minting.
`--stop-reason` is required rather than defaulted to the empty string, inheriting the
core writer's measured reason: "the run finished" and "we never found out" must not be
the same value.
Eight probes, each a subprocess with the subcommand in argv, each asserting on the FILE
the command wrote. The ground truth is composed in the test -- the payload it wrote and
counted itself, and the byte form the contract requires -- never `outbox._dump`, which
would have measured the module against itself. The refusal arm carries its rc-0 control.
Measured, own run of the B gate: row 1 8 of 17 -> 15 of 17, exit 1 unchanged, no other
row moved. 0 chat-client names reachable from the toolbox (known-positive control: 24 in
run.py).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The ledger row for toolbox step 3: what was built, the row-1 movement (5 -> 8 of 17), the
int/float measurement that decided how the door reads a proposal, the exit-3 carrier for a
blocked proposal, the fasit each probe uses outside the door, and the stage the door does NOT
expose (P7 input grounding) with the reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The entries for validering / verdict-id / dom were console-script pointers at run.py::main, which
the gate itself refused with a named reason: the way from that main() requires a chat client.
They now point at the toolbox subcommand that reaches each symbol, with the probe that drives it.
Measured, same denominator both ways, no other row moved:
row 1 5 av 17 -> 8 av 17 (RED both ways)
row 2 0/2 · row 3 15/15 GREEN over 514 published files · row 4 0/3 · row 5 4/5 · row 6 NOT
MEASURED -- unchanged, exit 1 unchanged.
_EXTERNAL_TODAY follows the measurement, and _WITH_A_DOOR_TODAY names WHICH eight, so a door that
moves without being built shows up as a red test rather than as a different number. The arm that
asserted "validering has no door" had to pick a step that still does not: kjorekonfig.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
B's premise applied to the three steps that DECIDE a proposal: a proposal authored outside po --
by a human, or by an agent that is not po -- now meets the blocking deterministic gate, mints the
learning key, and is captured as a Verdict, all without a chat client on the way.
Three thin adapters, no second implementation. The reason is the one the first four doors were
built on, but it bites harder here: the refusal SENTENCE is fed back verbatim into the next
attempt by step 5, so a door that reworded it would break the repair loop while still looking
correct. The probes assert the sentence, not a substring two stages share.
One measurement decided a design detail. The IR writes whole magnitudes as JSON integers
(30000), the run path carries the pydantic float, and verdicts._mint_id hashes the raw value --
so minting from the undeclared JSON would hand out a DIFFERENT verdict id than the debate does
for the same proposal. The door therefore reads the proposal through SavingsProposal and feeds
model_dump() to the public features_from_ir; the probe pins both forms and asserts they differ,
so the shortcut cannot come back silently.
A blocked proposal exits 3, carrying the verdict rather than an exception envelope. "You asked
right and the answer is no" is the same fact whether a file was missing or a claim was
infeasible, and a caller that only reads the exit code must not see a blocked proposal as a
cleared one.
Fasit outside the door in every arm: the base's own checked-in golden suite (written before the
toolbox existed, so it cannot have been fitted to it), a cost baseline authored in the test, a
method cap computed by hand from the fixture, and the public minting rule. Each refusal arm has
an rc-0 control on an argv that would otherwise be accepted.
STATED LIMIT: the input-grounding stage (P7, stage 0b) has no flag here. It falsifies a proposal
against the rendered prompt the model received, and an outside caller has no such prompt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The constant-sync gate is fail-closed on a name it cannot find, and it searches the package's top
level; `RUNBOOK_MIN_BODY` lives in `evals/b_gate.py`. Citing it as `NAME = value` therefore read as
a claim about a constant that does not exist. Same shape as the note at the top of this file about
the finding-99 row, and the same repair: the name in the span, the value in prose.
Caught by the suite, not by reading.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One row for the whole delivery: why `portfolio-optimiser-toolbox` is the third console script (it
is the one thing the other two cannot be used for), what each subcommand is bound to, why every
handler is a thin adapter and every dispatch an explicit branch, and how the probes assert on what
the door wrote rather than on the function it calls.
Plus the third repair of B-gate's binding with its measurements: the judge's seven forms at 0 of 1,
the naming rule turned into a discriminator, named-prose reasons, row 3 at 15 of 15 over 514 files,
and the runbook heading that is no longer a section. The row states its own limit (the gate reads
that the probe drives the door; it does not re-run the probe with the door broken) and records that
N7 stays open in the gate and closed in the suite.
The mutation run is in the row because two of its findings could not have come from reading: the
dead-code pruning survived two mutants until the arms that reach it were written, and three
mutants are named as non-measurements (two renamed a label, one was equivalent).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The measured escape: a runbook that was the five contract-named headings plus "x" (125 characters
in all), with a correctly checksummed attestation, read `2 av 2 GRØNN`. Only a file whose whole
content was "x" had been refused -- the rule asked whether the section NAME appeared in the text,
never whether anything stood under it. A table of contents is not a runbook anyone can follow.
Section names are now bound to a HEADING line, and what counts is the body beneath it
(`section_bodies`, 80 non-whitespace characters as a floor). The row says so itself, and says what
the floor is not: a length is never a measure of whether the runbook is true. That stays the
operator's, which is why row 6 is still `IKKE MÅLT`.
Two rc-0 controls in the suite carried section bodies of "noe" and "steg 1: naviger pakken" -- both
would now fail on the new rule rather than on the rule they were written for, so both got a real
body. A control that falls on the wrong rule has stopped controlling.
Also: the one very long line in the ledger (the arm name mid-paragraph) rewrapped. Cosmetic, named
in the 2026-09-20 checkpoint's leftovers.
NOT taken from that same list: freezing the outbox shape so the round-builder's denominator arm
stops skipping outside a tree with `scratchpad/`. It needs a checked-in fixture of the measured
form, which is not the "only if cheap" the order allowed, and the checkpoint proposed no round for
it either.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
M1 (`if False:` counted as live) and M2 (code after `return` counted as live) both SURVIVED the
whole suite. The pruning was real and nothing reached it: every dishonest form the arms already
carried fell on the data-dependence rule first, so removing the pruning changed no verdict.
The escape the pruning actually closes needs a probe that is GREEN in pytest: run a real process
that is NOT the door, bind the name the assert reads, and put the door call in dead code under
the same name. The taint then arrives from a line that can never run, and the probe "proves" a
step it never touched. Both forms are planted here, each with the honest rc-0 control beside it.
[mutation-ok] -- no guard is switched off. The `if False:` in this diff is INSIDE a Python string
that the arm writes to a tmp_path file; the gate only PARSES that file, never runs it, and the
arm asserts the form gives 0 of 1. The commit is the arm that makes dead-code pruning fall when
it is removed.
Found by mutating, not by reading. A rule no mutant can kill is a rule the suite has not measured.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`_PUBLISHED_TODAY` is the row-3 denominator, pinned in the suite against `git ls-files`. It read
512 and the arm was green in the worktree only because `toolbox.py` and `test_toolbox_doors.py`
were still untracked -- the clone measured 514 the moment they were committed. The pin is what it
is for: a surface that grows without anyone noticing is a surface the guard has not measured.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
B-gate row 1's premise, made callable. Every path through the framework CLI constructs a chat
client, so an outside caller -- a human at a terminal, or an agent that is NOT po -- could not
reach a single run-path step without paying for a model. These four steps need no model at all.
One CLI, four subcommands, one core call each:
navigate-bundle --bundle-dir -> okf.navigate_bundle
cost-baseline --bundle-dir --project-id -> okf.derive_cost_baseline
retrieve-chunks --query --docs-dir [--top-k] -> datasource.retrieve_chunks
prepass-admit --payload --bundle-dir [--dimension] -> prepass.admit_payload
Each handler is a thin adapter: strings in, the SAME function the run path calls, JSON on stdout,
and an exit code that says what happened (0 ran, 2 malformed call, 3 the step refused, named).
A handler that computed anything of its own would be a second implementation of a run-path step,
and the outside caller would stop getting what the debate gets.
Dispatch is an explicit branch per command, not argparse's `set_defaults(handler=...)`: the table
hides the one thing a reader wants to see, and B-gate row 1 asks the same question of the source
(it walks the call graph from `main` down to the step's symbol), where a callable in a Namespace
is a hop neither can follow.
Probes (`tests/test_toolbox_doors.py`, 10 arms): each starts the door as a SUBPROCESS with the
subcommand in argv and asserts on what it wrote -- never by importing the core function, which is
the whole difference the gate exists to measure. The yardstick is outside the door in every arm:
the filesystem (navigate-bundle, including the one deliberate outside-bundle link), a table
transcribed from the priced fixture (cost-baseline), the in-process seam it must equal byte for
byte (retrieve-chunks), and the producer's own checked-in payload (prepass-admit). Every refusal
arm has an rc-0 control beside it.
`portfolio-optimiser-toolbox` is the THIRD console script, and the pin test now says why: it is
the door the other two cannot be used for. README and CLAUDE.md updated with the command and the
reason it exists; every documented invocation was run.
Row 1: 1 -> 5 of 17 (four subcommands + `gate`, which the class fix in e47be68 stopped rejecting
on a name technicality). No other row moved; exit 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Third repair of the same failure, and the first that does not read a NAME. The judge measured
row 1 to 17 of 17 with 17 one-line probes and a door module without a single import: "the probe
touches the door" was still a text check, one notch further in.
What is measured now, in the probe's own source: an EXECUTION SITE (a process starter carrying
the door's name in its arguments, or the entry imported from the door's module and called), and
an assert that is DATA-DEPENDENT on that call. Dead code is pruned first -- a call in a branch
that never runs, or after a return, is no execution. The honest forms the suite already uses
keep counting: the command built in a variable first, and the subprocess started in a helper
that returns it.
All seven forms the judge measured (a1, a2, b1, b2, c1, c2 + the one-liner recipe) now give
0 of 1, each with an rc-0 control in the same setup.
[mutation-ok] -- no guard is switched off. The two `if False:` lines in this diff are the
judge's b1 probe form planted as test DATA (a Python string written into a tmp_path file the
gate only PARSES, never runs) and the docstring naming why dead code is pruned. The commit adds
the rule that makes `if False:` around a door call stop counting.
Naming the step becomes a DISCRIMINATOR, as a class: it is required only when more than one step
stands behind the same door. Driving a door with a sole step IS driving that step -- which is why
`gate` (a real end-to-end door probe against v1_gate) was rejected on a name technicality. Row 1:
0 -> 1 of 17. For a subcommand door the command name must stand in what was actually executed,
and the CLI itself must be registered or `-m`-runnable: an add_parser in a module nobody can
start is no door.
held_out: a reason must be named prose. "-", "todo", "x", "." were all accepted as a reason by
`.strip()`; measured against the contract's own 28 reasons (shortest: 30 chars, three words).
Row 3: the five forms the checkpoint measured as open AND cheap to close are closed -- importlib
for both SDK names, deno/npm exec/yarn dlx, and the official TypeScript SDK. 12 of 12 -> 15 of 15,
still 0 hits over 512 published files. The limit sentence stops enumerating what remains ("EXACTLY
TWO", then "THREE" -- both falsified by the first new measurement) and says what the guard IS.
N7 stays open in the gate and is closed in the suite: a deleted off-path step still shrinks the
denominator silently there, so the four are written out in an arm that goes red when one goes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured, not asserted: PM's P14 reproduced in BOTH directions (survived 43 of 43 pre-round arms,
felled by three arms after), 7 of 7 own mutants felled on AssertionError about behaviour with 0
IndexError, and the class the order named -- "an assert built from the binder's own table" --
counted at 4 of 12 rb.<table> assert sites and closed, with the independent side named for the
other eight.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Third writing of the same sentence, third time untrue. Measured in this working tree, by the arm
committed red before this one: FOUR of the five types outside the measured seven do not sit in an
outbox that also holds a coverage -- and for `plan-review` that says nothing at all, because the
type has 0 artefacts in the repo. `multibase` DOES sit there, in four outboxes
(`p17b-multibase/`, `p20-stress/`, `p21-stress/`, `p22-stress/`, all `lindaas`), and each of those
four holds TWO coverage files. That is why they fall outside the "exactly one coverage" rule the
union of seven is counted over -- and the same fact is the proof that seven is a FLOOR and not a
ceiling, which is the argument the false sentence was trying to make.
The comment above `_RUN_LEVEL_TYPES` carried the same claim in English and is corrected with it.
The arm now counts the outboxes and holds the ledger's row to the count, so a fourth writing is a
failing test rather than a review finding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED FIRST, and only one of these can be red in the ordinary way: the ledger's own sentence about
the five artefact types outside the measured seven is UNTRUE, so the new arm that counts the
outboxes fails on ASSERT (1 of 44 arms red, 0 on ImportError/AttributeError/collection). The
other changes fix ARMS, not code -- src/ is untouched -- so their discrimination is shown by the
mutant campaign in the next commit's message: surviving before, felled after.
- `_STAGE_SENTENCES`: stage -> (the clause the refusal heading opens with, a word only that
stage's explanation carries), written out here the way `_STATUS_WORDS` is. The arm that names
where an approach fell built its expected heading from `rb.STAGE_PROSE` and then agreed with
it: PM's mutant P14 (the sentences for `stage4-p90` and `stage0b-grounding` swapped) passed all
43 arms, and the report then said an approach fell on the grounding when it fell on the
uncertainty calculation. Both halves are compared against this file's table, on the ONE heading
line located by prefix, and the whole table is covered -- not only the two stages the fixture
exercises, because a fixture is not a denominator.
- That each mark belongs to exactly ONE stage is COUNTED in the test, so swapping explanations
alone is a failure too.
- `_section(report, heading)`: every `split("## ...")[1]` in the file goes through it. Renaming a
heading failed four arms on 19.09, three of them on IndexError -- a traceback that says a list
was too short, not that the report lost the section the expert reads first.
- New arm: `multibase` DOES lie in outboxes that also hold coverage (four of them, two coverage
files each). The arm counts the outboxes and requires the ledger's sentence to say what the
count says; the false clause is pinned verbatim so its fourth writing is a failing test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Row updated, not appended to: the B-gate invariant now carries both repairs of
19.09 and the numbers that moved in each. Two precision errors the checkpoint
named are corrected in place -- "no row became greener" is true of colour and not
of numbers (row 5 went 1 of 2 -> 4 of 5 when the unit was split), and ENTRY_KINDS
has three arts, not four.
What the new paragraph records: the probe binding and the two traps measured while
building it (substring, and a local module alias), the price in row 1 (3 of 17 ->
0 of 17, each of the three named with its reason), the limit statement that claimed
two remaining ways where six were measured -- four closed, three named and left
open with their reason -- the held_out door out of the denominator, the five small
holes closed and the two limits stated rather than closed, and the thirteenth
mutant: it survived twice, and the first arm written to fell it proved something
already true.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First attempt at M12 still passed 86 of 86 -- the arm planted a docstring that
MENTIONED the module in prose, and the gate compares whole strings, so prose never
reached the check either way. The arm proved something already true and left the
mutant alive.
The cheapest cheat is the bare name: a string that IS the dotted module, standing
alone. That is what the arm plants now, and the same for the step's symbol.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mutant M12 against the delivered tree: let _meaningful return the whole body, so a
string standing alone counts. 85 of 85 GREEN. That is a finding, not a pass: the
repair's own rule -- "the gate never reads a string that stands alone" -- had no arm
watching it, and without one the fix would have drifted back to "mentioning the door
is touching it", which is the same defect one floor in.
The arm carries its rc-0 control first (a real probe WITH a docstring still counts),
then breaks each half on its own: a probe whose docstring names the real door while
its call drives another module, and one whose docstring names the step while its call
passes another word. Both must be refused, each with its own reason.
12 of 12 mutants now felled, control 86 of 86.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Row 1 goes 3 of 17 -> 0 of 17. Nothing was removed from the product and no row
changed colour; the three that counted stopped counting because the contract they
satisfied was satisfiable without capability.
BEARING 1 -- the probe is bound to the step, and the binding is MEASURED in the
probe's own source. Chosen: read the probe (ast) rather than demand it live in a
contract-named file, because a file name is a convention a stub meets as easily as a
real probe. Three traits, each measured: it drives the DOOR (the dotted module or the
registered command name appears as a string it uses -- anywhere but a docstring,
because the honest form assembles argv in a variable first), it names the STEP (the
symbol, id or subcommand as a whole WORD in what it passes INTO a call or calls), and
it asserts at all. A probe claimed by two steps proves at most one and the gate cannot
tell which -- so neither.
Two traps found while measuring, both closed:
- substring: "gate" is not named by portfolio_optimiser.evals.v1_gate
- local alias: the first cut accepted step `gate` because the probe file imports the
module AS `gate`. Names are therefore read only where they are sent or called.
What that costs, measured against the contract that stands:
rundebinding drives the door, names no step (was green)
rapport never goes through the door at all (was green)
gate drives the door, names no step (was green)
BEARING 2 -- the limit statement said exactly TWO ways remained; the checkpoint
planted 21 call forms and measured SIX. Four are closed with a guard each (the
official Python SDK in both spellings, the node and uv runners, a dynamic import);
three remain and are now named: a runtime-composed name, a name from an environment
variable, a base64-decoded name. Left open deliberately -- the encodings are not
enumerable and our own contract stores base64 by design. Row 3: 9 of 9 -> 12 of 12,
still GREEN, 0 hits over 512 files. One of the three caught a command written in this
round's own test docstring; it was rewritten, not exempted.
BEARING 3 -- held_out accepted an EMPTY reason and shrank the denominator, while the
summary said "held out with a reason" either way. A blank reason is no reason: the
symbol stays in the denominator as a call without a door, the summary counts reasons,
and the four steps declared OUTSIDE the run path are now named one by one as having no
derived source instead of being counted in silence.
Five small rests, closed: a pruned manifest (451 of 512 was still GREEN) is now NOT
MEASURED, one sentinel per area the old handlist missed; a non-UTF-8 file is read as
byte text instead of counted and skipped; a symlink out of the tree is named and
fails the row; a runbook whose whole content is "x" no longer passes, the contract
names its sections; and the row states that its ratio is not a coverage measure.
Two stated, not closed, each with its reason in the row's own attestation: a po call
moved one floor down into a helper leaves the denominator (following helpers would
pull private ones in and make the denominator the curated list this row exists to
avoid), and row 3's k/n can still be padded by a guard with no measured escape behind
it. The ledger's two precision errors are corrected: "no row became greener" is true
of colour, not of numbers, and ENTRY_KINDS has three arts, not four.
Suite: 2172 passed, 5 skipped, 5 xfailed in 645 s. ruff and mypy clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED FIRST, second repair. PM's checkpoint on c1bdb37 measured N2b: row 1 goes from
3 of 17 to 17 of 17 GREEN with fourteen stub doors and ONE unrelated green test used
as the behaviour probe for all of them. The gate only looked up whether a nodeid was
`passed`; nothing bound the probe to its step. EXTERNAL_DOOR promised the probe
"calls the door and reads the artefact it writes" and there was no code for it --
the same class of defect as entry["kind"] one round ago.
20 new arms, 19 RED, every one on an assert about behaviour (43 assert lines, 0
ImportError, 0 AttributeError, 0 collection errors). The twentieth is green by
construction: every held_out entry in the contract does carry a reason today, and
the arm exists so that stops being true loudly.
Each refusal arm carries its own rc-0 control FIRST -- the same probe, the same
surface, accepted -- then breaks exactly one trait. The constructed probe is now
WRITTEN (`_write_probe`), not merely named: the repaired gate reads the probe's own
source, so a nodeid that does not exist cannot be a behaviour.
Red on the three bearing rests:
probe not bound to its step door untouched, step unnamed, nothing asserted,
test missing, one probe shared by two steps, and
N2b against the contract that stands (17 -> k=0)
limit statement untrue the six escapes PM measured as unseen, planted one
at a time, and the "TWO ways remain" sentence
held_out accepts empty a blank reason shrinks the denominator; the summary
counts entries, not reasons; the four steps outside
the run path are counted without being named
Red on five of the nine small rests: pruned manifest (451 of 512 still GREEN), a
non-UTF-8 file counted but never scanned, a symlink out of the tree, a runbook whose
whole content is "x", and the row's ratio read as a coverage measure.
One test-hygiene change, not a product fix: the git-dependent arm now skips itself
without .git instead of failing, so "red in a clean extract" stops growing with each
new arm of that class.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The round binder's row gets its two corrected denominators, and the first screen gets a row of
its own: what survived, why none of it was a code defect, and why that makes "red first"
impossible to satisfy with a commit rather than with a mutant run.
The 7-type sentence said "en ekte kjøring" and measured four archived runs. Re-counted: `run.py`
calls TEN of outbox.py's ten writers, so five more types exist; and the union over every outbox
in this repo with exactly one coverage (15 directories, of 25 coverage files in 20 directories)
is also 7. Seven is the floor the measurement gives, not a ceiling — the binder copies on a glob,
not on that list.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RED FIRST, HONESTLY: 0 of the 3 new arms can be red at HEAD, because neither finding is a code
defect. The binder already printed every one of these things correctly; what was missing was an
arm that looked. There is no fix to be red before. The proof that the arms discriminate is the
mutant run, not this commit — and that run is reported here in full.
REPRODUCED AS SURVIVORS FIRST, against the 40 arms as they stood: the checkpoint's N3
(`_status_word`, "rejected" -> "validert") and N2 (the heading's amount truncated to whole
kroner) both passed 40 of 40. Eight more of my own against the same first screen: the verdict
word dropped entirely, the label paired with the NEXT row's word, `not_evaluated` read as
"avvist", `unsupported` read as "validert", the commissioned count off by one, and the heading's
"kroner" changed to "kr" — 8 of 10 survived. Two were caught (the sum off by one øre, the
overview sorted).
AFTER: 12 of 12 felled, every one on an AssertionError about behaviour, 0 on
ImportError/AttributeError/ModuleNotFoundError. Control unmutated: 43 of 43. Mutants lived only
in a scratch clone of `git archive HEAD`; the work tree was never mutated.
THE CAUSE WAS MEASURED, NOT GUESSED. `"60 000,01" in text` also matched the "Berørte
kostnadslinjer" line, which is built from `unit_cost` and which the mutant never touched. Counted
over the fixture's own report, 4 of this file's 11 positive substring-assert sites have their
needle on MORE THAN ONE line, and 3 of those 4 are asserts about one particular line. Those three
now compare the WHOLE line: `- **{label}** — {word}`, `### {label} — {amount} kroner`,
`### {label} — falt på {stage}`. The fourth claims only presence, and presence is what it
measures, so it still says `in`. The docstring's "cannot pass by accident" now says which half it
covers.
The word table is written out HERE rather than imported from `_status_word`: an arm that asks the
binder which word it chose and then agrees with it cannot tell "avvist" from "validert". It is
pinned to the MANDATE's own status list (`ApproachOutcome.status`, four values) the way the stage
vocabulary is pinned to the validator's — `_status_word` falls back on `str(row["status"])`, so a
status added there without a word here reaches the expert as a bare English identifier.
"validert" is a PREFIX of "validert, men uten erklært krav", which is exactly why no substring
assert can separate those two.
Also, two prose claims that promised more than the measurement:
- "Every artefact TYPE a real run leaves in an outbox" -> the types the runs MEASURED left.
`run.py` calls TEN of outbox.py's ten writers; `exploration`, `prepass`, `multibase`,
`plan-review` and `proposal-reviews` are five more types. Re-counted over every outbox in this
repo holding exactly one coverage — 15 directories, out of 25 coverage files in 20 directories
— the union is still 7, so 7 is a measured floor and not a ceiling. The arm is renamed to say
so.
- "Four of the seven are RUN-level" -> FIVE. `_RUN_LEVEL_TYPES` has five members.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The row claimed three things the checkpoint measured as false: that every denominator comes from
the source (row 1's M=13 was a curated list in the gate's own b_gate.json), that the surface is
what is published (it was a hand list of eleven roots seeing 433 of 512 files), and that the gate
counts an MCP-registered door as callable (there was no code for it — "kind" appeared 0 times).
The 435-file figure was the working tree, two gitignored .local.md files included, so it was not
reproducible from the commit either.
Rewritten to what is true after the repair, with the old numbers kept as history so the row reads
as a correction and not as a clean slate: 3 of 17 · 0 of 2 · 9 of 9 · 0 of 3 · 4 of 5 · IKKE MAALT.
The markdown-fence rule's own measurement is refreshed too: on the bigger surface it now carries
TWO prose lines, not one — the second is a plan document quoting a command the operator ran in his
own session.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three of the new row-3 arms pinned 512 files and the literal source label, which only a tree WITH
git metadata can answer; a clean extract (git archive | tar -x) would have turned all three red for
the same reason the three already-named extraction artefacts go red. The property they test holds
in both worlds, so they now assert it there: main.py, examples/, spikes/ and contexts/ are inside
the guard whichever manifest was read, and the row prints whatever denominator it measured.
The exact number and the label belong to ONE arm — the one whose subject IS the git manifest. In an
extract it is an extraction artefact of the same class as the others, and says so in its docstring.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Row 1's M was a curated list of 13 in the gate's OWN b_gate.json. It is now DERIVED: run_path_calls
reads what run.py::run_project actually calls (41 po-functions, re-counted here against PM's 39 —
the two that differ are generate_via_llm and fresh_workflow, held out because they need a chat
client). A call that is neither a declared step nor named-with-a-reason counts in the denominator
WITHOUT a door, so the number cannot shrink in silence: removing a step from the contract now
leaves N unchanged and names the orphan. The four outbox writers the run path uses and nobody had
declared (write_prepass, write_parse_failures, write_proposal_reviews, write_debate_tools) are
steps now; 28 calls are held out, each with its reason printed under the row.
A door must be REGISTERED and PROVEN. entry["kind"] is read (it was read 0 times before) and must
be one of three kinds the gate has code to verify: console-script in pyproject, module-main with
its own __main__ guard, subcommand registered in the module's argparse. On top of that every step
needs a named probe that calls the door and reads the artefact. The MCP sentence is struck from
EXTERNAL_DOOR: it claimed a capability with no code behind it.
Row 3 now scans the repo manifest (git ls-files, or the tree itself in an extract), not a hand
list of 11 roots: 512 published files instead of 433, so main.py, examples/, spikes/, contexts/,
CLAUDE.md and llms.txt are inside the guard for the first time. Three new patterns catch the
indirect invocations that walked past the old six — absolute path, list in a variable, constant,
shell string — 5 of 5 of PM's variants are refused now, with 0 false positives measured over the
whole surface. An empty surface is IKKE MAALT, not GREEN: the row demands a sentinel file and
prints the file count and the manifest it read.
Rows 4, 5 and 6 get sources for their denominators. Row 4 counts only checks whose named source
symbol resolves in the code. Row 5 counts each structural requirement on its own (2 profile
members + factory + seam + probe = 5) instead of collapsing three into one unit. Row 6's N comes
from the artefacts the contract names, and the attestation is VALIDATED: it must name the
contract's runbook, carry its sha256, say who ran it, and bear a real ISO date that is not in the
future — v1_gate's own date rule, reused, BOM tolerated as there.
Measured in a scratch clone (/tmp/claude-po/bgate-mut): 12 of 12 mutants felled, control 65 of 65.
All six of PM's broken attacks reproduced as refused, with the rc-0 control green.
No row got greener: 3 of 17 (was 3 of 13), 0 of 2, GREEN, 0 of 3, 4 of 5 (was 1 of 2), IKKE MAALT.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The PM checkpoint on 207337c judged the gate DELVIS: row 1's M=13 is a curated list in the
gate's OWN b_gate.json (the run path has 41 po-calls, 7 of 10 outbox writers), rows 4, 5 and 6
have denominators with no source at all, 6 of 10 cheat-attacks got through, and the
never-Claude guard sees 433 of 512 published files.
This commit is the red half. Every arm fails on an ASSERT about behaviour, never at collection:
the four names that do not exist yet (ENTRY_KINDS, run_path_calls, registered_entry,
published_files) are stubbed here with DELIBERATELY wrong values — everything is a door, the
run path calls nothing, the surface is empty — so each arm measures the defect rather than the
absence of a symbol.
30 av 31 red on assert. The one that is green is the rc-0 control
(test_a_valid_attestation_is_the_only_thing_that_turns_row6_green): a valid attestation must
turn row 6 green both before and after, or the row refuses everything, which proves as little
as refusing nothing. All 34 pre-existing arms stay green — measured, not assumed.
The planted claude-invocations are base64 in the test file for the same reason the contract's
patterns are: tests/ is itself part of the surface row 3 scans, and a cleartext variant here
would register as its own finding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The round builder's row claimed "et sitat bærer ANTALLET siterte steder. Load-bearing
MÅLT" for a half that was not measured: the mutant dropping the count survived the whole
suite. The claim is true from today, and the row now says from WHEN — a ledger that
back-dates a measurement is worth less than one that admits the gap.
The new row records what the checkpoint found and what it cost to close: seven mutants
survived, none of them because the code was wrong, all of them because no arm looked.
Three were unreachable rather than merely unmeasured — the fixture wrote three of seven
artefact types, and every proposal carried the same citation stamp, so "1 av 1 siterte
steder" could not tell a dropped count from a kept one.
The denominator is written with its method, not as a number to be remembered: over the
four archived runs, -06/-07/-08 hold all seven types and -04 holds six (parse-failures is
written only when something failed to parse, so its absence is the signal). Union = 7,
from two commands.
And the citation cause is recorded as a measurement rather than a diagnosis, because
"the builder reads the wrong field" and "the outbox says the same thing five times"
want opposite fixes: one sha256 across all five proposals, in all four runs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A green mutation is a finding. Of the twelve planted against this file, eleven fell and
one survived: dropping `row["validated"]` from validated_ore's filter changed nothing.
It is equivalent — but only for as long as the OTHER guard holds. derive_outcome already
refuses to put a refused row's figure into (d), so on every outcome this builder writes,
validated_nok is None wherever validated is False and the two guards are
indistinguishable from the outside. Deleting either one alone is free today; deleting
both is the leak, and nothing witnessed that.
validated_ore is public and takes any outcome mapping, so the arm that separates them
hands it the outcome a future coverage writer could produce: a refused row whose amount
already sits in validated_nok. 137 000 002 øre, counted from the table, against the
227 000 002 a leak would give.
12 of 12 mutants now fall; control in the clone is 40 of 40.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
