The context sets, the packaged knowledge bases and the example bundles are
replaced by one fictitious example set about IT operations in an invented
organisation: three context sets (serverrom-2027, driftsavtale-2027 and the
two-base drift-og-avtale-2027), two synthetic knowledge bases under
src/portfolio_optimiser/data/kunnskapsbaser and two example bundles under
src/portfolio_optimiser/data/bundles. Numbers, codes and structural values in
tests and fixtures are kept; names, ids and wording change. Dated measurement
documents that only recorded runs on the replaced material are deleted.
Gate figures measured on the new set are not comparable with earlier ones.
The exclusion gate from the previous commit is green: 0 tracked files hit
outside the shared/ subtree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 18.09 re-measurement took row 2 to 3 of 3 GREEN on a tree this product had never
run in: four handwritten outcome.json, four handwritten <run_id>-coverage.json in an
outbox the forger named in those same files, and os.utime for the ordering. "Round 0
must be a named real run" was implemented as "a file with that name exists" — which
touch satisfies. The attack is committed as a red test in b769537.
Three bindings, chosen because each removes one of the forger's three moves:
1. The outbox is DERIVED, never declared. It is <rounds-dir>/<n>/outbox/, and an
outcome.json that names one is refused by name. A path a round file chooses is a
path it can point at a directory the same hand just wrote.
2. "The run exists" now means the run's own artefacts agree WITH EACH OTHER on content
the gate recomputes (verify_run). Every evaluated approach has the proposal/outcome
pair write_outbox actually persists; outcome_type IS the coverage status, reason IS
its detail, a validated row's figure IS the proposal's own claimed_saving_nok, the
provenance stamp follows the same branch, and verdict_id is RE-MINTED here from the
proposal's own IR with the product's one minting rule (A5) rather than read. A
not_evaluated approach wrote neither file, and an artefact naming an approach the
coverage omits belongs to another run. verdicts._features_from_ir is made public for
this: a second private copy in the gate could drift from the rule the run stamped
with, which would turn the binding into a coincidence.
3. mtime decides nothing. The run's time is the round's declared ran_at (ISO-8601 with
zone, required). An mtime is not evidence — it is a filesystem attribute one utime
call sets.
What this does NOT do is prove a run happened. Nothing in a directory can: the outbox
writers are byte-deterministic and carry no clock by contract. So row 2 states its own
limit on every run instead of leaving GREEN to imply it (Row.attests / RUN_ATTESTATION):
that a run was actually made, and when, is the operator's to confirm. The cost of a
forgery moves from touch to reproducing the product's own artefact set, minting rule
included.
Measured, in a scratch copy, never in the work tree — four new mutants in this class,
each one line, each felled by the whole suite:
M7 if "outbox" in data: -> if False: 1 failed (declares its own outbox)
M8 if verdict_id != minted: -> if False: 1 failed (key is not the IR's)
M9 if strays: -> if False: 1 failed (artefact of another run)
M10 ran_at -> coverage mtime 18 failed (incl. the utime test)
Control, same scratch copy, unmutated: 1993 passed, 10 skipped, 5 xfailed.
Work tree, re-run after git add: uv run pytest -q -> 1998 passed, 5 skipped, 5 xfailed.
Gate: uv run python -m portfolio_optimiser.evals.v1_gate -> exit 1, row 2 RED (0 of 3).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`cosine`'s docstring claimed its guard was load-bearing because "a NaN reaching the
ranking sort key would corrupt ordering silently rather than failing loudly" — but the
guard tested `norm == 0.0` only, which a NaN or inf norm passes straight through. The
claim was prose, not behaviour.
Measured, not assumed: `cosine(unit, nan_vector)` AND `cosine(unit, inf_vector)` both
returned `nan`, and a NaN sort key made ranking INPUT-ORDER-DEPENDENT — six permutations
of the same three candidates produced four distinct orderings. That defeats the total
order `HybridRanker` documents ("`id` makes the result independent of input order").
Refuse rather than coerce, and deliberately NOT symmetric with the zero-norm branch: a
zero vector is a legitimate handled state (`FakeEmbedder` returns `np.zeros` by design),
whereas a non-finite component only ever means the INJECTED embedder is broken. Scoring
it `0.0` would launder that into "no semantic similarity" while ranking proceeded on a
forged signal — validation, never repair, mirroring `read_spend`.
Reachable via the documented `Embedder` extension point, not the shipped fake; scoped to
the norms (90% principle — a finite-normed dot-product overflow is not chased).
Also corrects `docs/extending.md`, which stated `SEMANTIC_WEIGHT_DEFAULT = 0.5` while the
code has said `0.25` since the weight was lowered.
625 -> 630 tests. Load-bearing MEASURED against the WHOLE suite, five mutations all red:
detach the guard entirely · coerce to 0.0 instead of raising · check only the first norm ·
drop "non-finite" from the message · (control) detach the zero-norm branch, which fails
ONLY the zero-norm test — the new guard does not mask the existing one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V9vNBmxAmgJ2JMoHByiHS
