Row 7 left three mutants standing after v1.1 (11 of 14, bar 13): the passage
signal reading no body, the field signal weighing no title and no path, and a
flattened fusion (bm25.RRF_K = 10 000). Each switches off a mechanism the
default ranking runs, and each moved 0 ranks, because every synthetic concept
was short and opened with its own title -- none of the three ever decided a
delivery here.
Three bundles, one per mechanism, and one pinned set (set-mechanisms.json):
- PASSAGE: a long concept answered in one window of its body, against ten
short concepts whose titles carry the question's words. Rank 1; with no
body windows it falls out of k (class b).
- PATH: a concept named by its path alone, against ten decoys denser in the
body. Rank 4 at k = 6; with no title/path weight it falls out of k.
- FUSION: gold 1st in the passage signal and 21st in the field signal, one
decoy 10th and 11th (rank sum 19 < 20). At RRF_K = 60 the gold leads, and
it keeps the lead through K = 180 (measured in steps of 10); flattened, the
decoy wins k = 1.
Chose separate fixtures over one combined one because each fixture's comment
can then name the single mechanism it pins. Verified per question against all
14 mutants: no previously felled mutant is lost. src/ is untouched, MUTANT_BAR
and the roster are unchanged, the corpus pin moved with the corpus.
Gate: rows 1 and 6 go 10/10 -> 13/13, row 7 11/14 -> 14/14 GREEN, rows 2/3/4
unchanged, GATE RED: rows 5, 8. Suite: 2444 passed, 1 skipped (+3), measured
with FORCE_COLOR unset -- with FORCE_COLOR=3 in the environment Python 3.14
colours argparse help and test_each_arm_flag_carries_its_attribution fails,
independently of this change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three mutants that survive v1.1 each switch off a mechanism the default
BM25 ranking runs: the passage signal's body windows, the field signal's
title and path weight, and the rank-fusion constant. A gate that cannot see
them removed cannot see them break. One test per survivor, driven through
the gate's own `deterministic_rows` on the pinned sets and corpus: it fails
today with an AssertionError (no row got worse), and the same test passes
for M04, which the gate already fells.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Operator decision 2026-09-21: the test track built on material tied to the
operator's employer (K2) is retired -- not re-measured, not frozen. Public
tests and gates run on invented material.
Retrieval gate:
- The four FUSION_PREMISE xfails are gone and pass through their INPUTS: the
synthetic MISS, LOOKUP and QUOTA bundles were re-measured for BM25 (the
miss fasit no longer shares the rare word `maa`; lookup and quota decoys
carry the question's words so each partition and the quota decide their
own fixture). SPECS_SHA256 moved with them. Rows 2 and 3 green again.
- Row 7's mutants M04, M06, M07, M08, M10 now patch `bm25`, the code the
default runs. Three survive with 0 ranks moved (passage body, title
weight, bm25.RRF_K), each with its mechanism printed. M07 was not forced:
every synthetic body carries its title as a heading.
- Row 9 (K2) removed; row 8 requires `wiki-20` alone, the `r761` and
`vegnormal` adapters are gone. Chose the broad reading of "K2" because the
operator decision defines it as employer-tied material and the order's
grep includes `vegnormal`.
Also removed: tests/test_default_bundle_pin.py, the K2 arms of
test_okf_consume, the four real-arm tests of test_quality, the R761 soft
hyphen test, the N101/N200 delivery tests and okf_accounting_gate's default
real corpus (and H5's guard, which only existed for those defaults). Two
fixtures carrying road-standard identifiers are rewritten with invented ones.
Gate after: 1 10/10, 2 7/7, 3 5/5, 4 6/6, 5 0/1, 6 10/10, 7 11/14,
8 NOT RUN -> GATE RED: rows 5, 7, 8. Suite 2423 passed, 1 skipped,
0 xfailed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
C4. `coverage` gains two keys, and every key it had keeps its bytes:
- `absent_terms`: the question's words the bundle holds in NO form -- not as
written, and not through a relative it uses (`bm25.query_groups`, the same
bridge the ranking reads through);
- `weak`: true when one such word exists or nothing was delivered.
A reading with its rule in the open, never a verdict about the bundle. It is
computed for both rankings (`bm25.absent_terms` serves the fusion). The
retrieval gate's `marked` -- the one reading both gates share -- reads `weak`
beside its own bar, never instead of it; the known-negative that strips the
payload's words now strips both readings.
Words that only FRAME a question are stopwords in both languages (`how
often`, `hvor ofte`, `hva står i`, `what does it say`), and so are the
Norwegian function words spelled without their letters (`naar`, `paa`), the way
ASCII-only text writes them. Read as topic words they would be "absent" from
any collection that never uses them, which is what the synthetic sets showed
on three answered questions before the list was extended.
The working method says what to do with it, in one sentence each: the skill
template's step 3 and the MCP server's instructions (1 253 bytes, under the
2 048 a client keeps) -- rephrase in the bundle's words, and if it stays weak,
say the bundle does not cover the question.
The search gate's table for this commit is kept in local state.
Suite on a clean tree after `git add`: 2397 passed, 2 skipped, 4 xfailed.
ruff, ruff format, mypy --strict clean. Retrieval gate unchanged at the rows
the previous commit left red.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
C1. `okf consume` and MCP's `okf_ask` now rank with BM25 (`bm25.py`) instead
of the three-signal fusion. Two signals, fused by reciprocal rank:
- passage: every body cut into 500-character windows every 250, a concept
scored by its BEST window -- a narrow question is answered in one place;
- field: title three times, the id path and source name twice, then the body
-- a broad question is answered by what a section is called.
The document prior and the rarity weight are gone from the default: the first
favoured big documents full of common words, the second gave its largest
weight to a word the collection does not hold. Under BM25 such a word weighs
exactly zero. A signal that scores a concept zero adds nothing to it, and ties
share a rank, so alphabetical order lifts nothing either.
Three rules carried over from the fusion, each with its own test, because the
suite showed what BM25 alone lost:
- a directory every concept shares is not read (K3-20's defect, one signal on);
- a number a section is known by (`4.2`, `10.2-2`) is kept as one token, or
a question naming a section by its number matches nothing in it;
- a question word the collection does NOT hold is read as the collection's
words it shares a leading word with (`consume.tokens_match`) -- Norwegian
inflection and compounding -- at that word's idf, never at its own.
The lookup and title-covered partitions are shared with the fusion
(`_partitioned`). `ranking="fusion"` / `--ranking fusion` keeps the old order
reachable; `--cost-vocabulary` and `--rarity-weight` widen only the fusion and
are refused with the default (`ranking_flag_conflict`) rather than ignored.
C3. A concept longer than `PASSAGE_CHARS` (4 000) is delivered as the span
around its best window, snapped to whole lines, under the nearest heading
above it, with `[...]` where text was left out. `passage: {start, end, of}`
says so, `text_sha256` covers what was delivered, and `sha256` stays the
file's, so the whole can be fetched by `concept_id`. 4 000 because eight
excerpts of it stay far under a tool response's limit even with several
sub-questions merged, while a 500-character window keeps 3 500 characters of
surroundings. The budget pays for the passage, not the file.
Tests moved with the default, each stated rather than silenced:
- fusion-mechanism tests (cost vocabulary, rarity weight, reservation, shared
rank, the reference-bundle pins) ask for `ranking="fusion"`, the order they
were measured on; the BM25 reading of the reference bundle is a separate
measurement, kept in local state;
- the retrieval gate still measures the shipped default. Row 1 holds. Four of
its premises were built against the fusion (a concept forced below k that
BM25 now delivers, a quota that no longer decides, mutants patching fusion
code) and are `xfail(strict=True)` until the fixtures are re-measured;
- the shipped example payload is regenerated; the shipped skill is unchanged.
README's Consume section and CLAUDE.md state the new default and that the
flags described after it belong to the fusion.
The search gate's table for this commit is kept in local state: the question
sets belong to a consumer whose content does not go on a public mirror.
Suite on a clean tree after `git add`: 2390 passed, 2 skipped, 4 xfailed.
ruff, ruff format, mypy --strict clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three guards for the code the two step-0 commits added, each written
against a mutation that would otherwise survive: a wiki-shaped file
carrying the pinned 20 questions and 29 fasit entries but invented content
(so the counts alone are a one-line forgery), a threshold of 80 or 1.5
(a share or a typo, and guessing is not this row's job), and a hold-out set
of no questions -- the shape every row-5 test used until today, which would
have made the new comparison vacuous the moment it was added.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`coverage` carries three lists: the terms the pre-pass read the question as,
the terms no concept in the bundle answers, and the terms no delivered
excerpt answers. Without it a reader holding eight excerpts cannot tell a
bundle that ANSWERED its question from one that merely ranked something --
the two payloads have the same shape.
FACTS, AND NO VERDICT, which is a measurement and not caution. Two readings
were built and both falsified over 81 questions (16 synthetic, 65 across the
three real sets, 2026-09-20): the share of question terms a delivered
excerpt answers separates the synthetic controls at 0.33 against 0.50 and
REVERSES on real data (covered questions down to 0.27, one genuinely
uncovered question at 0.71); the share of a bundle tying the best lexical
match is ~0.00 for every real question either way. Question style dominates
the first, corpus size the second.
The one bar this repository declares is the gate's: `UNANSWERED_BAR = 2/3`
over `unanswered_in_bundle`, swept and collapsing at both ends -- at 0.50
eleven real covered questions are marked, at 0.70 the row falls to 5 of 6,
at 2/3 the row is 6 of 6 and 0 of 65 real questions are marked. The margin
is thin (0.6087 against 0.6667) and is published that way, together with
what it does not catch: r761-sk2's own known-negative sits at 0.2857.
Row 4: 3 of 6 RED -> 6 of 6 GREEN, with the 10 answered synthetic questions
held unmarked as the known-negative. The contract's SS 8 gains point 7, the
consumption skill is told to read the block, and the SS 7.4 known-positive
moves with the document (14 721/375 -> 16 389/417). Suite 2292 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Red first, on behaviour: three asserts about what the payload carries and
what row 4 scores, none on an import or an attribute.
Measured 2026-09-19: N3, N4 and N5 come back with 8, 8 and 1 excerpts and
nothing that says they are weak, so `marked = nothing delivered` reads
three uncovered questions as answered ones. The second gate test is the
known-negative that a marking firing on everything would fail: the 10
questions the three synthetic sets DO answer must stay unmarked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The quota filters the WHOLE candidate list, so every over-quota candidate
came back `source_quota_exceeded` -- including the ones the RANK had
already put outside k, which the quota only reached because it ran first.
`_fates_without_quota` asks the same cut what would have become of each
candidate with no quota, and the drop keeps THAT rule; only a candidate the
quota-off cut would have delivered is named as the quota's. The packer is
lifted into `_pack` and used by both, so the quota-off fate is decided by
the code the run itself uses and never by a second implementation.
The retrieval gate's row 3 goes 2 of 5 RED to 5 of 5 GREEN. Row 7 is
unchanged at 12 of 14; M01 and M02 lose their row-3 credit, which was the
lying label moving and not the ranking. Suite: 2288 passed, 0 failed; no
committed payload moves a byte.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two checks replace `bool(threshold)`: `_as_share` parses it as a share in
[0, 1] -- so `report-only; any number is acceptable for v1` is a NO, and so
is `80` -- and `_hold_out_verdict` RUNS the registered set against the
registered bundle and prints `answered of asked = share against threshold`.
A registration naming an absent set, an unreadable bundle or an empty set
is a NO with its reason, never an exception and never a silent pass. The
row is 11 checks; it stays RED today because no registration exists.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Red first, on behaviour: both assertions are about the checks the row
prints, neither is an import or an attribute error.
Measured by PM on 23588e5: `bool(threshold)` was the whole check, so the
threshold `report-only; any number is acceptable for v1` read as `a
threshold is written: yes`. A threshold that is never compared with a
number cannot fell anything.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Red first, on behaviour: the identity assertion fails on the rendered
details, and the two pin assertions fail with DID NOT RAISE. Neither dies
on an import or an attribute.
PM measured both on 23588e5: one pinned r761 set gives 4 of 7 on three
bundles and 0 of 7 on a fourth while the session recorded 7 of 7, from a
bundle the output never names; and three self-written one-question files
plus a self-written bundle read `3 of 3 GREEN`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
X2 of this round's own mutation run SURVIVED: put the headline back to
`quoted + concept` over `quoted_units + concept_units` and all 64 tests stayed
green. The reason is the fixture, not the row -- three sets of one fasit entry
each make "questions answered" and "units hit" the same number, so the test
agreed with the mutant by construction. The same shape as round 158's X5, and
the same lesson as `konstruert-rad-kan-vaere-fasitradens-speil`.
One set now carries TWO fasit entries, which is the only shape where the two
readings differ: the headline must be 3 of 3 questions where the sum would be
4 of 4. `_hitting_set` takes `entries` and says why in its docstring.
64 passed. Mutation run in a scratch copy of HEAD, twelve mutants plus a
control that must survive: killed 11 of 12 before this commit, and X2 is the
twelfth.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM ran eight cheating attacks and four were already refused. Three of those
four already had a test (a tampered sha is exit 2, a moved set is refused, a
tuned corpus now meets a pin as well as row 2's forced classes); the fourth --
a cut that delivers nothing, which came back as an error from `consume` rather
than as a quiet row -- did not. The table is 8 of 8 now, not 4 of 4.
64 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's checkpoint left seven small findings beside the four bearing ones. Six
are closed here (the seventh, running row 8 against the real sets, follows).
G9 -- THE CONFIRMATION TERM IS OBSERVED NOW. `hit = bool(hit_ids) and
bool(confirmed)` survived all 46 tests, because every mutation of the text
empties `hit_ids` one step earlier. The shape that reaches it is a delivery
that still CARRIES the citation and is no longer the concept file's bytes:
`M14` is that mutation and it is FELLED (row 7 goes 11 of 13 to 12 of 14, bar
12 of 13 to 13 of 14, still RED, the same two survivors), and a test drives it
with its known-positive in the same test. No production line changed: the term
was always observable, it was unobserved.
AND THAT MEASURES THE JUDGE'S INDEPENDENCE RATHER THAN ASSERTING IT. The judge
does read the bundle through `consume.read_concept` and `delivered_text` --
PM's finding -- but the index is warmed BEFORE the first mutation, so the two
sides do not move together. Measured both ways: index warmed first, every unit
is a miss with `confirmed False`; index built UNDER the same patch, every unit
is a hit. The gate never builds one under a mutation. Stated in `LIMITS` with
that measurement, rather than closed by re-implementing a normalisation rule
this repository already owns once.
SPECS -- the synthetic corpus is pinned like the sets (`SPECS_SHA256` over
`specs_digest`). PM's corpus tuning was caught by row 2's forced classes and
not by a pin, and a more careful tuning was left standing.
ROW 9 TAKES AN INPUT. `--k2 SET SHA256 BUNDLE` reads a gold set in this gate's
own set shape; `K2_QUESTIONS` stays the denominator whatever the file carries,
and a set of another size is refused (exit 2) as another set wearing this
one's name. Without a set the row stays RED and not NOT RUN -- ITS denominator
is known, six recorded questions, so the absence is measured; row 8's is not
known until the sets arrive. Both fail the gate identically. This is a
deliberate divergence from the order's parenthetical, stated here and in the
row.
MYPY. `mypy --strict` on this file goes 8 errors to 0, the four in
`read_real_set` among them (`questions = []` against a name inferred
`tuple[Question, ...]`) -- the adapters that meet the real sets.
63 passed. The verdict is unchanged: GATE RED: rows 3, 4, 5, 7, 8, 9.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's J3. `MUTANT_ROSTER` carries the thirteen labels and `MUTANT_COUNT` their
number, both written apart from `MUTANTS`, and `row_seven` refuses to run
unless the labels it was handed ARE that roster, in order, with no duplicate.
The bar is taken from the roster's length, not from `len(mutants)`.
Why a pin and not a share: the bar is a percentage, so a longer list is a
lower bar per survivor. Seven copies of `M03 k = 1` took the row to 18 of 20
GREEN with the same two survivors -- nothing new felled, the bar lowered.
Lengthening the list honestly now costs three edits in three places, each
readable as what it is; a duplicate label is refused outright, because two
copies of one mutation are one mutation whatever the roster says.
Row 7 is unchanged on the shipped list: 11 of 13, bar 12 of 13, RED, the same
two survivors with the same measured notes. 56 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's J3: seven duplicate `M03 k = 1` mutants appended to `MUTANTS` took row 7
from `11 of 13 RED` to `18 of 20 GREEN` with the SAME two survivors. The bar
is `MUTANT_BAR` of `len(mutants)`, and `MUTANTS` is a module constant in the
file a capability session edits -- a gate whose pass mark is a list the
candidate writes.
Reproduced here at 18 of 20 exactly. 1 red on an assertion about behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's J1. Three of row 5's ten checks now read the repository's history instead
of the registration's own fields:
- the registration is COMMITTED and unmodified against HEAD -- a file written
in a working tree is not a pre-registration;
- the commit that ADDED it is not itself a change to the ranking -- a
threshold and the change it is supposed to bind, in one commit, is neither;
- a change to the ranking landed AFTER it. This is the one that cannot be
self-attested: it is satisfied only in the order a pre-registration actually
happens, and it is red today because neither half has happened.
The three come from `git_provenance`, a `Provenance` the row receives the way
it already receives a path, so both directions are driven from the test and
the default is the real git. Outside a git tree, or with no git on PATH, the
fields come back in their REFUSING form with the reason printed -- unknown is
not unknown-and-therefore-fine.
WHAT GIT CANNOT PROVE IS STATED IN THE ROW, not implied: history shows ORDER,
never that nobody read the number before writing the threshold -- a number can
be read from an uncommitted tree. Order is what these three checks are.
Row 5 is unchanged where it matters today: `0 of 1 RED`, no registration. The
two existing direction tests move 7 -> 10 checks and a third arm is added that
drives each git check red on its own. 54 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's J1: two files written by the session under test came back `7 of 7 GREEN`.
Every one of row 5's checks reads a field the registration owns. "Written by
another session than the ranking change" is `bool()` of a string the file
sets; "no reading predates the threshold" reads a `readings` list from the
same file. The order's point -- report-only without a pre-written threshold is
not a protection -- is not met by a protection that is itself a claim.
1 red on an assertion about behaviour (GREEN for a registration this test
wrote seconds earlier), and it names what a session cannot write about
itself: that git already carried the file before the ranking moved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's J10 and J8. Two mechanisms, one per attack, and neither is a pin a
capability session can edit in the same breath as the code.
J10 -- THE DENOMINATOR IS THE SET'S. `Unit` now carries the class its question
DECLARES. Row 2's denominator is the misses plus every forced fixture that
came back a hit, and row 3's is every unit whose set declares a withheld class
(b, c, e -- (a) is not in the bundle and (d) was delivered, so neither can
carry a printed reason) plus whatever the run withheld besides. A fixture that
stops producing its declared class is a BROKEN PREMISE, printed as one, and it
counts against its row: at `k = 32` row 2 stays RED with its denominator held
and row 3 keeps 5 units where it had shrunk to 2 and called that green.
J8 -- THE ROW CARRIES A KNOWN-POSITIVE. With `--source-quota` off, every
printed reason is true; that reading is not a lie, it is an empty measurement,
and row 3 must say so rather than print `6 of 6 GREEN` beside row 1 falling to
8 of 9. A set may now declare `source_quota_in_force`, and the row is NOT RUN
for such a set when the default cut and the quota-off cut deliver the same
concepts everywhere.
THE CONTROL'S OWN PREMISE WAS MEASURED FIRST, and it was false where it was
first put: over the five existing sets the two cuts deliver the SAME concepts
(the quota is topped back up), so 52 labels move `source_quota_exceeded` ->
`below_k` without one delivery changing. `set-quota.json` is the set where the
quota genuinely decides -- measured, `oversikt-08` is delivered without the
quota and withheld with it, and the fasit `svar/broennproeve` is delivered
only with it -- so the requirement is declared there and nowhere else. It
survives the honest fix, which changes labels and not the cut.
Rows 1 and 6 go 9 of 9 to 10 of 10: one added fixture, one added hit, both
green before and after. Rows 2 (7 of 7), 3 (2 of 5), 4, 5, 7, 8, 9 and the
verdict `GATE RED: rows 3, 4, 5, 7, 8, 9` are unchanged. 52 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's J10 and J8, the two attacks on the denominator, written as tests that
must refuse them.
J10 (`k = 32`): rows 1, 2, 3 and 6 all came back green at once and not one
label had become true. Rows 2 and 3 count against the misses and the withheld,
so a cut that delivers more broadly does not answer their question -- it
shrinks their denominator to the cases that were already honest. The three
fixtures that DECLARE class b are delivered under that cut, so their premise
is broken, and a broken premise is not an absence.
J8 (`--source-quota` off): every printed reason became true and row 3 read
`6 of 6 GREEN` while row 1 fell to 8 of 9. That reading is not a lie -- with
no quota there is no quota to name falsely -- but the row measured nothing,
because the label it judges was never printed.
3 red on an assertion about behaviour (GREEN where RED or NOT RUN is required,
and a broken premise counted as 0 of 0), 49 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two of PM's eight attacks, closed where they were measured.
J2 -- a set left out. `REQUIRED_REAL_SETS` names the three (`wiki-20`,
`r761-sk2`, `vegnormal-32`) and the row is NOT RUN until all three are given,
whatever the ones that ran scored. The numbers the run DID measure are still
printed: a missing set must not cost the reader the set that was measured, and
"not run" is the row's status, never a reason to withhold a figure.
J2b -- the headline was `quoted_hits + concept_hits` over
`quoted_units + concept_units`, written one line above the detail that says
the two are not summed. The three sets do not share a unit: `wiki-20` names a
citation, `r761-sk2` a section title, `vegnormal-32` a requirement number, and
a citation hit plus a concept hit is a number that is neither. A QUESTION is
the one unit all three have, so the headline is questions answered of
questions asked -- answered meaning at least one of the question's fasit
entries arrived, the reading row 1 already prints beside its own units -- and
the two unit totals keep their own denominators below it, unsummed.
49 passed (was 46 passed, 3 failed). The gate is unchanged where it did not
run: still exit 1, still `0 of 3 NOT RUN` with PM's recorded figures carried.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PM's checkpoint on 2c8296b ran eight cheating attacks at this gate and four
went through: a row came back GREEN without one label becoming true or one
concept ranking better. This is the first of them, written as a test that must
refuse it.
J2: `row_eight` iterates over whatever `--real` handed it and counts. One set
of three came back `6 of 6 GREEN`, and the realistic route is `--real r761`
alone -- the set PM's own registration puts at 7 of 7, with the two that miss
left out. The docstring already said "never green by leaving a set out"; the
code did not, and THIS REPOSITORY'S OWN TEST asserted the opposite at
`tests/test_retrieval_gate.py:486`, `(1, 1, GREEN)` for a single wiki set.
That assertion is corrected here rather than worked around: a test that pins
the defect is the defect.
J2b: the headline `k of N` is `quoted_hits + concept_hits` over
`quoted_units + concept_units`, written on the line directly above the detail
that says the two granularities are NOT summed into one number.
3 red on an assertion about behaviour (GREEN where NOT RUN is required, and a
headline that is a sum), 46 passed. The green direction -- all three named
sets present -- is in the same commit and passes already, so the fix cannot
close the rows by making the row unreachable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One command, one exit code, and no capability: nothing here changes the
ranking, the fusion, the tokenisation or the cut. For a frozen question set
against a bundle it asks how many of N measurement units the payload carries
the fasit for, gives every miss exactly ONE class, and asks the thing the
payload does not do at all -- say so when it does not know.
Today, on the synthetic corpus this repository generates and the five sets it
ships pinned by sha256: rows 1 (9 of 9), 2 (7 of 7) and 6 (9 of 9) green;
3 (2 of 5), 4 (3 of 6), 5 (0 of 1), 7 (11 of 13), 8 (not run) and 9 (0 of 6)
red. GATE RED, exit 1.
- Row 3 is the finding to read first. In a bundle built from ONE source
document, every concept past the first two carries that document's
`source_file`, so a concept the RANK had already lost is withheld as
`source_quota_exceeded`. The truth is decided by the SAME cut run without
the quota -- PM measured the same lie on 13 of 25 real misses 2026-09-17.
- Row 4 is the second. No payload key says "this bundle does not answer
that", so the only honest reading today is `delivered == 0`; three of six
controls come out right and the three that do not get eight excerpts and no
statement. A control the bundle DOES answer must come back unmarked, or the
marking says nothing, so both directions sit in the same row.
- A miss has exactly one class -- (a) not in the bundle, (b) below k, (c) cut
by quota or budget, (d) delivered without the citation, (e) other -- and
each is FORCED by its own fixture. An unplaced miss takes the row to 0 of N.
- The judge opens the bundle. Every concept is read off disk before a payload
is built, a delivered excerpt whose text is not the concept file's bytes is
not a hit, and the two title forms come from `quality` rather than being
written a second time.
- FELLED means a row got WORSE, never merely that something changed: removing
the quota makes row 3 green, and counting that as a kill would credit this
gate with a check it does not have. The two survivors are reported with what
they measured -- the document prior dead and RRF_K flattened each moved 0
ranks and 0 deliveries on these fixtures -- and with a mechanism for each.
- Rows 8 and 9 are never green by leaving something out. The three real sets
are read from path + sha256 and never committed: a gold set names a
consumer's documents and this repository is public. K2 is 0 of 6 because the
gold set exists nowhere while its bundles do.
Denominators re-measured against the sources 2026-09-19, and all four hold:
wiki 20 questions / 29 fasit entries (sha 972d0f57...), R761 8 (S1-S6 + KP +
KN), vegnormal 32 questions / 43 must_cite, K2 6. PM's 45 of 70 is itself a
mixed sum -- wiki at question level, vegnormal at citation level -- so this
gate carries it labelled as recorded and never sums the two granularities.
46 tests, each failing row driven both ways through inputs rather than by
editing the row. No network (the socket guard is fired before its silence
counts), no private corpus, no clock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
