Kjell Tore Guttormsen
2c8296b807
One command, one exit code, and no capability: nothing here changes the ranking, the fusion, the tokenisation or the cut. For a frozen question set against a bundle it asks how many of N measurement units the payload carries the fasit for, gives every miss exactly ONE class, and asks the thing the payload does not do at all -- say so when it does not know. Today, on the synthetic corpus this repository generates and the five sets it ships pinned by sha256: rows 1 (9 of 9), 2 (7 of 7) and 6 (9 of 9) green; 3 (2 of 5), 4 (3 of 6), 5 (0 of 1), 7 (11 of 13), 8 (not run) and 9 (0 of 6) red. GATE RED, exit 1. - Row 3 is the finding to read first. In a bundle built from ONE source document, every concept past the first two carries that document's `source_file`, so a concept the RANK had already lost is withheld as `source_quota_exceeded`. The truth is decided by the SAME cut run without the quota -- PM measured the same lie on 13 of 25 real misses 2026-09-17. - Row 4 is the second. No payload key says "this bundle does not answer that", so the only honest reading today is `delivered == 0`; three of six controls come out right and the three that do not get eight excerpts and no statement. A control the bundle DOES answer must come back unmarked, or the marking says nothing, so both directions sit in the same row. - A miss has exactly one class -- (a) not in the bundle, (b) below k, (c) cut by quota or budget, (d) delivered without the citation, (e) other -- and each is FORCED by its own fixture. An unplaced miss takes the row to 0 of N. - The judge opens the bundle. Every concept is read off disk before a payload is built, a delivered excerpt whose text is not the concept file's bytes is not a hit, and the two title forms come from `quality` rather than being written a second time. - FELLED means a row got WORSE, never merely that something changed: removing the quota makes row 3 green, and counting that as a kill would credit this gate with a check it does not have. The two survivors are reported with what they measured -- the document prior dead and RRF_K flattened each moved 0 ranks and 0 deliveries on these fixtures -- and with a mechanism for each. - Rows 8 and 9 are never green by leaving something out. The three real sets are read from path + sha256 and never committed: a gold set names a consumer's documents and this repository is public. K2 is 0 of 6 because the gold set exists nowhere while its bundles do. Denominators re-measured against the sources 2026-09-19, and all four hold: wiki 20 questions / 29 fasit entries (sha 972d0f57...), R761 8 (S1-S6 + KP + KN), vegnormal 32 questions / 43 must_cite, K2 6. PM's 45 of 70 is itself a mixed sum -- wiki at question level, vegnormal at citation level -- so this gate carries it labelled as recorded and never sums the two granularities. 46 tests, each failing row driven both ways through inputs rather than by editing the row. No network (the socket guard is fired before its silence counts), no private corpus, no clock. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
14 lines
1 KiB
JSON
14 lines
1 KiB
JSON
{
|
|
"set_id": "synthetic-controls",
|
|
"bundle": "positive",
|
|
"note": "Row 4's denominator: the five negative control classes measured 2026-09-17, plus one control the bundle DOES answer. A marking that also fires on a covered question says nothing, so both directions are in the same row.",
|
|
"questions": [],
|
|
"controls": [
|
|
{"id": "N1", "kind": "an unrelated topic", "question": "Hvilken safran passer til fiskesuppe?"},
|
|
{"id": "N2", "kind": "a foreign language", "question": "How do I rotate the database credentials?"},
|
|
{"id": "N3", "kind": "a plausible term the bundle does not cover", "question": "Hvilken kontroll gjelder for broeytebilen om vinteren?"},
|
|
{"id": "N4", "kind": "a lookup on an identifier that does not exist", "question": "Hva sier punkt 99.9 i haandboka?"},
|
|
{"id": "N5", "kind": "the set's own known-negative", "question": "Hva koster et doegn paa hytta for gjester?"},
|
|
{"id": "P0", "kind": "covered by the bundle", "question": "Naar kontrolleres vinterberedskapen?", "covered": true}
|
|
]
|
|
}
|