Kjell Tore Guttormsen
2c8296b807
One command, one exit code, and no capability: nothing here changes the ranking, the fusion, the tokenisation or the cut. For a frozen question set against a bundle it asks how many of N measurement units the payload carries the fasit for, gives every miss exactly ONE class, and asks the thing the payload does not do at all -- say so when it does not know. Today, on the synthetic corpus this repository generates and the five sets it ships pinned by sha256: rows 1 (9 of 9), 2 (7 of 7) and 6 (9 of 9) green; 3 (2 of 5), 4 (3 of 6), 5 (0 of 1), 7 (11 of 13), 8 (not run) and 9 (0 of 6) red. GATE RED, exit 1. - Row 3 is the finding to read first. In a bundle built from ONE source document, every concept past the first two carries that document's `source_file`, so a concept the RANK had already lost is withheld as `source_quota_exceeded`. The truth is decided by the SAME cut run without the quota -- PM measured the same lie on 13 of 25 real misses 2026-09-17. - Row 4 is the second. No payload key says "this bundle does not answer that", so the only honest reading today is `delivered == 0`; three of six controls come out right and the three that do not get eight excerpts and no statement. A control the bundle DOES answer must come back unmarked, or the marking says nothing, so both directions sit in the same row. - A miss has exactly one class -- (a) not in the bundle, (b) below k, (c) cut by quota or budget, (d) delivered without the citation, (e) other -- and each is FORCED by its own fixture. An unplaced miss takes the row to 0 of N. - The judge opens the bundle. Every concept is read off disk before a payload is built, a delivered excerpt whose text is not the concept file's bytes is not a hit, and the two title forms come from `quality` rather than being written a second time. - FELLED means a row got WORSE, never merely that something changed: removing the quota makes row 3 green, and counting that as a kill would credit this gate with a check it does not have. The two survivors are reported with what they measured -- the document prior dead and RRF_K flattened each moved 0 ranks and 0 deliveries on these fixtures -- and with a mechanism for each. - Rows 8 and 9 are never green by leaving something out. The three real sets are read from path + sha256 and never committed: a gold set names a consumer's documents and this repository is public. K2 is 0 of 6 because the gold set exists nowhere while its bundles do. Denominators re-measured against the sources 2026-09-19, and all four hold: wiki 20 questions / 29 fasit entries (sha 972d0f57...), R761 8 (S1-S6 + KP + KN), vegnormal 32 questions / 43 must_cite, K2 6. PM's 45 of 70 is itself a mixed sum -- wiki at question level, vegnormal at citation level -- so this gate carries it labelled as recorded and never sums the two granularities. 46 tests, each failing row driven both ways through inputs rather than by editing the row. No network (the socket guard is fired before its silence counts), no private corpus, no clock. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
26 lines
1.1 KiB
JSON
26 lines
1.1 KiB
JSON
{
|
|
"set_id": "synthetic-signals",
|
|
"bundle": "lookup",
|
|
"note": "Three questions the shipped ranker must answer through three different mechanisms: the identifier lookup, the covered title, and the source quota. Each bundle answers the question's other words several times over, so a mechanism that stops firing takes its own fasit out of k and row 1 sees it.",
|
|
"questions": [
|
|
{
|
|
"id": "L1",
|
|
"question": "Hva staar i punkt 4.2 om kontrollen av hytta?",
|
|
"bundle": "lookup",
|
|
"fasit": [{"by": "concept", "value": "rutiner/vakthold-4-2", "quote": "fredag til soendag"}]
|
|
},
|
|
{
|
|
"id": "L2",
|
|
"question": "Hva sier noekkelrutine om kontrollen av hytta?",
|
|
"bundle": "lookup",
|
|
"fasit": [{"by": "concept", "value": "rutiner/noekkelrutine", "quote": "skifte av laas"}]
|
|
},
|
|
{
|
|
"id": "Q1",
|
|
"question": "Hva sier oversikten om dugnaden og kontrollen av broennen?",
|
|
"bundle": "quota",
|
|
"fasit": [{"by": "concept", "value": "svar/broennproeve", "quote": "proevetas i juni"}]
|
|
}
|
|
],
|
|
"controls": []
|
|
}
|