open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

fix(retrieval-gate): row 5 reads the hold-out set in the schema the set declares

Browse source 
Row 5 read every registered set with the gate's synthetic reader, so a
set written in a consumer's own question-set schema came back "not a
question set this gate can read: 'set_id'" -- a NO no such set could
turn, whatever it measured.

The SET now says how it is read, and the registration carries nothing
about it: `read_hold_out_set` checks the pinned sha256, then routes on
the set's own `schema` field. `fase-sporsmaal/1` goes through the same
adapter and hit_rule as row 8 (`_wiki_questions`, factored out of
`read_real_set` so the two rows share one reading); a set with no
`schema` is the gate's synthetic form through `load_set`, unchanged;
any other schema is a NO that names it (fail closed).

Chosen to recognise the set's schema rather than add a field to the
registration because the registration then needs to know nothing about
the reading, and an unknown schema still fails with a named reason.

The row still prints only answered of asked against the threshold, never
which hold-out question missed; a test holds that. src/ untouched,
MUTANT_BAR unchanged. CLAUDE.md gains one sentence on row 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-23 08:49:26 +02:00
commit a381361d99
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
2 changed files with 73 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

8
CLAUDE.md
View file

@ -1385,7 +1385,13 @@ and fixtures, never code.
itself; three now read GIT (committed and unmodified, its commit is not
itself a ranking change, a ranking change landed AFTER it), the last being
the one that cannot be self-attested. What git cannot prove -- that nobody
read the number first -- is stated, not implied. **Row 7's bar** was 90 % of
read the number first -- is stated, not implied. Since 2026-09-23 row 5
reads the hold-out set in the schema the SET declares
(`HOLD_OUT_SCHEMAS`: `fase-sporsmaal/1` goes through row 8's own adapter
and `hit_rule`, no `schema` is the gate's synthetic form, any other schema
is a NO naming it) -- it read every set with the synthetic reader, so a set
in a consumer's schema was a NO nothing could turn. It prints the share and
never which hold-out question missed, and a test holds that. **Row 7's bar** was 90 % of
a list in the file a capability session edits, so seven duplicate `k = 1`
mutants read `18 of 20 GREEN`; `MUTANT_ROSTER` and `MUTANT_COUNT` are pinned
apart from `MUTANTS`, duplicates are refused, and the bar is the roster's.