Commit graph

1 commit

Author SHA1 Message Date
f2c739da75 test(eval): a frozen-set directory for the search gate, sets ignored not absent
`eval/soek/` is where the search gate's question sets are placed by hand. The
sets themselves are `.gitignore`d rather than committed, and that is this
repository's existing rule rather than a new one: `okf_retrieval_gate.py`
already states that a gold set names documents in someone's corpus, so it
arrives as a path and is never a constant here.

Measured before it was decided: the sets carry coordination ids and an
absolute path under a home directory, both of which this repository's own
frames forbid in a tracked file. A set that is absent is reported
`IKKE KJOERT -- fixture mangler` and counted RED, never as 0 hits and never as
a quiet pass, so a machine without them cannot read silence as a green row.

The README states the SHAPE each set must have and nothing about where any of
them came from or what any of them contains.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 04:03:11 +02:00