Kjell Tore Guttormsen
f2c739da75
`eval/soek/` is where the search gate's question sets are placed by hand. The sets themselves are `.gitignore`d rather than committed, and that is this repository's existing rule rather than a new one: `okf_retrieval_gate.py` already states that a gold set names documents in someone's corpus, so it arrives as a path and is never a constant here. Measured before it was decided: the sets carry coordination ids and an absolute path under a home directory, both of which this repository's own frames forbid in a tracked file. A set that is absent is reported `IKKE KJOERT -- fixture mangler` and counted RED, never as 0 hits and never as a quiet pass, so a machine without them cannot read silence as a green row. The README states the SHAPE each set must have and nothing about where any of them came from or what any of them contains. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
16 lines
841 B
Text
16 lines
841 B
Text
# The frozen question sets are INPUTS, never constants in this repository.
|
|
#
|
|
# `tools/okf_retrieval_gate.py` states the rule this directory inherits: "a
|
|
# gold set names documents in a consumer's corpus, so a real set arrives as a
|
|
# path plus an expected sha256 and is never committed". The sets named here
|
|
# additionally carry prose quoted verbatim from a consumer whose own status is
|
|
# unsettled, and CLAUDE.md names that class as not publishable from here.
|
|
#
|
|
# So they are ignored, not absent: place them in this directory and the gate
|
|
# reads them. Nothing here reaches a public mirror by accident, and the gate
|
|
# reports `IKKE KJOERT -- fixture mangler` (RED) for any set that is not here,
|
|
# so a machine without them cannot mistake silence for a pass.
|
|
fase-sporsmaal.json
|
|
holdout-sporsmaal.json
|
|
norske-sporsmaal.json
|
|
delsporsmaal.json
|