test(eval): a frozen-set directory for the search gate, sets ignored not absent

`eval/soek/` is where the search gate's question sets are placed by hand. The
sets themselves are `.gitignore`d rather than committed, and that is this
repository's existing rule rather than a new one: `okf_retrieval_gate.py`
already states that a gold set names documents in someone's corpus, so it
arrives as a path and is never a constant here.

Measured before it was decided: the sets carry coordination ids and an
absolute path under a home directory, both of which this repository's own
frames forbid in a tracked file. A set that is absent is reported
`IKKE KJOERT -- fixture mangler` and counted RED, never as 0 hits and never as
a quiet pass, so a machine without them cannot read silence as a green row.

The README states the SHAPE each set must have and nothing about where any of
them came from or what any of them contains.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

eval/soek/.gitignore

@@ -0,0 +1,16 @@
+# The frozen question sets are INPUTS, never constants in this repository.
+#
+# `tools/okf_retrieval_gate.py` states the rule this directory inherits: "a
+# gold set names documents in a consumer's corpus, so a real set arrives as a
+# path plus an expected sha256 and is never committed". The sets named here
+# additionally carry prose quoted verbatim from a consumer whose own status is
+# unsettled, and CLAUDE.md names that class as not publishable from here.
+#
+# So they are ignored, not absent: place them in this directory and the gate
+# reads them. Nothing here reaches a public mirror by accident, and the gate
+# reports `IKKE KJOERT -- fixture mangler` (RED) for any set that is not here,
+# so a machine without them cannot mistake silence for a pass.
+fase-sporsmaal.json
+holdout-sporsmaal.json
+norske-sporsmaal.json
+delsporsmaal.json

eval/soek/README.md

@@ -0,0 +1,55 @@
+# Frozen question sets for the search gate
+
+`tools/okf_soek_gate.py` measures what a reader actually RECEIVES from a
+collection at the shipped defaults. It reads its question sets from this
+directory and the collection from `--bundle`; **neither is committed here.**
+
+**The sets are `.gitignore`d and are placed here by hand, by whoever runs the
+gate.** They are INPUTS, never constants: that is the rule
+`tools/okf_retrieval_gate.py` already states for a gold set, and this
+directory inherits it. A set that is absent is reported
+`IKKE KJOERT -- fixture mangler` and counted RED — never as 0 hits, and never
+as a quiet pass. So a machine without the sets cannot mistake silence for a
+green row, and nothing about anyone's corpus reaches a public mirror.
+
+A set is **FROZEN**: it is copied verbatim from wherever it was written,
+nothing here is edited to make a row go green, and a set whose bytes move is a
+new set with a new name.
+
+## The four files and the shape each must have
+
+| File | Shape | Units |
+|---|---|---|
+| `fase-sporsmaal.json` | `{"questions": [{id, question, fasit: [{doc, quote}], release_only}]}` | the phase set, with a `release_only` class |
+| `holdout-sporsmaal.json` | the same shape | a blind hold-out |
+| `norske-sporsmaal.json` | `{"sporsmaal": {id: text}}` — the same ids, asked in plain Norwegian; the fasit is read from the phase set by id | the phase set's ids |
+| `delsporsmaal.json` | `{"delsporsmaal": {id: [...]}, "operator": {question, hit_rule, gold: [{doc, section}]}, "negative": {pass_rule, questions: [{id, question}]}}` | one list per id, the operator's question by two routes, known negatives |
+
+Each set carries its own `hit_rule`, and the gate implements it verbatim
+rather than restating it: an excerpt whose `source_file` matches a fasit
+entry's document **and** whose text contains that entry's quote (case folded,
+whitespace collapsed). Any one fasit entry suffices. The negative set carries
+a `pass_rule` instead, because a question the collection does not cover is
+passed by SAYING so, not by returning nothing.
+
+## The collection
+
+The collection is not a fixture here either — it is far too large to commit.
+Build one with today's code and point the gate at it. It is built with
+`--gate none` and otherwise at the shipped defaults: the sources are the
+vendor's own public documentation, and the screen is for material of unknown
+origin — at the default tier it refuses some of these pages outright, and every
+refused page would read here as a build failure the search never had a chance
+at:
+
+```sh
+okf project <sources> --out <scratch> --gate none
+python3 tools/okf_soek_gate.py --bundle <scratch>/.okf/<id>
+```
+
+`okf build <sources> --bundle <dir> --bundle-id <id> --okf-version 0.2
+--gate none` writes the same concept bytes.
+
+A gate measuring a collection nobody builds measures nothing. If the
+collection is missing the gate exits **2** with its reason rather than
+reporting 0 hits against nothing.