test(fixtures): replace sector-specific example material with generic, fictitious examples — green

Every fixture, test document, tool example and document now uses an invented
kitchen-and-baking handbook series, written in this repository. The package's
behaviour is unchanged; src/ changes are comments and help text only.

- Generated fixtures are regenerated from their generators. Their structural
  counts are identical before and after: elements, images, rows, cells,
  headings, bookmarks and the witness inventory's per-document totals. The
  image-inbox and accounting documents are renamed kapittel-84-*.
- tools/okf_accounting_gate.py: the two options that named one real corpus
  each are replaced by a generic, repeatable --corpus PATH with no default.
  Row 5 compares the PDF pair alone. Gate verdict unchanged: RED rows 2, 3, 6.
- tools/okf_witness.py: the STS JSON reader for one publisher's delivery is
  removed, along with its three twins and five tests. The mutation harness
  loses W09.
- docs/: 13 dated reports that documented runs on a retired reference corpus
  are removed, and 40 are neutralized. Dead links are removed, and no new
  dangling path is introduced.
- The synthetic MCP-gate corpus and the residual probe words are neutral.

Valgt: keep the `okf quality --fasit` bar value (the measured fraction, one corpus) and
rewrite only its provenance, because the verdict stays unchanged and the
number names nothing.

Term check with the local list: 0 of 411 tracked files, 0 file names, 0 of
27 binary fixtures. Suite after git add: 2457 passed, 1 skipped. The base
tree had 2460 passed and 2 skipped; five tests went with the JSON reader and
four were added by the term check. ruff, ruff format and mypy --strict src/
are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-23 13:54:57 +02:00
commit 9d1f4b14ed
174 changed files with 1889 additions and 6512 deletions

View file

@ -239,7 +239,7 @@ cd /tmp/c1-scratch && claude -p "<the question>"
|---|---|---|---|---|
| i | fresh, 54 concepts | "Hvordan skal prisene fylles ut?" | yes | **pass** |
| ii | fresh, 54 concepts | a topic measured absent from the bundle | yes | **pass** — refused |
| iii | third-party, 270 concepts | "Hva krever Krav 10.2—2 i N500? Gjengi det sentrale vilkåret." | **no** | **pass** |
| iii | third-party, 270 concepts | a question naming one requirement number and its document | **no** | **pass** |
| iv | fresh, 54 concepts | "Hvordan skal prisene fylles ut?" | **no** | **pass** — and selected the right one of two |
**(i) The known-positive.** Rank 1 of 8 delivered. The answer opened by naming
@ -261,7 +261,7 @@ used to check the delivered texts, and with the 46 withheld concepts named as
`[unverifiable-from-bundle]`.
**(iii) The third-party bundle.** Measured on the tree on disk at the time:
`vegnormal-n500-2024`, 270 concepts, ref
a third-party bundle, 270 concepts, ref
`sha256-tree:673a0c2c…d6b5b3ad` — **not** the V1 tree the order named, because
that repository landed a rebuild during the day. The gold concept came back at
rank 1, and the answer quoted the requirement verbatim against the concept body,
@ -312,21 +312,20 @@ denominator, corrected here rather than published.
## 5. A finding about the ranking, not about the skill
The first attempt at (ii) asked the normative bundle about water and frost
protection in a subsea tunnel. The model answered
The first attempt at (ii) asked the normative bundle about a compound topic
it was believed not to cover. The model answered
`[sourced-not-sufficient]`, said the relevant requirements "if they exist, are
among these 262 withheld", and refused to go around the cut. Checked
afterwards, **the bundle does cover it**: 17 files match `frostsikring`, 10 match
"vann- og frostsikring", 13 match `undersjøisk`, and none of them was ranked
into the cut.
afterwards, **the bundle does cover it**: 17, 10 and 13 files match the
question's three key terms, and none of them was ranked into the cut.
So it was never a known-negative — it was a **retrieval miss**, and it is
recorded as one rather than quietly replaced. Two things follow, and they point
in opposite directions. The consumer behaved exactly as the contract asks: it
did not fabricate, it named its denominator, and it reported its own zero as
unmeasured because the `withheld` entries carry only ids and rules, no titles.
The ranker did not: a compound Norwegian question ("vann- og frostsikring",
"undersjøisk") was out-ranked by concepts sharing only "tunnel" and "vann". That
The ranker did not: a compound Norwegian question was out-ranked by concepts sharing only two
common words with it. That
belongs with the fusion work already open, and it is the same shape as the
`MIN_SHARED_PREFIX` finding.