open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

test(fixtures): replace sector-specific example material with generic, fictitious examples — green

Browse source 
Every fixture, test document, tool example and document now uses an invented
kitchen-and-baking handbook series, written in this repository. The package's
behaviour is unchanged; src/ changes are comments and help text only.

- Generated fixtures are regenerated from their generators. Their structural
  counts are identical before and after: elements, images, rows, cells,
  headings, bookmarks and the witness inventory's per-document totals. The
  image-inbox and accounting documents are renamed kapittel-84-*.
- tools/okf_accounting_gate.py: the two options that named one real corpus
  each are replaced by a generic, repeatable --corpus PATH with no default.
  Row 5 compares the PDF pair alone. Gate verdict unchanged: RED rows 2, 3, 6.
- tools/okf_witness.py: the STS JSON reader for one publisher's delivery is
  removed, along with its three twins and five tests. The mutation harness
  loses W09.
- docs/: 13 dated reports that documented runs on a retired reference corpus
  are removed, and 40 are neutralized. Dead links are removed, and no new
  dangling path is introduced.
- The synthetic MCP-gate corpus and the residual probe words are neutral.

Valgt: keep the `okf quality --fasit` bar value (the measured fraction, one corpus) and
rewrite only its provenance, because the verdict stays unchanged and the
number names nothing.

Term check with the local list: 0 of 411 tracked files, 0 file names, 0 of
27 binary fixtures. Suite after git add: 2457 passed, 1 skipped. The base
tree had 2460 passed and 2 skipped; five tests went with the JSON reader and
four were added by the term check. ruff, ruff format and mypy --strict src/
are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-23 13:54:57 +02:00
commit 9d1f4b14ed
174 changed files with 1889 additions and 6512 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/plan/references-cue-rule.md
View file

@ -48,7 +48,7 @@ markdown link target, or a number immediately preceded by a reference cue from
a closed set.** Everything else number-shaped in body prose is not a reference.
The rule is positive rather than a blacklist, and that is forced by the data,
not chosen for taste: corpus A proves `V221` and `V240` are genuine document
not chosen for taste: corpus A proves `W221` and `W240` are genuine document
numbers, corpus B proves `V0.3.0` is a software version, and the two are the
same token shape. Nothing structural separates them. Only the words in front of
them do, so only a positive cue can be defended.
@ -98,12 +98,12 @@ reader, a false one is not.
Two recall losses inside the corpora measured, both accepted: a reference whose
cue is a preposition (`i henhold til …` where the noun is absent) and one
written as a prefixed compound (`N-V240`) are dropped.
written as a prefixed compound (`Q-W240`) are dropped.
## What was deliberately not done
**The prefix-resolution rule (`N130` in a body not finding a document numbered
`N130.2`) stays open.** The order allowed it in the same pass only if the data
**The prefix-resolution rule (`Q130` in a body not finding a document numbered
`Q130.2`) stays open.** The order allowed it in the same pass only if the data
bore it. It does not: of 2 589 documents across both corpora, 2 562 carry a
number and **0** of those numbers are dotted, so a unique-prefix match has
nothing here to be defended against. Closing two things badly is worse than