open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

test(fixtures): replace sector-specific example material with generic, fictitious examples — green

Browse source 
Every fixture, test document, tool example and document now uses an invented
kitchen-and-baking handbook series, written in this repository. The package's
behaviour is unchanged; src/ changes are comments and help text only.

- Generated fixtures are regenerated from their generators. Their structural
  counts are identical before and after: elements, images, rows, cells,
  headings, bookmarks and the witness inventory's per-document totals. The
  image-inbox and accounting documents are renamed kapittel-84-*.
- tools/okf_accounting_gate.py: the two options that named one real corpus
  each are replaced by a generic, repeatable --corpus PATH with no default.
  Row 5 compares the PDF pair alone. Gate verdict unchanged: RED rows 2, 3, 6.
- tools/okf_witness.py: the STS JSON reader for one publisher's delivery is
  removed, along with its three twins and five tests. The mutation harness
  loses W09.
- docs/: 13 dated reports that documented runs on a retired reference corpus
  are removed, and 40 are neutralized. Dead links are removed, and no new
  dangling path is introduced.
- The synthetic MCP-gate corpus and the residual probe words are neutral.

Valgt: keep the `okf quality --fasit` bar value (the measured fraction, one corpus) and
rewrite only its provenance, because the verdict stays unchanged and the
number names nothing.

Term check with the local list: 0 of 411 tracked files, 0 file names, 0 of
27 binary fixtures. Suite after git add: 2457 passed, 1 skipped. The base
tree had 2460 passed and 2 skipped; five tests went with the JSON reader and
four were added by the term check. ruff, ruff format and mypy --strict src/
are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-23 13:54:57 +02:00
commit 9d1f4b14ed
174 changed files with 1889 additions and 6512 deletions
Download patch file Download diff file Expand all files Collapse all files

85
CHANGELOG.md
View file

@ -230,7 +230,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **Every carried image is now one a model can be SHOWN, and the ones that
cannot be are refused out loud.** Until this round the asset path carried
whatever format a publisher shipped. Measured 2026-09-19 over the frozen
R761 delivery's own `assets/` (denominator 50): 29 JPEG, 2 PNG and **19 "PC
reference standard's own `assets/` (denominator 50): 29 JPEG, 2 PNG and **19 "PC
bitmap, Windows 3.x, 8-bit, compression 1"** — RLE8 BMP. The 19 are
byte-correct files nothing reads, so 19 of that document's figures were
present and invisible at once, with `images: N` reporting that they had
@ -249,7 +249,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
bundle's identity move with that library's version — the property 0.10.0
felled page rasterisation over. Pillow is the INDEPENDENT decoder in the
tests instead.
- **Lossless, measured on the real files:** all **19 of 19** R761 RLE8
- **Lossless, measured on the real files:** all **19 of 19** reference-standard RLE8
assets convert with RGB identical to Pillow's decoding of the source,
**2 366 365 pixels** compared.
- **Traceability per converted asset**, on the pointer line where the rest
@ -279,9 +279,9 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
COUNTED** (operator decision 2026-09-18). `llm-ingestion-guard` 1.4.0 keeps
the soft hyphen in `_ZERO_WIDTH_CPS`, and `output:zero-width-present` is an
any-tier carrier — `fail_secure` at every trust level, no sanitisation, no
exception. R761 Prosesskoden:2025 carries 71 of them and 0 of the four real
exception. A 701-page reference standard carries 71 of them and 0 of the four real
zero-width characters; all 71 are Norwegian hyphenation points inside words,
so a 701-page process code was unreadable for the whole chain over
so a 701-page reference standard was unreadable for the whole chain over
typography. `extract.normalise_extracted` removes that one character from
every extracted text; `ExtractedDocument.soft_hyphens`,
`InboxResult.normalised` and the accounting's `normalised_soft_hyphen` carry
@ -309,7 +309,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **The judge is `tools/okf_accounting_gate.py`**, written red first
against an independent witness (`tools/okf_witness.py`, which imports
nothing from this package). At this change it is green on all six rows,
including R761 Prosesskoden:2025: 110 of 110 units under both the default
including a 701-page reference standard: 110 of 110 units under both the default
gate and `--gate none`.
- **Opt-in, measured.** On the 43-document reference corpus the build took
+744 s (+19 %) and +0.53 GB peak RSS.
@ -330,7 +330,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
number, because no witness counts a heading in a PDF.
- **Limit, measured:** the check proves that a string is present, not
where. Short elements such as a section label or a one-word title are
often found elsewhere in the same document. With R761's concept text cut
often found elsewhere in the same document. With the reference standard's concept text cut
to half, 4 823 paragraphs and 3 621 sections were reported lost, but only
3 titles and 16 labels.
@ -339,7 +339,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **`okf build` exits 1 when it extracted at least one document and
persisted none.** Until now such a run exited 0, because every refusal was
coded and the conservation identity held. The bundle was nonetheless empty.
Measured case: guard 1.4.0 refuses R761 Prosesskoden:2025 whole, because of
Measured case: guard 1.4.0 refuses a 701-page reference standard whole, because of
its 71 soft hyphens (U+00AD). Door B's library function
(`process_inbox`) and `corpus.measure` are unchanged; for a hostile inbox,
"all rejected" is a correct outcome.
@ -350,14 +350,14 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **A file carried through a document is no longer also a coded rejection.**
Since 0.10.0, an image beside a document was carried into `assets/` through
that document and was ALSO counted as `extractor_unknown`, so one file had
two fates. On R761 under `--gate none` that was 50 files.
two fates. On the reference standard under `--gate none` that was 50 files.
- The conservation identity is now `merged + files carried through a
document + coded rejections = N`.
- `log.md` writes the middle term only when it is non-zero, so a corpus with
no such files keeps its line byte for byte.
- The carried files are the references the reader actually resolved and
carried (`ExtractedDocument.files`), never a byte match. A byte match
would credit R761's 7 unpointed duplicates.
would credit the reference standard's 7 unpointed duplicates.
- An unpointed file beside a document stays a coded rejection.
- **`log.md`'s `Images: C carried of F found`**: with `--accounting`, F is
what the SOURCES declare. A refused document's pictures therefore no longer
@ -512,8 +512,8 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
becoming true or one concept ranking better. An eval written before the
capability has one job beyond being red today, so the gate was repaired
before anything is built against it. `src/` is untouched.
- **Row 8 requires all three named sets** (`wiki-20`, `r761-sk2`,
`vegnormal-32`) and is NOT RUN otherwise. It counted whatever `--real`
- **Row 8 requires all three named sets** (`wiki-20` and two sets built on a
retired reference corpus) and is NOT RUN otherwise. It counted whatever `--real`
gave it, so one set of three read `6 of 6 GREEN` — and this repository's
own test asserted `(1, 1, GREEN)` for a single set. The numbers the run
DID measure are still printed: a missing set must not cost the reader the
@ -561,8 +561,8 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
one it stays RED rather than NOT RUN: its denominator is known.
- **Row 8 ran, against all three real sets**: **44 of 64 questions**, 7 of
29 at citation granularity and 38 of 50 at concept granularity, 33 of 34
misses class b. wiki (6 of 20) and r761 (7 of 7) reproduce PM's recorded
figures exactly; vegnormal measures 31 of 43 citations where PM recorded
misses class b. wiki (6 of 20) and the first retired set (7 of 7) reproduce PM's recorded
figures exactly; the second retired set measures 31 of 43 citations where PM recorded
32, a one-citation disagreement between two instruments over the same
pinned bytes, stated and not resolved here.
- Rows 1 and 6 go 9 of 9 to 10 of 10 (one added fixture, one added hit).
@ -595,7 +595,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
reading the gate had before the conversion route existed.
- Measured: the three arms PM reproduced go forged → refused, 3 of 3, with
the known-positive (a BMP the run really does convert) True in all three.
The text-level regression guard goes 3 arms to 13. R761 rebuilt is
The text-level regression guard goes 3 arms to 13. The reference standard rebuilt is
`diff -r`-identical, 50 assets (29 JPEG + 21 PNG), 19 of 19 conversions
confirmed against 19 declared, SHY 71, u = 0, d = 0.
- **An RLE8 stream that stops before the frame is refused (0.10.1).** The
@ -609,7 +609,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
end-of-line escape state their skip, so the pixels they pass over keep
index 0 and every decoder produces the same picture; a pixel-coverage
count would refuse both constructions the format defines. The corpus
cannot choose between the two rules — over the 25 RLE8 BMPs the R761
cannot choose between the two rules — over the 25 RLE8 BMPs the reference
delivery ships, 25 of 25 paint every pixel, 25 of 25 reach the end of the
frame and 0 of 25 use a delta — and an independent decoder can: Pillow
reads 5 of the 8 streams in the table and refuses the same 3 the new rule
@ -655,7 +655,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
`unaccounted` or `double_booked`, which the door had written for a round
longer. A door built from the contract writes a ledger the gate reads as
"nothing was converted", and every converted image comes out
claimed-and-not-found — 19 of 50 on R761. Two tests hold the sketch against
claimed-and-not-found — 19 of 50 on the reference standard. Two tests hold the sketch against
both sides: what the gate LOOKS UP (measured with a ledger that records its
own lookups, not by grep) and what the door SERIALISES.
- **A bundle built without the door now says why a converted image cannot be
@ -694,7 +694,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
(0.10.1).** `_bmp_rle8_rows` painted what the stream held and left the rest
of the frame at palette index 0 — which is what the format says about a
SKIPPED pixel, so no decoder disagreed and the picture was wrong with no
code and no row. Measured by PM 2026-09-19 on a real R761 asset (352x548 =
code and no row. Measured by PM 2026-09-19 on a real reference-standard asset (352x548 =
192 896 pixels): cut to 90 % it was carried with 13 923 pixels wrong, to
50 % with 95 890, to 10 % with 166 525. The uncompressed path already
refused the same shape.
@ -705,7 +705,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- The rule is the terminator rather than `biSizeImage` (a claim by the same
untrusted header) or a coverage count (which would refuse the delta escape
the format defines), and it is read off the corpus: over the 19 real RLE8
assets of the frozen R761 delivery, **19 of 19** end at an explicit
assets of the frozen reference delivery, **19 of 19** end at an explicit
end-of-bitmap, on **19 of 19** it is the stream's last two bytes, and on
**19 of 19** `biSizeImage` equals the available bytes. A whole stream that
omits the terminator is refused alongside a cut one.
@ -723,8 +723,8 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
in the note.
- **The STS JSON role map reads the publisher's own tags.** `count_sts_json`
compared the raw tag string where the XML witness has always used `_local`,
so `mml:math` reached `tag == "math"` on nothing — 74 formulas in N200
Vegbygging:2024 counted as 0. And the publisher's JSON writes a figure's
so `mml:math` reached `tag == "math"` on nothing — 74 formulas in a second
regulatory reference document counted as 0. And the publisher's JSON writes a figure's
caption as `figcaption` under the `graphic`, not as the `fig/caption`
NISO-STS writes — 430 of them over the eight deliveries measured. No other
count moves, measured role by role over those eight and the committed twins.
@ -735,7 +735,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **The skipped-row guard measures the machine, not the argument.** Row 6 is
SKIPPED exactly when the corpora the arguments name are absent, so asking the
arguments made the branch unreachable.
- **Row 6 says when a corpus measures no element class at all.** On N200 the
- **Row 6 says when a corpus measures no element class at all.** On that second document the
build proposes 0 plans and exits 2 before the accounting door, so 16 549
elements land as unaccounted with no declared fate — a finding about the run
that read as a finding about the build.
@ -749,7 +749,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **Three sentences this release publishes are now held by tests.** PM's
checkpoint on `43331fc` found three mutants surviving the entire suite: a
normalisation door that ALSO removes U+00A0 NBSP — which would have eaten
all 6 633 of them in R761 while `log.md` went on saying "No other character
all 6 633 of them in the reference standard while `log.md` went on saying "No other character
is touched" — and row 3 of the accounting gate losing either its
`refused={n}` column or its "N element(s) lost with R of D document(s)
refused whole" clause. Each is killed now by a test that counts its own
@ -764,7 +764,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
the source bytes would disagree by construction on every type extraction
does not carry verbatim, and the gate would have to decide which difference
was a loss.
- **Two published numbers were wrong and are corrected.** N101 ships **568**
- **Two published numbers were wrong and are corrected.** A third reference document ships **568**
`tbx:` tags, not 574 — three independent counts agree (raw substring, regex
over the JSON `tag` field, node traversal), and a test now counts them over
the delivery instead of repeating the number in a fourth docstring. And the
@ -806,9 +806,9 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
a PDF was opened for its text alone, the converter's markdown writer dropped
every picture, and the only writer into a bundle took `content: str`. The two
lossiness warnings said so on every run, which made the loss honest and did
not make it smaller. Measured on R761 Prosesskoden:2025: the process text is
not make it smaller. Measured on a 701-page reference standard: the section text is
carried in full while 12 `Tabell N-N` and 9 `Figur N-N` captions stand over
nothing, so process 84's "toleranseklasse ... er gitt i tabell 84-2" points
nothing, so a section's "is given in table 84-2" points
at empty space.
**Five readers place, one module decides.** `pdf` (embedded image XObjects),
`docx`/`pptx`/`odt`/`rtf` (the converter's media, through `--extract-media`),
@ -945,7 +945,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
not the bundle. The bar rests on **one product**, and the run says so on every
boundary row. `docs/2026-09-12-g37-terskler.md` SS 7 carries the premises
re-measured, the seven bundles, the interval any bar could sit in, and the two
R761 builds this one fails.
reference-standard builds this one fails.
### Unchanged
@ -959,7 +959,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **`okf quality <bundle>` -- a per-file-type verdict, with the denominator
(G37).** `okf check` is a CONTRACT check, and a green one is not a quality
gate: measured 2026-09-10 by `vegnormal-okf`, three arms over one corpus all
gate: measured 2026-09-10 by a downstream consumer repository, three arms over one corpus all
returned 0 findings and exit 0 while their hit@k ranged from 6 of 6 to 0 of 6.
The new command asks the other question. Three verdicts and no fourth --
`PASS`, `FAIL`, `UNMEASURED` -- and a type with no measured threshold is never
@ -1014,7 +1014,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
parameter, `link_in_signal`, default `True`, no CLI flag, no payload moved:
it is the instrument that separates a ranking movement from a budget
displacement on ONE bundle, and the measurement it was built for is
`docs/2026-09-12-k3-runde23-stien-i-kroppssignalet.md`. Measured on R761
`docs/2026-09-12-k3-runde23-stien-i-kroppssignalet.md`. Measured on the reference standard
(2 761 concepts, 710 heading-only sections, 675 carrying the line): of the
**39** newly delivered concepts the line ever added a question token to,
**39** gained it from the bundle-absolute PATH and **0** from the link's
@ -1175,12 +1175,12 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
(K3-22).** SPEC § 11 point 1 requires "a parseable YAML frontmatter block" in
every file. Measured with PyYAML 6.0.3 before the change, the pinned K2
default bundle failed `safe_load` on 41 of 455 blocks (and a 42nd parsed to
a truncated title), and each R761 build on 1 -- every one a block scalar
a truncated title), and each reference-standard build on 1 -- every one a block scalar
written verbatim: a title with `": "` or `" #"`, a leading `- `, `*` or `**`,
a trailing `:`.
- **Block scalars:** a value K3-19's plain-scalar rule refuses is written
double-quoted, `\` and `"` escaped; every other value keeps its bytes.
Rebuilt, the five-document project moves 0 files, each R761 build 1 line
Rebuilt, the five-document project moves 0 files, each reference-standard build 1 line
and the K2 default bundle 42 `title` lines, after which all 454 of its
frontmatters parse and read back the same.
- **Flow leaves (`sources`, Door A's list, a run-stated flow value):** the
@ -1191,7 +1191,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
`source_reference_unquotable`, `run_frontmatter_invalid`).
- **Behaviour change:** `okf build --frontmatter 'sources=[{ resource: <URL
with a query string>, … }]'` now exits 2 and writes nothing. K3-19's own
flagged R761 build used such a URL and wrote 2 761 of 2 761 frontmatters
flagged reference-standard build used such a URL and wrote 2 761 of 2 761 frontmatters
PyYAML refuses.
- **Readers** (`parse_frontmatter`, the index and structure readers, both
`read_sources` branches) unquote a `"`-wrapped value; `'`-wrapped values
@ -1199,7 +1199,6 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
exactly what they returned before.
- The generated `SKILL.md` header goes through the same block rule.
- PyYAML joins the `dev` dependency group only; `src/` imports no yaml.
- Report: `docs/2026-09-11-k3-runde22-yaml-lesbar-frontmatter.md`.
## [0.8.3] — 2026-09-11
@ -1210,7 +1209,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
ancestor that holds text: the nearest preceding plan entry at a smaller
level, passing over an ancestor that is empty too. Nothing is copied and no
boundary moves. It reads the plan's level and order, never the row. Measured
on one process code, 710 of 2 761 concepts are heading-only; the route names
on one reference standard, 710 of 2 761 concepts are heading-only; the route names
the ancestor the document's own nesting names on 708 of them (two sit at
depth 7, which a markdown heading clips to 6), where reading section numbers
gets 686.
@ -1299,8 +1298,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
a title.
`--ingested-at` alone was confirmed to stamp every concept, on the segmented
route too, and to date `log.md`: 2 761 of 2 761. Report:
[`docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md`](docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md).
route too, and to date `log.md`: 2 761 of 2 761.
## [0.8.2] — 2026-09-11
@ -1426,8 +1424,8 @@ route too, and to date `log.md`: 2 761 of 2 761. Report:
section titled with that subject PLUS a qualifier scores, and then loses on
the body count. Decomposed per miss: S1 turns on `hvordan`, an interrogative
pronoun; S5 on `hvilke` and `stilles` in a body 7x the gold's, and on
`betonghvelv ~ betongkonstruksjoner` through the four-character stem
`betong`; S6 on an exact tie broken by `concept_id`.
a compound pair sharing a six-character first
element; S6 on an exact tie broken by `concept_id`.
- **A PARTITION, never a fourth RRF signal, and the arithmetic is why.** RRF
consumes ranks alone, so with shared ranks a rule whose positive group has
`m` members is worth `1/61 - 1/(61 + m)` -- a rule firing on ONE concept of
@ -1448,11 +1446,11 @@ route too, and to date `log.md`: 2 761 of 2 761. Report:
known-positive from 1 to 4 every time it does.
- **NOTHING ELSE MOVES AND IT IS MEASURED ON THE BYTES.** The pinned K2 bundle
keeps `(1,1,1,1,1,5)` and its 7 pin tests, Arm B keeps `(1,1,1,1,1,5)`, and
the payloads on both are **byte-identical on 6 of 6 questions**; n100/n200/
n500 payloads are byte-identical on 5 questions each; the 828-file HTML
the payloads on both are **byte-identical on 6 of 6 questions**; the three
downstream bundles' payloads are byte-identical on 5 questions each; the 828-file HTML
corpus still gives 828 plans, 0 unreadable and 6 015 md with `diff -rq`
empty; the five-document folder is `diff -r`-identical at 26 concepts / 52
md; `okf project` stays byte-equal to `okf build`. hit@k on n100/n200/n500
md; `okf project` stays byte-equal to `okf build`. hit@k on those three bundles
is **NOT MEASURED** -- this repository holds no gold set for them, which is
0 gold sets and not 0 hits.
- **THIS IS THE FOURTH READING-SIDE CHANGE THAT MOVES A PAYLOAD WITH NO BUNDLE
@ -1554,7 +1552,6 @@ route too, and to date `log.md`: 2 761 of 2 761. Report:
a success. Separating "0 plans, 0 unreadable" from "0 plans because nothing
could be read" would change the outcome on **0 of the 4** reference
corpora, so it is not separated.
- Report: `docs/2026-09-11-k3-runde13-xml-sts.md`.
### Fixed
@ -1569,7 +1566,7 @@ route too, and to date `log.md`: 2 761 of 2 761. Report:
- **`--pdf-outline` (OFF): cut a PDF at the boundaries its own `/Outlines`
bookmark tree declares.** Measured outside this repository on one 701-page
process code whose publisher also ships a NISO-STS structure for it: the
reference standard whose publisher also ships a NISO-STS structure for it: the
shipped default recovers **1 967 of 2 761** titled sections, **0 of its 28**
chapters, and **794 of 794** misses have their heading text present in the
extracted text -- the line was read, the boundary was never opened. The same
@ -1811,7 +1808,7 @@ subcommand to five, and one build default moved.
block form. Measured before the bump so the test discriminates rather than
merely passes, and pinned by
`tests/test_guard_adapter.py::test_the_guard_parses_the_flow_form_sources_our_goldens_emit`.
Two rows of the gate table in `docs/okf-nokkelinventar.md` moved, not one:
Two rows of the gate table moved, not one:
the BLOCK form of `sources` now passes too, which retires G30 -- though it
changes nothing about what we emit, since our own parser is still the binding
constraint. `resource` is allowlisted only inside a `sources` entry, so
@ -1891,7 +1888,7 @@ subcommand to five, and one build default moved.
dependency, still stdlib otherwise — and a packaging test enforces that.
**The parser choice was forced by a measurement, not by preference**
(`docs/2026-08-21-g2-pdf-extraction-measurement.md`). On a real Vegnormalene
(`docs/2026-08-21-g2-pdf-extraction-measurement.md`). On a real regulatory
requirement table, `pdfplumber` keeps 4 of 4 rows with label and value on the
same line; `pypdf`, `pdfminer.six` and `pymupdf` each keep 0 of 4, emitting
all labels and then all values. A downstream reader can only re-pair those by