open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

test(fixtures): the STS fixtures and fixture codes are fictitious

Browse source 
Three STS fixtures still carried the section titles and labels of one real
reference document, and three identifiers were copies of its codes with a
letter or a word swapped. They now describe an invented kitchen counter and
cookbook series: the titles, labels and descriptions of sts-identity.xml,
sts-inherit.xml and sts-empty-label.xml, the P350/P351 document codes, the
99-0001 delivery prefix and chapter 7 of the image and accounting corpora.
Generated fixtures are regenerated and the witness inventory's per-document
totals are identical before and after; only names and text move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-23 14:50:44 +02:00
commit 88cf67f12e
65 changed files with 396 additions and 390 deletions
Download patch file Download diff file Expand all files Collapse all files

4
tests/fixtures/README.md vendored
View file

@ -17,7 +17,7 @@ and reproducible from that one file.
|---|---|
| `two-line-krav.pdf` | One heading plus one requirement row with label and value on the **same line**. That pairing is the property `pdfplumber` was chosen for. |
| `no-text-layer.pdf` | A structurally valid page with no text operators — the shape a scanned or image-only PDF presents. Must fail fast (`extractor_empty_pdf`), never persist as an empty concept. |
| `outline-collision.pdf` | **Two bookmarks whose destinations resolve to the same line** — the tree's root node and a front-matter node, both on line 0, which is the shape a 701-page reference standard carries. The marks are collected in a dict keyed on the line index, so without this fixture the second node is dropped with nothing counting it: 2 763 nodes in, 2 762 marks out, `unresolved` at 0. A one-bookmark-per-line fixture cannot see that. |
| `outline-collision.pdf` | **Two bookmarks whose destinations resolve to the same line** — the tree's root node and a front-matter node, both on line 0, which is the shape a long reference standard carries. The marks are collected in a dict keyed on the line index, so without this fixture the second node is dropped with nothing counting it: N + 2 nodes in (N its declared sections), N + 1 marks out, `unresolved` at 0. A one-bookmark-per-line fixture cannot see that. |
| `three-page-krav.pdf` | Three pages, one line of text each, and **the middle page carries no text operators**. The extractor drops empty pages, so the last page's text belongs to page 3 — which is what separates a page NUMBER from a count of the pages that produced text. Two pages could not tell those apart. |
## The XML fixtures
@ -31,7 +31,7 @@ round-trip-symmetric defect.
| Fixture | What it is for |
|---|---|
| `sts-mini.xml` | The known-positive. A `<standard>` root with `<sec>` at three nesting levels carrying `<label>`+`<title>`, two lettered points (`a)`, `b)`) with a **label and no title**, one `<table-wrap>` with a label and two rows, `<p>` bodies, a `<list>`, and **one unnumbered section** (`Forord`, `<title>` with no `<label>`) mirroring the single such section in that reference standard. The lettered points are what the 4 954 label-only `<sec>` in that document look like: promoted to headings they would bury its own 2 761. |
| `sts-mini.xml` | The known-positive. A `<standard>` root with `<sec>` at three nesting levels carrying `<label>`+`<title>`, two lettered points (`a)`, `b)`) with a **label and no title**, one `<table-wrap>` with a label and two rows, `<p>` bodies, a `<list>`, and **one unnumbered section** (`Forord`, `<title>` with no `<label>`) mirroring the single such section in that reference standard. The lettered points are what the label-only `<sec>` in that document look like -- most (64 %) of its `<sec>`: promoted to headings they would bury its own N titled sections. |
| `sts-empty-label.xml` | A `<sec>` carrying a `<label>` and **nothing else**, between a lettered point that has a body and the next titled section. The label is held as a prefix for a body line that never arrives, so it was overwritten and lost: measured on the reference standard that is exactly one `x)`, two characters of 1 283 395, ratio 0.999998. An exact invariant does not get to be 0.999998. |
| `sts-identity.xml` | A document that **states who it is**: exactly one `<std-ident>` with a `<doc-number>` and a `<year>`, one `<title-wrap>` whose `<full>` carries a **comma** (as the reference standard's does, which is why that title cannot be written into a `sources` flow mapping verbatim), and a `<std-ref type="dated">`. Its body carries the `sec-type="spec"` shape the `description` rule reads: a titled `<sec>` whose first spec point has one `<p>`, a second spec point that must never become the description, a titled child with no spec point of its own, and a spec point with **two** `<p>` of which only the first counts. `sts-mini.xml` is the half identity (a `<title-wrap>`, no `<doc-number>`) and `sts-empty-label.xml` the absent one. |
| `generic-feed.xml` | Known-negative: XML that is **not** STS. It must produce text and ONE plan — never zero, never a crash, and never element names promoted to headings. |