open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

test(fixtures): the STS fixtures and fixture codes are fictitious

Browse source 
Three STS fixtures still carried the section titles and labels of one real
reference document, and three identifiers were copies of its codes with a
letter or a word swapped. They now describe an invented kitchen counter and
cookbook series: the titles, labels and descriptions of sts-identity.xml,
sts-inherit.xml and sts-empty-label.xml, the P350/P351 document codes, the
99-0001 delivery prefix and chapter 7 of the image and accounting corpora.
Generated fixtures are regenerated and the witness inventory's per-document
totals are identical before and after; only names and text move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-23 14:50:44 +02:00
commit 88cf67f12e
65 changed files with 396 additions and 390 deletions
Download patch file Download diff file Expand all files Collapse all files

4
tests/fixtures/README.md vendored
View file

@ -17,7 +17,7 @@ and reproducible from that one file.
|---|---|
| `two-line-krav.pdf` | One heading plus one requirement row with label and value on the **same line**. That pairing is the property `pdfplumber` was chosen for. |
| `no-text-layer.pdf` | A structurally valid page with no text operators — the shape a scanned or image-only PDF presents. Must fail fast (`extractor_empty_pdf`), never persist as an empty concept. |
| `outline-collision.pdf` | **Two bookmarks whose destinations resolve to the same line** — the tree's root node and a front-matter node, both on line 0, which is the shape a 701-page reference standard carries. The marks are collected in a dict keyed on the line index, so without this fixture the second node is dropped with nothing counting it: 2 763 nodes in, 2 762 marks out, `unresolved` at 0. A one-bookmark-per-line fixture cannot see that. |
| `outline-collision.pdf` | **Two bookmarks whose destinations resolve to the same line** — the tree's root node and a front-matter node, both on line 0, which is the shape a long reference standard carries. The marks are collected in a dict keyed on the line index, so without this fixture the second node is dropped with nothing counting it: N + 2 nodes in (N its declared sections), N + 1 marks out, `unresolved` at 0. A one-bookmark-per-line fixture cannot see that. |
| `three-page-krav.pdf` | Three pages, one line of text each, and **the middle page carries no text operators**. The extractor drops empty pages, so the last page's text belongs to page 3 — which is what separates a page NUMBER from a count of the pages that produced text. Two pages could not tell those apart. |
## The XML fixtures
@ -31,7 +31,7 @@ round-trip-symmetric defect.
| Fixture | What it is for |
|---|---|
| `sts-mini.xml` | The known-positive. A `<standard>` root with `<sec>` at three nesting levels carrying `<label>`+`<title>`, two lettered points (`a)`, `b)`) with a **label and no title**, one `<table-wrap>` with a label and two rows, `<p>` bodies, a `<list>`, and **one unnumbered section** (`Forord`, `<title>` with no `<label>`) mirroring the single such section in that reference standard. The lettered points are what the 4 954 label-only `<sec>` in that document look like: promoted to headings they would bury its own 2 761. |
| `sts-mini.xml` | The known-positive. A `<standard>` root with `<sec>` at three nesting levels carrying `<label>`+`<title>`, two lettered points (`a)`, `b)`) with a **label and no title**, one `<table-wrap>` with a label and two rows, `<p>` bodies, a `<list>`, and **one unnumbered section** (`Forord`, `<title>` with no `<label>`) mirroring the single such section in that reference standard. The lettered points are what the label-only `<sec>` in that document look like -- most (64 %) of its `<sec>`: promoted to headings they would bury its own N titled sections. |
| `sts-empty-label.xml` | A `<sec>` carrying a `<label>` and **nothing else**, between a lettered point that has a body and the next titled section. The label is held as a prefix for a body line that never arrives, so it was overwritten and lost: measured on the reference standard that is exactly one `x)`, two characters of 1 283 395, ratio 0.999998. An exact invariant does not get to be 0.999998. |
| `sts-identity.xml` | A document that **states who it is**: exactly one `<std-ident>` with a `<doc-number>` and a `<year>`, one `<title-wrap>` whose `<full>` carries a **comma** (as the reference standard's does, which is why that title cannot be written into a `sources` flow mapping verbatim), and a `<std-ref type="dated">`. Its body carries the `sec-type="spec"` shape the `description` rule reads: a titled `<sec>` whose first spec point has one `<p>`, a second spec point that must never become the description, a titled child with no spec point of its own, and a spec point with **two** `<p>` of which only the first counts. `sts-mini.xml` is the half identity (a `<title-wrap>`, no `<doc-number>`) and `sts-empty-label.xml` the absent one. |
| `generic-feed.xml` | Known-negative: XML that is **not** STS. It must produce text and ONE plan — never zero, never a crash, and never element names promoted to headings. |

2
tests/fixtures/accounting/corpus/bilde.rtf vendored
View file

@ -1 +1 @@
{\rtf1\ansi\deff0{\fonttbl{\f0 Times New Roman;}}\pard Figur 84-1 viser prinsippet.\par\pard{\pict\pngblip\picw16\pich16 89504e470d0a1a0a}\par}
{\rtf1\ansi\deff0{\fonttbl{\f0 Times New Roman;}}\pard Figur 7-1 viser prinsippet.\par\pard{\pict\pngblip\picw16\pich16 89504e470d0a1a0a}\par}

6
tests/fixtures/accounting/corpus/figur.html vendored
View file

@ -1,10 +1,10 @@
<!DOCTYPE html>
<html lang="no">
<head><title>Figur 84-1</title></head>
<head><title>Figur 7-1</title></head>
<body>
<h1>Figur 84-1</h1>
<h1>Figur 7-1</h1>
<p>Prinsippet for hevetidsklasser.</p>
<img src="graphics/figur-84-1.png" alt="Prinsippskisse">
<img src="graphics/figur-7-1.png" alt="Prinsippskisse">
<table><tr><th>Klasse</th><th>Avvik</th></tr><tr><td>A</td><td>5 min</td></tr></table>
<ul><li>Klasse A</li><li>Klasse B</li></ul>
</body>

0
tests/fixtures/accounting/corpus/graphics/figur-84-1.png → tests/fixtures/accounting/corpus/graphics/figur-7-1.png vendored
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 94 B

After

Width:  |  Height:  |  Size: 94 B

Before After
Before After

0
tests/fixtures/accounting/corpus/graphics/tabell-84-2.png → tests/fixtures/accounting/corpus/graphics/tabell-7-2.png vendored
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 122 B

After

Width:  |  Height:  |  Size: 122 B

Before After
Before After

BIN
tests/fixtures/accounting/corpus/kapittel-7-notat.docx vendored Normal file
View file

Binary file not shown.

0
tests/fixtures/accounting/corpus/kapittel-84-presentasjon.pptx → tests/fixtures/accounting/corpus/kapittel-7-presentasjon.pptx vendored
View file

18
tests/fixtures/accounting/corpus/kapittel-7-sts.xml vendored Normal file
View file

@ -0,0 +1,18 @@
<?xml version="1.0" encoding="UTF-8"?>
<standard xmlns:xlink="http://www.w3.org/1999/xlink">
<front><std-ident><doc-number>P350</doc-number></std-ident></front>
<body>
<sec>
<label>7</label>
<title>Brød og boller</title>
<sec>
<label>7.1</label>
<title>Hevetider</title>
<p>Hevetidsklasse er gitt i tabell 7-2.</p>
<graphic xlink:href="graphics/tabell-7-2.png"/>
<p>Figur 7-1 viser prinsippet.</p>
<graphic xlink:href="figur-7-1.png"/>
</sec>
</sec>
</body>
</standard>

BIN
tests/fixtures/image-inbox/kapittel-84-tabell.pdf → tests/fixtures/accounting/corpus/kapittel-7-tabell.pdf vendored
View file

Binary file not shown.

14
tests/fixtures/accounting/corpus/kapittel-7-web.html vendored Normal file
View file

@ -0,0 +1,14 @@
<!doctype html>
<html><head><title>Kapittel 7</title></head>
<body>
<h1>7 Brød og boller</h1>
<p>Hevetidsklasse for de enkelte deigtyper er gitt i tabell 7-2.</p>
<img src="graphics/tabell-7-2.png" alt="Tabell 7-2 Hevetidsklasser">
<p>Figuren under viser prinsippet.</p>
<figure>
<img src="graphics/figur-7-1.png" alt="Figur 7-1 Prinsippskisse">
<figcaption>Figur 7-1 Prinsippskisse</figcaption>
</figure>
<p>Og en som ligger et annet sted:</p>
<img src="https://example.invalid/ekstern.png" alt="Ekstern figur">
</body></html>

BIN
tests/fixtures/accounting/corpus/kapittel-84-notat.docx vendored
View file

Binary file not shown.

18
tests/fixtures/accounting/corpus/kapittel-84-sts.xml vendored
View file

@ -1,18 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<standard xmlns:xlink="http://www.w3.org/1999/xlink">
<front><std-ident><doc-number>P761</doc-number></std-ident></front>
<body>
<sec>
<label>84</label>
<title>Brød og boller</title>
<sec>
<label>84.1</label>
<title>Hevetider</title>
<p>Hevetidsklasse er gitt i tabell 84-2.</p>
<graphic xlink:href="graphics/tabell-84-2.png"/>
<p>Figur 84-1 viser prinsippet.</p>
<graphic xlink:href="figur-84-1.png"/>
</sec>
</sec>
</body>
</standard>

14
tests/fixtures/accounting/corpus/kapittel-84-web.html vendored
View file

@ -1,14 +0,0 @@
<!doctype html>
<html><head><title>Kapittel 84</title></head>
<body>
<h1>84 Brød og boller</h1>
<p>Hevetidsklasse for de enkelte deigtyper er gitt i tabell 84-2.</p>
<img src="graphics/tabell-84-2.png" alt="Tabell 84-2 Hevetidsklasser">
<p>Figuren under viser prinsippet.</p>
<figure>
<img src="graphics/figur-84-1.png" alt="Figur 84-1 Prinsippskisse">
<figcaption>Figur 84-1 Prinsippskisse</figcaption>
</figure>
<p>Og en som ligger et annet sted:</p>
<img src="https://example.invalid/ekstern.png" alt="Ekstern figur">
</body></html>

BIN
tests/fixtures/accounting/corpus/liste-og-bilde.odt vendored
View file

Binary file not shown.

4
tests/fixtures/accounting/corpus/mengder.csv vendored
View file

@ -1,3 +1,3 @@
Post,Mengde,Enhet
84.1,120,m3
84.2,45,m2
7.1,120,m3
7.2,45,m2

1 Post Mengde Enhet
2 84.1 7.1 120 m3
3 84.2 7.2 45 m2

2
tests/fixtures/accounting/corpus/notat.md vendored
View file

@ -11,7 +11,7 @@ Vask skjer to ganger i året.
| Øverst | 6 mnd |
| Nederst | 12 mnd |
![Figur 1 Vaskeplan](graphics/tabell-84-2.png)
![Figur 1 Vaskeplan](graphics/tabell-7-2.png)
```bash
# ikke en overskrift

BIN
tests/fixtures/accounting/corpus/notater-og-skjult.pptx vendored
View file

Binary file not shown.

2
tests/fixtures/accounting/corpus/side.htm vendored
View file

@ -3,7 +3,7 @@
<body>
<h1>Ovnen</h1>
<p>Steketemperaturen er 145 grader.</p>
<img src="graphics/tabell-84-2.png" alt="Hyllekart">
<img src="graphics/tabell-7-2.png" alt="Hyllekart">
<h2>Hyller</h2>
<ul><li>Øverst</li><li>Nederst</li></ul>
<table><tr><th>Hylle</th><th>Temperatur</th></tr><tr><td>Øverst</td><td>180 C</td></tr></table>

4
tests/fixtures/accounting/corpus/sts-rikt.xml vendored
View file

@ -1,11 +1,11 @@
<standard>
<front><std-ident><doc-number>P762</doc-number><year>2025</year></std-ident></front>
<front><std-ident><doc-number>P351</doc-number><year>2025</year></std-ident></front>
<body>
<sec><label>85</label><title>Bakeplater</title>
<p>Platen skal ha jevn varme etter <mixed-citation>EKS 1234-1:2030</mixed-citation>.</p>
<p>Kravet regnes som <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>T</mml:mi><mml:mo>&lt;</mml:mo><mml:mn>2</mml:mn></mml:math>.</p>
<fig><label>Figur 85-1</label><caption><p>Maalepunkter langs bakeplaten.</p></caption>
<graphic xlink:href="figur-84-1.png" xmlns:xlink="http://www.w3.org/1999/xlink"/></fig>
<graphic xlink:href="figur-7-1.png" xmlns:xlink="http://www.w3.org/1999/xlink"/></fig>
<table-wrap><label>Tabell 85-1</label>
<table><tr><th>Klasse</th><th>T</th></tr><tr><td>1</td><td>1,5</td></tr></table>
</table-wrap>

112
tests/fixtures/accounting/inventory.json vendored
View file

@ -22,7 +22,7 @@
],
"paragraph": [
[
"Figur 84-1 viser prinsippet."
"Figur 7-1 viser prinsippet."
],
[]
],
@ -42,8 +42,8 @@
"images": [
{
"kind": "local",
"ref": "graphics/figur-84-1.png",
"target": "graphics/figur-84-1.png"
"ref": "graphics/figur-7-1.png",
"target": "graphics/figur-7-1.png"
}
],
"suffix": ".html",
@ -64,7 +64,7 @@
],
"heading": [
[
"Figur 84-1"
"Figur 7-1"
]
],
"image": [
@ -94,7 +94,7 @@
},
"witness": "html.parser"
},
"kapittel-84-notat.docx": {
"kapittel-7-notat.docx": {
"elements": {
"cell": 0,
"comment": 0,
@ -127,7 +127,7 @@
],
"paragraph": [
[
"Hevetidsklasse er gitt i tabell 84-2."
"Hevetidsklasse er gitt i tabell 7-2."
],
[
"Etter tabellen kommer steketidene."
@ -138,7 +138,7 @@
},
"witness": "docx zip xml"
},
"kapittel-84-presentasjon.pptx": {
"kapittel-7-presentasjon.pptx": {
"elements": {
"cell": 0,
"hidden_slide": 0,
@ -179,7 +179,7 @@
},
"witness": "pptx zip xml"
},
"kapittel-84-sts.xml": {
"kapittel-7-sts.xml": {
"elements": {
"cell": 0,
"citation": 0,
@ -199,13 +199,13 @@
"images": [
{
"kind": "local",
"ref": "graphics/tabell-84-2.png",
"target": "graphics/tabell-84-2.png"
"ref": "graphics/tabell-7-2.png",
"target": "graphics/tabell-7-2.png"
},
{
"kind": "local",
"ref": "figur-84-1.png",
"target": "graphics/figur-84-1.png"
"ref": "figur-7-1.png",
"target": "graphics/figur-7-1.png"
}
],
"suffix": ".xml",
@ -223,34 +223,34 @@
"math": [],
"paragraph": [
[
"Hevetidsklasse er gitt i tabell 84-2."
"Hevetidsklasse er gitt i tabell 7-2."
],
[
"Figur 84-1 viser prinsippet."
"Figur 7-1 viser prinsippet."
]
],
"section": [
[
"84.1",
"7.1",
"Hevetider",
"Hevetidsklasse er gitt i tabell 84-2.",
"Figur 84-1 viser prinsippet."
"Hevetidsklasse er gitt i tabell 7-2.",
"Figur 7-1 viser prinsippet."
],
[
"84",
"7",
"Brød og boller",
"84.1",
"7.1",
"Hevetider",
"Hevetidsklasse er gitt i tabell 84-2.",
"Figur 84-1 viser prinsippet."
"Hevetidsklasse er gitt i tabell 7-2.",
"Figur 7-1 viser prinsippet."
]
],
"section_label": [
[
"84"
"7"
],
[
"84.1"
"7.1"
]
],
"table": [],
@ -266,7 +266,7 @@
},
"witness": "xml.etree"
},
"kapittel-84-tabell.pdf": {
"kapittel-7-tabell.pdf": {
"elements": {
"image": 2,
"page": 1
@ -291,13 +291,13 @@
],
"page": [
[
"Hevetidsklasse er gitt i tabell 84-2"
"Hevetidsklasse er gitt i tabell 7-2"
]
]
},
"witness": "pdfplumber objects"
},
"kapittel-84-web.html": {
"kapittel-7-web.html": {
"elements": {
"cell": 0,
"heading": 1,
@ -309,13 +309,13 @@
"images": [
{
"kind": "local",
"ref": "graphics/tabell-84-2.png",
"target": "graphics/tabell-84-2.png"
"ref": "graphics/tabell-7-2.png",
"target": "graphics/tabell-7-2.png"
},
{
"kind": "local",
"ref": "graphics/figur-84-1.png",
"target": "graphics/figur-84-1.png"
"ref": "graphics/figur-7-1.png",
"target": "graphics/figur-7-1.png"
},
{
"kind": "remote",
@ -328,7 +328,7 @@
"cell": [],
"heading": [
[
"84 Brød og boller"
"7 Brød og boller"
]
],
"image": [
@ -339,7 +339,7 @@
"list_item": [],
"paragraph": [
[
"Hevetidsklasse for de enkelte deigtyper er gitt i tabell 84-2."
"Hevetidsklasse for de enkelte deigtyper er gitt i tabell 7-2."
],
[
"Figuren under viser prinsippet."
@ -1047,7 +1047,7 @@
"texts": {
"cell": [
[
"84.1"
"7.1"
],
[
"120"
@ -1056,7 +1056,7 @@
"m3"
],
[
"84.2"
"7.2"
],
[
"45"
@ -1078,12 +1078,12 @@
],
"row": [
[
"84.1",
"7.1",
"120",
"m3"
],
[
"84.2",
"7.2",
"45",
"m2"
]
@ -1103,8 +1103,8 @@
"images": [
{
"kind": "local",
"ref": "graphics/tabell-84-2.png",
"target": "graphics/tabell-84-2.png"
"ref": "graphics/tabell-7-2.png",
"target": "graphics/tabell-7-2.png"
}
],
"suffix": ".md",
@ -1170,13 +1170,13 @@
"Post"
],
[
"84.1"
"7.1"
],
[
"Post"
],
[
"84.1"
"7.1"
]
],
"hidden_slide": [
@ -1184,7 +1184,7 @@
"Utgaatt lysbilde",
"Ikke vis dette.",
"Post",
"84.1"
"7.1"
]
],
"image": [],
@ -1203,25 +1203,25 @@
],
"slide": [
[
"Kapittel 84 Boller",
"Kapittel 7 Boller",
"Hevetider er gitt i tabell.",
"Post",
"84.1"
"7.1"
]
],
"table": [
[
"Post",
"84.1"
"7.1"
],
[
"Post",
"84.1"
"7.1"
]
],
"title": [
[
"Kapittel 84 Boller"
"Kapittel 7 Boller"
],
[
"Utgaatt lysbilde"
@ -1429,8 +1429,8 @@
"images": [
{
"kind": "local",
"ref": "graphics/tabell-84-2.png",
"target": "graphics/tabell-84-2.png"
"ref": "graphics/tabell-7-2.png",
"target": "graphics/tabell-7-2.png"
}
],
"suffix": ".htm",
@ -1586,8 +1586,8 @@
"images": [
{
"kind": "local",
"ref": "figur-84-1.png",
"target": "graphics/figur-84-1.png"
"ref": "figur-7-1.png",
"target": "graphics/figur-7-1.png"
}
],
"suffix": ".xml",
@ -1795,18 +1795,18 @@
}
},
"files": {
"graphics/figur-84-1.png": {
"graphics/figur-7-1.png": {
"pointed_at_by": [
"figur.html",
"kapittel-84-sts.xml",
"kapittel-84-web.html",
"kapittel-7-sts.xml",
"kapittel-7-web.html",
"sts-rikt.xml"
]
},
"graphics/tabell-84-2.png": {
"graphics/tabell-7-2.png": {
"pointed_at_by": [
"kapittel-84-sts.xml",
"kapittel-84-web.html",
"kapittel-7-sts.xml",
"kapittel-7-web.html",
"notat.md",
"side.htm"
]

18
tests/fixtures/accounting/make_accounting_fixtures.py vendored
View file

@ -124,7 +124,7 @@ def _slide(title: str, body: str, *, hidden: bool = False) -> str:
+ 'uri="http://schemas.openxmlformats.org/drawingml/2006/table">'
+ "<a:tbl><a:tr>"
+ "<a:tc><a:txBody><a:p><a:r><a:t>Post</a:t></a:r></a:p></a:txBody></a:tc>"
+ "<a:tc><a:txBody><a:p><a:r><a:t>84.1</a:t></a:r></a:p></a:txBody></a:tc>"
+ "<a:tc><a:txBody><a:p><a:r><a:t>7.1</a:t></a:r></a:p></a:txBody></a:tc>"
+ "</a:tr></a:tbl></a:graphicData></a:graphic></p:graphicFrame>"
+ "</p:spTree></p:cSld></p:sld>"
)
@ -164,7 +164,7 @@ _PPTX_PARTS: dict[str, str | bytes] = {
+ f"<p:presentation {_P}><p:sldIdLst>"
+ '<p:sldId id="256" r:id="rId1"/><p:sldId id="257" r:id="rId2"/>'
+ "</p:sldIdLst></p:presentation>",
"ppt/slides/slide1.xml": _slide("Kapittel 84 Boller", "Hevetider er gitt i tabell."),
"ppt/slides/slide1.xml": _slide("Kapittel 7 Boller", "Hevetider er gitt i tabell."),
"ppt/slides/slide2.xml": _slide("Utgaatt lysbilde", "Ikke vis dette.", hidden=True),
"ppt/notesSlides/notesSlide1.xml": _XML
+ f"<p:notes {_P} {_A}><p:cSld><p:spTree><p:sp><p:txBody>"
@ -272,7 +272,7 @@ _ODT_PARTS: dict[str, str | bytes] = {
+ "<text:list><text:list-item><text:p>Kant</text:p></text:list-item>"
+ "<text:list-item><text:p>Flate</text:p></text:list-item></text:list>"
+ "<text:p>Se figuren under."
+ '<draw:frame><draw:image xlink:href="graphics/figur-84-1.png"/></draw:frame></text:p>'
+ '<draw:frame><draw:image xlink:href="graphics/figur-7-1.png"/></draw:frame></text:p>'
+ "<office:annotation><text:p>Sjekk denne mot Q400 foer utsendelse.</text:p>"
+ "</office:annotation>"
+ "<table:table><table:table-row>"
@ -294,7 +294,7 @@ _ODT_PARTS: dict[str, str | bytes] = {
_RTF = (
r"{\rtf1\ansi\deff0{\fonttbl{\f0 Times New Roman;}}"
r"\pard Figur 84-1 viser prinsippet.\par"
r"\pard Figur 7-1 viser prinsippet.\par"
r"\pard{\pict\pngblip\picw16\pich16 89504e470d0a1a0a}\par"
"}"
)
@ -304,11 +304,11 @@ _RTF = (
_HTML = """<!DOCTYPE html>
<html lang="no">
<head><title>Figur 84-1</title></head>
<head><title>Figur 7-1</title></head>
<body>
<h1>Figur 84-1</h1>
<h1>Figur 7-1</h1>
<p>Prinsippet for hevetidsklasser.</p>
<img src="graphics/figur-84-1.png" alt="Prinsippskisse">
<img src="graphics/figur-7-1.png" alt="Prinsippskisse">
<table><tr><th>Klasse</th><th>Avvik</th></tr><tr><td>A</td><td>5 min</td></tr></table>
<ul><li>Klasse A</li><li>Klasse B</li></ul>
</body>
@ -319,13 +319,13 @@ _HTML = """<!DOCTYPE html>
# --- sts: a citation, a formula, a figure with a caption, a table, a footnote -
_STS = """<standard>
<front><std-ident><doc-number>P762</doc-number><year>2025</year></std-ident></front>
<front><std-ident><doc-number>P351</doc-number><year>2025</year></std-ident></front>
<body>
<sec><label>85</label><title>Bakeplater</title>
<p>Platen skal ha jevn varme etter <mixed-citation>EKS 1234-1:2030</mixed-citation>.</p>
<p>Kravet regnes som <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>T</mml:mi><mml:mo>&lt;</mml:mo><mml:mn>2</mml:mn></mml:math>.</p>
<fig><label>Figur 85-1</label><caption><p>Maalepunkter langs bakeplaten.</p></caption>
<graphic xlink:href="figur-84-1.png" xmlns:xlink="http://www.w3.org/1999/xlink"/></fig>
<graphic xlink:href="figur-7-1.png" xmlns:xlink="http://www.w3.org/1999/xlink"/></fig>
<table-wrap><label>Tabell 85-1</label>
<table><tr><th>Klasse</th><th>T</th></tr><tr><td>1</td><td>1,5</td></tr></table>
</table-wrap>

0
tests/fixtures/image-inbox/graphics/figur-84-1.png → tests/fixtures/image-inbox/graphics/figur-7-1.png vendored
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 94 B

After

Width:  |  Height:  |  Size: 94 B

Before After
Before After

0
tests/fixtures/image-inbox/graphics/tabell-84-2.png → tests/fixtures/image-inbox/graphics/tabell-7-2.png vendored
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 122 B

After

Width:  |  Height:  |  Size: 122 B

Before After
Before After

BIN
tests/fixtures/image-inbox/kapittel-7-notat.docx vendored Normal file
View file

Binary file not shown.

0
tests/fixtures/image-inbox/kapittel-84-presentasjon.pptx → tests/fixtures/image-inbox/kapittel-7-presentasjon.pptx vendored
View file

18
tests/fixtures/image-inbox/kapittel-7-sts.xml vendored Normal file
View file

@ -0,0 +1,18 @@
<?xml version="1.0" encoding="UTF-8"?>
<standard xmlns:xlink="http://www.w3.org/1999/xlink">
<front><std-ident><doc-number>P350</doc-number></std-ident></front>
<body>
<sec>
<label>7</label>
<title>Brød og boller</title>
<sec>
<label>7.1</label>
<title>Hevetider</title>
<p>Hevetidsklasse er gitt i tabell 7-2.</p>
<graphic xlink:href="graphics/tabell-7-2.png"/>
<p>Figur 7-1 viser prinsippet.</p>
<graphic xlink:href="figur-7-1.png"/>
</sec>
</sec>
</body>
</standard>

BIN
tests/fixtures/accounting/corpus/kapittel-84-tabell.pdf → tests/fixtures/image-inbox/kapittel-7-tabell.pdf vendored
View file

Binary file not shown.

14
tests/fixtures/image-inbox/kapittel-7-web.html vendored Normal file
View file

@ -0,0 +1,14 @@
<!doctype html>
<html><head><title>Kapittel 7</title></head>
<body>
<h1>7 Brød og boller</h1>
<p>Hevetidsklasse for de enkelte deigtyper er gitt i tabell 7-2.</p>
<img src="graphics/tabell-7-2.png" alt="Tabell 7-2 Hevetidsklasser">
<p>Figuren under viser prinsippet.</p>
<figure>
<img src="graphics/figur-7-1.png" alt="Figur 7-1 Prinsippskisse">
<figcaption>Figur 7-1 Prinsippskisse</figcaption>
</figure>
<p>Og en som ligger et annet sted:</p>
<img src="https://example.invalid/ekstern.png" alt="Ekstern figur">
</body></html>

BIN
tests/fixtures/image-inbox/kapittel-84-notat.docx vendored
View file

Binary file not shown.

18
tests/fixtures/image-inbox/kapittel-84-sts.xml vendored
View file

@ -1,18 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<standard xmlns:xlink="http://www.w3.org/1999/xlink">
<front><std-ident><doc-number>P761</doc-number></std-ident></front>
<body>
<sec>
<label>84</label>
<title>Brød og boller</title>
<sec>
<label>84.1</label>
<title>Hevetider</title>
<p>Hevetidsklasse er gitt i tabell 84-2.</p>
<graphic xlink:href="graphics/tabell-84-2.png"/>
<p>Figur 84-1 viser prinsippet.</p>
<graphic xlink:href="figur-84-1.png"/>
</sec>
</sec>
</body>
</standard>

14
tests/fixtures/image-inbox/kapittel-84-web.html vendored
View file

@ -1,14 +0,0 @@
<!doctype html>
<html><head><title>Kapittel 84</title></head>
<body>
<h1>84 Brød og boller</h1>
<p>Hevetidsklasse for de enkelte deigtyper er gitt i tabell 84-2.</p>
<img src="graphics/tabell-84-2.png" alt="Tabell 84-2 Hevetidsklasser">
<p>Figuren under viser prinsippet.</p>
<figure>
<img src="graphics/figur-84-1.png" alt="Figur 84-1 Prinsippskisse">
<figcaption>Figur 84-1 Prinsippskisse</figcaption>
</figure>
<p>Og en som ligger et annet sted:</p>
<img src="https://example.invalid/ekstern.png" alt="Ekstern figur">
</body></html>

14
tests/fixtures/make_fixtures.py vendored
View file

@ -113,19 +113,19 @@ OUTLINED_TREE = (
)
#: TWO bookmarks whose destinations resolve to the SAME line, mirroring what a
#: 701-page reference standard carries: its tree's root node and its front-matter
#: node both land on line 0. Measured on that document, 2 763 nodes
#: entered the bridge and 2 762 marks came out with `unresolved` at 0 -- the
#: difference was a dict keyed on the line index, dropping the second node with
#: nothing counting it. A one-bookmark-per-line fixture cannot see that.
#: long reference standard carries: its tree's root node and its front-matter
#: node both land on line 0. Measured on that document, N + 2 nodes (N its
#: declared sections) entered the bridge and N + 1 marks came out with
#: `unresolved` at 0 -- the difference was a dict keyed on the line index,
#: dropping the second node with nothing counting it. A one-bookmark-per-line fixture cannot see that.
COLLISION_CONTENTS = (
b"BT /F1 12 Tf 20 170 Td (P761 Oppskriftsboka) Tj ET\n"
b"BT /F1 12 Tf 20 170 Td (P350 Oppskriftsboka) Tj ET\n"
b"BT /F1 12 Tf 20 150 Td (Innledende tekst om grunnlaget.) Tj ET\n",
)
#: Both point at `/XYZ 20 185`, which is above the page's first line.
COLLISION_TREE = (
("P761 Oppskriftsboka", 1, 0, 185),
("P350 Oppskriftsboka", 1, 0, 185),
("EKS - Forside", 2, 0, 185),
)

56
tests/fixtures/make_image_fixtures.py vendored
View file

@ -88,7 +88,7 @@ PDF_GRAY_SAMPLES = bytes([0, 60, 120, 180, 20, 80, 140, 200, 40, 100, 160, 255])
PDF_JPEG_WIDTH, PDF_JPEG_HEIGHT = 360, 269
PDF_CONTENT = (
b"BT /F1 12 Tf 20 170 Td (Hevetidsklasse er gitt i tabell 84-2) Tj ET\n"
b"BT /F1 12 Tf 20 170 Td (Hevetidsklasse er gitt i tabell 7-2) Tj ET\n"
b"q 80 0 0 60 20 90 cm /ImFlate Do Q\n"
b"q 80 0 0 60 20 20 cm /ImJpeg Do Q\n"
)
@ -153,15 +153,15 @@ def build_image_pdf() -> bytes:
# the gate counts it as found-and-not-carried rather than as absent.
HTML_DOCUMENT = """<!doctype html>
<html><head><title>Kapittel 84</title></head>
<html><head><title>Kapittel 7</title></head>
<body>
<h1>84 Brød og boller</h1>
<p>Hevetidsklasse for de enkelte deigtyper er gitt i tabell 84-2.</p>
<img src="graphics/tabell-84-2.png" alt="Tabell 84-2 Hevetidsklasser">
<h1>7 Brød og boller</h1>
<p>Hevetidsklasse for de enkelte deigtyper er gitt i tabell 7-2.</p>
<img src="graphics/tabell-7-2.png" alt="Tabell 7-2 Hevetidsklasser">
<p>Figuren under viser prinsippet.</p>
<figure>
<img src="graphics/figur-84-1.png" alt="Figur 84-1 Prinsippskisse">
<figcaption>Figur 84-1 Prinsippskisse</figcaption>
<img src="graphics/figur-7-1.png" alt="Figur 7-1 Prinsippskisse">
<figcaption>Figur 7-1 Prinsippskisse</figcaption>
</figure>
<p>Og en som ligger et annet sted:</p>
<img src="https://example.invalid/ekstern.png" alt="Ekstern figur">
@ -178,18 +178,18 @@ HTML_DOCUMENT = """<!doctype html>
STS_DOCUMENT = """<?xml version="1.0" encoding="UTF-8"?>
<standard xmlns:xlink="http://www.w3.org/1999/xlink">
<front><std-ident><doc-number>P761</doc-number></std-ident></front>
<front><std-ident><doc-number>P350</doc-number></std-ident></front>
<body>
<sec>
<label>84</label>
<label>7</label>
<title>Brød og boller</title>
<sec>
<label>84.1</label>
<label>7.1</label>
<title>Hevetider</title>
<p>Hevetidsklasse er gitt i tabell 84-2.</p>
<graphic xlink:href="graphics/tabell-84-2.png"/>
<p>Figur 84-1 viser prinsippet.</p>
<graphic xlink:href="figur-84-1.png"/>
<p>Hevetidsklasse er gitt i tabell 7-2.</p>
<graphic xlink:href="graphics/tabell-7-2.png"/>
<p>Figur 7-1 viser prinsippet.</p>
<graphic xlink:href="figur-7-1.png"/>
</sec>
</sec>
</body>
@ -217,20 +217,20 @@ def build_docx() -> bytes:
"word/_rels/document.xml.rels": _XML
+ '<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">'
+ '<Relationship Id="rIdImg" Type="http://schemas.openxmlformats.org/officeDocument'
+ '/2006/relationships/image" Target="media/tabell-84-2.png"/></Relationships>',
"word/media/tabell-84-2.png": image,
+ '/2006/relationships/image" Target="media/tabell-7-2.png"/></Relationships>',
"word/media/tabell-7-2.png": image,
"word/document.xml": _XML
+ '<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"'
+ ' xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"'
+ ' xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing"'
+ ' xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main"'
+ ' xmlns:pic="http://schemas.openxmlformats.org/drawingml/2006/picture"><w:body>'
+ "<w:p><w:r><w:t>Hevetidsklasse er gitt i tabell 84-2.</w:t></w:r></w:p>"
+ "<w:p><w:r><w:t>Hevetidsklasse er gitt i tabell 7-2.</w:t></w:r></w:p>"
+ '<w:p><w:r><w:drawing><wp:inline><wp:extent cx="381000" cy="285750"/>'
+ '<wp:docPr id="1" name="Bilde 1" descr="Tabell 84-2 Hevetidsklasser"/>'
+ '<wp:docPr id="1" name="Bilde 1" descr="Tabell 7-2 Hevetidsklasser"/>'
+ '<a:graphic><a:graphicData uri="http://schemas.openxmlformats.org/drawingml/2006'
+ '/picture"><pic:pic><pic:nvPicPr><pic:cNvPr id="1" name="tabell-84-2.png"'
+ ' descr="Tabell 84-2 Hevetidsklasser"/><pic:cNvPicPr/></pic:nvPicPr>'
+ '/picture"><pic:pic><pic:nvPicPr><pic:cNvPr id="1" name="tabell-7-2.png"'
+ ' descr="Tabell 7-2 Hevetidsklasser"/><pic:cNvPicPr/></pic:nvPicPr>'
+ '<pic:blipFill><a:blip r:embed="rIdImg"/><a:stretch><a:fillRect/></a:stretch>'
+ '</pic:blipFill><pic:spPr><a:xfrm><a:off x="0" y="0"/>'
+ '<a:ext cx="381000" cy="285750"/></a:xfrm>'
@ -310,17 +310,17 @@ if __name__ == "__main__":
# FIVE DISTINCT STEMS. One stem across five extensions is refused by the
# door's own SS 3 collision rule -- measured while building this fixture:
# `kapittel-84.{pdf,docx,pptx}` gave `inbox_slug_collision: 2/7` and two of
# `kapittel-7.{pdf,docx,pptx}` gave `inbox_slug_collision: 2/7` and two of
# the five readers were never exercised at all, with the gate reporting a
# carrying defect that was really a fixture defect.
written: list[tuple[str, bytes]] = [
("graphics/tabell-84-2.png", png(120, 90, value=0x20)),
("graphics/figur-84-1.png", png(64, 48, value=0x80)),
("kapittel-84-web.html", HTML_DOCUMENT.encode("utf-8")),
("kapittel-84-sts.xml", STS_DOCUMENT.encode("utf-8")),
("kapittel-84-tabell.pdf", build_image_pdf()),
("kapittel-84-notat.docx", build_docx()),
("kapittel-84-presentasjon.pptx", build_pptx()),
("graphics/tabell-7-2.png", png(120, 90, value=0x20)),
("graphics/figur-7-1.png", png(64, 48, value=0x80)),
("kapittel-7-web.html", HTML_DOCUMENT.encode("utf-8")),
("kapittel-7-sts.xml", STS_DOCUMENT.encode("utf-8")),
("kapittel-7-tabell.pdf", build_image_pdf()),
("kapittel-7-notat.docx", build_docx()),
("kapittel-7-presentasjon.pptx", build_pptx()),
]
for name, payload in written:
(IMAGES / name).write_bytes(payload)

4
tests/fixtures/outline-collision.pdf vendored
View file

@ -11,7 +11,7 @@ endobj
4 0 obj
<< /Length 114 >>
stream
BT /F1 12 Tf 20 170 Td (P761 Oppskriftsboka) Tj ET
BT /F1 12 Tf 20 170 Td (P350 Oppskriftsboka) Tj ET
BT /F1 12 Tf 20 150 Td (Innledende tekst om grunnlaget.) Tj ET
endstream
endobj
@ -22,7 +22,7 @@ endobj
<< /Type /Outlines /First 7 0 R /Last 7 0 R /Count 2 >>
endobj
7 0 obj
<< /Title (P761 Oppskriftsboka) /Parent 6 0 R /First 8 0 R /Last 8 0 R /Count 1 /Dest [3 0 R /XYZ 20 185 0] >>
<< /Title (P350 Oppskriftsboka) /Parent 6 0 R /First 8 0 R /Last 8 0 R /Count 1 /Dest [3 0 R /XYZ 20 185 0] >>
endobj
8 0 obj
<< /Title (EKS - Forside) /Parent 7 0 R /Dest [3 0 R /XYZ 20 185 0] >>

8
tests/fixtures/sts-empty-label.xml vendored
View file

@ -1,8 +1,8 @@
<standard>
<body>
<sec id="s-1">
<label>84.61</label>
<title>Beskyttelse av betong</title>
<label>14.61</label>
<title>Beskyttelse av benkeplate</title>
<sec id="s-1-a" sec-type="spec">
<label>a)</label>
<p>Omfatter beskyttelse mot kjemikalier.</p>
@ -12,8 +12,8 @@
</sec>
</sec>
<sec id="s-2">
<label>84.62</label>
<title>Rengjoering av betongoverflate</title>
<label>14.62</label>
<title>Rengjoering av benkeflate</title>
<p>Overflaten rengjoeres foer behandling.</p>
</sec>
</body>

34
tests/fixtures/sts-identity.xml vendored
View file

@ -8,31 +8,31 @@
</std-doc-meta>
</front>
<body>
<sec id="s-88" sec-type="procedure">
<label>88</label>
<title>Gelender</title>
<sec id="s-88-612" sec-type="procedure">
<label>88.612</label>
<title>Utskifting av enkeltkomponenter</title>
<sec id="s-88-612-a" sec-type="spec">
<sec id="s-17" sec-type="procedure">
<label>17</label>
<title>Kjokkenbenk</title>
<sec id="s-17-612" sec-type="procedure">
<label>17.612</label>
<title>Utskifting av enkeltdeler</title>
<sec id="s-17-612-a" sec-type="spec">
<label>a)</label>
<p>Omfatter utskifting av skadde enkeltkomponenter i gelender.</p>
<p>Omfatter utskifting av skadde enkeltdeler i kjokkenbenk.</p>
</sec>
<sec id="s-88-612-b" sec-type="spec">
<sec id="s-17-612-b" sec-type="spec">
<label>b)</label>
<p>Et andre punkt blir aldri beskrivelsen.</p>
</sec>
<sec id="s-88-6121" sec-type="procedure">
<label>88.6121</label>
<title><italic>Utskifting av handlist</italic></title>
<sec id="s-17-6121" sec-type="procedure">
<label>17.6121</label>
<title><italic>Utskifting av handtak</italic></title>
</sec>
</sec>
<sec id="s-88-613" sec-type="procedure">
<label>88.613</label>
<title>Maling av gelender</title>
<sec id="s-88-613-a" sec-type="spec">
<sec id="s-17-613" sec-type="procedure">
<label>17.613</label>
<title>Maling av kjokkenbenk</title>
<sec id="s-17-613-a" sec-type="spec">
<label>a)</label>
<p>Omfatter maling av gelender.</p>
<p>Omfatter maling av kjokkenbenk.</p>
<p>Et andre avsnitt i samme punkt blir ikke med.</p>
</sec>
</sec>

6
tests/fixtures/sts-inherit.xml vendored
View file

@ -8,12 +8,12 @@
<body>
<sec sec-type="procedure" id="s7"><label>7</label><title>Vedlikehold av utstyr</title>
<sec sec-type="spec"><label>a)</label><p>Omfatter vedlikehold av alt utstyr i kjøkkenet.</p></sec>
<sec sec-type="procedure" id="s7-6"><label>7.6</label><title>Gelender</title>
<sec sec-type="procedure" id="s7-6"><label>7.6</label><title>Kjokkenbenk</title>
<sec sec-type="procedure" id="s7-61"><label>7.61</label><title>Utskifting</title>
<sec sec-type="procedure" id="s7-612"><label>7.612</label><title>Utskifting av enkeltdeler</title>
<sec sec-type="spec"><label>a)</label><p>Omfatter utskifting av skadde enkeltdeler i gelender.</p></sec>
<sec sec-type="spec"><label>a)</label><p>Omfatter utskifting av skadde enkeltdeler i kjokkenbenk.</p></sec>
<sec sec-type="procedure" id="s7-6121"><label>7.6121</label><title>Utskifting av list</title></sec>
<sec sec-type="procedure" id="s7-6122"><label>7.6122</label><title>Utskifting av stolpe</title></sec>
<sec sec-type="procedure" id="s7-6122"><label>7.6122</label><title>Utskifting av hengsel</title></sec>
</sec>
</sec>
</sec>

36
tests/test_accounting_gate.py
View file

@ -63,7 +63,7 @@ from pathlib import Path
fixtures = Path({fixtures!r})
w.witness_inbox(fixtures / "corpus")
w.witness_inbox(fixtures / "rejected")
w.pdf_poppler(fixtures / "corpus" / "kapittel-84-tabell.pdf")
w.pdf_poppler(fixtures / "corpus" / "kapittel-7-tabell.pdf")
print(sorted(m for m in sys.modules if m.split(".")[0] == "llm_ingestion_okf"))
"""
@ -126,7 +126,7 @@ HAND_COUNTS = {
},
"krav-rikt-tekstformat.rtf": {"cell": 56, "image": 0, "paragraph": 3, "table_row": 24},
"bilde.rtf": {"cell": 0, "image": 1, "paragraph": 2, "table_row": 0},
"kapittel-84-sts.xml": {
"kapittel-7-sts.xml": {
"cell": 0,
"citation": 0,
"figure": 0,
@ -191,8 +191,8 @@ HAND_COUNTS = {
"logg.txt": {"line": 4, "paragraph": 3},
"mengder.csv": {"cell": 6, "header_cell": 3, "row": 2},
"parametre.json": {"key": 6, "value": 6},
"kapittel-84-tabell.pdf": {"image": 2, "page": 1},
"kapittel-84-web.html": {
"kapittel-7-tabell.pdf": {"image": 2, "page": 1},
"kapittel-7-web.html": {
"cell": 0,
"heading": 1,
"image": 3,
@ -200,7 +200,7 @@ HAND_COUNTS = {
"paragraph": 3,
"table": 0,
},
"kapittel-84-notat.docx": {
"kapittel-7-notat.docx": {
"cell": 0,
"comment": 0,
"endnote": 0,
@ -212,7 +212,7 @@ HAND_COUNTS = {
"table": 0,
"text_box": 0,
},
"kapittel-84-presentasjon.pptx": {
"kapittel-7-presentasjon.pptx": {
"cell": 0,
"hidden_slide": 0,
"image": 1,
@ -292,8 +292,8 @@ def test_a_fenced_heading_is_not_a_heading_to_the_witness() -> None:
def test_the_sts_image_reference_resolves_through_the_graphics_directory() -> None:
inventory = witness.witness_file(gate.CORPUS, gate.STS_FIXTURE)
assert [ref.target for ref in inventory.images] == [
"graphics/tabell-84-2.png",
"graphics/figur-84-1.png",
"graphics/tabell-7-2.png",
"graphics/figur-7-1.png",
]
@ -559,9 +559,9 @@ def test_a_document_cannot_forge_a_conversion_claim(tmp_path: Path) -> None:
other = "0" * 64
arms = {
"plain body text": f"{pointer}\n\nKapittel 84. {clause}. Se figuren over.\n",
"a table cell": f"{pointer}\n\n| Krav | Kilde |\n| --- | --- |\n| 84-1 | {clause} |\n",
"a figure caption": f"{pointer}\n\nFigur 84-1 -- {clause}\n",
"plain body text": f"{pointer}\n\nKapittel 7. {clause}. Se figuren over.\n",
"a table cell": f"{pointer}\n\n| Krav | Kilde |\n| --- | --- |\n| 7-1 | {clause} |\n",
"a figure caption": f"{pointer}\n\nFigur 7-1 -- {clause}\n",
"a fenced code block": f"{pointer}\n\n```\n{clause}\n```\n",
"link text with a URL": f"{pointer}\n\n[{clause}](https://example.invalid/x)\n",
"the clause with non-breaking spaces": f"{pointer}\n\n{nbsp_clause}\n",
@ -652,12 +652,12 @@ def test_the_build_never_writes_a_claim_the_document_supplied(tmp_path: Path) ->
after = gate._sha256(real)
clause = f"converted from image/bmp sha256:{before} to image/png sha256:{after}"
(corpus / "kapittel.html").write_text(
"<!doctype html>\n<html><head><title>Kapittel 84</title></head>\n<body>\n"
"<h1>84 Brød og boller</h1>\n"
"<!doctype html>\n<html><head><title>Kapittel 7</title></head>\n<body>\n"
"<h1>7 Brød og boller</h1>\n"
"<p>Hevetidsklassene staar i figuren under.</p>\n"
f'<img src="graphics/ekte.png" alt="Figur 84-1 -- {clause}">\n'
f'<img src="graphics/ekte.png" alt="Figur 7-1 -- {clause}">\n'
"<p>Og den store figuren:</p>\n"
'<img src="graphics/figur.bmp" alt="Figur 84-2">\n'
'<img src="graphics/figur.bmp" alt="Figur 7-2">\n'
f"<p>{clause}</p>\n"
"</body></html>\n",
encoding="utf-8",
@ -707,8 +707,8 @@ def _forgery_corpus(root: Path, body: str, note: str | None = None) -> tuple[Pat
before = gate._sha256(never_carried)
after = gate._sha256(real)
(inbox / "kapittel.html").write_text(
"<!doctype html>\n<html><head><title>Kapittel 84</title></head>\n<body>\n"
"<h1>84 Brød og boller</h1>\n"
"<!doctype html>\n<html><head><title>Kapittel 7</title></head>\n<body>\n"
"<h1>7 Brød og boller</h1>\n"
'<img src="graphics/stor.bmp" alt="Stor">'
'<img src="graphics/ekte.png" alt="Ekte">'
'<img src="graphics/figur.bmp" alt="Figur">\n'
@ -1228,7 +1228,7 @@ def test_an_asset_under_a_reduced_name_still_proves_the_carry(tmp_path: Path) ->
corpus = _corpus(tmp_path)
source = corpus / "graphics" / "x.png"
digest = gate._sha256(source)
reduced = {f"{digest[:12]}-25-0143-tabeller-p761-p762.jpeg": digest}
reduced = {f"{digest[:12]}-99-0001-tabeller-p350-p351.jpeg": digest}
build = _build(accounting=_declared(fate="carried"), assets=reduced)
assert gate.account(_inventory(), build, corpus)[1].clean

24
tests/test_asset_gate.py
View file

@ -14,11 +14,11 @@ editable tree -- an editable install reads `src/` live, so a "before" run taken
in this working tree would have been measuring the change it was supposed to
predate:
carried 0 of 2 local (2 declared) kapittel-84-tabell.pdf
carried 0 of 1 local (1 declared) kapittel-84-notat.docx
carried 0 of 1 local (1 declared) kapittel-84-presentasjon.pptx
carried 0 of 2 local (3 declared) kapittel-84-web.html
carried 0 of 2 local (2 declared) kapittel-84-sts.xml
carried 0 of 2 local (2 declared) kapittel-7-tabell.pdf
carried 0 of 1 local (1 declared) kapittel-7-notat.docx
carried 0 of 1 local (1 declared) kapittel-7-presentasjon.pptx
carried 0 of 2 local (3 declared) kapittel-7-web.html
carried 0 of 2 local (2 declared) kapittel-7-sts.xml
---------------------------------------------------------------
carried 0 of 8 local images across 5 documents (9 declared),
and the bundle held no `assets/` directory at all.
@ -32,7 +32,7 @@ is this one.
WHY THE GATE IS A BUNDLE-LEVEL COUNT and not a per-function assertion: the
failure this exists to stop is not "the reader returned nothing". It is a
bundle that looks complete and is not -- a real reference standard saying
"... er gitt i tabell 84-2" over an empty space, with `okf check` green
"... er gitt i tabell 7-2" over an empty space, with `okf check` green
and `okf quality` unable to see it. So the gate runs the real command, walks
the written bundle, and asks whether the bytes are there.
@ -62,11 +62,11 @@ OKF_VERSION = "0.2"
#: Every document in the fixture inbox, with the reader it exercises.
DOCUMENTS = (
"kapittel-84-tabell.pdf",
"kapittel-84-notat.docx",
"kapittel-84-presentasjon.pptx",
"kapittel-84-web.html",
"kapittel-84-sts.xml",
"kapittel-7-tabell.pdf",
"kapittel-7-notat.docx",
"kapittel-7-presentasjon.pptx",
"kapittel-7-web.html",
"kapittel-7-sts.xml",
)
@ -218,7 +218,7 @@ def test_a_remote_image_is_a_pointer_without_a_file_never_a_silent_drop(
) -> None:
"""The network boundary, stated in the artifact rather than implied by absence."""
bundle, _ = built
html = [path for path in _concepts(bundle) if "kapittel-84-web.html" in path.read_text("utf-8")]
html = [path for path in _concepts(bundle) if "kapittel-7-web.html" in path.read_text("utf-8")]
assert html, "the html document produced no concept at all"
body = "\n".join(path.read_text("utf-8") for path in html)
assert "https://example.invalid/ekstern.png" in body, (

2
tests/test_asset_import.py
View file

@ -3,7 +3,7 @@
Measured on 2026-09-17, before this was fixed: importing a bundle built with
`--assets` merged 6 of 6 concepts -- the guard cleared every pointer block as
ordinary body text -- and wrote **no `assets/` directory at all**. Every
`![Tabell 84-2](/assets/...)` in the imported bundle pointed at a file that was
`![Tabell 7-2](/assets/...)` in the imported bundle pointed at a file that was
not there.
That is the same defect one door over: a bundle that reads as complete and is

18
tests/test_asset_limits.py
View file

@ -60,7 +60,7 @@ def test_a_remote_reference_is_not_a_markdown_image_link() -> None:
def test_the_pointer_block_of_a_carried_image_is_unchanged() -> None:
"""The known-positive beside it: a local image keeps its image block, or
the fix has merely disarmed the whole capability."""
image = assets.read_image((FIXTURES / "graphics" / "figur-84-1.png").read_bytes(), name="f.png")
image = assets.read_image((FIXTURES / "graphics" / "figur-7-1.png").read_bytes(), name="f.png")
assert IMAGE_POINTER.search(assets.render_block(image)) is not None
@ -89,7 +89,7 @@ def test_no_reader_writes_a_live_link_for_a_remote_reference(document: str, ref:
def test_a_data_uri_image_is_carried_and_leaves_no_foreign_link() -> None:
png = (FIXTURES / "graphics" / "figur-84-1.png").read_bytes()
png = (FIXTURES / "graphics" / "figur-7-1.png").read_bytes()
import base64
uri = "data:image/png;base64," + base64.b64encode(png).decode("ascii")
@ -482,13 +482,19 @@ def test_the_caption_of_a_remote_reference_is_still_stated() -> None:
against 0.10.0 and against this module's own reason for writing the line:
a reader cannot weigh an absence they were never shown."""
line = render_missing(
"p.gif", reason="the source is off this machine", label="Figur 84-1 Tverrprofil", href=None
"p.gif",
reason="the source is off this machine",
label="Figur 7-1 Prinsippskisse",
href=None,
)
assert "Figur 84-1 Tverrprofil" in line
assert "Figur 7-1 Prinsippskisse" in line
with_href = render_missing(
REMOTE, reason="the source is off this machine", label="Figur 84-1 Tverrprofil", href=REMOTE
REMOTE,
reason="the source is off this machine",
label="Figur 7-1 Prinsippskisse",
href=REMOTE,
)
assert "Figur 84-1 Tverrprofil" in with_href
assert "Figur 7-1 Prinsippskisse" in with_href
# --- BLOCKER of the 18.09 PM checkpoint: the CHAIN, not only its first link --

18
tests/test_asset_structure.py
View file

@ -24,32 +24,32 @@ from __future__ import annotations
from llm_ingestion_okf.structure import derive_document_structure
POINTER = (
"# 84 Boller\n\n"
"Hevetidsklasse er gitt i tabell 84-2, jf. Q500.\n\n"
"![Tabell 84-2](/assets/e54e5f5da0e8-tabell-84-2.png)\n"
"Image: graphics/tabell-84-2.png (120x90 px) -- Tabell 84-2\n\n"
"# 7 Boller\n\n"
"Hevetidsklasse er gitt i tabell 7-2, jf. Q500.\n\n"
"![Tabell 7-2](/assets/e54e5f5da0e8-tabell-7-2.png)\n"
"Image: graphics/tabell-7-2.png (120x90 px) -- Tabell 7-2\n\n"
"Se ellers [Q100](q100.md).\n"
)
def test_an_image_pointer_is_not_a_reference() -> None:
structure = derive_document_structure(POINTER, source_file="p761.xml")
assert "/assets/e54e5f5da0e8-tabell-84-2.png" not in structure.references
structure = derive_document_structure(POINTER, source_file="p350.xml")
assert "/assets/e54e5f5da0e8-tabell-7-2.png" not in structure.references
def test_the_links_beside_it_still_are() -> None:
"""The known-positive in the same text: masking must not eat real edges."""
structure = derive_document_structure(POINTER, source_file="p761.xml")
structure = derive_document_structure(POINTER, source_file="p350.xml")
assert "q100.md" in structure.references
def test_the_asset_name_does_not_leak_a_number_into_references() -> None:
"""A masked span is still masked: `84-2` inside the file name is not a subject.
"""A masked span is still masked: `7-2` inside the file name is not a subject.
The span is blanked rather than deleted for exactly this reason -- deleting
would move every later offset and break first-appearance order, and leaving
it would let the number scan read an identifier out of a file name.
"""
structure = derive_document_structure(POINTER, source_file="p761.xml")
structure = derive_document_structure(POINTER, source_file="p350.xml")
assert all("ASSETS" not in subject.upper() for subject in structure.references)
assert "Q500" in structure.references

10
tests/test_asset_viewable.py
View file

@ -227,14 +227,14 @@ def _inbox(root: Path) -> Path:
(inbox / "graphics" / "figur-flat.bmp").write_bytes(bmp_24(RGB_ROWS))
(inbox / "graphics" / "skjema.tiff").write_bytes(TIFF_STUB)
(inbox / "kapittel.html").write_text(
"<!doctype html>\n<html><head><title>Kapittel 84</title></head>\n<body>\n"
"<h1>84 Brød og boller</h1>\n"
"<!doctype html>\n<html><head><title>Kapittel 7</title></head>\n<body>\n"
"<h1>7 Brød og boller</h1>\n"
"<p>Hevetidsklassene staar i figuren under.</p>\n"
'<img src="graphics/figur-rle8.bmp" alt="Figur 84-1 Hevetidsklasser">\n'
'<img src="graphics/figur-rle8.bmp" alt="Figur 7-1 Hevetidsklasser">\n'
"<p>Og prinsippet:</p>\n"
'<img src="graphics/figur-flat.bmp" alt="Figur 84-2 Prinsipp">\n'
'<img src="graphics/figur-flat.bmp" alt="Figur 7-2 Prinsipp">\n'
"<p>Og skjemaet:</p>\n"
'<img src="graphics/skjema.tiff" alt="Skjema 84-3">\n'
'<img src="graphics/skjema.tiff" alt="Skjema 7-3">\n'
"</body></html>\n",
encoding="utf-8",
)

40
tests/test_assets.py
View file

@ -4,10 +4,10 @@ Until 0.10.0 no reader in this package fetched, named, described or copied a
single image, and the only writer into a bundle was
`materialize.write_bytes(bundle_dir, name, content: str)` -- UTF-8, text, no
binary path anywhere. A document whose table is a raster picture therefore
reached a concept as an absence with no denominator: measured on a 701-page
reached a concept as an absence with no denominator: measured on a long
reference standard, its text is carried in full while 12 `Tabell N-N` and 9
`Figur N-N` captions stand over nothing, and a section saying "... er gitt i
tabell 84-2" stands above an empty space.
tabell 7-2" stands above an empty space.
THIS MODULE IS THE ONE PLACE THAT DECIDES WHAT AN IMAGE IS. Every reader hands
it bytes and gets back either a carried image or a coded rejection, so a
@ -125,7 +125,7 @@ def test_sniff_refuses_what_is_not_an_image() -> None:
def test_the_claimed_extension_never_decides() -> None:
"""A PNG named `.jpg` is carried as a PNG, under a `.png` name."""
image = assets.read_image(_png(4, 3), name="tabell-84-2.jpg")
image = assets.read_image(_png(4, 3), name="tabell-7-2.jpg")
assert image.media_type == "image/png"
assert assets.asset_name(image).endswith(".png")
@ -178,7 +178,7 @@ def test_dimensions_absent_is_absent_never_zero() -> None:
def test_asset_name_is_digest_plus_a_reduced_original() -> None:
data = _jpeg(360, 269)
image = assets.read_image(data, name="25-0143 - Tabeller - Oppskriftsboka (P761-P762).jpg")
image = assets.read_image(data, name="99-0001 - Tabeller - Oppskriftsboka (P350-P351).jpg")
name = assets.asset_name(image)
assert name.startswith(hashlib.sha256(data).hexdigest()[: assets.DIGEST_PREFIX] + "-")
assert name.endswith(".jpg")
@ -212,15 +212,15 @@ def test_a_nameless_image_still_gets_a_name() -> None:
def test_block_names_the_original_the_size_and_the_label() -> None:
image = assets.read_image(
_jpeg(360, 269),
name="25-0143 - Tabeller - Oppskriftsboka (P761-P762).jpg",
label="Tabell 84-2",
name="99-0001 - Tabeller - Oppskriftsboka (P350-P351).jpg",
label="Tabell 7-2",
)
block = assets.render_block(image)
first, second = block.split("\n")
assert first == f"![Tabell 84-2]({assets.asset_href(image)})"
assert second.startswith("Image: 25-0143 - Tabeller - Oppskriftsboka (P761-P762).jpg")
assert first == f"![Tabell 7-2]({assets.asset_href(image)})"
assert second.startswith("Image: 99-0001 - Tabeller - Oppskriftsboka (P350-P351).jpg")
assert "360x269 px" in second
assert second.endswith("Tabell 84-2")
assert second.endswith("Tabell 7-2")
def test_the_label_falls_back_to_the_original_name() -> None:
@ -236,13 +236,13 @@ def test_the_label_falls_back_to_the_original_name() -> None:
def test_a_bracket_in_a_label_cannot_break_the_link() -> None:
image = assets.read_image(_png(4, 3), name="f.png", label="Tabell [84-2] jf. pkt (3)")
image = assets.read_image(_png(4, 3), name="f.png", label="Tabell [7-2] jf. pkt (3)")
first = assets.render_block(image).split("\n")[0]
assert first == f"![Tabell (84-2) jf. pkt (3)]({assets.asset_href(image)})"
assert first == f"![Tabell (7-2) jf. pkt (3)]({assets.asset_href(image)})"
def test_a_newline_in_a_label_cannot_open_a_third_line() -> None:
image = assets.read_image(_png(4, 3), name="f.png", label="Tabell\n84-2")
image = assets.read_image(_png(4, 3), name="f.png", label="Tabell\n7-2")
assert len(assets.render_block(image).split("\n")) == 2
@ -271,7 +271,7 @@ def test_a_document_supplied_label_cannot_emit_a_checksum_field() -> None:
bundle must not state a conversion in a sentence the run did not write.
"""
forged = (
"Tabell 84-2 -- converted from image/bmp sha256:"
"Tabell 7-2 -- converted from image/bmp sha256:"
+ "a" * 64
+ " to image/png sha256:"
+ "b" * 64
@ -309,7 +309,7 @@ def test_the_pointer_regex_finds_every_shipped_block() -> None:
assets.read_image(_png(4, 3), name="a.png", label="Figur 1"),
assets.read_image(_jpeg(9, 9), name="b.jpg"),
]
text = "Kapittel 84\n\n" + "\n\n".join(assets.render_block(i) for i in images) + "\n\nSlutt\n"
text = "Kapittel 7\n\n" + "\n\n".join(assets.render_block(i) for i in images) + "\n\nSlutt\n"
found = assets.IMAGE_POINTER.findall(text)
assert [match[1] for match in found] == [assets.asset_name(i) for i in images]
@ -374,7 +374,7 @@ def test_the_restated_reduction_is_materializes_own() -> None:
from llm_ingestion_okf.materialize import reduce_to_id_grammar
for value in (
"25-0143 - Tabeller - Oppskriftsboka (P761-P762)",
"99-0001 - Tabeller - Oppskriftsboka (P350-P351)",
"Figur 11.1 Hevetidskrav",
"grafikk_med_æøå",
"---",
@ -387,15 +387,15 @@ def test_one_image_reached_by_two_paths_is_one_asset() -> None:
"""The asset name reads the BASENAME, never the path the document used.
Measured on the fixture inbox before this rule existed: the HTML document
points at `graphics/figur-84-1.png` and the STS document at
`figur-84-1.png` (resolved through the `graphics/` sibling convention), so
points at `graphics/figur-7-1.png` and the STS document at
`figur-7-1.png` (resolved through the `graphics/` sibling convention), so
one image was written twice, under two names, from one run -- with the
digest in both of them announcing that the bytes were identical. The path a
document happened to use is not a property of the picture.
"""
data = _png(6, 4)
through_directory = assets.read_image(data, name="graphics/figur-84-1.png")
bare = assets.read_image(data, name="figur-84-1.png")
through_directory = assets.read_image(data, name="graphics/figur-7-1.png")
bare = assets.read_image(data, name="figur-7-1.png")
assert assets.asset_name(through_directory) == assets.asset_name(bare)
# The full original is not lost -- it moves to the line a person reads.
assert "graphics/figur-84-1.png" in assets.render_block(through_directory)
assert "graphics/figur-7-1.png" in assets.render_block(through_directory)

2
tests/test_block_sources_flat_readers.py
View file

@ -4,7 +4,7 @@
the line-oriented frontmatter grammar read only the flow one. On a block
sequence they return the key with an EMPTY value -- not a `KeyError` a
consumer can catch, an address that disappears. Measured 2026-09-12 over four
bundles a producer ships: 2 756 of 2 757, 446 of 447, 1 133 of 1 134 and 270
bundles a producer ships: all but one of a few thousand, 446 of 447, 1 133 of 1 134 and 270
of 271 concept files lost the address that way, while PyYAML 6.0.3 and the
pinned guard 1.4.0 both read it.

12
tests/test_content_accounting.py
View file

@ -66,7 +66,7 @@ def test_a_file_carried_through_a_document_is_not_a_coded_rejection(tmp_path: Pa
def test_an_unpointed_image_beside_a_document_stays_a_coded_rejection(tmp_path: Path) -> None:
pytest.importorskip("pypandoc")
inbox = _inbox(tmp_path, CORPUS)
shutil.copy(inbox / "graphics" / "figur-84-1.png", inbox / "graphics" / "ubrukt.png")
shutil.copy(inbox / "graphics" / "figur-7-1.png", inbox / "graphics" / "ubrukt.png")
report = cli.build(inbox, tmp_path / "b", bundle_id="acc", okf_version="0.2")
assert report.carried_files == 2
assert dict(report.codes)["extractor_unknown"] == 1
@ -195,12 +195,12 @@ def test_the_classes_this_package_does_not_count_are_named(name: str) -> None:
def test_the_inventory_resolves_the_files_a_document_points_at() -> None:
got = accounting.inventory(CORPUS, CORPUS / "kapittel-84-sts.xml")
assert got.pointed_files() == {"graphics/tabell-84-2.png", "graphics/figur-84-1.png"}
got = accounting.inventory(CORPUS, CORPUS / "kapittel-7-sts.xml")
assert got.pointed_files() == {"graphics/tabell-7-2.png", "graphics/figur-7-1.png"}
def test_the_inventory_of_an_unread_type_is_empty() -> None:
got = accounting.inventory(CORPUS, CORPUS / "graphics" / "figur-84-1.png")
got = accounting.inventory(CORPUS, CORPUS / "graphics" / "figur-7-1.png")
assert got.counts() == {}
@ -245,8 +245,8 @@ def test_the_door_books_every_element_of_the_fixture_corpus_but_the_known_losses
found[(document["source_file"], kind)] = (booked, number)
assert found == KNOWN_LOSSES
files = {entry["source_file"]: entry["fate"] for entry in data["files"]}
assert files == {"graphics/figur-84-1.png": "carried", "graphics/tabell-84-2.png": "carried"}
web = next(d for d in data["documents"] if d["source_file"] == "kapittel-84-web.html")
assert files == {"graphics/figur-7-1.png": "carried", "graphics/tabell-7-2.png": "carried"}
web = next(d for d in data["documents"] if d["source_file"] == "kapittel-7-web.html")
assert web["fates"]["image"] == {"carried": 2, "pointer": 1, "rejected": {}}

2
tests/test_depth_seven.py
View file

@ -5,7 +5,7 @@ heading it emits, where markdown has six levels and the clip is the grammar's,
and the `OutlineMark` the declared route builds its plan from, where nothing
clips and the source's own depth is the point (`OutlineMark`'s docstring:
the level is what the tree declares, "reported rather than fixed up"). On one
standard 9 of 2 761 titled sections sit at depth 7, and the plan read them at
standard 9 of its N titled sections sit at depth 7, and the plan read them at
6 -- so `--shell-parent` gave its two depth-7 shells the ancestor one level
too high.

2
tests/test_document_identity.py
View file

@ -1,7 +1,7 @@
"""A document that states who it is names its own directory (K3-19 a).
MEASURED OUTSIDE THIS REPOSITORY: a NISO-STS delivery built with `okf build`
put all 2 761 concepts under a directory named for the delivery path's file
put all of its concepts under a directory named for the delivery path's file
name, a UUID that occurs **0 times** in the document itself. The document
carries exactly one `<std-ident>` whose `<doc-number>` says what it is, and one
`<title-wrap>` -- and nothing in this package read either. Every concept's

2
tests/test_error_codes.py
View file

@ -815,7 +815,7 @@ def test_asset_pdf_unsupported() -> None:
from llm_ingestion_okf.extract import extract_document
data = (
Path(__file__).parent / "fixtures" / "image-inbox" / "kapittel-84-tabell.pdf"
Path(__file__).parent / "fixtures" / "image-inbox" / "kapittel-7-tabell.pdf"
).read_bytes()
stencil = data.replace(
b"/BitsPerComponent 8 /Filter /FlateDecode", b"/BitsPerComponent 1 /Filter /FlateDecode", 1

22
tests/test_extract.py
View file

@ -61,8 +61,8 @@ def test_json_is_verbatim_inside_a_fenced_block() -> None:
def test_html_text_via_htmlparser() -> None:
"""CHANGED 2026-09-09 BECAUSE THE BEHAVIOUR CHANGED, not to go green.
This asserted the collapsed one-line form -- the very form that made 828
of 828 real HTML sections unsegmentable. Block tags now open lines and
This asserted the collapsed one-line form -- the very form that made every
one of M real HTML sections unsegmentable. Block tags now open lines and
headings carry their ATX level; inline tags are still word boundaries,
which is the half of the old assertion that still holds and is still here.
"""
@ -85,13 +85,13 @@ def test_htm_is_an_html_alias() -> None:
# --- html: block structure survives extraction (round 11) -------------------
#
# The defect this closes, measured OUTSIDE this repo on 828 real sections:
# The defect this closes, measured OUTSIDE this repo on M real sections:
# `text()` used to be `" ".join("".join(parts).split())`, and `str.split()`
# with no argument splits on newlines too, so every HTML file extracted to
# UNCONDITIONALLY ONE LINE. Every boundary grammar in `propose` is
# line-anchored (`_ATX`, `_NUMBERED`, `_TABLE_ROW`, `_GRID_RULE`, `_OUTLINE`,
# each with `^`), so 828 of 828 documents got zero boundaries and the run
# exited 2. The same 828 sections as markdown gave 828 plans.
# each with `^`), so M of M documents got zero boundaries and the run
# exited 2. The same M sections as markdown gave M plans.
_ADDED_ATX = re.compile(r"(?m)^#{1,6} ")
@ -206,10 +206,10 @@ def test_html_line_structure_preserves_every_non_whitespace_character() -> None:
def test_the_invariant_holds_over_a_whole_corpus_not_one_document() -> None:
"""One document proves the invariant is reachable, not that it holds.
Run over the SK1 corpus (828 `.html` sections of a published handbook) this
was 828 of 828 exact and a character ratio of 1.000000 against SK1's own
Run over the SK1 corpus (M `.html` sections of a published handbook) this
was M of M exact and a character ratio of 1.000000 against SK1's own
>= 99.8 % bar, with 7600 ATX markers added and 31 141 lines produced where
the old extractor produced 828 -- one per file, which was the defect.
the old extractor produced M -- one per file, which was the defect.
"""
root = Path(os.environ["OKF_HTML_CORPUS"])
files = sorted(root.rglob("*.html"))
@ -344,9 +344,9 @@ def test_evidence_class_is_asserted_not_commented() -> None:
The table is NOT the converter's rows alone: `.html` and `.pdf` are
core-supported and carry a class here too -- `.html` since 2026-09-09
(`measured`, 828 files), `.pdf` since 2026-09-10 (`measured`, eight corpus
documents with a hand-counted fasit plus a publisher's own 2 761-section
structure for a 701-page one). `.pdf` was the row with the most measurement
(`measured`, M files), `.pdf` since 2026-09-10 (`measured`, eight corpus
documents with a hand-counted fasit plus a publisher's own section
structure for a long one). `.pdf` was the row with the most measurement
behind it and no row in the table at all, which is the one way a table like
this can mislead while every entry in it is true.
"""

42
tests/test_link_line_signal.py
View file

@ -37,7 +37,7 @@ from llm_ingestion_okf.inbox import ENCLOSING_SECTION # noqa: E402
QUESTION = "Hvilke krav gjelder for hevekurver?"
_FRONTMATTER = (
"---\ntype: reference\ntitle: {title}\nsource_file: p761.xml\n"
"---\ntype: reference\ntitle: {title}\nsource_file: p350.xml\n"
"source_sha256: {digest}\ningested_at: 2026-09-01T00:00:00Z\n"
"adjudication: proposed\nbundle_id: link-fixture\n"
"verified: [{{ by: process:okf-check, at: 2026-09-01T00:00:00Z }}]\n---\n\n"
@ -45,7 +45,7 @@ _FRONTMATTER = (
#: Exactly what the door writes, built from the door's own constant so a rename
#: there breaks this file rather than silently loosening the rule.
DOOR_LINE = f"{ENCLOSING_SECTION}: [Hevekurver](/p761/33-2/hevekurver.md)"
DOOR_LINE = f"{ENCLOSING_SECTION}: [Hevekurver](/p350/33-2/hevekurver.md)"
def _bundle(root: Path) -> Path:
@ -55,16 +55,16 @@ def _bundle(root: Path) -> Path:
question's token -- so every lexical hit it has comes from the door's line.
The two known-negatives carry text that merely LOOKS like the line.
"""
(root / "p761").mkdir(parents=True)
(root / "p350").mkdir(parents=True)
(root / "index.md").write_text(
"---\nokf_version: 0.2\nbundle_id: link-fixture\n---\n\n- [p761 (index)](p761/index.md)\n",
"---\nokf_version: 0.2\nbundle_id: link-fixture\n---\n\n- [p350 (index)](p350/index.md)\n",
encoding="utf-8",
)
entries: list[str] = []
def add(slug: str, title: str, body: str) -> None:
entries.append(f"- [{title}]({slug}.md) — adjudication: proposed\n")
(root / "p761" / f"{slug}.md").write_text(
(root / "p350" / f"{slug}.md").write_text(
_FRONTMATTER.format(title=title, digest="1" * 64) + body,
encoding="utf-8",
)
@ -83,7 +83,7 @@ def _bundle(root: Path) -> Path:
"33.214 Deigtype D",
f"## 33.214 Deigtype D\n\n{DOOR_LINE}\n\nDenne teksten staar under lenka.\n",
)
(root / "p761" / "index.md").write_text("".join(entries), encoding="utf-8")
(root / "p350" / "index.md").write_text("".join(entries), encoding="utf-8")
return root
@ -127,14 +127,14 @@ def test_the_doors_line_is_the_shells_only_lexical_hit(tmp_path: Path) -> None:
concepts, QUESTION, okf_consume.document_scores(root, QUESTION), link_in_signal=True
)
scored = {concept.concept_id: lexical for concept, _, lexical in ranked}
assert scored["p761/skall"] > 0
assert scored["p350/skall"] > 0
def test_the_instrument_takes_the_line_out_of_the_signal(tmp_path: Path) -> None:
root = _bundle(tmp_path / "bundle")
# Two different numbers, not one predicate read twice.
assert _lexical(root, link_in_signal=True)["p761/skall"] == 1
assert _lexical(root, link_in_signal=False)["p761/skall"] == 0
assert _lexical(root, link_in_signal=True)["p350/skall"] == 1
assert _lexical(root, link_in_signal=False)["p350/skall"] == 0
def test_both_readings_run_on_the_same_concept_objects(tmp_path: Path) -> None:
@ -151,7 +151,7 @@ def test_both_readings_run_on_the_same_concept_objects(tmp_path: Path) -> None:
without = okf_consume.concept_scores(concepts, QUESTION, prior, link_in_signal=False)
assert with_line != without
# The instrument reads; it does not edit what it measures.
shell = next(c for c in concepts if c.concept_id == "p761/skall")
shell = next(c for c in concepts if c.concept_id == "p350/skall")
assert DOOR_LINE in shell.body
@ -162,14 +162,14 @@ def test_a_human_line_opening_with_the_same_two_words_is_not_stripped(tmp_path:
root = _bundle(tmp_path / "bundle")
on = _lexical(root, link_in_signal=False)
off = _lexical(root, link_in_signal=True)
assert off["p761/menneske"] == on["p761/menneske"] > 0
assert off["p350/menneske"] == on["p350/menneske"] > 0
def test_the_doors_form_anywhere_but_last_is_not_stripped(tmp_path: Path) -> None:
root = _bundle(tmp_path / "bundle")
on = _lexical(root, link_in_signal=False)
off = _lexical(root, link_in_signal=True)
assert off["p761/midt"] == on["p761/midt"] > 0
assert off["p350/midt"] == on["p350/midt"] > 0
def test_the_stripper_leaves_a_body_that_never_had_a_link(tmp_path: Path) -> None:
@ -249,7 +249,7 @@ def _unflagged(root: Path) -> Path:
deterministic, and without a corpus.
"""
_bundle(root)
(root / "p761" / "skall.md").write_text(
(root / "p350" / "skall.md").write_text(
_FRONTMATTER.format(title="33.212 Deigtype B", digest="1" * 64) + "## 33.212 Deigtype B\n",
encoding="utf-8",
)
@ -268,7 +268,7 @@ def test_the_default_reading_takes_the_doors_line_out_of_the_signal(tmp_path: Pa
"""`searchable_text` called with NO parameter: the door's line is gone."""
root = _bundle(tmp_path / "bundle")
texts = _default_texts(root)
assert DOOR_LINE not in texts["p761/skall"]
assert DOOR_LINE not in texts["p350/skall"]
def test_the_default_reading_scores_the_shell_at_zero(tmp_path: Path) -> None:
@ -285,7 +285,7 @@ def test_the_default_reading_scores_the_shell_at_zero(tmp_path: Path) -> None:
concepts, QUESTION, okf_consume.document_scores(root, QUESTION)
)
scored = {concept.concept_id: lexical for concept, _, lexical in ranked}
assert scored["p761/skall"] == 0
assert scored["p350/skall"] == 0
def test_the_default_payload_reads_the_body_without_the_line(tmp_path: Path) -> None:
@ -326,7 +326,7 @@ def test_the_old_reading_is_still_reachable_and_still_does_the_old_thing(
default's. Both halves, because either alone would pass on a no-op.
"""
root = _bundle(tmp_path / "bundle")
assert _lexical(root, link_in_signal=True)["p761/skall"] == 1
assert _lexical(root, link_in_signal=True)["p350/skall"] == 1
default = okf_consume.serialise(okf_consume.build_payload(root, question=QUESTION))
old = okf_consume.serialise(
okf_consume.build_payload(root, question=QUESTION, link_in_signal=True)
@ -357,8 +357,8 @@ def test_a_human_line_with_the_same_two_words_survives_the_default_reading(
"""Known-negative 1, restated against the default path."""
root = _bundle(tmp_path / "bundle")
texts = _default_texts(root)
assert DOOR_LINE not in texts["p761/skall"]
assert "Enclosing section: se kapittelet om hevekurver." in texts["p761/menneske"]
assert DOOR_LINE not in texts["p350/skall"]
assert "Enclosing section: se kapittelet om hevekurver." in texts["p350/menneske"]
def test_the_doors_form_anywhere_but_last_survives_the_default_reading(
@ -367,9 +367,9 @@ def test_the_doors_form_anywhere_but_last_survives_the_default_reading(
"""Known-negative 2, restated against the default path."""
root = _bundle(tmp_path / "bundle")
texts = _default_texts(root)
assert DOOR_LINE not in texts["p761/skall"]
assert DOOR_LINE in texts["p761/midt"]
assert "Denne teksten staar under lenka." in texts["p761/midt"]
assert DOOR_LINE not in texts["p350/skall"]
assert DOOR_LINE in texts["p350/midt"]
assert "Denne teksten staar under lenka." in texts["p350/midt"]
def test_the_two_readings_differ_in_rank_and_never_in_an_excerpts_bytes(

8
tests/test_okf_consume.py
View file

@ -1111,11 +1111,11 @@ def _quota_concept(concept_id: str, *, source_file: str) -> okf_consume.Concept:
def test_a_source_quota_caps_how_many_places_one_document_takes() -> None:
"""One source document taking most of the payload is a MEASURED defect.
Measured outside this repo on a 3206-concept bundle of a published
handbook: the code's own process overview contributes 28 of 3206 concepts
(0.87 %) and 8.0 % of the source characters, and takes 22 of 43 delivered
Measured outside this repo on a bundle of a published handbook, a few
thousand concepts: the code's own process overview contributes 0.87 % of
its concepts and 8.0 % of the source characters, and takes 22 of 43 delivered
places on one question and 24 of 45 on the known-positive. Identical at
343 and 1651 concepts, so it is the corpus's COMPOSITION -- that it holds
half and at a tenth of the corpus, so it is the corpus's COMPOSITION -- that it holds
its own table of contents -- and not its size.
The quota cuts where the shortlist is cut, so `k` is still delivered in

6
tests/test_parent_reaches_reader.py
View file

@ -42,10 +42,10 @@ CONTRACT = PROJECT_ROOT / "docs" / "consumption-contract.md"
#: shell title -> the section its link and its excerpt name. The same pairs
#: `tests/test_shell_parent.py` holds for the plan.
LINKED = {
"Gelender": "Vedlikehold av utstyr",
"Kjokkenbenk": "Vedlikehold av utstyr",
"Utskifting": "Vedlikehold av utstyr",
"Utskifting av list": "Utskifting av enkeltdeler",
"Utskifting av stolpe": "Utskifting av enkeltdeler",
"Utskifting av hengsel": "Utskifting av enkeltdeler",
}
#: The one line a linked body gains, in the spec's recommended form.
@ -164,7 +164,7 @@ def test_a_pointer_resolves_inside_its_own_document(tmp_path: Path) -> None:
"--shell-parent",
)
payload = consume.build_payload(
bundle, question="Utskifting av list og stolpe i gelender", k=50
bundle, question="Utskifting av list og hengsel i kjokkenbenk", k=50
)
carried = [e for e in payload["excerpts"] if "parent" in e] # type: ignore[attr-defined]
documents = {str(e["concept_id"]).split("/")[0] for e in carried}

4
tests/test_parent_text.py
View file

@ -30,7 +30,7 @@ from llm_ingestion_okf import cli, consume, contract_check, skill
FIXTURE = Path(__file__).parent / "fixtures" / "sts-inherit.xml"
SHELL = "Utskifting av list"
#: The one lettered point the shell's enclosing section states.
INHERITED = "Omfatter utskifting av skadde enkeltdeler i gelender."
INHERITED = "Omfatter utskifting av skadde enkeltdeler i kjokkenbenk."
def _flagged(tmp_path: Path) -> Path:
@ -131,7 +131,7 @@ def test_a_parent_the_payload_already_holds_is_not_delivered_twice(tmp_path: Pat
assert "text" not in excerpt["parent"]
# Two shells sharing one enclosing section: its text travels once.
two = consume.build_payload(
bundle, question="Utskifting av list og stolpe", k=2, follow_parent=True
bundle, question="Utskifting av list og hengsel", k=2, follow_parent=True
)
texts = [e["parent"].get("text") for e in two["excerpts"] if "parent" in e]
assert len(texts) == 2

22
tests/test_pdf_outline.py
View file

@ -1,19 +1,19 @@
"""The PDF's own `/Outlines` tree as a segmentation source.
MEASURED, OUTSIDE THIS REPOSITORY, ON ONE DOCUMENT: a 701-page process code
MEASURED, OUTSIDE THIS REPOSITORY, ON ONE DOCUMENT: a long reference standard
whose publisher also ships a NISO-STS structure for it. `okf build` on the
default recovered 1 967 of its 2 761 titled sections, 0 of 28 at the top level,
default recovered 1 967 of its N titled sections, 0 of 28 at the top level,
and **794 of 794 misses had their heading text present in the extracted text**
-- the boundary was never opened, the line was read. The same file carries an
`/Outlines` tree of 2 763 nodes which, after `re.sub(r"\\s+","",s).lower()`,
matches 2 761 of 2 761 STS titles exactly. The index shipped inside the file
`/Outlines` tree of N + 2 nodes which, after `re.sub(r"\\s+","",s).lower()`,
matches N of N STS titles exactly. The index shipped inside the file
and the build discarded it unopened.
WHAT THESE TESTS PIN, and why each one exists rather than "it found more":
- the LEVEL a node declares, not its distance from the root. A two-level tree
cannot tell those apart, so the fixture has three.
- the LINE a mark lands on. 2 706 of 2 761 bookmarks in that document share a
- the LINE a mark lands on. N - 55 of N bookmarks in that document share a
destination page with another bookmark, so a bridge that resolved to the page
and stopped would be wrong on almost every node while still looking like it
worked. The fixture's last page carries four lines and its second bookmark
@ -21,7 +21,7 @@ WHAT THESE TESTS PIN, and why each one exists rather than "it found more":
- a PDF with no `/Outlines` behaving IDENTICALLY with the arm on -- `pdfminer`
raises `PDFNoOutlines` there, and "this file has no index" is not an error.
- an unresolvable `/Dest` being dropped and COUNTED. That document has 0 of
2 763; a PDF in the wild has them, so without this the branch would ship
N + 2; a PDF in the wild has them, so without this the branch would ship
having never run.
"""
@ -93,8 +93,8 @@ def test_an_unresolvable_destination_is_dropped_and_counted() -> None:
def test_two_bookmarks_on_one_line_are_counted_not_lost() -> None:
"""The node that shares a line is DROPPED on purpose -- and never silently.
Measured on a 701-page reference standard: 2 763 nodes entered the bridge,
2 762 marks came out and `unresolved` was 0, so one node left no trace
Measured on a long reference standard with N declared sections: N + 2 nodes
entered the bridge, N + 1 marks came out and `unresolved` was 0, so one node left no trace
anywhere. The line it shared was line 0, between the tree's own root node and
its front-matter node.
@ -108,7 +108,7 @@ def test_two_bookmarks_on_one_line_are_counted_not_lost() -> None:
outline = extract.pdf_outline(COLLISION.name, data)
assert [(mark.line, mark.level, mark.title) for mark in outline.marks] == [
(0, 1, "P761 Oppskriftsboka"),
(0, 1, "P350 Oppskriftsboka"),
]
assert outline.unresolved == 0
assert outline.collided == 1
@ -148,11 +148,11 @@ def test_the_outline_replaces_the_text_heuristics_when_it_is_given() -> None:
def test_a_declared_section_with_no_prose_of_its_own_survives() -> None:
"""The orphan check asks the wrong question of a publisher's own tree.
Measured on that 701-page document: 683 of its 2 762 bookmark marks are
Measured on that document: 683 of its N + 1 bookmark marks are
followed immediately by their first subsection, with no prose between. They
are container sections, not mis-detected headings -- the check exists to
catch a heuristic's false positive, and there is no heuristic here. Left
in, the arm scores 2 079 of 2 761 instead of 2 762 of 2 763.
in, the arm scores 2 079 of N instead of N + 1 of N + 2.
"""
text = "1 Grunnlag\n1.1 Omfang\nOmfanget dekker alt.\n"
marks = (

4
tests/test_run_frontmatter.py
View file

@ -28,7 +28,7 @@ value a publisher's address needs carries `=` itself.
K3-22 moved one thing here, and it is a refusal: a flow value is written as
given, so every leaf in it must be one a YAML reader and the guard both read
back. The publisher's address with `?languageCode=nb` is not -- PyYAML refused
all 2 761 frontmatters K3-19's flagged build wrote with it -- and the build
all N frontmatters K3-19's flagged build wrote with it (one per concept) -- and the build
tests below now write an address without `?`.
"""
@ -242,7 +242,7 @@ def test_a_pair_that_would_not_read_back_verbatim_is_refused(tmp_path: Path, pai
def test_a_stated_address_no_yaml_reader_reads_back_is_refused(
tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""K3-19's own flagged build wrote this form, and 2 761 of 2 761 of its
"""K3-19's own flagged build wrote this form, and every one of its
frontmatters then failed `yaml.safe_load`. It is refused, never written
(K3-22): `?` ends a PyYAML flow scalar, and the guard refuses the quote."""
bundle = tmp_path / "bundle"

10
tests/test_shell_parent.py
View file

@ -1,10 +1,10 @@
"""A section whose body is its heading alone points at the ancestor holding its text.
THE SHAPE, in the mechanism rather than in a corpus. A process code states its
THE SHAPE, in the mechanism rather than in a corpus. A reference standard states its
lettered points once, on the section that owns them, and every section nested
below inherits them; the nested section itself carries a title and nothing
else. Built faithfully, it becomes a concept whose body is one heading line --
measured on one 2 761-concept standard, 710 such concepts -- and a reader
measured on one standard of a few thousand concepts, 710 such concepts -- and a reader
handed one has no way to reach the text it inherits: the directory tree is two
levels deep, so the parent is not in the path either.
@ -46,11 +46,11 @@ BUILD_DEFAULTS = dict(
#: shell title -> title of the nearest ancestor holding text, or None.
EXPECTED = {
"Gelender": "Vedlikehold av utstyr",
# Its parent `Gelender` is a shell too, so the pointer goes one further up.
"Kjokkenbenk": "Vedlikehold av utstyr",
# Its parent `Kjokkenbenk` is a shell too, so the pointer goes one further up.
"Utskifting": "Vedlikehold av utstyr",
"Utskifting av list": "Utskifting av enkeltdeler",
"Utskifting av stolpe": "Utskifting av enkeltdeler",
"Utskifting av hengsel": "Utskifting av enkeltdeler",
# A shell whose only ancestor is empty, and one with no ancestor at all.
"Kapittel uten tekst": None,
"Underkapittel uten tekst": None,

6
tests/test_soft_hyphen_door.py
View file

@ -1,6 +1,6 @@
"""The soft-hyphen normalisation door, in front of the persist gate.
Operator decision 2026-09-18. A 701-page reference standard was refused WHOLE
Operator decision 2026-09-18. A long reference standard was refused WHOLE
by `llm-ingestion-guard` 1.4.0 -- `output:zero-width-present`, HIGH, an any-tier
carrier and therefore `fail_secure` at every trust level -- because the
publisher's source carries 71 U+00AD SOFT HYPHEN and 0 real zero-width
@ -16,7 +16,7 @@ before -- which is the known-negative every test here carries.
The three alternatives and why they are not here: weakening the guard is the
security repo's call and weakens every consumer's screen; delivering a cleaned
corpus fixes one document and no other; and doing nothing leaves a 701-page
corpus fixes one document and no other; and doing nothing leaves a long
reference standard unreadable for the whole chain.
U+00A0 NBSP is NOT touched (6 633 of them in that document). The guard has no rule
@ -134,7 +134,7 @@ def _build(inbox: Path, out: Path, accounting: Path) -> tuple[int, str]:
_SOURCE = (
"# Kapit{s}tel 84\n\nAr{s}beider med sjoko{s}lade og surdeigs{s}bakst.\n\n"
"# Kapit{s}tel 7\n\nAr{s}beider med sjoko{s}lade og surdeigs{s}bakst.\n\n"
"# Krav\n\nTilsvar{s}ende krav gjelder.\n"
)

34
tests/test_sts_description.py
View file

@ -5,7 +5,7 @@ concept. Used by `index.md` generators, search snippets, and previews." The
spec sets NO length limit, in SS 4.1, SS 8 or SS 11. The limit here is this
package's own and it is structural: the FIRST `<p>` of the FIRST direct-child
`<sec sec-type="spec">` of a titled `<sec>`, whole. Measured on the one STS
document this row has: 2 026 of 2 761 titled sections carry such a point.
document this row has: 2 026 of its N titled sections carry such a point.
WHAT THESE TESTS PIN:
@ -38,13 +38,13 @@ from llm_ingestion_okf.segmentation import parse_segmentation_plan
FIXTURES = Path(__file__).parent / "fixtures"
IDENTITY = FIXTURES / "sts-identity.xml"
FIRST = "Omfatter utskifting av skadde enkeltkomponenter i gelender."
SECOND = "Omfatter maling av gelender."
FIRST = "Omfatter utskifting av skadde enkeltdeler i kjokkenbenk."
SECOND = "Omfatter maling av kjokkenbenk."
EXPECTED = {
"Gelender": None,
"Utskifting av enkeltkomponenter": FIRST,
"Utskifting av handlist": None,
"Maling av gelender": SECOND,
"Kjokkenbenk": None,
"Utskifting av enkeltdeler": FIRST,
"Utskifting av handtak": None,
"Maling av kjokkenbenk": SECOND,
}
STAMP = "2026-09-08T12:00:00Z"
@ -77,10 +77,10 @@ def test_each_titled_section_carries_its_own_first_spec_paragraph() -> None:
marks = extract.xml_outline(IDENTITY.name, IDENTITY.read_bytes())
assert [(mark.title, mark.description) for mark in marks] == [
("88 Gelender", None),
("88.612 Utskifting av enkeltkomponenter", FIRST),
("88.6121 Utskifting av handlist", None),
("88.613 Maling av gelender", SECOND),
("17 Kjokkenbenk", None),
("17.612 Utskifting av enkeltdeler", FIRST),
("17.6121 Utskifting av handtak", None),
("17.613 Maling av kjokkenbenk", SECOND),
]
@ -118,13 +118,13 @@ def test_a_spec_point_a_yaml_reader_could_not_read_verbatim_is_not_written(
"""
data = (
IDENTITY.read_bytes()
.replace(b"Omfatter maling av gelender.", b"Omfatter maling: gelender og gjerder.")
.replace(b"Omfatter maling av kjokkenbenk.", b"Omfatter maling: kjokkenbenk og skap.")
.replace(b"Omfatter utskifting av skadde", b"Omfatter 1:2 utskifting av skadde")
)
concepts = _by_title(_build(tmp_path, IDENTITY.name, data))
assert "description" not in concepts["Maling av gelender"]
assert concepts["Utskifting av enkeltkomponenter"]["description"] == FIRST.replace(
assert "description" not in concepts["Maling av kjokkenbenk"]
assert concepts["Utskifting av enkeltdeler"]["description"] == FIRST.replace(
"Omfatter utskifting", "Omfatter 1:2 utskifting"
)
@ -208,8 +208,8 @@ def test_a_description_the_gate_refuses_is_dropped_and_the_concept_is_kept(
concepts = _door(tmp_path, gate)
assert "description" not in concepts["Utskifting av enkeltkomponenter"]
assert concepts["Maling av gelender"]["description"] == SECOND
assert "description" not in concepts["Utskifting av enkeltdeler"]
assert concepts["Maling av kjokkenbenk"]["description"] == SECOND
def test_the_description_written_is_the_gates_sanitized_text(tmp_path: Path) -> None:
@ -218,6 +218,6 @@ def test_the_description_written_is_the_gates_sanitized_text(tmp_path: Path) ->
concepts = _door(tmp_path, gate)
assert concepts["Utskifting av enkeltkomponenter"]["description"] == FIRST.replace(
assert concepts["Utskifting av enkeltdeler"]["description"] == FIRST.replace(
"skadde", "[fjernet]"
)

2
tests/test_title_covered.py
View file

@ -19,7 +19,7 @@ WHY A PARTITION AND NOT A FOURTH SIGNAL. RRF consumes RANKS ONLY, so any one
signal contributes at most `1/(RRF_K + 1)`, and with shared ranks a signal
whose positive group is SMALL separates least of all: the group's members take
position 1 and everyone else position `len(group) + 1`, so a rule that fires on
two concepts of 2 761 is worth `1/61 - 1/63` to them. Precision is exactly such
two concepts of thousands is worth `1/61 - 1/63` to them. Precision is exactly such
a rule. `lookup_hits` above it is the same shape and was made a partition for
the same measured reason.
"""

16
tests/test_xml_declared_route.py
View file

@ -1,24 +1,24 @@
"""A structure the publisher DECLARED takes the route declared structure takes.
MEASURED, AND THE MEASUREMENT IS THE WHOLE REASON THIS FILE EXISTS. Round 13
gave `.xml` a reader that emits **2 761 of 2 761** ATX lines for a reference
standard's titled `<sec>` elements, with the text preserved exactly. At SHIPPED DEFAULTS the
build then delivered **23 concepts and 15 of 2 761 boundaries** -- everything
gave `.xml` a reader that emits **N of N** ATX lines for a reference
standard's N titled `<sec>` elements, with the text preserved exactly. At SHIPPED DEFAULTS the
build then delivered **23 concepts and 15 of N boundaries** -- everything
after the reader ate it. Two steps, each measured on that document and
reproduced in miniature by `sts-container.xml`:
- the **orphan check** removes a heading with nothing under its own first line.
It took **710 of 2 761**, and 710 of 710 removed headings are followed
It took **710 of N**, and 710 of 710 removed headings are followed
immediately by another heading while **0 of 2 051** delivered ones are: they
are container sections, not false positives. The check asks whether a GUESS
was a heading, which is the wrong question for a section a publisher named.
- **Arm F** (`--unit-fold`, a build default) folds every level deeper than the
shallowest repeated one into its parent. It took **2 066 more**, 2 089 -> 23,
because a process code whose chapters are all `14.121 ...` reads as a unit
because a reference standard whose chapters are all `14.121 ...` reads as a unit
sheet.
`find_candidates` already skips both for `outline_marks`, the PDF bookmark arm
-- which is why that arm reaches 2 762. This file pins the same route for the
-- which is why that arm reaches N + 1. This file pins the same route for the
one type whose reader KNOWS the structure because it wrote the heading itself,
under its OWN rule name, so an artifact still says which of the two a boundary
came from. Nothing else may move: the rule is reached only from the `.xml` row.
@ -59,7 +59,7 @@ def test_the_reader_reports_the_line_every_declared_section_landed_on() -> None:
"""No bridge, and that is the difference from the PDF arm.
The PDF arm has to map (page, `/XYZ` top) onto a line index and was wrong
on 1 840 of 2 762 under the naive rule. Here the reader WROTE the line, so
on 1 840 of N + 1 under the naive rule. Here the reader WROTE the line, so
the index is the one it appended at -- asserted against the text itself
rather than against a count, because a mark naming the right title at the
wrong offset passes any count.
@ -92,7 +92,7 @@ def test_a_declared_section_is_a_rule_of_its_own_and_not_the_bookmark_arms() ->
def test_a_container_section_survives_where_the_orphan_check_deletes_it() -> None:
"""710 of 2 761 on the reference standard; 1 of 9 here, and it is the chapter above the rest."""
"""710 of N on the reference standard; 1 of 9 here, and it is the chapter above the rest."""
text = extract.extract_text(CONTAINER.name, CONTAINER.read_bytes())
marks = extract.xml_outline(CONTAINER.name, CONTAINER.read_bytes())

22
tests/test_xml_sts.py
View file

@ -4,24 +4,24 @@ MEASURED OUTSIDE THIS REPOSITORY AND REPRODUCED HERE: the zip a publisher's
own viewer delivers as "Html" holds 0 html, 1 xml and 109 images, and `okf
build` on it gave **110 of 110 unreadable, 0 plans, exit 2** -- `.xml` was in
neither extractor registry. The one xml file IS the whole product: a large
regulatory reference document, the same one round 12 met as a 701-page PDF,
in NISO-STS form. It carries 7 715 `<sec>`, **2 761 of them with a `<title>`**,
and a `<sec>`-nesting depth distribution of 28/118/500/1141/868/97/9 that is
row-for-row identical to the fasit built from that same file.
regulatory reference document, the same one round 12 met as a PDF,
in NISO-STS form. It carries several thousand `<sec>`, **N (36 %) of them with a
`<title>`**, and a `<sec>`-nesting depth distribution that is row for row the
fasit's own, the fasit being built from that same file.
THE CEILING IS THEREFORE STRUCTURAL AND NOT COMPUTED: 2 761 of 2 761. The PDF
arm reached 2 759 of 2 761 by bridging from (page, y) to a line index; here the
THE CEILING IS THEREFORE STRUCTURAL AND NOT COMPUTED: N of N. The PDF
arm reached N - 2 of N (99.9 %) by bridging from (page, y) to a line index; here the
publisher states the structure in elements, so nothing is recovered and nothing
is guessed.
WHAT THESE TESTS PIN, each because the measurement said it could go wrong:
- `<label>` carries the number and `<title>` carries the text. Emitting only
`<title>` scores 0 of 2 761 while every line of the code looks right, because
`<title>` scores 0 of N while every line of the code looks right, because
the number is what okf reduces to a directory name.
- a `<sec>` with a `<label>` and NO `<title>` is never a heading. 4 954 of the
7 715 are lettered points (`a)`, `c)`) inside a process description, and one
heading each would bury the document's own 2 761.
- a `<sec>` with a `<label>` and NO `<title>` is never a heading. Most (64 %) of
the `<sec>` are lettered points (`a)`, `c)`) inside a process description, and one
heading each would bury the document's own N.
- the ONE titled section with no label -- `Forord` -- emitted without a
numbering token, because the corrected instrument key looks it up by title.
- a `<table-wrap>` as ONE markdown table block. The PDF path delivered 0 of 10.
@ -111,7 +111,7 @@ def test_the_label_and_the_title_reach_the_proposer_as_one_heading() -> None:
def test_a_lettered_point_is_a_body_line_and_never_a_heading() -> None:
"""4 954 of the reference standard's 7 715 `<sec>` are these. One heading each buries 2 761."""
"""64 % of the reference standard's `<sec>` are these. One heading each buries its N."""
text = extract.extract_text(STS.name, STS.read_bytes())
assert "a) Betong skal ha fasthetsklasse B35." in text.split("\n")

8
tests/test_yaml_frontmatter.py
View file

@ -24,7 +24,7 @@ Three readers must agree on every value written, each for its own reason:
MEASURED before any code moved (PyYAML 6.0.3): the K2 default bundle this
repository pins fails `safe_load` on 41 of its 455 frontmatter blocks, every
one a `title` (a leading `- `, `**` or `*`, a `": "`), and the reference-standard build on 1
of 2 763 (a title ending in `:`). No `sources` value okf itself wrote failed.
of a few thousand (a title ending in `:`). No `sources` value okf itself wrote failed.
"""
from __future__ import annotations
@ -97,7 +97,7 @@ def _concept(title: str, **overrides: object) -> str:
#: Plain today and plain after: a colon with no space, a `#` with no space
#: before it, `=`, `&`. Written byte-for-byte as before -- the known-negative
#: that keeps the fix from quoting what never needed it.
BLOCK_PLAIN = ["Q100:2023", "Kap#3", "P761 Oppskriftsboka:2025", "a=b&c", "1:2 utskifting"]
BLOCK_PLAIN = ["Q100:2023", "Kap#3", "P350 Oppskriftsboka:2025", "a=b&c", "1:2 utskifting"]
#: Each is refused or MISREAD by PyYAML as a plain block scalar.
BLOCK_NOT_PLAIN = [
@ -147,7 +147,7 @@ def test_a_run_stated_description_with_a_colon_space_reads_back(tmp_path: Path)
"""K3-19's 217: a DERIVED spec point carrying `": "` is still omitted
(`test_sts_description.py`); a STATED one is the caller's words and is
written in a form a YAML reader returns verbatim."""
value = "Omfatter maling: gelender og gjerder."
value = "Omfatter maling: kjokkenbenk og skap."
text = _concept(
"T", profile=SEGMENTED_OKF_V0_2, concept_frontmatter_values={"description": value}
)
@ -157,7 +157,7 @@ def test_a_run_stated_description_with_a_colon_space_reads_back(tmp_path: Path)
# --- the `sources` flow mapping ---------------------------------------------
#: Written verbatim inside the flow mapping, and every reader agrees.
FLOW_KEPT = ["Q100:2023", "P761 Oppskriftsboka 2025", "Kap#3", "a=b&c"]
FLOW_KEPT = ["Q100:2023", "P350 Oppskriftsboka 2025", "Kap#3", "a=b&c"]
#: No form both PyYAML and the guard read: plain, PyYAML refuses or misreads
#: it; quoted, the guard refuses it. Refused rather than written.

4
tools/okf_accounting_gate.py
View file

@ -97,8 +97,8 @@ CORPUS = FIXTURES / "corpus"
REJECTED = FIXTURES / "rejected"
INVENTORY = FIXTURES / "inventory.json"
REJECTED_INVENTORY = FIXTURES / "rejected-inventory.json"
STS_FIXTURE = CORPUS / "kapittel-84-sts.xml"
PDF_FIXTURE = CORPUS / "kapittel-84-tabell.pdf"
STS_FIXTURE = CORPUS / "kapittel-7-sts.xml"
PDF_FIXTURE = CORPUS / "kapittel-7-tabell.pdf"
README = REPO / "README.md"
#: Row 6 has NO default corpus: it measures a real corpus only when one is

4
tools/okf_retrieval_gate.py
View file

@ -1464,8 +1464,8 @@ def row_three(cases: Sequence[Case]) -> Row:
#:
#: WHAT IT DOES NOT CATCH, measured on the same data: one retired set's own
#: known-negative, a question naming a section number the document does not
#: have, sits at 0.2857 -- every other word of it is in a 2 756-concept
#: bundle. One term absent among seven is not a share, and no bar over this
#: have, sits at 0.2857 -- every other word of it is in a one-document bundle
#: of a few thousand concepts. One term absent among seven is not a share, and no bar over this
#: list reaches it. Row 4's denominator is the synthetic controls, so this
#: costs the row nothing and is stated rather than implied.
UNANSWERED_BAR = 2 / 3