open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

test(pdf): two bookmarks on one line, and nothing counts the one that is lost

Browse source 
RED. Measured on R761: 2 763 /Outlines nodes entered the bridge, 2 762 marks
came out, and `unresolved` was 0 -- so one node left no trace in the return
value at all. The lost one is `SVV - Forside`, sharing line 0 with the tree's
own root node `R761 Prosesskoden`. The cause is structural, not an edge value:
the marks are collected in a dict keyed on the destination line index, so a
second bookmark on a line is discarded by `setdefault` in silence. One node
today; a document with several bookmarks per line loses several and reports
none of them.

`outline-collision.pdf` is the fixture no existing one could stand in for --
every other outline fixture puts one bookmark on one line, which is exactly
the case where the defect cannot appear. It is laid out by hand through
`make_fixtures.py` like the other PDF fixtures, and regenerating that file
left every other fixture byte-identical.

The known-negative is in the same commit: `outline-collided.pdf` aside, the
two existing trees must report `collided == 0`, so the counter can be measured
at zero and is not green for an unstated reason.

pytest -q: 2 failed, 1554 passed, 1 skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-10 03:16:14 +02:00
commit 3f8f03cff5
4 changed files with 106 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
tests/fixtures/README.md vendored
View file

@ -17,6 +17,7 @@ and reproducible from that one file.
|---|---|
| `two-line-krav.pdf` | One heading plus one requirement row with label and value on the **same line**. That pairing is the property `pdfplumber` was chosen for. |
| `no-text-layer.pdf` | A structurally valid page with no text operators — the shape a scanned or image-only PDF presents. Must fail fast (`extractor_empty_pdf`), never persist as an empty concept. |
| `outline-collision.pdf` | **Two bookmarks whose destinations resolve to the same line** — the tree's root node and a front-matter node, both on line 0, which is the shape R761 carries. The marks are collected in a dict keyed on the line index, so without this fixture the second node is dropped with nothing counting it: 2 763 nodes in, 2 762 marks out, `unresolved` at 0. A one-bookmark-per-line fixture cannot see that. |
| `three-page-krav.pdf` | Three pages, one line of text each, and **the middle page carries no text operators**. The extractor drops empty pages, so the last page's text belongs to page 3 — which is what separates a page NUMBER from a count of the pages that produced text. Two pages could not tell those apart. |
## The office fixtures

22
tests/fixtures/make_fixtures.py vendored
View file

@ -113,6 +113,23 @@ OUTLINED_TREE = (
("1.2.1 Materialer", 3, 2, 150),
)
#: TWO bookmarks whose destinations resolve to the SAME line, mirroring what
#: R761 carries: its tree's root node `R761 Prosesskoden` and the node
#: `SVV - Forside` both land on line 0. Measured on that document, 2 763 nodes
#: entered the bridge and 2 762 marks came out with `unresolved` at 0 -- the
#: difference was a dict keyed on the line index, dropping the second node with
#: nothing counting it. A one-bookmark-per-line fixture cannot see that.
COLLISION_CONTENTS = (
b"BT /F1 12 Tf 20 170 Td (R761 Prosesskoden) Tj ET\n"
b"BT /F1 12 Tf 20 150 Td (Innledende tekst om grunnlaget.) Tj ET\n",
)
#: Both point at `/XYZ 20 185`, which is above the page's first line.
COLLISION_TREE = (
("R761 Prosesskoden", 1, 0, 185),
("SVV - Forside", 2, 0, 185),
)
#: One resolvable bookmark and one whose `/Dest` names an object that is not a
#: page. A PDF in the wild carries these; R761 carries none of them, so
#: without this fixture the "drop it, count it, do not fabricate a boundary"
@ -608,6 +625,11 @@ if __name__ == "__main__":
(HERE / "outlined-krav.pdf").write_bytes(build_outlined_pdf(OUTLINED_CONTENTS, OUTLINED_TREE))
print("wrote outlined-krav.pdf")
(HERE / "outline-collision.pdf").write_bytes(
build_outlined_pdf(COLLISION_CONTENTS, COLLISION_TREE)
)
print("wrote outline-collision.pdf")
(HERE / "outline-broken-dest.pdf").write_bytes(
build_outlined_pdf(BROKEN_DEST_CONTENT, (("1 Grunnlag", 1, 0, 185),), broken_dest=True)
)

45
tests/fixtures/outline-collision.pdf vendored Normal file
View file

@ -0,0 +1,45 @@
%PDF-1.4
1 0 obj
<< /Type /Catalog /Pages 2 0 R /Outlines 6 0 R >>
endobj
2 0 obj
<< /Type /Pages /Kids [3 0 R] /Count 1 >>
endobj
3 0 obj
<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >>
endobj
4 0 obj
<< /Length 112 >>
stream
BT /F1 12 Tf 20 170 Td (R761 Prosesskoden) Tj ET
BT /F1 12 Tf 20 150 Td (Innledende tekst om grunnlaget.) Tj ET
endstream
endobj
5 0 obj
<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>
endobj
6 0 obj
<< /Type /Outlines /First 7 0 R /Last 7 0 R /Count 2 >>
endobj
7 0 obj
<< /Title (R761 Prosesskoden) /Parent 6 0 R /First 8 0 R /Last 8 0 R /Count 1 /Dest [3 0 R /XYZ 20 185 0] >>
endobj
8 0 obj
<< /Title (SVV - Forside) /Parent 7 0 R /Dest [3 0 R /XYZ 20 185 0] >>
endobj
xref
0 9
0000000000 65535 f
0000000009 00000 n
0000000074 00000 n
0000000131 00000 n
0000000257 00000 n
0000000419 00000 n
0000000516 00000 n
0000000587 00000 n
0000000711 00000 n
trailer
<< /Size 9 /Root 1 0 R >>
startxref
797
%%EOF