llm-ingestion-okf/docs/2026-08-21-g2-pdf-extraction-measurement.md

G2 measurement: one handbook PDF page through the extraction registry

Date: 2026-08-21 · Order: 20260821T170054Z-486638087-from-.claude (from .claude) Question this document answers: is a PDF parser behind the [extract] extra a small, bounded job, or is it a project?

This is a measurement, not an implementation. No production code was written, no version bumped, no tag cut, no guard pin moved. Filename is English per this repo's docs convention; the order suggested a Norwegian one. Grep anchor for the gap register: G2.

Verdict

Two answers, and the fork is the finding — but only one of the two branches is needed to unblock the trial stone.

1. Text extraction is small. One extractor function plus an import probe, at a seam that is two source lines wide. One library (pdfplumber, MIT) reads 308 of 308 pages of the measured handbook, is deterministic across runs, and recovers the requirement table on the sample page as correctly paired text lines (4 of 4 rows). This is a bounded job.
2. Recovering tables as structured markdown is a project, and it is not a parser choice. pdfplumber.extract_tables() and PyMuPDF.find_tables() — two independent implementations — return the same wrong structure for the sample table: one 2x6 grid where four logical rows collapse into a single cell with embedded newlines. The breakage is in the document's ruling geometry, not in either parser. Fixing it means layout heuristics this repo would own, specify and test. Across the whole handbook, only 45 of 196 detected table objects are structurally clean enough to hand to render_table unchanged.

Recommendation: take branch 1, decline branch 2 for now. The handbook enters the chain as prose with correctly paired table lines, which is what the trial stone needs to reach leg 3. Branch 2 is a separate decision that nothing currently waits on.

The choice of library is forced by branch 1, not free. Cheaper candidates parse the page but destroy the requirement table's row-to-value pairing — in a krav document that is a correctness hazard, not a cosmetic one. See Table pairing.

What was measured

Item	Value
Source	a public Norwegian technical handbook, one edition, PDF
Book bytes	9 879 066
Book pages	308
Sample page	page index 150 (printed page 151), in a chapter on dimensioning
Sample bytes	316 356
Why this page	running prose + one numbered requirement table (Tabell 524.1) + one real figure (Figur 524.1) + skal-requirements

The PDF lives under a scratch path, not in this repo — it is test material, not content. Everything below ran on Python 3.14.0 in throwaway virtualenvs; nothing was added to this repo's environment or its dependency set.

Probe 1 — the registry as it stands today

extract_text called on the page, with two known-positive controls in the same call, so the negative result is measured rather than assumed:

handbook-page.pdf: RAISED ExtractionError code='extractor_extra_missing' msg=extracting '.pdf'
  requires the optional 'extract' extra (pip install 'llm-ingestion-okf[extract]');
  it is not installed
control.md:  OK 48 chars -> '# Kontroll\n\nDette er en kjent-positiv kontroll.\n'
control.csv: OK 54 chars -> '| krav | verdi |\n| --- | --- |\n| Q200 5.4.1 | fritt |\n'

The probe can succeed; the .pdf rejection is real and typed, not a broken probe.

Probe 2 — the full Door B path

The same page dropped in an inbox next to a .md control, through process_inbox with the real guard adapter:

persisted : [PersistedFile(source_file='kontroll.md',
             path=.../bundle/inbox-kontroll.md, reasons=('clean: no findings',))]
failed    : [FailedFile(source_file='handbook-page.pdf',
             error=ExtractionError("extracting '.pdf' requires the optional 'extract' extra ..."))]
rejected  : ()   quarantined: ()

The behaviour is exactly what the contract promises: one bad file does not abort the run, the control persists and is indexed, and the PDF is reported per-file. Nothing here is a defect — leg 2 stops because the extra ships no parser, by design.

Candidates

License, release date and required dependencies read from PyPI metadata; package count and disk size measured by installing each into a clean virtualenv and listing it.

Candidate	Version	Released	License	Packages installed	site-packages
pypdf	6.16.1	2026-08-14	BSD-3-Clause	1	16 MB
pdfminer.six	20260107	2026-01-07	MIT	5	32 MB
pdfplumber	0.11.10	2026-06-15	MIT	8	56 MB
pymupdf	1.28.2	2026-08-06	AGPL-3.0 or Artifex commercial	1	71 MB

Three facts that matter more than the table:

• pymupdf is a licensing decision, not a technical one. AGPL-3.0 propagates to whoever installs the extra. This repo is MIT; shipping an AGPL default in an optional extra hands a consumer a copyleft obligation they did not choose. It is out on that ground alone, and it performs no better on the sample page than the MIT alternatives.
• pdfplumber pins pdfminer.six==20260107 exactly. A hard == in a transitive dependency is a co-installation hazard for any consumer that already uses pdfminer.six.
• pdfplumber pulls binary wheels (pillow, pypdfium2). Acceptable behind an opt-in extra; it would not be acceptable in core, and this repo's "exactly one runtime dependency" rule is untouched either way.

For completeness, the other two types the extra covers: python-docx 1.2.0 (MIT, 3 packages, 26 MB, pulls lxml) and openpyxl 3.1.5 (MIT, 2 packages, 15 MB). Both are materially simpler problems than PDF — the source formats are already structured XML — so they do not change the verdict, but they do mean [extract] is three parsers, not one.

What each candidate does to this page

pdfplumber.extract_text() gave about 30 lines, not reproduced here (they are the source's own text): a chapter heading, a section heading, four paragraphs of skal-requirements, the caption and four rows of Tabell 524.1 -- each label with its value on one line -- a figure caption, two more paragraphs and the running footer.

That is readable Norwegian technical prose, in reading order, with the table's rows intact as lines. Two fidelity defects are visible and neither is fatal: the subscript in F10 breaks across two lines, and the figure is gone — only its caption survives, because the figure is a vector drawing, not text. A figure whose content is only in the drawing is lost at leg 2 regardless of parser, and that is worth knowing before anyone promises a complete bundle.

Character counts on the same page: pypdf 1819, pdfminer.six 1834, pdfplumber 1761, pymupdf 1820. Volume is not the discriminator — order is.

Table pairing: the decisive number

Tabell 524.1 has four requirement rows (a class → a maximum value). Counting how many survive with label and value on the same line:

Parser	Paired rows
pdfplumber.extract_text()	4 / 4
pypdf.extract_text()	0 / 4
pdfminer.six.extract_text()	0 / 4
pymupdf.get_text()	0 / 4

The three zero-scorers emit all four labels, then all four values, as separate blocks. A downstream reader — human or model — can only re-pair them by guessing. In a requirements document, a row's value turning into its neighbour's value is a wrong answer that looks right. This is why the cheapest candidate is not the correct one.

What happens to the table as structure

Feeding render_table needs rows and cells, not lines. Both structural extractors return the same thing for this table:

pdfplumber.extract_tables() -> 1 table, 2 rows x 6 cols
['', '<label header>', '', '', '<value header>', '']
['<label 1>\n<label 2>\n<label 3>\n<label 4>', None, None,
 '<value 1>\n<value 2>\n<value 3>\n<value 4>', None, None]

pymupdf.find_tables()       -> 1 table, 2 rows x 6 cols   (byte-identical shape)

Four logical rows collapse into one cell each side; three of six columns are empty padding. render_table would faithfully render a two-row table that misrepresents the requirement. Two independent implementations agreeing on the same wrong shape is the evidence that this is the document's geometry, not a library bug — the handbook draws ruling lines around blocks, not rows.

So: handbooks of this kind are usable as prose, and are not usable as structured concept tables without a layer this repo does not have.

Determinism

Two runs of each configuration, compared byte-for-byte:

pypdf                     identical=True
pdfminer.six              identical=True
pdfplumber.text           identical=True
pdfplumber.tables         identical=True
pdfplumber slice[140:160] identical=True

Within one parser version, extraction is deterministic — the bit-exact promise survives. Across parser versions it is not guaranteed (ASSUMED, not measured): pdfminer.six uses date-stamped releases with no stability contract, so any golden fixture built on extracted text would be pinned to an exact parser version, and a parser upgrade becomes a fixture migration. That is a real, recurring maintenance cost and it belongs in the decision.

Denominators

Whole-book run, all 308 pages:

Measure	Count
Pages total	308
Pages yielding non-empty text	308 / 308
Lines matching ^Tabell <n> (tables the document claims)	149
Table objects detected by extract_tables()	196
Detected tables clean enough for render_table unchanged	45 / 196

"Clean enough" is defined in the measurement script, not by eye: at least two rows, rectangular, no None cell, no cell containing a newline. Anything failing that would either crash or silently misrepresent if handed to render_table.

Scope of these denominators: one handbook, one edition. They are not a claim about its series as a whole. Nothing here was measured on a scanned or image-only PDF; every page of this book carries a text layer, and a scanned handbook would score 0 and need OCR — a different project entirely.

Size of the wiring

The seam is small and already shaped for this. grep finds the gate at exactly two source lines — extract.py:30 (_OPTIONAL_EXTENSIONS) and extract.py:135 (the raise) — plus two test files asserting the current rejection (tests/test_extract.py, tests/test_error_codes.py). Adding a type means one _extract_pdf(data) function in the same shape as _extract_csv, an import probe replacing the frozenset membership test, and the same error code kept for the still-uninstalled case. The module docstring already describes this exact transition.

The wiring is hours. The cost is the choice above it: which library, which fidelity promise, which fixtures, and what happens to figures and tables that do not survive.

What this order deliberately did not do

• No parser implemented, no [extract] populated, no error message changed.
• No version bump, no tag, no publish, no guard pin move.
• No writes in any other repo. The PDF stays in scratch.

Verification log

#	Claim	Status	Evidence
1	The handbook PDF is public and downloadable from its publisher	VERIFIED	curl -L -w '%{http_code} %{content_type} %{size_download}' → 200 application/pdf 9879066; magic bytes %PDF-1.6
2	Book has 308 pages; sample is index 150	VERIFIED	pypdf.PdfReader(...).pages length; makepage.py
3	.pdf fails with extractor_extra_missing, controls pass in the same call	VERIFIED	probe_registry.py output, quoted above
4	Full Door B path reports the PDF as failed and persists the control	VERIFIED	probe_inbox.py output, quoted above
5	[extract] is declared and empty	VERIFIED	pyproject.toml [project.optional-dependencies] extract = []
6	Candidate licenses and release dates	VERIFIED	PyPI JSON API per package (pypi.org/pypi/<name>/json)
7	Package counts and disk sizes	VERIFIED	one clean python3 -m venv per candidate; pip list --format=freeze, du -sk
8	pdfplumber pins pdfminer.six==20260107	VERIFIED	PyPI requires_dist
9	Table row pairing 4/4 vs 0/4	VERIFIED	pairing.py / pairing_mupdf.py, ground truth read off the rendered page
10	extract_tables() and find_tables() return the same 2x6 shape	VERIFIED	cand_pdfplumber.py, cand_pymupdf.py
11	Determinism within a version, 5 configurations	VERIFIED	determinism.py, sha256 per run
12	308/308 pages with text; 149 captions; 196 detected; 45 clean	VERIFIED	denominator.py, whole book, ~41 s
13	Gate seam is two source lines plus two test files	VERIFIED	grep -rn '_OPTIONAL_EXTENSIONS|extractor_extra_missing' src/ tests/
14	Cross-version output stability is not guaranteed	ASSUMED	inferred from pdfminer.six date-stamped versioning; not measured across versions
15	Figures are lost because they are vector drawings	VERIFIED (this page)	caption present, no figure text in any parser's output; not generalised to the book
16	Denominators generalise beyond this handbook	NOT CLAIMED	one handbook, one edition measured