llm-ingestion-okf/docs/2026-08-21-g2-pdf-extraction-measurement.md

# G2 measurement: one handbook PDF page through the extraction registry

**Date:** 2026-08-21 · **Order:** `20260821T170054Z-486638087-from-.claude` (from `.claude`)
**Question this document answers:** is a PDF parser behind the `[extract]` extra a small,
bounded job, or is it a project?

This is a measurement, not an implementation. No production code was written, no version
bumped, no tag cut, no guard pin moved. Filename is English per this repo's docs convention;
the order suggested a Norwegian one. Grep anchor for the gap register: **G2**.

## Verdict

**Two answers, and the fork is the finding — but only one of the two branches is needed
to unblock the trial stone.**

1. **Text extraction is small.** One extractor function plus an import probe, at a seam that
   is two source lines wide. One library (`pdfplumber`, MIT) reads 308 of 308 pages of the
   measured handbook, is deterministic across runs, and recovers the requirement table on the
   sample page as correctly paired text lines (4 of 4 rows). This is a bounded job.
2. **Recovering tables as *structured* markdown is a project, and it is not a parser choice.**
   `pdfplumber.extract_tables()` and `PyMuPDF.find_tables()` — two independent
   implementations — return the *same* wrong structure for the sample table: one 2x6 grid
   where four logical rows collapse into a single cell with embedded newlines. The breakage
   is in the document's ruling geometry, not in either parser. Fixing it means layout
   heuristics this repo would own, specify and test. Across the whole handbook, only
   **45 of 196** detected table objects are structurally clean enough to hand to
   `render_table` unchanged.

**Recommendation: take branch 1, decline branch 2 for now.** The handbook enters the chain as
prose with correctly paired table lines, which is what the trial stone needs to reach leg 3.
Branch 2 is a separate decision that nothing currently waits on.

**The choice of library is forced by branch 1, not free.** Cheaper candidates parse the page
but destroy the requirement table's row-to-value pairing — in a `krav` document that is a
correctness hazard, not a cosmetic one. See [Table pairing](#table-pairing-the-decisive-number).

## What was measured

| Item | Value |
|---|---|
| Source | a public Norwegian technical handbook, one edition, PDF |
| Book bytes | 9 879 066 |
| Book pages | 308 |
| Sample page | page index 150 (printed page 151), in a chapter on dimensioning |
| Sample bytes | 316 356 |
| Why this page | running prose + one numbered requirement table (`Tabell 524.1`) + one real figure (`Figur 524.1`) + `skal`-requirements |

The PDF lives under a scratch path, not in this repo — it is test material, not content.
Everything below ran on Python 3.14.0 in throwaway virtualenvs; nothing was added to this
repo's environment or its dependency set.

## Probe 1 — the registry as it stands today

`extract_text` called on the page, with two known-positive controls **in the same call**, so
the negative result is measured rather than assumed:

```
handbook-page.pdf: RAISED ExtractionError code='extractor_extra_missing' msg=extracting '.pdf'
  requires the optional 'extract' extra (pip install 'llm-ingestion-okf[extract]');
  it is not installed
control.md:  OK 48 chars -> '# Kontroll\n\nDette er en kjent-positiv kontroll.\n'
control.csv: OK 54 chars -> '| krav | verdi |\n| --- | --- |\n| Q200 5.4.1 | fritt |\n'
```

The probe can succeed; the `.pdf` rejection is real and typed, not a broken probe.

## Probe 2 — the full Door B path

The same page dropped in an inbox next to a `.md` control, through `process_inbox` with the
real guard adapter:

```
persisted : [PersistedFile(source_file='kontroll.md',
             path=.../bundle/inbox-kontroll.md, reasons=('clean: no findings',))]
failed    : [FailedFile(source_file='handbook-page.pdf',
             error=ExtractionError("extracting '.pdf' requires the optional 'extract' extra ..."))]
rejected  : ()   quarantined: ()
```

The behaviour is exactly what the contract promises: one bad file does not abort the run, the
control persists and is indexed, and the PDF is reported per-file. Nothing here is a defect —
leg 2 stops because the extra ships no parser, by design.

## Candidates

License, release date and required dependencies read from PyPI metadata; package count and
disk size measured by installing each into a clean virtualenv and listing it.

| Candidate | Version | Released | License | Packages installed | site-packages |
|---|---|---|---|---|---|
| `pypdf` | 6.16.1 | 2026-08-14 | BSD-3-Clause | 1 | 16 MB |
| `pdfminer.six` | 20260107 | 2026-01-07 | MIT | 5 | 32 MB |
| `pdfplumber` | 0.11.10 | 2026-06-15 | MIT | 8 | 56 MB |
| `pymupdf` | 1.28.2 | 2026-08-06 | AGPL-3.0 **or** Artifex commercial | 1 | 71 MB |

Three facts that matter more than the table:

- **`pymupdf` is a licensing decision, not a technical one.** AGPL-3.0 propagates to whoever
  installs the extra. This repo is MIT; shipping an AGPL default in an optional extra hands a
  consumer a copyleft obligation they did not choose. It is out on that ground alone, and it
  performs no better on the sample page than the MIT alternatives.
- **`pdfplumber` pins `pdfminer.six==20260107` exactly.** A hard `==` in a transitive
  dependency is a co-installation hazard for any consumer that already uses `pdfminer.six`.
- **`pdfplumber` pulls binary wheels** (`pillow`, `pypdfium2`). Acceptable behind an opt-in
  extra; it would not be acceptable in core, and this repo's "exactly one runtime dependency"
  rule is untouched either way.

For completeness, the other two types the extra covers: `python-docx` 1.2.0 (MIT, 3 packages,
26 MB, pulls `lxml`) and `openpyxl` 3.1.5 (MIT, 2 packages, 15 MB). Both are materially
simpler problems than PDF — the source formats are already structured XML — so they do not
change the verdict, but they do mean `[extract]` is three parsers, not one.

## What each candidate does to this page

`pdfplumber.extract_text()` gave about 30 lines, not reproduced here (they are the
source's own text): a chapter heading, a section heading, four paragraphs of
`skal`-requirements, the caption and four rows of `Tabell 524.1` -- each label with its
value on one line -- a figure caption, two more paragraphs and the running footer.

That is readable Norwegian technical prose, in reading order, with the table's rows intact as
lines. Two fidelity defects are visible and neither is fatal: the subscript in `F10` breaks
across two lines, and the figure is gone — only its caption survives, because the figure is a
vector drawing, not text. A figure whose content is only in the drawing is lost at leg 2
regardless of parser, and that is worth knowing before anyone promises a complete bundle.

Character counts on the same page: `pypdf` 1819, `pdfminer.six` 1834, `pdfplumber` 1761,
`pymupdf` 1820. **Volume is not the discriminator — order is.**

### Table pairing: the decisive number

`Tabell 524.1` has four requirement rows (a class → a maximum value). Counting how many
survive with label and value on the same line:

| Parser | Paired rows |
|---|---|
| `pdfplumber.extract_text()` | **4 / 4** |
| `pypdf.extract_text()` | 0 / 4 |
| `pdfminer.six.extract_text()` | 0 / 4 |
| `pymupdf.get_text()` | 0 / 4 |

The three zero-scorers emit all four labels, then all four values, as separate blocks. A
downstream reader — human or model — can only re-pair them by guessing. In a requirements
document, a row's value turning into its neighbour's value is a wrong answer that looks right.
This is why the cheapest candidate is not the correct one.

### What happens to the table as *structure*

Feeding `render_table` needs rows and cells, not lines. Both structural extractors return the
same thing for this table:

```
pdfplumber.extract_tables() -> 1 table, 2 rows x 6 cols
['', '<label header>', '', '', '<value header>', '']
['<label 1>\n<label 2>\n<label 3>\n<label 4>', None, None,
 '<value 1>\n<value 2>\n<value 3>\n<value 4>', None, None]

pymupdf.find_tables()       -> 1 table, 2 rows x 6 cols   (byte-identical shape)
```

Four logical rows collapse into one cell each side; three of six columns are empty padding.
`render_table` would faithfully render a two-row table that misrepresents the requirement.
Two independent implementations agreeing on the same wrong shape is the evidence that this is
the document's geometry, not a library bug — the handbook draws ruling lines around blocks, not rows.

**So: handbooks of this kind are usable as prose, and are not usable as structured concept tables
without a layer this repo does not have.**

## Determinism

Two runs of each configuration, compared byte-for-byte:

```
pypdf                     identical=True
pdfminer.six              identical=True
pdfplumber.text           identical=True
pdfplumber.tables         identical=True
pdfplumber slice[140:160] identical=True
```

Within one parser version, extraction is deterministic — the bit-exact promise survives.
**Across parser versions it is not guaranteed** (ASSUMED, not measured): `pdfminer.six` uses
date-stamped releases with no stability contract, so any golden fixture built on extracted
text would be pinned to an exact parser version, and a parser upgrade becomes a fixture
migration. That is a real, recurring maintenance cost and it belongs in the decision.

## Denominators

Whole-book run, all 308 pages:

| Measure | Count |
|---|---|
| Pages total | 308 |
| Pages yielding non-empty text | **308 / 308** |
| Lines matching `^Tabell <n>` (tables the document claims) | 149 |
| Table objects detected by `extract_tables()` | 196 |
| Detected tables clean enough for `render_table` unchanged | **45 / 196** |

"Clean enough" is defined in the measurement script, not by eye: at least two rows,
rectangular, no `None` cell, no cell containing a newline. Anything failing that would either
crash or silently misrepresent if handed to `render_table`.

Scope of these denominators: **one handbook, one edition**. They are not a
claim about its series as a whole. Nothing here was measured on a scanned or image-only
PDF; every page of this book carries a text layer, and a scanned handbook would score 0 and
need OCR — a different project entirely.

## Size of the wiring

The seam is small and already shaped for this. `grep` finds the gate at exactly two source
lines — `extract.py:30` (`_OPTIONAL_EXTENSIONS`) and `extract.py:135` (the raise) — plus two
test files asserting the current rejection (`tests/test_extract.py`,
`tests/test_error_codes.py`). Adding a type means one `_extract_pdf(data)` function in the
same shape as `_extract_csv`, an import probe replacing the frozenset membership test, and
the same error code kept for the still-uninstalled case. The module docstring already
describes this exact transition.

**The wiring is hours. The cost is the choice above it**: which library, which fidelity
promise, which fixtures, and what happens to figures and tables that do not survive.

## What this order deliberately did not do

- No parser implemented, no `[extract]` populated, no error message changed.
- No version bump, no tag, no publish, no guard pin move.
- No writes in any other repo. The PDF stays in scratch.

## Verification log

| # | Claim | Status | Evidence |
|---|---|---|---|
| 1 | The handbook PDF is public and downloadable from its publisher | VERIFIED | `curl -L -w '%{http_code} %{content_type} %{size_download}'` → `200 application/pdf 9879066`; magic bytes `%PDF-1.6` |
| 2 | Book has 308 pages; sample is index 150 | VERIFIED | `pypdf.PdfReader(...).pages` length; `makepage.py` |
| 3 | `.pdf` fails with `extractor_extra_missing`, controls pass in the same call | VERIFIED | `probe_registry.py` output, quoted above |
| 4 | Full Door B path reports the PDF as `failed` and persists the control | VERIFIED | `probe_inbox.py` output, quoted above |
| 5 | `[extract]` is declared and empty | VERIFIED | `pyproject.toml` `[project.optional-dependencies] extract = []` |
| 6 | Candidate licenses and release dates | VERIFIED | PyPI JSON API per package (`pypi.org/pypi/<name>/json`) |
| 7 | Package counts and disk sizes | VERIFIED | one clean `python3 -m venv` per candidate; `pip list --format=freeze`, `du -sk` |
| 8 | `pdfplumber` pins `pdfminer.six==20260107` | VERIFIED | PyPI `requires_dist` |
| 9 | Table row pairing 4/4 vs 0/4 | VERIFIED | `pairing.py` / `pairing_mupdf.py`, ground truth read off the rendered page |
| 10 | `extract_tables()` and `find_tables()` return the same 2x6 shape | VERIFIED | `cand_pdfplumber.py`, `cand_pymupdf.py` |
| 11 | Determinism within a version, 5 configurations | VERIFIED | `determinism.py`, sha256 per run |
| 12 | 308/308 pages with text; 149 captions; 196 detected; 45 clean | VERIFIED | `denominator.py`, whole book, ~41 s |
| 13 | Gate seam is two source lines plus two test files | VERIFIED | `grep -rn '_OPTIONAL_EXTENSIONS\|extractor_extra_missing' src/ tests/` |
| 14 | Cross-version output stability is not guaranteed | ASSUMED | inferred from `pdfminer.six` date-stamped versioning; not measured across versions |
| 15 | Figures are lost because they are vector drawings | VERIFIED (this page) | caption present, no figure text in any parser's output; not generalised to the book |
| 16 | Denominators generalise beyond this handbook | NOT CLAIMED | one handbook, one edition measured |