llm-ingestion-okf/src/llm_ingestion_okf
Kjell Tore Guttormsen 0f308c1f56
fix(assets): bound what the run pays, not what the document claims (0.10.1)
A second independent review read `230d1cb` -- the commit that closed the
`v0.10.0` review's two MAJOR findings -- and found one of them open. The
bound read `/Width` and `/Height`, which an untrusted document writes, while
`get_data()` pays for the stream beside them; `/Length` is the COMPRESSED
length and the two numbers are independent.

Re-measured here on `ed8d9d7` before anything changed, in its own
interpreter: a 408 516-byte PDF declaring 1x1 and carrying 400 MB of deflated
zeros was CARRIED, no rejection, 891 904 000 B peak RSS. After: 0 carried,
`asset_too_large`, 57 065 472 B. At 1,2 GB of zeros, 2 436 MB -> 64 569 344 B
-- the cost no longer scales with the bomb. End to end through the CLI with
the shipped defaults: 838 000 640 B and an asset written -> exit 0,
79 650 816 B, `0 carried of 1 found`, no `assets/`.

Three numbers are bounded now, not one: what a container DECLARES, what a
carried FILE measures (`read_image`, so a 49 MP PNG of 47 705 bytes is not
passed on to a consumer), and what a PDF stream DECOMPRESSES to
(`assets.inflated_size`, a chunk at a time, output discarded, before
`get_data()`). The limit is stated rather than implied: the stream
measurement runs where `FlateDecode` is the first filter and the document is
not encrypted; every other chain is a check on the decoded length AFTER the
decode, a counted refusal and not a bounded one.

A non-positive declared dimension is `asset_size_invalid`, its own code,
raised before the stream is read. `-1 x 40000000000` is a NEGATIVE pixel
count, under which every `>` bound read as satisfied, so the check returned
silently and the refusal arrived from `encode_png` as
`asset_samples_invalid`. Its own code because a publisher shipping a picture
bigger than this package carries and a dictionary written to be read wrong
are different facts about a document.

Two smaller findings in the line that says what is missing, both introduced
by the first fix: the address was written twice, once bare, and a linkifying
renderer autolinks a bare URL -- written once now, in one code span; and
`label` became a dead parameter, so the figure's caption was dropped, a
regression against 0.10.0. It is written again in the `-- <label>` form a
carried pointer uses.

Version bumped to 0.10.1 across all ten places. Nine were unbound and stale:
four README install lines naming the previous release, two prose lines, the
"current tag" entry, `uv.lock`, and a CHANGELOG whose 0.10.1 content sat
under `[Unreleased]`. Two new packaging tests bind them to `__version__`, and
the README's guard tag to `[tool.uv.sources]`.

Every test was red first. The fate of every image is identical with and
without the new bound on three K2 PDFs carrying 800 images (464/464, 311/311
with the same 12 rejections, 25/25), and the second inflate is below the
noise floor there. 0 shipped artifacts move: no bundle under `examples/`,
`skills/` or `tests/fixtures/` carries an image pointer at all, measured
against a known-positive control.

`asset_too_large` was undocumented in the error registry; both codes are
there now. `tools/okf_accounting_gate.py` gains the new code in its closed
list -- one string, no behaviour change, stated because that file belongs to
another order.

Suite 2141 passed / 1 skipped, ruff + format + mypy --strict clean.
Report: docs/2026-09-18-bildestien-holder-0-10-1.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 13:41:18 +02:00
..
__init__.py fix(assets): bound what the run pays, not what the document claims (0.10.1) 2026-09-18 13:41:18 +02:00
_pandoc.py feat(extract): resolve the vendored converter and refuse an unpinned version 2026-09-02 14:07:20 +02:00
accounting.py feat(accounting): okf build accounts for every source element 2026-09-17 18:35:20 +02:00
assets.py fix(assets): bound what the run pays, not what the document claims (0.10.1) 2026-09-18 13:41:18 +02:00
cli.py feat(accounting): okf build accounts for every source element 2026-09-17 18:35:20 +02:00
connectors.py fix(errors): type the NUL-byte path failure as SourceError 2026-07-20 10:30:42 +02:00
consume.py feat(consume): the door's link line is out of the default reading 2026-09-12 18:56:53 +02:00
contract_check.py feat(consume): parent reaches the reader -- excerpt field, body link, checker rule 2026-09-11 12:36:23 +02:00
corpus.py feat(accounting): okf build accounts for every source element 2026-09-17 18:35:20 +02:00
errors.py fix(assets): bound what the run pays, not what the document claims (0.10.1) 2026-09-18 13:41:18 +02:00
extract.py fix(assets): bound what the run pays, not what the document claims (0.10.1) 2026-09-18 13:41:18 +02:00
guard_adapter.py fix(gate,propose): okf build runs a real guard; a code fence declares no structure 2026-09-16 00:19:50 +02:00
importer.py feat(assets): a bundle carries the images its sources declare (0.10.0) 2026-09-17 10:01:31 +02:00
inbox.py feat(accounting): okf build accounts for every source element 2026-09-17 18:35:20 +02:00
manifest.py feat(profiles): materialize_bundle takes a keyword-only profile (req 6) 2026-07-31 15:37:17 +02:00
materialize.py docs(k3): round 24, a reason that became false is struck and the emitter stays flow 2026-09-12 16:50:26 +02:00
profiles.py feat(assets): a bundle carries the images its sources declare (0.10.0) 2026-09-17 10:01:31 +02:00
project.py feat(accounting): okf build accounts for every source element 2026-09-17 18:35:20 +02:00
propose.py feat(assets): a bundle carries the images its sources declare (0.10.0) 2026-09-17 10:01:31 +02:00
py.typed feat(typing): ship the PEP 561 py.typed marker 2026-07-17 03:46:51 +02:00
quality.py feat(quality): okf quality --fasit, boundary recall against a declared structure 2026-09-13 07:27:33 +02:00
render.py feat(errors): stable machine-readable codes on IngestError 2026-07-17 04:01:04 +02:00
segmentation.py feat(consume): parent reaches the reader -- excerpt field, body link, checker rule 2026-09-11 12:36:23 +02:00
skill.py feat(consume): parent reaches the reader -- excerpt field, body link, checker rule 2026-09-11 12:36:23 +02:00
structure.py feat(assets): a bundle carries the images its sources declare (0.10.0) 2026-09-17 10:01:31 +02:00