fix(assets): bound what the run pays, not what the document claims (0.10.1)

A second independent review read `230d1cb` -- the commit that closed the
`v0.10.0` review's two MAJOR findings -- and found one of them open. The
bound read `/Width` and `/Height`, which an untrusted document writes, while
`get_data()` pays for the stream beside them; `/Length` is the COMPRESSED
length and the two numbers are independent.

Re-measured here on `ed8d9d7` before anything changed, in its own
interpreter: a 408 516-byte PDF declaring 1x1 and carrying 400 MB of deflated
zeros was CARRIED, no rejection, 891 904 000 B peak RSS. After: 0 carried,
`asset_too_large`, 57 065 472 B. At 1,2 GB of zeros, 2 436 MB -> 64 569 344 B
-- the cost no longer scales with the bomb. End to end through the CLI with
the shipped defaults: 838 000 640 B and an asset written -> exit 0,
79 650 816 B, `0 carried of 1 found`, no `assets/`.

Three numbers are bounded now, not one: what a container DECLARES, what a
carried FILE measures (`read_image`, so a 49 MP PNG of 47 705 bytes is not
passed on to a consumer), and what a PDF stream DECOMPRESSES to
(`assets.inflated_size`, a chunk at a time, output discarded, before
`get_data()`). The limit is stated rather than implied: the stream
measurement runs where `FlateDecode` is the first filter and the document is
not encrypted; every other chain is a check on the decoded length AFTER the
decode, a counted refusal and not a bounded one.

A non-positive declared dimension is `asset_size_invalid`, its own code,
raised before the stream is read. `-1 x 40000000000` is a NEGATIVE pixel
count, under which every `>` bound read as satisfied, so the check returned
silently and the refusal arrived from `encode_png` as
`asset_samples_invalid`. Its own code because a publisher shipping a picture
bigger than this package carries and a dictionary written to be read wrong
are different facts about a document.

Two smaller findings in the line that says what is missing, both introduced
by the first fix: the address was written twice, once bare, and a linkifying
renderer autolinks a bare URL -- written once now, in one code span; and
`label` became a dead parameter, so the figure's caption was dropped, a
regression against 0.10.0. It is written again in the `-- <label>` form a
carried pointer uses.

Version bumped to 0.10.1 across all ten places. Nine were unbound and stale:
four README install lines naming the previous release, two prose lines, the
"current tag" entry, `uv.lock`, and a CHANGELOG whose 0.10.1 content sat
under `[Unreleased]`. Two new packaging tests bind them to `__version__`, and
the README's guard tag to `[tool.uv.sources]`.

Every test was red first. The fate of every image is identical with and
without the new bound on three K2 PDFs carrying 800 images (464/464, 311/311
with the same 12 rejections, 25/25), and the second inflate is below the
noise floor there. 0 shipped artifacts move: no bundle under `examples/`,
`skills/` or `tests/fixtures/` carries an image pointer at all, measured
against a known-positive control.

`asset_too_large` was undocumented in the error registry; both codes are
there now. `tools/okf_accounting_gate.py` gains the new code in its closed
list -- one string, no behaviour change, stated because that file belongs to
another order.

Suite 2141 passed / 1 skipped, ruff + format + mypy --strict clean.
Report: docs/2026-09-18-bildestien-holder-0-10-1.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-18 13:41:18 +02:00
commit 0f308c1f56
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
15 changed files with 842 additions and 81 deletions

View file

@ -48,6 +48,7 @@ from .assets import (
check_payload,
check_size,
encode_png,
inflated_size,
read_image,
render_block,
render_missing,
@ -1365,6 +1366,40 @@ def _pdf_alpha(attrs: dict[str, object], width: int, height: int) -> bytes | Non
return alpha if len(alpha) >= width * height else False
def _check_inflated(stream: object, name: str) -> None:
"""Refuse a PDF image stream whose DECOMPRESSED size is over the bound.
`check_size` reads `/Width` and `/Height`, which an untrusted document
writes and which say nothing about what `get_data()` will return: `/Length`
is the COMPRESSED length, and a dictionary declaring 1x1 may hang 400 MB of
deflated zeros off it. Measured by an independent review of 0.10.1 on
`230d1cb`: that document is 408 516 bytes, was carried without a rejection,
and cost 892 MB of peak RSS -- the same failure mode the 0.10.0 review
named, reached through the other number.
WHAT THIS BOUNDS, and what it does not. The measurement runs on the RAW
stream, so it applies where `FlateDecode` is the first filter -- the
overwhelming case, and the one every deflate bomb needs. It does not run
when the stream is already decoded, when the document is encrypted (the raw
bytes are ciphertext until pdfminer deciphers them), or when a chain puts
another filter in front. Those are caught by `check_payload` AFTER
`get_data()`, which makes them a counted refusal rather than a bounded one:
the memory is spent and then the picture is dropped. Stated rather than
implied, because the difference is exactly what the review found missing.
"""
from pdfminer.pdftypes import LITERALS_FLATE_DECODE
if getattr(stream, "decipher", None) is not None:
return
raw = stream.get_rawdata() if hasattr(stream, "get_rawdata") else None
if not raw:
return
filters = stream.get_filters() if hasattr(stream, "get_filters") else []
if not filters or filters[0][0] not in LITERALS_FLATE_DECODE:
return
inflated_size(raw, name=name)
def _pdf_image(stream: object, name: str) -> ExtractedImage:
"""One image XObject, carried verbatim where it already is a file.
@ -1394,16 +1429,22 @@ def _pdf_image(stream: object, name: str) -> ExtractedImage:
# THE DECLARED SIZE IS READ FIRST, and the stream is not touched until it
# is within the bound: `get_data()` decompresses, so a check after it has
# already paid for a picture of compressed zeros.
# already paid for a picture of compressed zeros. `channels=1` because the
# colour space is not resolved until further down and one channel is the
# LOWEST estimate of the cost -- a bound that under-counts refuses nothing
# it should carry, and the pixel count beside it does not depend on it.
# What that leaves unbounded is the stream, which `_check_inflated` reads
# for what it actually costs rather than for what it claims.
declared = dict(getattr(stream, "attrs", {}))
declared_width = resolve1(declared.get("Width"))
declared_height = resolve1(declared.get("Height"))
check_size(
resolve1(declared.get("Width")) if isinstance(resolve1(declared.get("Width")), int) else 0,
resolve1(declared.get("Height"))
if isinstance(resolve1(declared.get("Height")), int)
else 0,
declared_width if isinstance(declared_width, int) else None,
declared_height if isinstance(declared_height, int) else None,
name=name,
channels=1,
)
_check_inflated(stream, name)
try:
data = stream.get_data() # type: ignore[attr-defined]
except Exception as exc:
@ -1411,6 +1452,10 @@ def _pdf_image(stream: object, name: str) -> ExtractedImage:
f"the PDF image stream behind {name!r} could not be decoded: {exc}",
code="asset_pdf_unsupported",
) from exc
# THE BACKSTOP, and it is a weaker guarantee than the one above it: this
# one COUNTS a stream that `_check_inflated` could not bound in advance
# (see its docstring for which those are), after the memory has been spent.
check_payload(len(data), name=name)
if data and sniff(data) is not None:
return read_image(data, name=name)