fix(assets): bound what the run pays, not what the document claims (0.10.1)
A second independent review read `230d1cb` -- the commit that closed the `v0.10.0` review's two MAJOR findings -- and found one of them open. The bound read `/Width` and `/Height`, which an untrusted document writes, while `get_data()` pays for the stream beside them; `/Length` is the COMPRESSED length and the two numbers are independent. Re-measured here on `ed8d9d7` before anything changed, in its own interpreter: a 408 516-byte PDF declaring 1x1 and carrying 400 MB of deflated zeros was CARRIED, no rejection, 891 904 000 B peak RSS. After: 0 carried, `asset_too_large`, 57 065 472 B. At 1,2 GB of zeros, 2 436 MB -> 64 569 344 B -- the cost no longer scales with the bomb. End to end through the CLI with the shipped defaults: 838 000 640 B and an asset written -> exit 0, 79 650 816 B, `0 carried of 1 found`, no `assets/`. Three numbers are bounded now, not one: what a container DECLARES, what a carried FILE measures (`read_image`, so a 49 MP PNG of 47 705 bytes is not passed on to a consumer), and what a PDF stream DECOMPRESSES to (`assets.inflated_size`, a chunk at a time, output discarded, before `get_data()`). The limit is stated rather than implied: the stream measurement runs where `FlateDecode` is the first filter and the document is not encrypted; every other chain is a check on the decoded length AFTER the decode, a counted refusal and not a bounded one. A non-positive declared dimension is `asset_size_invalid`, its own code, raised before the stream is read. `-1 x 40000000000` is a NEGATIVE pixel count, under which every `>` bound read as satisfied, so the check returned silently and the refusal arrived from `encode_png` as `asset_samples_invalid`. Its own code because a publisher shipping a picture bigger than this package carries and a dictionary written to be read wrong are different facts about a document. Two smaller findings in the line that says what is missing, both introduced by the first fix: the address was written twice, once bare, and a linkifying renderer autolinks a bare URL -- written once now, in one code span; and `label` became a dead parameter, so the figure's caption was dropped, a regression against 0.10.0. It is written again in the `-- <label>` form a carried pointer uses. Version bumped to 0.10.1 across all ten places. Nine were unbound and stale: four README install lines naming the previous release, two prose lines, the "current tag" entry, `uv.lock`, and a CHANGELOG whose 0.10.1 content sat under `[Unreleased]`. Two new packaging tests bind them to `__version__`, and the README's guard tag to `[tool.uv.sources]`. Every test was red first. The fate of every image is identical with and without the new bound on three K2 PDFs carrying 800 images (464/464, 311/311 with the same 12 rejections, 25/25), and the second inflate is below the noise floor there. 0 shipped artifacts move: no bundle under `examples/`, `skills/` or `tests/fixtures/` carries an image pointer at all, measured against a known-positive control. `asset_too_large` was undocumented in the error registry; both codes are there now. `tools/okf_accounting_gate.py` gains the new code in its closed list -- one string, no behaviour change, stated because that file belongs to another order. Suite 2141 passed / 1 skipped, ruff + format + mypy --strict clean. Report: docs/2026-09-18-bildestien-holder-0-10-1.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ed8d9d709f
commit
0f308c1f56
15 changed files with 842 additions and 81 deletions
|
|
@ -133,6 +133,22 @@ class ExtractionError(IngestError):
|
|||
- `asset_samples_invalid` — the sample buffer does not fit the dimensions
|
||||
the image dictionary declares. Refused rather than padded: a short buffer
|
||||
means the dictionary was read wrong
|
||||
- `asset_too_large` — the picture is over this package's bound, either
|
||||
because it DECLARES a size beyond `MAX_IMAGE_PIXELS`, because the file
|
||||
itself is that large, or because the stream behind it DECOMPRESSES to
|
||||
more than `MAX_IMAGE_BYTES`. The three are one code because they are one
|
||||
decision — this run will not hold that picture — and because a consumer
|
||||
counting refusals wants the picture, not the mechanism. The bound is read
|
||||
off the corpora and sits an order of magnitude above anything measured
|
||||
- `asset_size_invalid` — the container declares a size that is not a size:
|
||||
a zero or negative `/Width` or `/Height`. DISTINCT from
|
||||
`asset_too_large`, because the two say different things about the
|
||||
document — one is a legitimate publisher shipping a picture bigger than
|
||||
this package carries, the other is a dictionary written wrong or written
|
||||
to be read wrong — and counting them together would make a corpus
|
||||
statistic about the first untrue. Refused before the stream is read: a
|
||||
negative dimension multiplies to a negative pixel count, under which
|
||||
every bound reads as satisfied
|
||||
"""
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue