fix(assets): bound what the run pays, not what the document claims (0.10.1)
A second independent review read `230d1cb` -- the commit that closed the `v0.10.0` review's two MAJOR findings -- and found one of them open. The bound read `/Width` and `/Height`, which an untrusted document writes, while `get_data()` pays for the stream beside them; `/Length` is the COMPRESSED length and the two numbers are independent. Re-measured here on `ed8d9d7` before anything changed, in its own interpreter: a 408 516-byte PDF declaring 1x1 and carrying 400 MB of deflated zeros was CARRIED, no rejection, 891 904 000 B peak RSS. After: 0 carried, `asset_too_large`, 57 065 472 B. At 1,2 GB of zeros, 2 436 MB -> 64 569 344 B -- the cost no longer scales with the bomb. End to end through the CLI with the shipped defaults: 838 000 640 B and an asset written -> exit 0, 79 650 816 B, `0 carried of 1 found`, no `assets/`. Three numbers are bounded now, not one: what a container DECLARES, what a carried FILE measures (`read_image`, so a 49 MP PNG of 47 705 bytes is not passed on to a consumer), and what a PDF stream DECOMPRESSES to (`assets.inflated_size`, a chunk at a time, output discarded, before `get_data()`). The limit is stated rather than implied: the stream measurement runs where `FlateDecode` is the first filter and the document is not encrypted; every other chain is a check on the decoded length AFTER the decode, a counted refusal and not a bounded one. A non-positive declared dimension is `asset_size_invalid`, its own code, raised before the stream is read. `-1 x 40000000000` is a NEGATIVE pixel count, under which every `>` bound read as satisfied, so the check returned silently and the refusal arrived from `encode_png` as `asset_samples_invalid`. Its own code because a publisher shipping a picture bigger than this package carries and a dictionary written to be read wrong are different facts about a document. Two smaller findings in the line that says what is missing, both introduced by the first fix: the address was written twice, once bare, and a linkifying renderer autolinks a bare URL -- written once now, in one code span; and `label` became a dead parameter, so the figure's caption was dropped, a regression against 0.10.0. It is written again in the `-- <label>` form a carried pointer uses. Version bumped to 0.10.1 across all ten places. Nine were unbound and stale: four README install lines naming the previous release, two prose lines, the "current tag" entry, `uv.lock`, and a CHANGELOG whose 0.10.1 content sat under `[Unreleased]`. Two new packaging tests bind them to `__version__`, and the README's guard tag to `[tool.uv.sources]`. Every test was red first. The fate of every image is identical with and without the new bound on three K2 PDFs carrying 800 images (464/464, 311/311 with the same 12 rejections, 25/25), and the second inflate is below the noise floor there. 0 shipped artifacts move: no bundle under `examples/`, `skills/` or `tests/fixtures/` carries an image pointer at all, measured against a known-positive control. `asset_too_large` was undocumented in the error registry; both codes are there now. `tools/okf_accounting_gate.py` gains the new code in its closed list -- one string, no behaviour change, stated because that file belongs to another order. Suite 2141 passed / 1 skipped, ruff + format + mypy --strict clean. Report: docs/2026-09-18-bildestien-holder-0-10-1.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ed8d9d709f
commit
0f308c1f56
15 changed files with 842 additions and 81 deletions
60
CHANGELOG.md
60
CHANGELOG.md
|
|
@ -5,7 +5,7 @@ All notable changes to this project will be documented in this file.
|
|||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
## [0.10.1] — 2026-09-18
|
||||
|
||||
### Added
|
||||
|
||||
|
|
@ -84,21 +84,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
consumer into an SSRF. The guard refuses such a line at
|
||||
`guard-user-upload` and the build's default tier does not, so the same bytes
|
||||
were persisted under the default and refused one tier up. A remote reference
|
||||
is now inert text with the address in a code span, and a property test over
|
||||
is now inert text with the address in ONE code span, and a property test over
|
||||
the readers asserts that no reference produces a markdown image link outside
|
||||
`assets/`. Found by an independent review of 0.10.0 before it was pushed.
|
||||
- **A declared image size is bounded (0.10.1).** Nothing limited a PDF image's
|
||||
size: a 9.6 KB file declaring 3 000 x 3 000 grayscale zeros took 83 MB of
|
||||
peak RSS and a 63 KB one declaring 8 000 x 8 000 took 276 MB, linear in the
|
||||
pixel count, so one document could exhaust memory and take a whole batch
|
||||
build with it — before any gate, because the guard never sees image bytes.
|
||||
`MAX_IMAGE_PIXELS` (40 000 000) and `MAX_IMAGE_BYTES` (256 MiB) are now
|
||||
checked against what the container DECLARES, before anything is
|
||||
decompressed, and a larger image is refused with `asset_too_large` and
|
||||
counted. The bound is read off the corpora: the largest image in the
|
||||
43-document reference corpus is 18.6 MP and R761's largest is 1.4 MP. The
|
||||
same bound closes the inline `data:` URI, which the review flagged as the
|
||||
same class and did not measure.
|
||||
- The first fix wrote the address **twice** — once in a code span and once
|
||||
bare — and a GFM/linkify renderer autolinks a bare URL into `<a href>`.
|
||||
It takes a click rather than a render, so it is weaker than an image link,
|
||||
but "inert" was half true. The address is now written once.
|
||||
- The first fix also **dropped the caption**: `label` stayed in the
|
||||
signature of the line that says what is missing, and no branch read it, so
|
||||
the alt text or figure caption of an image the bundle does not carry was
|
||||
lost — a regression against 0.10.0 and against that line's own reason for
|
||||
existing. It is written again, in the same `-- <label>` form a carried
|
||||
pointer uses.
|
||||
- **An image is bounded in three places, and the third is what the run pays
|
||||
(0.10.1).** Nothing limited a PDF image's size: a 9.6 KB file declaring
|
||||
3 000 x 3 000 grayscale zeros took 83 MB of peak RSS and a 63 KB one
|
||||
declaring 8 000 x 8 000 took 276 MB, linear in the pixel count, so one
|
||||
document could exhaust memory and take a whole batch build with it — before
|
||||
any gate, because the guard never sees image bytes.
|
||||
- The size a container **declares** (`/Width` x `/Height`, an IHDR, a
|
||||
`data:` payload's encoded length) is checked against `MAX_IMAGE_PIXELS`
|
||||
(40 000 000) and `MAX_IMAGE_BYTES` (256 MiB) before anything is decoded.
|
||||
- The size a carried **file** has is checked the same way. This package
|
||||
never decodes such a file, so it pays nothing for it — but a 7 000 x 7 000
|
||||
PNG of 47 705 bytes written into a bundle hands the consumer the same bomb
|
||||
with `7000x7000 px` printed beside it.
|
||||
- What the **stream** behind a PDF image decompresses to is measured, a
|
||||
chunk at a time and discarded, before `get_data()` is called. That is an
|
||||
independent number from the declared size: `/Length` is the compressed
|
||||
length, and a second independent review measured a 408 516-byte PDF
|
||||
declaring a 1x1 picture and carrying 400 MB of deflated zeros being
|
||||
CARRIED, with no rejection, at 892 MB of peak RSS. With the bound: 0
|
||||
carried, `asset_too_large`, 54 MB — and 62 MB where the old path cost
|
||||
2 436 MB, so the cost no longer scales with the bomb.
|
||||
- The measurement runs where `FlateDecode` is the first filter and the
|
||||
document is not encrypted. Any other chain is caught by a check on
|
||||
`len(data)` AFTER `get_data()`, which is a counted refusal and not a
|
||||
bounded one. The difference is stated in the code rather than implied.
|
||||
- **A declared size that is not a size is refused with its own code
|
||||
(0.10.1).** `/Width -1 /Height 40000000000` multiplies to a NEGATIVE pixel
|
||||
count, under which every bound read as satisfied: the check returned
|
||||
silently, 400 MB was decompressed, and the refusal arrived from the PNG
|
||||
encoder as `asset_samples_invalid` — a code about a sample buffer for a
|
||||
defect in the declaration. A non-positive dimension is now `asset_size_invalid`,
|
||||
raised before the stream is read. Its own code because a legitimate
|
||||
publisher shipping a picture larger than this package carries and a
|
||||
dictionary written to be read wrong are different facts about a document.
|
||||
|
||||
### Fixed
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue