fix(assets): bound what the run pays, not what the document claims (0.10.1)
A second independent review read `230d1cb` -- the commit that closed the `v0.10.0` review's two MAJOR findings -- and found one of them open. The bound read `/Width` and `/Height`, which an untrusted document writes, while `get_data()` pays for the stream beside them; `/Length` is the COMPRESSED length and the two numbers are independent. Re-measured here on `ed8d9d7` before anything changed, in its own interpreter: a 408 516-byte PDF declaring 1x1 and carrying 400 MB of deflated zeros was CARRIED, no rejection, 891 904 000 B peak RSS. After: 0 carried, `asset_too_large`, 57 065 472 B. At 1,2 GB of zeros, 2 436 MB -> 64 569 344 B -- the cost no longer scales with the bomb. End to end through the CLI with the shipped defaults: 838 000 640 B and an asset written -> exit 0, 79 650 816 B, `0 carried of 1 found`, no `assets/`. Three numbers are bounded now, not one: what a container DECLARES, what a carried FILE measures (`read_image`, so a 49 MP PNG of 47 705 bytes is not passed on to a consumer), and what a PDF stream DECOMPRESSES to (`assets.inflated_size`, a chunk at a time, output discarded, before `get_data()`). The limit is stated rather than implied: the stream measurement runs where `FlateDecode` is the first filter and the document is not encrypted; every other chain is a check on the decoded length AFTER the decode, a counted refusal and not a bounded one. A non-positive declared dimension is `asset_size_invalid`, its own code, raised before the stream is read. `-1 x 40000000000` is a NEGATIVE pixel count, under which every `>` bound read as satisfied, so the check returned silently and the refusal arrived from `encode_png` as `asset_samples_invalid`. Its own code because a publisher shipping a picture bigger than this package carries and a dictionary written to be read wrong are different facts about a document. Two smaller findings in the line that says what is missing, both introduced by the first fix: the address was written twice, once bare, and a linkifying renderer autolinks a bare URL -- written once now, in one code span; and `label` became a dead parameter, so the figure's caption was dropped, a regression against 0.10.0. It is written again in the `-- <label>` form a carried pointer uses. Version bumped to 0.10.1 across all ten places. Nine were unbound and stale: four README install lines naming the previous release, two prose lines, the "current tag" entry, `uv.lock`, and a CHANGELOG whose 0.10.1 content sat under `[Unreleased]`. Two new packaging tests bind them to `__version__`, and the README's guard tag to `[tool.uv.sources]`. Every test was red first. The fate of every image is identical with and without the new bound on three K2 PDFs carrying 800 images (464/464, 311/311 with the same 12 rejections, 25/25), and the second inflate is below the noise floor there. 0 shipped artifacts move: no bundle under `examples/`, `skills/` or `tests/fixtures/` carries an image pointer at all, measured against a known-positive control. `asset_too_large` was undocumented in the error registry; both codes are there now. `tools/okf_accounting_gate.py` gains the new code in its closed list -- one string, no behaviour change, stated because that file belongs to another order. Suite 2141 passed / 1 skipped, ruff + format + mypy --strict clean. Report: docs/2026-09-18-bildestien-holder-0-10-1.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
ed8d9d709f
commit
0f308c1f56
15 changed files with 842 additions and 81 deletions
53
CLAUDE.md
53
CLAUDE.md
|
|
@ -890,27 +890,58 @@ and fixtures, never code.
|
|||
move (it is not delivered on the Arm B bundle either, by a different
|
||||
mechanism). hit@8 over the six published questions holds at **5 of 6 on both
|
||||
K2 bundles**, so the default move cost the retrieval side nothing.
|
||||
- **The image path has a size CEILING and no live links (0.10.1).** An
|
||||
independent review of `v0.10.0` found two MAJOR defects, both with the
|
||||
shipped defaults, both new in 0.10.0.
|
||||
- **The image path has a size CEILING and no live links (0.10.1).** TWO
|
||||
independent reviews, a day apart. The first found two MAJOR defects in
|
||||
`v0.10.0`, both with the shipped defaults, both new in 0.10.0. The second
|
||||
read the fix for them and found that ONE of the two was not closed --
|
||||
which is why the version below is 0.10.1 and not 0.10.0 with a patch note.
|
||||
- A remote `<img src>` / `xlink:href` became a LIVE markdown image link in
|
||||
the persisted concept. This package opens no socket; a consumer that
|
||||
renders the bundle does, and the guard refuses such a line at
|
||||
`user-upload` while the build's default tier persisted it. Now inert text
|
||||
with the address in a code span, pinned by a PROPERTY over the readers
|
||||
(`FOREIGN_IMAGE_LINK`), never one string.
|
||||
- Nothing bounded a declared image size: 9.6 KB of PDF declaring
|
||||
3 000 x 3 000 zeros took 83 MB peak RSS, linear in pixels.
|
||||
`MAX_IMAGE_PIXELS = 40 000 000` and `MAX_IMAGE_BYTES = 256 MiB` are
|
||||
checked on what the container DECLARES, before any decompression, and
|
||||
over them is `asset_too_large`, counted. Read off the corpora: the
|
||||
largest image measured is 18.6 MP (K2) and 1.4 MP (R761). The same bound
|
||||
closes the inline `data:` URI.
|
||||
- **THE FIRST FIX FOR IT WROTE THE ADDRESS TWICE**, once in a code span and
|
||||
once BARE, and a GFM/linkify renderer autolinks a bare URL. Weaker than an
|
||||
image link -- it takes a click -- but "inert" was half true. It also
|
||||
dropped the figure's CAPTION: `label` stayed in the signature and no
|
||||
branch read it, so an image the bundle does not carry lost the one fact
|
||||
that lets a reader weigh the absence. Both closed; the caption is written
|
||||
in the same `-- <label>` form a carried pointer uses.
|
||||
- **NOTHING BOUNDED WHAT THE RUN PAYS, only what the document CLAIMS.**
|
||||
`MAX_IMAGE_PIXELS = 40 000 000` and `MAX_IMAGE_BYTES = 256 MiB` were
|
||||
checked against `/Width` x `/Height`, which is an untrusted document's
|
||||
claim, while `get_data()` pays for the actual stream -- two independent
|
||||
numbers, since `/Length` is the COMPRESSED length. Measured by the second
|
||||
review: a **408 516-byte** PDF declaring **1x1** and carrying 400 MB of
|
||||
deflated zeros was CARRIED, no rejection, **892 MB peak RSS**, and 1,2 GB
|
||||
of zeros cost **2 436 MB** -- about 2 100x the file size, linear. The
|
||||
first fix's four mutations all died, because not one of them separated
|
||||
declared from actual.
|
||||
- **THREE NUMBERS ARE BOUNDED NOW.** What the container DECLARES (as
|
||||
before), what a carried FILE measures (`read_image`, so a 49 MP PNG of
|
||||
47 705 bytes is not passed on to the consumer with `7000x7000 px` beside
|
||||
it), and what a PDF image's STREAM decompresses to -- measured a chunk at
|
||||
a time and DISCARDED, before `get_data()`. With it: **0 carried,
|
||||
`asset_too_large`, 54 MB** at 400 MB and **62 MB** at 1,2 GB, so the cost
|
||||
no longer scales with the bomb. **The limit is stated rather than
|
||||
implied**: the stream measurement runs where `FlateDecode` is the first
|
||||
filter and the document is not encrypted; every other chain is a check on
|
||||
the decoded length AFTER the decode, which is a COUNTED refusal and not a
|
||||
bounded one.
|
||||
- **A non-positive declared dimension is `asset_size_invalid`**, its own
|
||||
code, raised before the stream is read. `-1 x 40 000 000 000` is a
|
||||
NEGATIVE pixel count, under which every `>` bound reads as satisfied: the
|
||||
check returned silently, 400 MB was decompressed, and the refusal came
|
||||
from the PNG encoder as `asset_samples_invalid`. Its own code because a
|
||||
publisher shipping a picture bigger than we carry and a dictionary written
|
||||
to be read wrong are different facts about a document.
|
||||
- **An inline PDF image is named from its position, not from `id()`** —
|
||||
pdfminer's name broke the bit-exact rebuild on two K2 concepts.
|
||||
- Documented rather than changed: `images: N` counts POINTER BLOCKS, and a
|
||||
pointer-only concept is substantive.
|
||||
- Report: `docs/2026-09-17-bildestien-0-10-1.md`.
|
||||
- Reports: `docs/2026-09-17-bildestien-0-10-1.md` and
|
||||
`docs/2026-09-18-bildestien-holder-0-10-1.md`.
|
||||
- **Account for content: `okf build --accounting PATH` and its judge
|
||||
`python3 tools/okf_accounting_gate.py` (2026-09-17).**
|
||||
- The gate was written RED at `0b00de4` and turned GREEN on all six rows by
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue