fix(assets): a remote reference is inert and a declared size is bounded

Two MAJOR findings of the independent v0.10.0 review, both with the
shipped defaults, both new in 0.10.0. Repros rebuilt as tests first.

- A remote <img src>/xlink:href became a LIVE markdown image link in the
  persisted concept, with the address and query string chosen by whoever
  wrote the document. Extraction opens no socket; a consumer rendering
  the bundle does. Now inert text with the address in a code span,
  pinned by a property over the readers rather than by one string. The
  tier asymmetry (user-upload refuses, trusted-source persisted) went to
  the guard repo with the repro.
- Nothing bounded a declared image size: 9.6 KB of PDF declaring
  3000x3000 grayscale zeros took 83 MB peak RSS, linear in pixels.
  MAX_IMAGE_PIXELS (40 000 000) and MAX_IMAGE_BYTES (256 MiB) are read
  off the corpora (largest measured 18.6 MP on K2, 1.4 MP on R761) and
  checked on what the container declares, before any decompression;
  over them is asset_too_large, counted. The same bound closes the
  inline data: URI, which the review flagged and did not measure.

Also fixed, added by PM to this order: an inline PDF image was named
from id() of a Python object, so two concept files of the reference
corpus differed between builds. It is now named from its position.

R761 unchanged: 50 carried of 50 found, assets diff -rq clean.

Report: docs/2026-09-17-bildestien-0-10-1.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-18 00:32:50 +02:00
commit 230d1cbccd
8 changed files with 580 additions and 3 deletions

View file

@ -0,0 +1,121 @@
# Two findings in the image path, closed before push (0.10.1)
Order `20260917T135034Z-4840022792`, from an independent review of `v0.10.0`
(`~/.claude/docs/2026-09-17-review-okf-0.10.0.local.md`, LOCAL-ONLY). The
review's verdict was "safe to push, with reservations": no blocker, and
byte-identity, determinism and `--no-assets` parity all hold. Both findings
land with the shipped defaults (`--assets` on, `--gate guard-trusted-source`),
and both are new in 0.10.0, because before it no reader read an `<img>`
attribute or opened an image stream.
Both repros were rebuilt here as tests before anything was fixed
(`tests/test_asset_limits.py`, 17 tests).
## MAJOR-1: a remote reference was a live markdown image link
Reproduced:
render_missing('https://collect.example.net/p.gif?u=S', href=…)
-> '![fig](https://collect.example.net/p.gif?u=S)\nImage: … (not carried: …)'
The first line is a live image link to an address the document's author
controls, query string included. This package opens no socket. A consumer that
renders the bundle, or an agent that fetches what it renders, does — which
turns "this bundle was opened" into a beacon (and a server-side consumer into
an SSRF). Measured with guard 1.4.0: `PRESET_USER_UPLOAD` fails the document
secure, `PRESET_TRUSTED_SOURCE` — the build's default — persists it.
**Fixed in `render_missing`:** a reference with an `href` is written as
`Image: <name> (not carried: <reason>) address: `<address>`` — inert text,
with the address still stated, because a reader has to know what stood there.
**Pinned as a property, not a string.** `FOREIGN_IMAGE_LINK` matches any
markdown image whose target is not this bundle's own `assets/`. It is asserted
over the two readers that resolve references (HTML, STS), over three shapes of
remote address (`https:`, `//host`, upper-case scheme), and over a whole built
bundle of the shipped fixture inbox. The known-positive beside it: a local
image still produces a pointer block that `IMAGE_POINTER` matches, and a
`data:` URI image is still carried.
The tier asymmetry is the guard's own question and was sent to
`llm-ingestion-pipeline-security` with the repro
(`20260917T221801Z-428505178`). Nothing was built there.
## MAJOR-2: nothing bounded a declared image size
Reproduced with the review's own generator, rebuilt in the test file: a PDF
declaring one grayscale image of compressed zeros.
| declared | PDF file | peak RSS |
|---|---:|---:|
| 3 000 x 3 000 | 9.6 KB | 83 MB |
| 8 000 x 8 000 | 63 KB | 276 MB (review's measurement) |
The cost is linear in the pixel count, so 50 000 x 50 000 is several GB. One
document — malicious, defective, or a legitimately enormous scan — could take
a whole batch build with it, before any gate, because the guard never sees
image bytes.
**The bound is read off the corpora, not chosen.** Over the 4 828 image
objects of the 43-document reference corpus the largest is 4 515 x 4 128
(18.6 MP, a landscape drawing). Over R761's 109 delivered pictures the largest
is 2 072 x 656 (1.4 MP).
- `MAX_IMAGE_PIXELS = 40 000 000` (2.1x the largest measured).
- `MAX_IMAGE_BYTES = 256 MiB` of samples.
- Over either: `asset_too_large`, counted like every other refusal, with the
declared size in the reason. Never a silent skip and never a killed build.
**Checked on what the container DECLARES, before anything is decompressed.**
`stream.get_data()` is what pays for the bomb, so the declared `Width` and
`Height` are read first. The order is observable, not asserted: the test feeds
a PDF whose image stream is corrupt AND whose declared size is over the bound.
Decoding first gives `asset_pdf_unsupported`; reading the size first gives
`asset_too_large`. `encode_png` refuses the same size on its own, so the
encoder does not trust its caller.
**The `data:` URI, which the review flagged and did not measure,** is closed by
the same bound, checked on the payload length before decoding (base64 expands
by 3/4). Measured: refused with `asset_too_large` as a row.
Not bounded, and stated rather than hidden: an image FILE carried verbatim is
never decoded, so its cost is its own file size.
## The determinism defect PM added to this order
pdfminer names an inline image (`BI … EI`) from `id()` of a Python object.
Measured 2026-09-17: two concept files of the reference corpus differed
between two builds of one commit, in the pointer line
(`Image: page-3-6344095824` against `page-3-4555034320`). That breaks the
bit-exact rebuild invariant.
**Fixed:** a name pdfminer derived from `id()` — all digits, no resource name —
is replaced by the image's position on the page (`page-3-inline-2`). The test
builds a PDF with an inline image and compares two extractions with the
`_pdf_pages` cache cleared between them.
## The three MINOR findings
- **`images: N` counts pointer blocks, not unique pictures** (12 pointers to
2 files is `images: 12`). Documented in the README rather than changed:
the number is the count of places a picture stands, dedup is on content, and
changing it would move bundle bytes and lose the placement count.
- **A concept that is only a pointer block is persisted as substantive.**
Documented: "degenerate" means zero characters after stripping whitespace,
and a pointer block is text. Changing the definition is a spec question.
- **`data:` URI:** closed, see above.
## Verification
- `tests/test_asset_limits.py`: 17 tests, both repros red first.
- Full suite 2 045 passed, 1 skipped; `ruff`, `ruff format --check` and
`mypy --strict` clean. Re-run after `git add` against a clean tree.
- **R761 unchanged (PM's measurement, repeated):** `--gate none` gives
`50 carried of 50 found`, 50 files in `assets/`, and `diff -rq` against the
bundle built before this change reports no difference.
- `tools/okf_accounting_gate.py`: green on all six rows.
## Proposal
Version `0.10.1`. No tag, no release, no push — the content accounting sits in
the same tree and PM wants the order of the two settled first.