feat(assets): every carried image is one a model can be shown

Chosen: a stdlib BMP reader, because `read_image` is on the CORE path and an
asset's name is its content digest. Measured first, as the order requires:
Pillow 12.3.0 IS in this tree (transitively under `pdfplumber`) and it DOES
decode RLE8 correctly -- a hand-written stdlib decoder and Pillow agree on
19 of 19 of R761's real files, RGB per pixel. So the choice does not rest on
capability. It rests on two properties of this package: `.html` and `.xml`
carry images with no `[extract]` extra installed, so a Pillow converter
either makes a core path depend on an optional binary wheel or buys the
second runtime dependency; and encoding through an installed library would
make a bundle's identity move with that library's version, which is the
property 0.10.0 felled page rasterisation over and `encode_png`'s docstring
already defends. Pillow keeps the job it is good for: the INDEPENDENT decoder
in the tests, on neither side of the conversion.

The defect, measured over the frozen R761 delivery's `assets/`, denominator
50: 29 JPEG, 2 PNG and 19 RLE8 BMP. The 19 are byte-correct files nothing
reads, so 19 figures were present and invisible while `images: N` reported
that they had arrived.

- `VIEWABLE_MEDIA_TYPES` is tested against every asset's SNIFFED type, so it
  is a property and not a list of formats we met. WebP is on it and `sniff`
  does not recognise one; the limit is stated, not implied.
- `bmp_to_png`: 8-bit uncompressed, 8-bit RLE8, 24-bit uncompressed. All five
  RLE8 opcodes. 19 of 19 real files convert with RGB identical to Pillow's
  decoding of the source, 2 366 365 pixels compared.
- `asset_not_viewable` and `asset_bmp_unsupported`, both published, both
  leaving the concept's "not carried" line.
- Traceability on the pointer's second line, where the rest of the asset
  metadata already lives: original media type, original sha256 in full, new
  sha256 in full. A converted asset is ONE asset.
- The ceiling is paid on the DECLARATION before a row is allocated, and an
  RLE run is one clipped slice -- painting pixel by pixel leaves the memory
  bounded and the CPU unbounded.

Two repairs the change forced, each measured rather than assumed:

- `tests/test_assets.py`'s "dimensions absent is absent" used a TIFF, which
  is now refused before `read_image` returns. The property still has a
  reachable case -- a JPEG whose frame header never arrives -- and uses it.
- `asset_holds` in the accounting gate proved a carry by hashing the SOURCE
  file, which a converted image's bundle cannot satisfy. It now also reads
  the two digests the bundle states and HASHES THE ASSET ITSELF, so a bundle
  claiming a conversion it did not perform still fails.

`tools/okf_asset_census.py` is the committed instrument for the
known-positive: one row per image, from two pinned trees. It was caught by
the rule it serves -- its first version handed `_pdf_images` the wrong page
object and reported 0 images over 67 PDFs with exit 0. The attribute is
asserted now and a known-positive runs before the sweep.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 08:07:04 +02:00
commit b0b5e71658
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
10 changed files with 826 additions and 5 deletions

View file

@ -156,11 +156,19 @@ def test_dimensions_absent_is_absent_never_zero() -> None:
"""A format whose size this module does not read says so, rather than 0x0.
`0x0 px` in a concept is a measurement nobody took, printed as a fact.
The vehicle used to be a TIFF. Since the viewable-asset round a TIFF never reaches
`read_image`'s return at all -- it is a format no model can be shown and
there is no lossless conversion for it, so it is refused as
`asset_not_viewable`. The property still has a reachable case, and this is
it: a JPEG whose frame header never arrives. `sniff` types it from the
first three bytes, `_jpeg_dimensions` walks to the end and finds no SOF,
and the pointer has to say so rather than print a size.
"""
tiff = b"II\x2a\x00" + b"\x00" * 16
assert assets.sniff(tiff) == ("image/tiff", ".tiff")
assert assets.dimensions(tiff) is None
image = assets.read_image(tiff, name="scan.tiff")
headless = b"\xff\xd8\xff\xfe\x00\x04ab\xff\xd9"
assert assets.sniff(headless) == ("image/jpeg", ".jpg")
assert assets.dimensions(headless) is None
image = assets.read_image(headless, name="scan.jpg")
assert image.width is None and image.height is None
assert "dimensions unknown" in assets.render_block(image)

View file

@ -357,6 +357,29 @@ def test_the_readme_states_that_image_bytes_are_not_screened() -> None:
assert "image bytes are not screened" in text.lower()
_VIEWABLE_LINE = re.compile(
r"^<!-- asset-viewable-media-types: ([a-z0-9/,+.-]+) -->$", re.MULTILINE
)
def test_the_readme_publishes_the_viewable_set_the_code_applies() -> None:
"""A published set is a test obligation, the same as a published bound.
Sorted on both sides so the marker states a SET and not an order, and
compared as a whole rather than by membership: a README naming three of
four would pass every containment check and still tell a consumer that a
format is refused when it is carried.
"""
from llm_ingestion_okf.assets import VIEWABLE_MEDIA_TYPES
match = _VIEWABLE_LINE.search(README.read_text(encoding="utf-8"))
assert match is not None, (
"README carries no `<!-- asset-viewable-media-types: ... -->` marker; without it "
"the set a consumer reads and the set the code applies can drift apart silently"
)
assert match.group(1).split(",") == sorted(VIEWABLE_MEDIA_TYPES)
_MAX_PIXELS_LINE = re.compile(r"^<!-- asset-max-pixels: (\d+) -->$", re.MULTILINE)