fix(assets): the stream must reach the end of the frame, not just say stop

Chose the CURSOR over a pixel-coverage count because the corpus cannot
choose -- 25 of 25 of the R761 delivery's RLE8 BMPs paint every pixel,
25 of 25 reach the end of the frame, 0 of 25 use a delta -- and an
independent decoder can: a delta and an end-of-line escape state their
skip, so every decoder agrees on the index-0 pixels they pass over, while
a pixel count would refuse both constructions the format defines.

`_bmp_rle8_rows` now refuses (`asset_samples_invalid`) when the terminator
arrives with the cursor short of the last row. Pillow reads 5 of the 8
streams in the table and refuses the same 3, one of them short by a
single pixel.

Both docstrings the round was sent to correct are rewritten: the test no
longer claims every pixel is decoded (it is not -- a stated skip keeps
index 0), and `_bmp_rle8_rows` no longer frames the delta argument as
read off the corpus, which it never was.

R761 rebuilt: bundle `diff -r`-identical to the build before this commit,
50 assets (29 JPEG + 21 PNG), 19 of 19 conversions, SHY 71, u = 0, d = 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 19:16:32 +02:00
commit 561b609a53
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
2 changed files with 44 additions and 16 deletions

View file

@ -696,14 +696,32 @@ def _bmp_rle8_rows(data: bytes, head: _BmpHeader, *, name: str) -> bytes:
HERE and nowhere else, and `_bmp_flat_rows` refuses the same shape with the
same code.
The rule is the terminator rather than `biSizeImage`, which is a claim by
the same untrusted header, and rather than a coverage count, which would
refuse the delta the format defines. Read off the corpus: over the 19 real
RLE8 assets of the frozen R761 delivery, 19 of 19 end at an explicit
end-of-bitmap, on 19 of 19 it is the stream's LAST two bytes, and on 19 of
19 `biSizeImage` equals the available bytes -- so requiring it costs
The terminator is checked rather than `biSizeImage`, which is a claim by
the same untrusted header. Read off the corpus: over the 25 RLE8 BMPs
the frozen R761 delivery ships (24 distinct; the bundle carries 19 of
them, the rest being an unpointed duplicate and four no concept names),
25 of 25 end at an explicit
end-of-bitmap, on 25 of 25 it is the stream's LAST two bytes, and on 25 of
25 `biSizeImage` equals the available bytes -- so requiring it costs
nothing measured here, and a whole stream that omits it is refused
alongside a cut one because from the reader's side they are the same bytes.
AND THE TERMINATOR ALONE IS NOT A COVERAGE PROOF, because a stream may say
it is finished anywhere: measured 2026-09-19, one whose FIRST two bytes are
the end-of-bitmap escape was carried with every pixel of the frame never
decoded. So the cursor must also stand at or past the end of the last row.
THE LINE IS THE CURSOR AND NOT THE PIXELS, and that is a format argument
rather than a corpus one -- the corpus cannot choose between the two, since
25 of 25 of those files paint every pixel, 25 of 25 reach the end of the
frame and 0 of 25 use a delta. A delta escape and an end-of-line escape
STATE their skip, so the pixels they pass over keep index 0 and every
decoder produces the same picture; a pixel-coverage count would refuse both
constructions the format defines. Pixels the stream never reached have no
agreed value at all, which is why an independent decoder refuses the file:
measured over eight streams for one frame, Pillow reads the five whose
cursor reaches the end and refuses the three whose does not, one of them
short by a single pixel (`tests/test_asset_viewable.py`).
"""
width, height = head.width, head.height
rows = [bytearray(width) for _ in range(height)]
@ -748,6 +766,12 @@ def _bmp_rle8_rows(data: bytes, head: _BmpHeader, *, name: str) -> bytes:
"-- refusing to carry a frame whose remaining pixels were never decoded",
code="asset_samples_invalid",
)
if y < height - 1 or (y == height - 1 and x < width):
raise ExtractionError(
f"the RLE8 stream in {name!r} ends at row {y} column {x} of a {width}x{height} "
"frame -- refusing to carry a picture whose last rows the stream never reached",
code="asset_samples_invalid",
)
if not head.top_down:
rows.reverse()
return b"".join(bytes(row) for row in rows)

View file

@ -576,14 +576,18 @@ def test_a_truncated_rle8_stream_is_refused_with_a_published_code() -> None:
a stream cut to 90 % carried with 13 923 pixels wrong, to 50 % with 95 890,
to 10 % with 166 525 -- no code, no row, a partly blank PNG under a name
that says it holds the source's pixels. The uncompressed path already
refuses the same shape. NO PIXEL MAY BE GUESSED: either every one of them
is decoded from the stream, or the picture is refused.
refuses the same shape. NO PIXEL IS GUESSED THAT THE STREAM DID NOT SKIP
ON PURPOSE: a stream must say it is finished, and
`test_a_stream_that_stops_before_the_frame_is_refused` adds the other half
-- when it says so, the cursor must have reached the end of the frame.
Pixels a delta or an end-of-line escape passed over keep index 0, which is
the format's own value and what an independent decoder produces.
THE RULE IS THE TERMINATOR, and it is read off the corpus rather than
chosen: over the 19 real RLE8 assets of the frozen R761 delivery, 19 of 19
end at an explicit end-of-bitmap escape, that escape is the stream's LAST
two bytes on 19 of 19, and `biSizeImage` equals the available bytes on
19 of 19. So a stream that runs out before its terminator is refused, and
THIS GUARD IS THE TERMINATOR, and it is read off the corpus rather than
chosen: over the 25 RLE8 BMPs the frozen R761 delivery ships (24 distinct;
the bundle carries 19), 25 of 25 end at an explicit end-of-bitmap escape, that escape is the stream's LAST
two bytes on 25 of 25, and `biSizeImage` equals the available bytes on
25 of 25. So a stream that runs out before its terminator is refused, and
so is one that is whole but never states it is finished -- the two are the
same bytes from a reader's side, and the measurement says no real writer
here produces the second.
@ -675,8 +679,8 @@ def test_a_stream_that_stops_before_the_frame_is_refused() -> None:
not -- including one that is short by a single pixel.
A PIXEL coverage count would be a different rule and a wrong one: it
refuses the delta the format defines, and 0 of the 25 real RLE8 sources in
the R761 delivery would be affected either way (25 of 25 paint every
refuses the delta the format defines, and 0 of the 25 RLE8 BMPs
the R761 delivery ships would be affected either way (25 of 25 paint every
pixel, 25 of 25 reach the end of the frame, 0 of 25 use a delta), so the
corpus cannot choose between them. The independent decoder can.
"""