fix(assets): an end-of-line escape at column 0 states no skip, and neither does a delta out of its row

Green: 23 of 23 in the suite that was 3 red, and the sweep's carried-here-
refused-there count goes 703 of 22 620 to 0, the drawn-differently count
1 492 to 32.

Two clauses, both refusing with the code the path already uses:

* an end-of-line escape at column 0. It closes no row, so the row it moves
  the cursor over is a row the stream never wrote -- PM's four-EOL frame,
  carried with 32 of 32 pixels never decoded while Pillow refuses the file.
* a delta whose horizontal offset would leave the row. The format puts that
  offset inside the line; this reader keeps the cursor past the row end and a
  flat decoder rolls it into the next row, which is two pictures from one
  stream.

CHOSEN OVER PM's RECOMMENDATION, and the recommendation was measured first:
refusing only a stream that painted nothing leaves 512 streams carried here
and refused there, and 1 171 drawn differently. It would have narrowed this
class for the third round running instead of closing it.

WHAT IS NOT CLOSED IS STATED, in the docstring and in the test: 32 of 22 620
streams are still drawn differently, every one of them a run or absolute
block that overruns its row. Refusing those gives 0 and 0 -- and costs 15 of
the 25 real RLE8 files, which would drop 15 figures and move a pinned
bundle's bytes. Measured on the corpus first: over 11 441 files scanned
across the four raw deliveries and the K2 reference corpus, the 25 BMPs on
this machine use an end-of-line at column 0 in 0 of 25 and a delta in 0 of
25, and 25 of 25 still decode to Pillow's pixels exactly (3 117 220 pixels
compared, after the change).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 22:47:29 +02:00
commit 574cf61272
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q

View file

@ -714,14 +714,37 @@ def _bmp_rle8_rows(data: bytes, head: _BmpHeader, *, name: str) -> bytes:
THE LINE IS THE CURSOR AND NOT THE PIXELS, and that is a format argument
rather than a corpus one -- the corpus cannot choose between the two, since
25 of 25 of those files paint every pixel, 25 of 25 reach the end of the
frame and 0 of 25 use a delta. A delta escape and an end-of-line escape
STATE their skip, so the pixels they pass over keep index 0 and every
decoder produces the same picture; a pixel-coverage count would refuse both
constructions the format defines. Pixels the stream never reached have no
agreed value at all, which is why an independent decoder refuses the file:
measured over eight streams for one frame, Pillow reads the five whose
cursor reaches the end and refuses the three whose does not, one of them
short by a single pixel (`tests/test_asset_viewable.py`).
frame and 0 of 25 use a delta. A pixel-coverage count would refuse both
constructions the format defines.
BUT A CURSOR THAT MOVED IS NOT A SKIP THAT WAS STATED, and the round that
wrote the cursor rule claimed more than it had measured: it said a delta
escape and an end-of-line escape both leave pixels every decoder agrees
on. Only the delta does. A delta states a DISTANCE, so an independent
decoder passes over exactly the same pixels; an end-of-line states "the
rest of THIS row", so an end-of-line at column 0 closes nothing and claims
a row the stream never wrote. Measured 2026-09-19: four end-of-line
escapes and an end-of-bitmap carried an 8x4 frame with 32 of 32 pixels
never decoded, and Pillow refuses those same bytes. So an end-of-line at
column 0 is REFUSED, and so is a delta whose horizontal offset would leave
the row -- the format puts that offset inside the line, and a reader that
keeps the cursor past the row end and a reader that rolls it into the next
row draw two different pictures from one stream.
THE DIRECTION THAT IS CLOSED, AND THE ONE THAT IS NOT. Over every opcode
sequence of length 1 to 4 on a 4x3 frame -- 22 620 streams, swept in
`tests/test_asset_viewable.py` rather than curated -- this reader carried
703 streams the independent decoder refuses and drew 1 492 more
differently. With these two clauses: 0 and 32. All 32 remaining are a run
or absolute block that OVERRUNS its row, which this reader clips at the
row end and Pillow spills into the next one. That last class is STATED
rather than closed, because closing it costs pictures: 15 of the 25 real
files overrun a row, and refusing them would drop 15 real figures and move
a pinned bundle's bytes. The two clauses that shipped cost nothing
measured -- over 11 441 files scanned across four raw standard deliveries
and the K2 reference corpus, the only 25 BMPs on this machine use an
end-of-line at column 0 in 0 of 25 and a delta in 0 of 25, and 25 of 25
still decode to Pillow's pixels exactly (3 117 220 compared).
"""
width, height = head.width, head.height
rows = [bytearray(width) for _ in range(height)]
@ -741,6 +764,14 @@ def _bmp_rle8_rows(data: bytes, head: _BmpHeader, *, name: str) -> bytes:
x += count
continue
if value == 0:
if x == 0:
raise ExtractionError(
f"the RLE8 stream in {name!r} ends a row at column 0 of a "
f"{width}x{height} frame -- an end-of-line escape states the rest of the "
"row it started, so one that started no row claims pixels the stream "
"never wrote",
code="asset_samples_invalid",
)
x = 0
y += 1
elif value == 1:
@ -749,6 +780,13 @@ def _bmp_rle8_rows(data: bytes, head: _BmpHeader, *, name: str) -> bytes:
elif value == 2:
if position + 2 > end:
break
if x + data[position] > width:
raise ExtractionError(
f"the RLE8 stream in {name!r} deltas from column {x} by "
f"{data[position]} in a {width}-wide frame -- the format puts that offset "
"inside the line, and readers disagree about where it lands",
code="asset_samples_invalid",
)
x += data[position]
y += data[position + 1]
position += 2