docs: the BMP rest, the terminator and the pointer block
CHANGELOG under the UNTAGGED `[0.10.1]` -- no version bump, no tag. Three entries: the forged carry under Security, the truncated RLE8 under Fixed, and the two small rests (the judge's fidelity limit, the lossless guard's core arm) under "Documented, not changed". The report carries the measurements with their denominators, including the two checks this round introduced and then REMOVED after measuring that no mutant can fell them -- with the numbers for each (the gate suite green without the line anchor; 95 passed with and 95 without the length check, so the mutant the order asked for survives). CLAUDE.md records both defects and both rules where the viewable-asset paragraph already stands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
beef436cbb
commit
ae441abec4
3 changed files with 267 additions and 0 deletions
176
docs/2026-09-19-bmp-resten-terminator-og-pekerblokk.md
Normal file
176
docs/2026-09-19-bmp-resten-terminator-og-pekerblokk.md
Normal file
|
|
@ -0,0 +1,176 @@
|
|||
# The BMP rest: a terminator, and a claim that has to come from a pointer block
|
||||
|
||||
2026-09-19. Two defects PM's checkpoint on `c39b666` found in the
|
||||
viewable-asset round, both shipped, both with the default flags. The round
|
||||
that introduced them was judged PARTIAL; this closes the two load-bearing
|
||||
rests and the two small ones. No new format, no new dependency, no version
|
||||
bump and no tag.
|
||||
|
||||
## 1. A truncated RLE8 stream was carried, silently
|
||||
|
||||
`_bmp_rle8_rows` painted what the stream held and left the rest of the frame
|
||||
at palette index 0. That is what the format says about a pixel a delta
|
||||
SKIPPED, so no decoder disagrees — every one of them agrees on the wrong
|
||||
picture. Measured by PM on a real R761 asset, `1631e526275d-7-graphic-0003.bmp`,
|
||||
352x548 = 192 896 pixels:
|
||||
|
||||
| cut | result |
|
||||
|---|---|
|
||||
| stream truncated to 90 % | carried, 13 923 pixels wrong, no code, no row |
|
||||
| to 50 % | carried, 95 890 wrong |
|
||||
| to 10 % | carried, 166 525 wrong |
|
||||
|
||||
The uncompressed path already refused the same shape
|
||||
(`asset_samples_invalid`, "refusing to pad, because a short buffer means the
|
||||
header was read wrong"). Eval point 6 of the round's own order says "corrupt
|
||||
-> published code, never silent"; this was the one place it did not hold.
|
||||
|
||||
### The rule, read off the corpus rather than chosen
|
||||
|
||||
The order named two candidates — compare available bytes against
|
||||
`biSizeImage`, or require an end-of-bitmap escape. Measured first, over the
|
||||
19 real RLE8 assets of the frozen R761 delivery (`~/corpora/po-frosne-bundles/
|
||||
r761-2025-58e1ecca6007/assets`, read only):
|
||||
|
||||
```
|
||||
denominator (RLE8 files): 19
|
||||
stream reaches an explicit EOB: 19 of 19
|
||||
the EOB is the stream's LAST 2 bytes: 19 of 19
|
||||
biSizeImage == available bytes: 19 of 19
|
||||
bytes after the EOB: Counter({0: 19})
|
||||
```
|
||||
|
||||
Both candidates are satisfied by every real file, so the choice is made on
|
||||
what each MEANS. `biSizeImage` is a claim by the same untrusted header that
|
||||
declares the size; the terminator is a statement inside the data that the
|
||||
picture is complete. A third candidate, requiring every pixel to be painted,
|
||||
is wrong by construction: it would refuse the delta escape the format
|
||||
defines. So the decode may end at an explicit end-of-bitmap and nowhere else,
|
||||
and a stream that runs out first is refused with `asset_samples_invalid`.
|
||||
|
||||
A stream that is WHOLE but omits the terminator is refused alongside a cut
|
||||
one. From a reader's side they are the same bytes, and the measurement says
|
||||
no writer in this corpus produces the first.
|
||||
|
||||
### Nothing real changes hands
|
||||
|
||||
The same 19 files still convert losslessly after the rule: **19 of 19,
|
||||
2 366 365 pixels compared** — PM's own figure, re-measured here with stdlib on
|
||||
BOTH sides (an independent BMP reader and an independent `zlib` +
|
||||
filter-reversal PNG decoder, no Pillow anywhere), with a one-byte control
|
||||
proving the comparison can fail.
|
||||
|
||||
The guard in the suite is pinned to a decoder written in the test file, which
|
||||
computes what each cut costs: 8, 11, 17, 0 and 0 of 32 pixels. **Two of the
|
||||
five cuts lose no pixel at all**, which is the whole reason the rule has to be
|
||||
the terminator and not a pixel count.
|
||||
|
||||
## 2. The judge's new route could be forged by the document
|
||||
|
||||
`asset_holds` gained a second route in the viewable-asset round, because a
|
||||
converted image's own bytes are not in `assets/` and never will be. It reads
|
||||
the two digests the bundle states on the pointer line — with an expression
|
||||
that ran over the WHOLE bundle text. So a document could write the sentence
|
||||
itself. Measured by PM, and reproduced here before any code moved:
|
||||
|
||||
```
|
||||
the 50000x50000 BMP was REFUSED and is not in assets/: True
|
||||
judge asset_holds(refused BMP) = True (via the image's alt text)
|
||||
forged via PLAIN BODY TEXT -> asset_holds(refused BMP) = True
|
||||
```
|
||||
|
||||
Before that route existed, `asset_holds` hashed the source file and looked in
|
||||
`assets/` — nothing a document wrote could reach it. The gate's own first
|
||||
sentence is THE FASIT NEVER COMES FROM THE READER IT JUDGES, and `claimed and
|
||||
not found` had become something a document could ask to have silenced.
|
||||
|
||||
### Two halves, and neither is sufficient alone
|
||||
|
||||
**The judge** reads the clause only from inside a pointer block — the markdown
|
||||
image line plus the detail line under it — and only where the clause names the
|
||||
asset that block points at. That closes ordinary body text and a table cell.
|
||||
It cannot close an image's own `alt` text, because a LABEL is document text
|
||||
that the build writes INSIDE a pointer block.
|
||||
|
||||
**The door** is `assets._inline`, and this is where the boundary runs:
|
||||
everything it returns came from the document (an `alt` attribute, an STS
|
||||
caption, a publisher's file name); everything `render_block` appends after it
|
||||
came from the run (the size it measured, the type it sniffed, the digests it
|
||||
computed). The second line carries both, so document text may not emit the
|
||||
metadata grammar the run writes there. A checksum field is disarmed: the
|
||||
digits are kept, because a reader is owed what the document said, and the
|
||||
colon that makes them a FIELD is not.
|
||||
|
||||
### The mutants, and one finding about the harness
|
||||
|
||||
Three mutants in `tools/okf_gate_mutants.py`, one per check, each felled by
|
||||
its own arm: the pre-round `_conversions` restored verbatim (X3), the
|
||||
asset-name tie dropped (X4), `_inline`'s disarming dropped (X5).
|
||||
|
||||
**X5 survived 112 green tests on its first run, and had never been applied.**
|
||||
The harness copies the tree and runs pytest in the copy with the venv's
|
||||
interpreter — which carries an EDITABLE install pointing at the working tree,
|
||||
so `import llm_ingestion_okf` in the copy resolved to the original `src/`. The
|
||||
subprocess now gets the copy's own `src/` on `PYTHONPATH`, which wins over the
|
||||
editable finder, and X5 dies on the first run under it. The gate and the
|
||||
witness were never affected: the suite reaches those through the copy's own
|
||||
`tools/`.
|
||||
|
||||
`killed 39 of 39, exit 0` (36 before this round).
|
||||
|
||||
## 3. Two small rests
|
||||
|
||||
**The judge proves carriage, not fidelity.** PM's M10: a mutated converter
|
||||
writing a BLANK PNG gives `asset_holds = True`, because the bundle is
|
||||
internally consistent — the digest it claims to have written really is the
|
||||
asset's digest. The suite fells that mutant by decoding both sides; the gate
|
||||
cannot. The docstring now says so, and the limit is asserted in BOTH
|
||||
directions: a test shows the blank PNG is accepted AND that the docstring
|
||||
names the gap. Teaching the judge pixels is a different job.
|
||||
|
||||
**The lossless guard now runs on a core install.** It decoded through Pillow,
|
||||
which this package neither uses nor ships and which arrives only as a
|
||||
transitive dependency of `pdfplumber` under `[extract]`. Measured by PM on a
|
||||
core environment: 4 of the 13 guards in `tests/test_asset_viewable.py` were
|
||||
SKIPPED, the lossless one among them, so a plain `pip install
|
||||
llm-ingestion-okf` never measured the property the round is named after. The
|
||||
new arm decodes the carried PNG with `zlib` and the five PNG SS 9.2 filters
|
||||
and compares against pixels written out in the test file, over `read_image`
|
||||
and over what the BUILD wrote. The environment is MADE rather than assumed: a
|
||||
`sys.meta_path` finder raising for every `PIL` name, with a known-positive
|
||||
that it fires. Control, run once and not committed: a Pillow-dependent line
|
||||
placed inside the blocked section turns the guard red.
|
||||
|
||||
## 4. Two checks this round introduced and then removed
|
||||
|
||||
Both were measured and found unkillable, and a guard nothing can fell is not
|
||||
a guard.
|
||||
|
||||
- The end-of-line anchor on the judge's clause expression. With the
|
||||
pointer-block route and the asset-name tie in place, the only case it covers
|
||||
is a label forging a clause inside a block that also carries a real one —
|
||||
which `_inline` makes unreachable from a document. The whole gate suite is
|
||||
green without it.
|
||||
- The length check on a short absolute run in `_bmp_rle8_rows`. Unobservable
|
||||
by construction: a run shorter than its declared count means the stream ran
|
||||
out, so the loop cannot reach an end-of-bitmap afterwards and the picture is
|
||||
refused either way. Measured in a scratch copy, both directions: **95 passed
|
||||
with the check, 95 passed without it**, so the mutant that removes it
|
||||
SURVIVES. The order asked for that mutant; this is the answer to it.
|
||||
|
||||
A fifth cut replaces them and makes an existing bound load-bearing: a stream
|
||||
cut in the middle of a delta escape, 17 of 32 pixels wrong. Without the
|
||||
two-byte bound in that branch it raises `IndexError` instead of a coded
|
||||
refusal.
|
||||
|
||||
## 5. What this round did NOT measure
|
||||
|
||||
- The 43-document reference corpus was not rebuilt. The accounting gate's
|
||||
rows are unchanged (`GATE RED: rows 2, 3, 6`, row 3 identical at u = 19,
|
||||
d = 2, 17 of 22) — the operator-accepted losses of 2026-09-17.
|
||||
- No R761 end-to-end build was run this round; the 19-file lossless
|
||||
measurement is over the frozen delivery's own assets, which is the input
|
||||
that half of the build.
|
||||
- 16 JPEG 2000 objects are still lost with a line in the concept, and WebP is
|
||||
in the viewable set while `sniff` does not recognise one. Both are stated
|
||||
limits from the previous round and neither moved.
|
||||
Loading…
Add table
Add a link
Reference in a new issue