open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

docs: the end-of-line stream that paints nothing, and the four smaller rests

Browse source 
The round's report, and the two published sentences it makes false.

CLAUDE.md said a delta and an end-of-line escape both state a skip every
decoder agrees on. True of the delta, false of the end-of-line, and the
paragraph now says what was measured instead: 703 of 22 620 swept streams
carried here and refused by Pillow, 1 492 drawn differently, 0 and 32 after
the two clauses, and the third clause that would give 0 and 0 priced at 15 of
the 25 real files.

The accounting-gate section gains the two rests about what a CONSUMER is
told: the published door contract now names the keys the gate reads, and a
bundle built without the door says why a converted image cannot be proved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 23:11:02 +02:00
commit c5bfb32ce6
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
2 changed files with 219 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

60
CLAUDE.md
View file

@ -1100,16 +1100,42 @@ and fixtures, never code.
anywhere: a stream whose FIRST two bytes are the EOB escape was carried with
**32 of 32** pixels never decoded while Pillow refuses the same file. The
cursor must now also stand at or past the end of the last row. **The line is
the CURSOR and not the pixels** -- a delta and an end-of-line escape state
their skip, so every decoder agrees on the index-0 pixels they pass over,
and a pixel-coverage count would refuse both constructions the format
defines. The corpus cannot choose (over the **25** RLE8 BMPs the delivery
ships, 25 of 25 paint every pixel, 25 of 25 reach the end, **0 of 25** use a
delta) and an independent decoder can: Pillow reads **5 of 8** streams and
refuses the same **3**, one short by a single pixel. The earlier `19 of 19`
figures above are the BUNDLE's carried assets; 25 is what the delivery
ships. Report:
the CURSOR and not the pixels** -- a pixel-coverage count would refuse the
delta the format defines. The corpus cannot choose (over the **25** RLE8
BMPs the delivery ships, 25 of 25 paint every pixel, 25 of 25 reach the end,
**0 of 25** use a delta) and an independent decoder can. The earlier
`19 of 19` figures above are the BUNDLE's carried assets; 25 is what the
delivery ships. Report:
`docs/2026-09-19-pekerblokken-bindes-til-kjoringen.md`.
**AND THAT ROUND'S OWN SENTENCE WAS TRUE OF THE DELTA AND FALSE OF THE
END-OF-LINE.** It said both escapes state a skip every decoder agrees on.
A delta states a DISTANCE; an end-of-line states only "the rest of THIS
row", so an end-of-line at column 0 closes nothing and claims a row the
stream never wrote -- four of them and an EOB carried an 8x4 frame with
**32 of 32** pixels never decoded, and Pillow refuses those bytes. The class
is wider than that construction and this round SWEPT it instead of patching
it: over every opcode sequence of length 1-4 on a 4x3 frame, **22 620**
streams, this reader carried **703** streams Pillow refuses and drew
**1 492** more differently. PM's recommendation alone (refuse a stream that
painted nothing) leaves **512** and **1 171** -- it would have narrowed the
class a third time. Two clauses ship instead, both `asset_samples_invalid`:
an end-of-line at column 0, and a delta whose horizontal offset would leave
the row (the format puts it inside the line; this reader keeps the cursor
past the row end where a flat decoder rolls it into the next row). After:
**0** carried-here-refused-there and **32** drawn differently. **WHAT IS NOT
CLOSED IS STATED**: all 32 are a run or absolute block that OVERRUNS its
row, which this reader clips at the row end and Pillow spills into the next;
refusing those gives 0 and 0 and costs **15 of the 25** real files, so it is
a limit and not a loss taken. Cost measured on the corpus FIRST: over
**11 441** files scanned across the four raw deliveries and the K2 reference
corpus, the only **25** BMPs on this machine use an end-of-line at column 0
in **0 of 25** and a delta in **0 of 25**, and 25 of 25 still decode to
Pillow's pixels exactly (**3 117 220** compared). R761 rebuilt from a pinned
pre-change tree and from this one is `diff -r`-IDENTICAL, 5 551 files, 50 of
50 viewable, 19 of 19 lossless. `CURSOR_CASES` goes 8 arms to 12 (one for
the cursor rule's ROW clause, PM's `P8`, which survived 51 tests) and the
mutant runner 45 to 47. Report:
`docs/2026-09-19-eol-stroemmen-som-ikke-maler.md`.
- **Account for content: `okf build --accounting PATH` and its judge
`python3 tools/okf_accounting_gate.py` (2026-09-17).**
- The gate was written RED at `0b00de4` and turned GREEN on all six rows by
@ -1203,6 +1229,22 @@ and fixtures, never code.
finding about the run that read as a finding about the build; what it
would take is an STS reader for the JSON delivery, and `standardContent`
occurs 0 times in `src/`).
- **THE PUBLISHED DOOR CONTRACT NAMES WHAT THE GATE READS (2026-09-19).**
The JSON sketch at the top of `okf_accounting_gate.py` is what a consumer
implements `--accounting` from, and it did not name `conversions` -- which
`asset_holds`' conversion route DEPENDS on -- nor `normalised_soft_hyphen`,
`unaccounted` or `double_booked`. A door built from it writes a ledger the
gate reads as "nothing was converted", and every converted image comes out
claimed-and-not-found: **19 of 50** on R761. Two tests hold the sketch
against both sides -- what the gate LOOKS UP (measured with a ledger that
records its own lookups, never by grep) and what the door SERIALISES -- so
"publish everything" is not a fix either.
- **A DOORLESS BUNDLE SAYS WHY (2026-09-19).** Without the flag there is no
ledger, so a converted image cannot be proved carried and is counted
claimed-and-not-found -- honest, and until now silent: the count was
printed and its cause was not. `_tally` names the missing ledger when, and
only when, something was claimed and not found. The gate always passes the
flag, so **no row moves** (RED 2, 3, 6, exit 1, unchanged).
- **Operator 2026-09-17:** the door stays OPT-IN until the reference
corpus's losses are fixed, and of the three proposed exceptions only the
PDF one (heading, paragraph, table) is APPROVED. It moves no number --

168
docs/2026-09-19-eol-stroemmen-som-ikke-maler.md Normal file
View file

@ -0,0 +1,168 @@
# The end-of-line stream that paints nothing, and four smaller rests
2026-09-19. PM's checkpoint on `44ad845` judged the previous round PARTIAL
with no load-bearing rest and five small ones. Four are code and one is
arithmetic in a closing message. This closes all five. No new format, no new
dependency, no version bump, no tag, and not one byte of any bundle moves --
measured, not asserted.
## 1. The rest that was a rule and not a sentence
The round before wrote a cursor rule into `_bmp_rle8_rows` and, beside it, a
sentence:
> a delta escape and an end-of-line escape both leave pixels at index 0 and
> every decoder agrees on them, because the stream stated the skip
PM measured it false for the end-of-line half: four end-of-line escapes and an
end-of-bitmap carry an 8x4 frame with **32 of 32** pixels never decoded, and
Pillow -- the independent decoder that file names -- refuses the same bytes.
The order offered two options: refuse a stream that reaches the end of the
frame without painting (recommended), or write the sentence true. **Both were
measured before either was chosen**, and the recommendation does not close the
class.
### 1.1 The class, swept rather than curated
Twice now this defect has been closed one construction at a time -- a
truncated stream, then an immediate end-of-bitmap -- and twice the class
stayed open one step down. So it was enumerated: every opcode sequence of
length 1 to 4 over twelve opcodes on a 4x3 frame, **22 620 streams** (encoded
runs that fit and runs that do not, absolute blocks, end-of-line, five
deltas), each decoded by this package and by Pillow.
| rule | carried here, refused by Pillow | carried by both, drawn differently | real RLE8 files carried |
|---|---|---|---|
| before this round | **703** | **1 492** | 25 of 25 |
| + refuse when nothing was painted (PM's recommendation) | 512 | 1 171 | 25 of 25 |
| + refuse an end-of-line at column 0 | **0** | 1 118 | 25 of 25 |
| + refuse a delta out of its row | **0** | **32** | 25 of 25 |
| + refuse a run that overruns its row | 0 | 0 | **10 of 25** |
The recommendation on its own would have narrowed the class for the third
round running. The last row is the one that cannot ship.
### 1.2 Why the end-of-line, and why the delta
Pillow's RLE8 decoder is a flat accumulator: an end-of-line pads its buffer to
the next row boundary, so an end-of-line at column 0 adds NOTHING, while this
reader advances a whole row. A delta appends `dx + dy * width` bytes, which is
the same skip this reader makes -- as long as `dx` stays inside the row.
So the two clauses are the two places where a stream can move this reader's
cursor without stating a skip another reader follows:
* **an end-of-line escape at column 0.** It closes no row, so the row it
passes over is a row the stream never wrote.
* **a delta whose horizontal offset would leave the row.** The format puts
that offset inside the line; this reader keeps the cursor past the row end
and a flat decoder rolls it into the next row. Two pictures, one stream.
Both refuse with `asset_samples_invalid`, the code the path already uses.
### 1.3 What is NOT closed, and what closing it would cost
All **32** residual disagreements are a run or absolute block that OVERRUNS
its row -- clipped at the row end here, spilled into the next row by Pillow.
Refusing those gives 0 and 0 and costs **15 of the 25** real RLE8 files: 15
real figures dropped and a pinned bundle's bytes moved. It is stated in the
docstring, in the test and here, and it is not taken.
### 1.4 The corpus, measured first and measured again
Over **11 441** files scanned across the four raw standard deliveries and the
K2 reference corpus, the only BMPs on this machine are R761's **25** (24
distinct), all RLE8. Of those:
```
end-of-line at column 0 : 0 of 25
delta escape (any) : 0 of 25
run overruns its row : 15 of 25
```
So both shipped clauses cost nothing measured, and after the change **25 of
25** still decode to Pillow's pixels exactly -- **3 117 220** pixels compared,
byte for byte, with the two readers independent on both sides.
## 2. The arms the table could not see
`CURSOR_CASES` goes 8 to 12. One arm per CLAUSE of the cursor rule: the ROW
clause had none, because every short stream in the table stops on the last
row, which is why PM's `P8` (`height - 1` -> `height - 2`) survived 51 tests.
The new arm stops one row early with that row complete. Three more arms are
the end-of-line class.
The table is CURATED and stays that way -- every arm in it is one where the
two readers agree -- and the general relation between them is now held by the
sweep, which asserts the direction that matters (nothing carried here is a
file the other decoder refuses) and requires every residual disagreement to be
an overrunning run, computed in the test by a walker written there.
`P8` and `P13` (the end-of-line clause removed) join the mutant runner:
**killed 47 of 47**, exit 0.
## 3. The contract a consumer implements the door from
`tools/okf_accounting_gate.py` publishes the `--accounting` JSON a consumer is
expected to write. It did not name `conversions` -- which `asset_holds`'
conversion route now DEPENDS on -- nor `normalised_soft_hyphen`, `unaccounted`
or `double_booked`. A door built from that contract writes a ledger this gate
reads as "nothing was converted", and every converted image comes out
claimed-and-not-found: **19 of 50** on R761.
Two tests hold the sketch, one per direction:
* what the gate READS. The ledger is handed to `_declared_conversions` as a
mapping that records every lookup at any depth, so the assertion is about
lookups and not about grep -- grep reads a rejection code as a JSON key.
* what the door WRITES, from `Accounting.to_json()` itself, so the fix cannot
be "publish everything": the two sets must be equal.
## 4. A doorless bundle says why
Measured by PM (N5): built without `--accounting` there is no ledger, so
`asset_holds` falls back to its first route and a converted image is counted
claimed-and-not-found. That is the honest reading and it was a silent one --
the count was printed, its cause was not, and a reader of such a report can
only conclude the bundle lost the pictures.
**Chosen: say it out loud, not only in a docstring.** A docstring is read by
whoever edits the file; the reader who is misled is reading a REPORT. `_tally`
now names the missing ledger when, and only when, something was claimed and
not found -- two known-negatives hold that condition -- and `asset_holds` gets
the sentence as well, since it costs nothing. The gate itself always passes
the flag, so no row moves.
## 5. The count in a closing message
The previous closing message said "the five other pending orders" where there
were six. Measured here with `find ~/.claude/coord/llm-ingestion-okf/orders
-maxdepth 1 -name '*.md' | wc -l` at closing time, not from memory.
## 6. The bytes, and every gate
| what | result |
|---|---|
| R761 built from a pinned pre-change tree and from this one, shared inbox | `diff -r` **IDENTICAL**, **5 551** files |
| assets in that bundle | **50**, 29 JPEG + 21 PNG, **50 of 50** viewable |
| conversions booked, and lossless against Pillow | **19 of 19**, 2 366 365 pixels |
| soft hyphens | 71 removed, 0 in the bundle |
| content-accounting gate | `GATE RED: rows 2, 3, 6`, exit 1 -- row 2 8 of 13, row 3 17 of 22 (u = 19, d = 2), row 6 110 of 111. **Unchanged** |
| mutant runner | `killed 47 of 47`, exit 0 |
| suite, clean extract of the commit | see the closing message |
| `ruff check` / `ruff format --check` / `mypy --strict src/` | clean |
The BEFORE bundle was built from `git archive 23588e5` with `PYTHONPATH`
pointing at that tree, and the module path was printed in both runs, because
an editable install would otherwise have measured the working tree twice.
## 7. What this round did not measure
* The 43-document reference corpus. Unchanged since before the round and not
rebuilt (cost); the argument for it is the same as R761's, since the two
clauses can only fire inside an RLE8 BMP and 0 of that corpus's files are
one.
* Fidelity, which neither route of the judge proves. Unchanged.
* Whether another decoder besides Pillow draws the residual 32 the way this
reader does. One independent decoder is the measurement; it is not a vote.