docs: the viewable-asset round, measured per image

README, CHANGELOG (under the untagged 0.10.1), CLAUDE.md and the round's
report. The numbers are the census's, not a summary of them.

- 9 714 image rows over 18 403 files and 67 PDFs, two pinned trees: exactly
  35 rows moved, 9 321 of 9 321 JPEG and PNG rows byte-identical.
- The 16 the round did not go looking for: JPEG 2000 objects out of PDF
  streams, also unreadable by a model, with no stdlib route to convert them.
  They become `asset_not_viewable` -- 16 pictures lost from a bundle, 16
  statements gained that a picture stood there. Stated in the README, the
  CHANGELOG, CLAUDE.md and the report rather than left in a total.
- The accounting gate: R761 0 -> 19 claimed-and-not-found when the conversion
  landed, 0 again after the judge learned its second route; row 6 90 of 111
  -> 110 of 111. The gate's verdict is the same `RED: rows 2, 3, 6` as before
  the round, with row 3's numbers identical.
- `<!-- asset-viewable-media-types: ... -->` is pinned to
  `assets.VIEWABLE_MEDIA_TYPES` by a test, compared as a whole set: a README
  naming three of four would pass every containment check and tell a consumer
  that a format is refused when it is carried.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 08:26:17 +02:00
commit c39b666c11
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
4 changed files with 104 additions and 5 deletions

View file

@ -50,9 +50,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
(RLE4, BITFIELDS, 16/32-bit, BITMAPCOREHEADER, over 256 palette entries).
Both leave a "not carried" line in the concept and a row in the run log.
- **The cost, measured with a committed script** (`tools/okf_asset_census.py`,
one row per image from two pinned trees): see the round's report for the
before/after counts, including the **16 of 9 714** JPEG 2000 objects that
stop being carried.
one row per image, run from two pinned trees over 18 403 files and 67
PDFs, **9 714 image rows**): exactly **35 rows moved** — 19 BMP now
carried as PNG, and **16 JPEG 2000 objects** that stop being carried and
become `asset_not_viewable`, because no stdlib route decodes JPEG 2000.
**9 321 of 9 321** JPEG and PNG rows are byte-identical on both sides, so
not one already-viewable picture changed hands.
- **One normalisation door in front of the persist gate: U+00AD is removed and
COUNTED** (operator decision 2026-09-18). `llm-ingestion-guard` 1.4.0 keeps

View file

@ -1038,8 +1038,23 @@ and fixtures, never code.
metadata already lives (original media type, original sha256 in full, new
sha256), and a converted asset is ONE asset: one file, one pointer, one
accounting row. Exposure measured with a committed script
(`tools/okf_asset_census.py`, one row per image, run from two pinned trees).
Report: `docs/2026-09-19-hvert-baaret-bilde-kan-vises.md`.
(`tools/okf_asset_census.py`, one row per image, run from two pinned trees
over 18 403 files and 67 PDFs, **9 714 image rows**): exactly **35 rows
moved**, **9 321 of 9 321** JPEG and PNG rows byte-identical, so not one
already-viewable picture changed hands. **16 of the 35 are the finding the
round did not go looking for**: JPEG 2000 objects out of PDF streams, a
format no model decodes and no stdlib route converts, so they go from
carried-and-invisible to `asset_not_viewable` with a line in the concept --
a consumer loses 16 pictures and gains 16 statements that a picture was
there. **The judge had to move with it and it caught itself**: the gate's
`asset_holds` proved a carry by hashing the SOURCE file, which a converted
image's bundle cannot satisfy, so R761 went 0 -> **19 claimed and not
found** (exactly its RLE8 count) before the judge learned to read the two
digests the bundle states and then hash the asset itself; after, R761 is
u = 0, d = 0, 0 claimed and not found, 50 asset files, 31 081 carried
elements, and the gate's verdict is the same `RED: rows 2, 3, 6` as before
the round. Report:
`docs/2026-09-19-hvert-baaret-bilde-kan-vises.md`.
- **Account for content: `okf build --accounting PATH` and its judge
`python3 tools/okf_accounting_gate.py` (2026-09-17).**
- The gate was written RED at `0b00de4` and turned GREEN on all six rows by

View file

@ -335,6 +335,15 @@ and the size in pixels already live — states the original media type, the
original sha256 in full and the new one, so a reader can take the original
delivery, run `shasum -a 256` and find the row.
The cost is measured per image rather than per bundle, with a committed
script (`tools/okf_asset_census.py`) run from two pinned trees over 9 714
image rows: exactly **35 rows moved**. Nineteen are the BMPs, now PNG. The
other **16 are JPEG 2000 objects** carried out of PDF streams — a format no
model decodes either, and one no stdlib route converts, so they are refused
with `asset_not_viewable` and stated in the concept instead of being carried
unreadably. **9 321 of 9 321** JPEG and PNG rows are byte-identical across the
move.
<!-- asset-viewable-media-types: image/gif,image/jpeg,image/png,image/webp -->
**A size CEILING, read off the same corpora (0.10.1).** An image over

View file

@ -154,3 +154,75 @@ It was itself caught by the rule it exists to serve. Its first version handed
clean-looking table that had measured nothing. The attribute is asserted now,
and the known-positive (one PDF, 8 images) is run before the sweep.
## 6. What the change costs, row by row
Two census runs over every image this machine holds — 18 403 files walked,
**67 PDFs opened, 9 714 image rows** — one from the tree before the
conversion and one after.
| Outcome | Before | After |
|---|---|---|
| `CARRY image/jpeg.jpg` (from PDF streams) | 3 053 | 3 053 |
| `CARRY image/jpeg.jpg` (standalone files) | 29 | 29 |
| `CARRY image/png.png` (from PDF streams) | 6 233 | 6 233 |
| `CARRY image/png.png` (standalone files) | 2 | 2 |
| `CARRY image/bmp.bmp` | **19** | 0 |
| `CARRY image/png.png in=image/bmp` | 0 | **19** |
| `CARRY image/jp2.jp2` | **16** | 0 |
| `REJECT asset_not_viewable` | 0 | **16** |
| `REJECT asset_pdf_unbounded` | 8 | 8 |
| `REJECT asset_pdf_unsupported` | 314 | 314 |
| `REJECT asset_samples_invalid` | 40 | 40 |
| **carried** | 9 352 | 9 336 |
**Exactly 35 of 9 714 rows moved**, and 9 679 are byte-identical — same
source digest, same carried digest, same origin. The known-positive is
stated on the rows rather than on the totals: **9 321 of 9 321** JPEG and PNG
rows are identical on both sides, so not one already-viewable picture changed
hands.
**The 16 are the finding this round did not go looking for.** JPEG 2000
objects, carried out of PDF streams by `JPXDecode`, in a format no model
decodes either. They were invisible before and are refused with a code and a
line now, which is the round's rule applied where it happens to bite. There
is no stdlib route to decoding JPEG 2000, so converting them is not
available: a consumer loses 16 pictures from a bundle and gains 16 statements
that a picture was there. That is the trade this round makes, stated rather
than buried in a total.
## 7. The accounting gate
The conversion broke the judge, and the judge caught it. `asset_holds` proved
a carry by hashing the SOURCE file and looking for those bytes in `assets/`;
a converted image's source bytes are not in the bundle. Measured on R761:
| | claimed and not found | row 6 |
|---|---|---|
| with the conversion, before the judge's second route | **19** | 90 of 111 |
| after | **0** | 110 of 111 |
19 is exactly R761's RLE8 count. The judge now also reads the two digests the
bundle states on the pointer line and then hashes the asset itself, so a
bundle claiming a conversion it did not perform still fails — three cases in
one test, one positive and two negative.
The gate's verdict is unchanged: **`GATE RED: rows 2, 3, 6`**, the same rows
STATE recorded before the round, with row 3's numbers (u = 19 unaccounted,
d = 2 double-booked, 17 of 22) identical across both runs — they are the
reference corpus's own losses, which the operator left standing red on
2026-09-17. R761 reads exit 0, 1 of 1 persisted, **50 asset files, 31 081
carried elements found, 0 claimed and not found**.
## 8. What this round does NOT do
- **JPEG 2000 is refused, not converted.** No stdlib route exists.
- **WebP is not sniffed**, so the viewable set's fourth member is unreachable
today. The set states what a model can be shown, not what this package
reads.
- **`.xlsx` still carries no image** — a pointer block inside a sheet's pipe
table would break the `source_rows` locator, unchanged from 0.10.0.
- **Nothing is tagged, pushed or released.** The CHANGELOG entries go under
the untagged `[0.10.1]`.
- The reference corpus is **not rebuilt**. The exposure is measured per image
by the census, which is the finer instrument for this change: a rebuild
reports that bytes moved, the census reports which image moved and why.