Three STS fixtures still carried the section titles and labels of one real
reference document, and three identifiers were copies of its codes with a
letter or a word swapped. They now describe an invented kitchen counter and
cookbook series: the titles, labels and descriptions of sts-identity.xml,
sts-inherit.xml and sts-empty-label.xml, the P350/P351 document codes, the
99-0001 delivery prefix and chapter 7 of the image and accounting corpora.
Generated fixtures are regenerated and the witness inventory's per-document
totals are identical before and after; only names and text move.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every fixture, test document, tool example and document now uses an invented
kitchen-and-baking handbook series, written in this repository. The package's
behaviour is unchanged; src/ changes are comments and help text only.
- Generated fixtures are regenerated from their generators. Their structural
counts are identical before and after: elements, images, rows, cells,
headings, bookmarks and the witness inventory's per-document totals. The
image-inbox and accounting documents are renamed kapittel-84-*.
- tools/okf_accounting_gate.py: the two options that named one real corpus
each are replaced by a generic, repeatable --corpus PATH with no default.
Row 5 compares the PDF pair alone. Gate verdict unchanged: RED rows 2, 3, 6.
- tools/okf_witness.py: the STS JSON reader for one publisher's delivery is
removed, along with its three twins and five tests. The mutation harness
loses W09.
- docs/: 13 dated reports that documented runs on a retired reference corpus
are removed, and 40 are neutralized. Dead links are removed, and no new
dangling path is introduced.
- The synthetic MCP-gate corpus and the residual probe words are neutral.
Valgt: keep the `okf quality --fasit` bar value (the measured fraction, one corpus) and
rewrite only its provenance, because the verdict stays unchanged and the
number names nothing.
Term check with the local list: 0 of 411 tracked files, 0 file names, 0 of
27 binary fixtures. Suite after git add: 2457 passed, 1 skipped. The base
tree had 2460 passed and 2 skipped; five tests went with the JSON reader and
four were added by the term check. ruff, ruff format and mypy --strict src/
are clean.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner could only run one test file, which is why PM's three
survivors from `43331fc` could not be added: two are held by the gate's
row 3 and one by the soft-hyphen door's suite. A mutant now names its
suite; the catalogue goes 39 to 45.
X3 and X4 rewritten against the code as it now stands -- a mutant table
is a copy of the code it mutates, and this round moved the lines both of
them quoted. X6 is the defeated state exactly, X7 cuts the ledger off at
its source, X8 removes the cursor rule, P6/P11/P12 are PM's three.
Two survivors on the first run, both findings, both closed:
- X4 survived because every forgery arm now fails on the ledger check
before the binding is reached. An arm was added where the run DID book
the pair and the block stating it points at another picture.
- X5 survived the WHOLE suite -- 2134 passed with the disarming removed
-- because a document-supplied field can no longer reach the gate. The
property is about the BUNDLE and not about one judge, so it is kept and
measured in `tests/test_assets.py`, with a known-positive counting the
run's own two fields on the same expression.
killed 45 of 45, exit 0. Report, CHANGELOG and CLAUDE.md written.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Chosen: a stdlib BMP reader, because `read_image` is on the CORE path and an
asset's name is its content digest. Measured first, as the order requires:
Pillow 12.3.0 IS in this tree (transitively under `pdfplumber`) and it DOES
decode RLE8 correctly -- a hand-written stdlib decoder and Pillow agree on
19 of 19 of R761's real files, RGB per pixel. So the choice does not rest on
capability. It rests on two properties of this package: `.html` and `.xml`
carry images with no `[extract]` extra installed, so a Pillow converter
either makes a core path depend on an optional binary wheel or buys the
second runtime dependency; and encoding through an installed library would
make a bundle's identity move with that library's version, which is the
property 0.10.0 felled page rasterisation over and `encode_png`'s docstring
already defends. Pillow keeps the job it is good for: the INDEPENDENT decoder
in the tests, on neither side of the conversion.
The defect, measured over the frozen R761 delivery's `assets/`, denominator
50: 29 JPEG, 2 PNG and 19 RLE8 BMP. The 19 are byte-correct files nothing
reads, so 19 figures were present and invisible while `images: N` reported
that they had arrived.
- `VIEWABLE_MEDIA_TYPES` is tested against every asset's SNIFFED type, so it
is a property and not a list of formats we met. WebP is on it and `sniff`
does not recognise one; the limit is stated, not implied.
- `bmp_to_png`: 8-bit uncompressed, 8-bit RLE8, 24-bit uncompressed. All five
RLE8 opcodes. 19 of 19 real files convert with RGB identical to Pillow's
decoding of the source, 2 366 365 pixels compared.
- `asset_not_viewable` and `asset_bmp_unsupported`, both published, both
leaving the concept's "not carried" line.
- Traceability on the pointer's second line, where the rest of the asset
metadata already lives: original media type, original sha256 in full, new
sha256 in full. A converted asset is ONE asset.
- The ceiling is paid on the DECLARATION before a row is allocated, and an
RLE run is one clipped slice -- painting pixel by pixel leaves the memory
bounded and the CPU unbounded.
Two repairs the change forced, each measured rather than assumed:
- `tests/test_assets.py`'s "dimensions absent is absent" used a TIFF, which
is now refused before `read_image` returns. The property still has a
reachable case -- a JPEG whose frame header never arrives -- and uses it.
- `asset_holds` in the accounting gate proved a carry by hashing the SOURCE
file, which a converted image's bundle cannot satisfy. It now also reads
the two digests the bundle states and HASHES THE ASSET ITSELF, so a bundle
claiming a conversion it did not perform still fails.
`tools/okf_asset_census.py` is the committed instrument for the
known-positive: one row per image, from two pinned trees. It was caught by
the rule it serves -- its first version handed `_pdf_images` the wrong page
object and reported 0 images over 67 PDFs with exit 0. The attribute is
asserted now and a known-positive runs before the sweep.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Until now no reader in this package fetched, named, described or copied a
single image. `<img>`'s attributes were never read, a NISO-STS `<graphic>`
was walked past, a PDF was opened for its text alone, the converter's
markdown writer dropped every picture, and the only writer into a bundle
took `content: str`. The two lossiness warnings said so on every run, which
made the loss honest and did not make it smaller.
Measured on R761 Prosesskoden:2025, published as a 701-page PDF and as a
NISO-STS delivery: the process text is carried in full while 12 `Tabell N-N`
and 9 `Figur N-N` captions stand over nothing, because that publisher ships
those tables as raster pictures in both. Process 84's "toleranseklasse ...
er gitt i tabell 84-2" points at empty space.
THE GATE WAS WRITTEN FIRST AND RED. `tests/test_asset_gate.py` reads its
denominator out of the source (`page.images`, `word/media/`, `ppt/media/`,
`<img`, `<graphic`), never from a constant here. Measured at 332961a, built
from `git archive` and not from the editable tree: carried 0 of 8 local
images across 5 documents (9 declared), and no `assets/` at all. After: 8 of
8, with the ninth a remote source carried as a pointer without a file.
FIVE READERS PLACE, ONE MODULE DECIDES. `assets.py` owns what an image is
(sniffed from the bytes, never from the claimed extension), what it is
called (`<sha256[:12]>-<the source's own basename>`) and how it is pointed
at (one two-line block, one regex). `.xlsx` is deliberately not a row: a
block inside its pipe tables would break the `source_rows` locator, and 0 of
4 K2 workbooks hold media.
A PDF stream that is already a file is carried VERBATIM (29 of R761's 50
objects are DCTDecode); raw samples are encoded to PNG with stdlib zlib, so
no new dependency. Rendering the page region was the alternative and was
felled on determinism: a rasterised crop's bytes, and therefore the asset's
content-addressed name and the bundle's digest, would depend on the
installed rasteriser. What the encoder cannot express exactly is refused
with a code and counted, never approximated.
NO SIZE FLOOR, and that is a measurement: over the 4 828 image objects of
the K2 corpus the size distribution is a broad spread with no gap, unlike
OCR_CID_SHARE's bimodal one, so a threshold would be a number we chose.
ON BY DEFAULT, AND THE CONTROL IS TWO WHOLE BUILDS. The 43-document
reference corpus at 332961a versus rebuilt at HEAD with `--no-assets`:
865 files on both sides, `diff -rq` reports ONE difference, the added
`Images: NOT CARRIED` line in log.md. Every concept byte-identical.
Against the default: 453 -> 454 concepts, 865 -> 867 md, 0 -> 2 964 assets
(2 964 carried of 3 145 found, 4 622 pointers), 4.7 MB -> 115 MB, 2 414 s ->
3 088 s, peak RSS 6.26 -> 8.74 GB, 422 of 865 md files differ. The one new
concept has a measured cause: the pointers are body text, so a section
holding 146 of that document's images grew from 19.0 % to 30.6 % of the
extracted text and crossed `--outline-gate`'s 0.20 share clause.
THE IMAGE BYTES ARE NOT SCREENED. The guard is text-only, the pointer block
passes the gate as body text, the picture beside it passes nothing, and
log.md says so on every run.
Also fixed, both found by measuring rather than by reading:
- a markdown image is no longer read as a cross-reference. `structure._LINK`
never looked at the character in front of the bracket, so every pointer
would have arrived in the index as an edge to a concept that cannot exist.
- Door C carries the assets its merged concepts point at. Before this,
importing a bundle built with `--assets` merged 6 of 6 concepts and wrote
no `assets/` at all, so every pointer named a missing file.
Report: docs/2026-09-17-bilder-i-bundlen-trinn1.md
Spec proposal: docs/plan/okf-assets-section-6-4.md
Suite 1 955 passed / 1 skipped (from 1 896), ruff and mypy --strict clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
