"""Binary assets: what an image IS, what it is called, and how it is pointed at. Until 0.10.0 this package had no image path at all. Every reader recovered text, every warning said so, and the only writer into a bundle was :func:`materialize.write_bytes`, whose signature is ``(bundle_dir, name, content: str)`` -- UTF-8 in, text out, no binary route anywhere. A document whose table is a raster picture therefore reached a concept as an absence with no denominator. Measured on R761 Prosesskoden:2025, the publisher's own NISO-STS delivery: the process text is carried in full, and 12 ``Tabell N-N`` and 9 ``Figur N-N`` captions stand over nothing, so process 84's "toleranseklasse ... er gitt i tabell 84-2" points at empty space. THIS MODULE IS THE ONE PLACE THAT DECIDES WHAT AN IMAGE IS. A reader hands it bytes and a name; it returns a carried image or raises a coded rejection. That is what makes "N carried of M found" mean the same thing for ``pdf``, ``docx``, ``html`` and ``xml``, and it is what keeps each format's quirks out of the bundle layout. THREE RULES, and each one exists because the alternative is a silent lie: - **The type is SNIFFED, never claimed.** Measured on the R761 delivery, the graphics directory holds ``.bmp``, ``.jpg`` and ``.png`` side by side and the document's ``xlink:href`` values are whatever the publisher's tool wrote. A name is a claim; the magic bytes are the fact. A consumer dispatching on the extension of a name that lies reads the file wrong with full confidence. - **The name is CONTENT-ADDRESSED** -- ``-``. Two drops of one image are one file, a rebuild of one corpus is one bundle, and the digest carries the uniqueness so the readable tail can be shortened without any risk of collision. That is the byte-determinism rule this package already holds for text, extended to the bytes beside it. - **The pointer is ONE GRAMMAR**, owned here. ``okf describe`` (step 2) has to find every pointer mechanically in order to write a transcription under it, so the block is a regex this module ships beside the writer rather than a shape each reader invents and each consumer re-derives. WHAT THIS MODULE DOES NOT DO: it never looks at a picture. Classifying an image as a table or a figure, and reading what it says, is a model call, and the invariant "no model calls anywhere in the run path" is not negotiated here -- step 2 is a separate command, outside the build path, and this module is importable without it. **The image BYTES are not screened.** The guard is text-only (its own boundary, not ours), so what passes a persist gate is the pointer block, as body text, like every other line. The bytes of a carried image are written to the bundle unscreened. Stated here rather than implied, because a consumer weighing an untrusted drop needs to know which half of the concept was looked at. """ from __future__ import annotations import hashlib import re import struct import unicodedata import zlib from collections.abc import Iterator from dataclasses import dataclass from .errors import ExtractionError #: The one directory a bundle keeps its binary assets in, at the bundle root. #: Fixed rather than configurable: a consumer resolving `/assets/...` out of a #: concept has only the bundle, and a per-profile directory would make that #: pointer unresolvable without also shipping the profile that wrote it. ASSETS_DIR = "assets" #: How much of the digest names the file. 12 hex characters is 48 bits; over #: the largest asset population measured here (4 828 image objects in one #: 33-document corpus) the birthday probability of a collision is about #: 4e-11. A collision would be caught anyway -- an occupied name is re-used #: only when the bytes there are already identical, the same content-identity #: rule Door C proves ownership with. DIGEST_PREFIX = 12 #: How much of the original name survives into the asset name. The digest #: carries uniqueness, so this is decoration and truncating it is safe -- which #: is the opposite of `materialize.check_filename_length`'s situation, where the #: name IS the identity and truncation would silently merge two documents. NAME_TAIL_MAX = 60 #: Magic bytes -> (media type, suffix). Sniffed in this order; the first match #: wins, and nothing here overlaps. _MAGIC: tuple[tuple[bytes, str, str], ...] = ( (b"\x89PNG\r\n\x1a\n", "image/png", ".png"), (b"\xff\xd8\xff", "image/jpeg", ".jpg"), (b"GIF87a", "image/gif", ".gif"), (b"GIF89a", "image/gif", ".gif"), (b"BM", "image/bmp", ".bmp"), (b"II\x2a\x00", "image/tiff", ".tiff"), (b"MM\x00\x2a", "image/tiff", ".tiff"), ) #: The formats a model can be SHOWN. Everything a document ships outside this #: set is converted losslessly to PNG, or refused with a code -- never carried #: silently, which is what this package did until this round of 0.10.1. #: #: MEASURED 2026-09-19 over the frozen R761 delivery's own `assets/` #: (denominator 50): 29 JPEG, 2 PNG and **19 "PC bitmap, Windows 3.x, 8-bit, #: compression 1"**. The 19 are byte-correct files that nothing reads, so 19 of #: that document's figures were present and invisible at the same time -- and #: the `images: N` count said they had arrived. An absence a reader is shown is #: information; a picture that is there and unreadable is worse than either. #: #: IT IS A PROPERTY, NOT A LIST OF FORMATS WE HAPPENED TO MEET. A carried #: asset's type is read off its bytes and tested against this set, so a format #: nobody here has seen is refused by the same rule that refuses TIFF. #: #: WebP is on the list and `sniff` does not recognise it: the set states what a #: model can be shown, not what this package can read. A WebP is therefore #: refused one step earlier, as `asset_type_unknown`, and never reaches this #: test. Stating that is cheaper than a set whose name is wider than its reach. VIEWABLE_MEDIA_TYPES = frozenset({"image/png", "image/jpeg", "image/gif", "image/webp"}) #: JPEG 2000, in both the forms a PDF `JPXDecode` stream hands back: the JP2 #: container and a bare codestream. _JP2_SIGNATURE = b"\x00\x00\x00\x0cjP \r\n\x87\n" _J2K_SIGNATURE = b"\xff\x4f\xff\x51" #: The frame markers that carry a JPEG's dimensions. Every SOF except the four #: that are not frame headers at all (`DHT` 0xC4, `JPG` 0xC8, `DAC` 0xCC). _JPEG_SOF = frozenset(range(0xC0, 0xD0)) - {0xC4, 0xC8, 0xCC} #: `materialize.reduce_to_id_grammar`'s rule, restated. Not imported: this #: module is reached from `extract.py`, whose registry must not import the #: contract layer, and `materialize` pulls in `manifest` and `profiles`. #: `tests/test_assets.py` holds the two forms equal on the same inputs, so the #: restatement cannot drift into a second grammar. _SEPARATOR_RUN = re.compile(r"[^a-z0-9]+") #: One pointer block, as this module writes it. Group 1 is the label, group 2 #: is the asset file name, group 3 is the whole second line. `okf describe` #: finds its work with this and writes under the match; a consumer wanting to #: strip pointers uses the same expression, so there is one definition of what #: a pointer looks like rather than one per reader. IMAGE_POINTER = re.compile( r"^!\[(?P