Until now no reader in this package fetched, named, described or copied a
single image. `<img>`'s attributes were never read, a NISO-STS `<graphic>`
was walked past, a PDF was opened for its text alone, the converter's
markdown writer dropped every picture, and the only writer into a bundle
took `content: str`. The two lossiness warnings said so on every run, which
made the loss honest and did not make it smaller.
Measured on R761 Prosesskoden:2025, published as a 701-page PDF and as a
NISO-STS delivery: the process text is carried in full while 12 `Tabell N-N`
and 9 `Figur N-N` captions stand over nothing, because that publisher ships
those tables as raster pictures in both. Process 84's "toleranseklasse ...
er gitt i tabell 84-2" points at empty space.
THE GATE WAS WRITTEN FIRST AND RED. `tests/test_asset_gate.py` reads its
denominator out of the source (`page.images`, `word/media/`, `ppt/media/`,
`<img`, `<graphic`), never from a constant here. Measured at 332961a, built
from `git archive` and not from the editable tree: carried 0 of 8 local
images across 5 documents (9 declared), and no `assets/` at all. After: 8 of
8, with the ninth a remote source carried as a pointer without a file.
FIVE READERS PLACE, ONE MODULE DECIDES. `assets.py` owns what an image is
(sniffed from the bytes, never from the claimed extension), what it is
called (`<sha256[:12]>-<the source's own basename>`) and how it is pointed
at (one two-line block, one regex). `.xlsx` is deliberately not a row: a
block inside its pipe tables would break the `source_rows` locator, and 0 of
4 K2 workbooks hold media.
A PDF stream that is already a file is carried VERBATIM (29 of R761's 50
objects are DCTDecode); raw samples are encoded to PNG with stdlib zlib, so
no new dependency. Rendering the page region was the alternative and was
felled on determinism: a rasterised crop's bytes, and therefore the asset's
content-addressed name and the bundle's digest, would depend on the
installed rasteriser. What the encoder cannot express exactly is refused
with a code and counted, never approximated.
NO SIZE FLOOR, and that is a measurement: over the 4 828 image objects of
the K2 corpus the size distribution is a broad spread with no gap, unlike
OCR_CID_SHARE's bimodal one, so a threshold would be a number we chose.
ON BY DEFAULT, AND THE CONTROL IS TWO WHOLE BUILDS. The 43-document
reference corpus at 332961a versus rebuilt at HEAD with `--no-assets`:
865 files on both sides, `diff -rq` reports ONE difference, the added
`Images: NOT CARRIED` line in log.md. Every concept byte-identical.
Against the default: 453 -> 454 concepts, 865 -> 867 md, 0 -> 2 964 assets
(2 964 carried of 3 145 found, 4 622 pointers), 4.7 MB -> 115 MB, 2 414 s ->
3 088 s, peak RSS 6.26 -> 8.74 GB, 422 of 865 md files differ. The one new
concept has a measured cause: the pointers are body text, so a section
holding 146 of that document's images grew from 19.0 % to 30.6 % of the
extracted text and crossed `--outline-gate`'s 0.20 share clause.
THE IMAGE BYTES ARE NOT SCREENED. The guard is text-only, the pointer block
passes the gate as body text, the picture beside it passes nothing, and
log.md says so on every run.
Also fixed, both found by measuring rather than by reading:
- a markdown image is no longer read as a cross-reference. `structure._LINK`
never looked at the character in front of the bracket, so every pointer
would have arrived in the index as an edge to a concept that cannot exist.
- Door C carries the assets its merged concepts point at. Before this,
importing a bundle built with `--assets` merged 6 of 6 concepts and wrote
no `assets/` at all, so every pointer named a missing file.
Report: docs/2026-09-17-bilder-i-bundlen-trinn1.md
Spec proposal: docs/plan/okf-assets-section-6-4.md
Suite 1 955 passed / 1 skipped (from 1 896), ruff and mypy --strict clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One rule explains every remaining `pdf` miss on the twelve-position reference:
where a document DECLARES headings, Arm D's RECOVERED headings are the whole of
the excess, and every declared one is a unit the reference wants. `--outline-gate`
admits recovery only where the document declares none of its own, plus any one
recovered heading covering OUTLINE_SHARE (0.20) of the text. It is `fold_units`
clause 2's own principle moved from voting to admission, and it filters at
ADMISSION so the text a removed mark opened is carried by the mark above it --
the post-filter form scores identically on all twelve positions and loses that
text, which is why only one of them shipped.
`--outline-gate` and `--drop-wrapped-outline` become the package default, one
decision because neither carries the reference alone: `pdf` 2 of 8 -> 5 of 8
alone, 7 of 8 together; the sheet 5 of 12 -> 10 of 12; `docx` unchanged at 3 of
3. Each keeps an explicit opt-out. The bar the move had to clear was not the
reference: hit@8 on a K2 bundle built with it holds 5 of 6 at ranks 1,1,1,1,1,-,
no row losing rank 1. `--sheet-section-rows --keep-table-heading` reaches 11 of
12 and does NOT ship, because on a bundle built with it row 1 falls rank 1 -> 2.
Cost to a consumer is a re-run: 492 concepts / 944 files -> 425 / 810.
DOCUMENT_PRIOR_EXPONENT makes the document prior sublinear (total/n**0.5). A sum
measures size and a density is diluted by every unit carrying none of the
question, so a document split 1 -> 12 lost its prior by 12. Swept over five
values on 18 rows it is at least as good as the delivered density everywhere and
strictly better on three. Stated plainly: end to end it moved NOT ONE hit@8 row
on any of four bundles, so it did not solve the knot it was adopted for -- what
did is that the `pdf` gain never needed `--sheet-section-rows`.
`--first-span-from-zero` is off and repairs a measured loss found while chasing
one position's 940 characters: 32 of the 32 documents that get a plan leave the
text above their first concept in no segment -- 159 704 characters, 9.18 % of
the corpus, 45 841 from one document. It changes nothing on the reference. Off
because it moves the first span of essentially every bundle with no hit@8 number
behind it yet.
vegnormal-okf FUNN 2: SPEC section 8's own star row parsed as prose, so every
concept behind one was unreachable to the section 9.2 walk. `IndexPolicy.also_reads`
carries it for the SEGMENTED profiles, read-only, after the emitted pattern
misses -- the asymmetry `sources` already has. DEFAULT and STRICT_V1 untouched (O2).
vegnormal-okf FUNN 1: Door C's own outcome was refused at exit 1,
`bundle_id_missing`. `import_bundle` now takes `root_frontmatter_values`,
keyword-only, rendered before any disk mutation, written only when the index is
created -- Door B's mechanism and ordering.
Report: docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md.
Suite 1478 passed (1449 before), ruff and mypy clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An index ordering a profile names must be honoured wherever this library
writes an index. Door B and Door C have separate index writers, so an
ordering built on Door B's `_index_sort_key` seam alone would have been a
profile field Door B obeys and Door C ignores -- silently, because
nothing raises and both files still parse. That is
`IndexPolicy.per_directory` again: a field that reads as global and acts
on one path.
`IndexPolicy` gains `sort_key`, `sort_order` and `sort_missing`. Both
order fields draw from CLOSED sets, and `sort_order` is deliberately not
a caller-supplied callable: a callable cannot be serialised into the
bundle, reproduced from it, or audited by a reader, which is the whole of
what a deterministic bundle claims. A `sort_key` the facet policy does
not name is refused too -- every entry would be missing the key and the
ordering would silently do nothing, which is this row's own defect class.
`IndexPolicy.sort_entries` is the one helper. Four stable passes, so each
is the tie-break of the next: concept path, then the named key, then the
missing group partitioned to whichever end the policy says, then
navigation last. Passes 2 and 3 are separate on purpose -- folding them
into one reversible key tuple would flip the missing group along with the
order, so `sort_missing="last"` would mean "first" under `descending`.
The tie-break is the CONCEPT PATH, not the link target, and that is
measured rather than assumed: `notes-beta.md` precedes `notes/alpha.md`
by concept path and follows it by generated filename, so ordering Door C
on the target would have re-ordered every existing Door C bundle.
`IndexEntry` carries the path for that reason; `parse_entry` leaves it
`None` and the ordering falls back to the target, which costs nothing
because no caller sorts entries it read back off disk.
Door B's two reprojection writers and Door C's index emission all route
through the helper. Door B's unfaceted path is not routed and does not
need to be: `sort_key` requires a facet policy, and a faceted profile
never reaches that writer. Door C's guarantee is bounded and stated in
the code -- `link_in_index` appends what is absent and leaves what is
present, so the order holds within a run and never re-orders entries an
earlier run wrote.
Default ordering, unchanged and now stated: with no `sort_key`, concepts
before navigation, each group ascending by concept path.
TDD, and the red was watched twice. First behaviourally with the fields
inert (both doors emitted the exact reverse of the named order), then
again with Door B routed and Door C not -- the broken world reproduced,
where a Door-B-only test would have passed.
882 tests (868 before). The five byte-pinned goldens are untouched and
green; no shipped profile moved.
Co-Authored-By: Claude <claude-opus-5>
vegnormal-okf measured the gap on 2026-08-27: the arm reading DEFAULT's
index.md scored 0 hits of 8, the arm reading a faceted index of the same
frontmatter scored 25 of 29. Same bundle, same concepts, same model. The
DEFAULT arm did not answer wrongly, it abstained -- the metadata is in the
bundle and the index throws it away (30 974 characters over 269
requirements, 0 occurrences of any of the eight facts).
FacetPolicy and STRUCTURED_V1 already did this. They did not reach Door C.
`import_bundle` now takes a keyword-only `profile` defaulting to DEFAULT, so
every existing call site emits the bytes it always did, and `link_in_index`
takes the facets to render.
Door C PROJECTS and never DERIVES, which is the answer to the objection this
work opened with: deriving structure for a document a third party wrote would
put our inference into an index entry ABOUT their bytes, where it reads as
their claim. The concept file was already verbatim; the entry describing it
now is too. Where the sender carries `derived`, THEIR list travels unchanged,
so a reader can still see which of the sender's facts the sender inferred.
The projection asks the policy which keys to carry and never what a key
means. That is what makes it work for a meeting note as well as a numbered
norm -- nothing in it can key off a numbering scheme -- and it is why a
consumer whose concepts are named by UUID can get `title` into the index by
naming the key, with no change here.
Two things measured during the work rather than assumed:
- A value carrying the policy's own joiner cannot be rendered. Door C's
tolerance is structural and it refuses no sender on form, so the FACET is
dropped and the concept still merges -- reported per concept and key in
`ImportResult.unrendered_facets`, never dropped silently.
- A faceted entry can go stale where a flat one never could: the collision
gate refuses an updated concept, so the operator's only route is to remove
the merged file and re-import, after which the file said `gjeldende` while
the index still said `utkast`. A faceted entry for a present target is now
refreshed in place instead of skipped. Unfaceted callers keep the early
return byte for byte.
Suite 695 -> 707; ruff and mypy --strict clean.
Order 20260826T224500Z-873805419-from-vegnormal-okf.
The operator's condition for revisiting the pin was met (v1.2.0 contains
the flow-mapping frontmatter fix, commit 5870483) and the operator has now
approved the move itself, dispatched as its own order because it changes
consumer-visible Requires-Dist and shifts golden-fixture admission.
Floor 1.2: this library needs the flow-mapping support (`generated: { by:
x, at: y }`) that landed there — without it Door C fail-secures every
concept carrying that stamp. Ceiling <2.0, not narrower: the guard's own
1.0.0 release freezes its exported surface until a 2.0.0, and explicitly
keeps calibration (severities, dispositions) free to move within 1.x, so a
tighter ceiling here would claim a stability guarantee neither side needs.
Re-measured through Door C against the guard's own default
(allow_reserved=True, matching how the earlier recommendation measured
it), over the 9 concept documents across all four Door-A goldens:
4/9 admitted at 0.3.4 -> 8/9 admitted at 1.2.0, confirming the number
already reported. The ninth (a `sources` block-list carrying `resource`)
stays refused by design (G30) and is not expected to move.
Failing-test-first: test_guard_adapter.py::test_guard_version_is_inside_the_pin
and test_packaging.py::test_the_only_runtime_dependency_is_the_security_boundary
were updated to the new pin first and confirmed red against the
still-installed 0.3.4, then `uv sync --extra extract` installed 1.2.0 and
both went green. Full suite (615), ruff and mypy --strict clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RxcxzNwpX1kDP53n1rLhM5
Measure first, widen after. The 19-fixture guard-surface suite was re-run
against v0.3.4 in a scratch venv before the range moved, and reproduced the
three deltas measured against v0.3.3 exactly, with none added. v0.3.4 is the
tag pinned rather than v0.3.3 because it shipped first and repairs a quadratic
regex (okf._MD_LINK_RE) that sits on Door C's own call path.
Door C now passes allow_reserved=False explicitly. The guard added the keyword
in the 0.3 line and defaults it True for received bundles, which would merge a
sender's index.md / log.md instead of rejecting them. The override keeps the
unconditional reserved-name refusal committed to before the keyword existed,
and the reason is structural rather than a second opinion on the guard's scan:
Door C generates the merged bundle's index.md from what it merged and writes
every merged concept verbatim, so a sender's index.md would be a second and
irreconcilable claim on one path.
This is not a behaviour change for anyone on the previous pin: under v0.2.0
the keyword did not exist and reserved names were refused by construction.
The floor is >=0.3 and not >=0.2 for a measured reason. allow_reserved is
absent in v0.2.0 and present from v0.3.0 onward, checked across all five tags:
a >=0.2 floor would admit a version that raises TypeError on every Door C
import. That measurement also corrects a recorded premise -- the plan said the
keyword "shipped in v0.3.3", which read the first version we ran the suite
against as the version it was introduced in. The conclusion held; the reason
did not, and the reason is what a future bump would have relied on.
test_door_c_pins_allow_reserved_false_against_the_guards_default locks both
halves: that the guard still defaults True, without which the override is a
no-op that would pass forever over nothing, and that Door C overrides it.
586 tests, mypy --strict clean, goldens byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V2v1hrDhrff2H3y2TNJHkF
V6, settled by the operator today: import and report, not refuse.
Door C imports the POINTER to executable code and never the code — it
writes concepts verbatim and skips every non-`.md` file. So an imported
Attested Computation can name an `executor`/`attester` resource that did
not arrive, or one that RESOLVES against a file the destination tree
already holds under that path. The second is the outcome worth
surfacing: it looks valid.
Refusing was the plan's leaning and is not what shipped. §14 forbids a
consumer to reject a bundle over a broken cross-link and does not settle
whether `executor.resource` is one; §10.5 asks a consumer to surface
rather than silently drop. Reporting honours the second without testing
the first, and leaves the door's one invariant — verbatim bytes — alone.
`ImportResult.unverified_references` is an advisory over the merged set,
not a fifth bucket: every concept it names has already merged, the
bytes are unchanged, and a refused concept is never named (there is no
imported pointer to check).
The report is at KEY level, and that is a measured limit rather than a
choice. Resolving the resource means reading `executor.resource` — the
value the line-oriented parser cannot recover in either canonical form:
a block mapping flattens and collides, a flow mapping stays one opaque
string. A resource-level report would be empty or wrong on exactly the
forms upstream writes. Precision arrives with the structured reader
(D1b); the key-level signal is robust in both forms today.
[skip-docs] is on the CLAUDE.md half only: README carries the new
public surface (`unverified_references`), and the invariant this work
put in CLAUDE.md — flow form, never block — landed with the previous
commit and needs no restatement.
584 tests, mypy --strict clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KKKMwi7e7PVHoFW6dJK5XP
Phase 3 step 1. What Phases 1 and 2 hard-coded about a valid bundle now
lives on one frozen `BundleProfile`, and `DEFAULT` states exactly the
ingest-spec v1 + Phase 2 contract. Nothing observable changes: the 425
existing tests are unmodified and green, and `git diff --stat examples/`
is empty, which is assumption C1's whole proof. Both oracles are real
rather than nominal — the golden suite compares `read_bytes()`, and Door
B pins its frontmatter block as an exact string.
Moved onto the profile, each one previously a constant with a reader:
the index name and its managed-link shape (template plus pattern, kept
honest by a round-trip test), the three filename namespaces
(`ingest-`/`inbox-`/`import-` and `.md`), the reserved `verdict` layer
(duplicated in `manifest` and `inbox` before this), the frontmatter key
order, and the `source_query` whitespace collapse.
Two details are worth naming. The DEFAULT key order spans both doors:
Door A's seven keys and Door B's six are subsequences of one nine-key
order, so a single canonical order reproduces both doors byte-for-byte.
And emission follows commons decision D1 — ordered prefix, then any
remaining keys sorted — which is the mechanism the proving consumer's
hash registry needs; under DEFAULT the tail is always empty.
`TypePolicy` refuses the reserved layer at CONSTRUCTION (assumption C3),
so a profile admitting `verdict` cannot be built, let alone passed to a
door. It reports refusals rather than raising them, because Door A
refuses with `ManifestError` and Door B with `MaterializationError` for
the same type — the wording and the stable code come from the policy,
the exception class stays each door's own.
Deliberately NOT moved, each for a stated reason: the required and
allowlisted key sets the proving consumer needs (no code reads them
until the STRICT_V1 validator exists, and a field nothing reads is a
claim nothing tests), the id grammar (a pattern the slugger derives a
separator class from, not a flat name), `NAME_MAX_BYTES` (a filesystem
fact, not a contract choice), and every disposition/origin/channel
vocabulary (guard territory, always). The profile is also not exported
from the package root yet — the optional `profile` argument on the
flows is step 3's plumbing change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Reads an external OKF bundle as {bundle-relative path -> document text},
hands it WHOLE to an injected gate over the guard's okf.import_bundle (a
bundle-level call: it resolves the cross-link graph across concepts), and
merges only concepts clearing the non-blocking floor. Same injection pattern
as Door B, so the core stays dependency-free while the CI channel for the
real guard is settled.
Two constraints shaped the design and are pinned by tests:
- A merged concept is written VERBATIM. Stamping provenance into it would
require round-tripping its frontmatter through this library's line-oriented
parser, which cannot represent the block lists the guard's parser accepts --
silent data loss -- and would persist bytes the guard never screened.
- Ownership is therefore proven by content identity: identical bytes at the
target name are a no-op re-merge (re-import of an unchanged bundle is
idempotent), and anything else at the name is refused. Curated content and
an updated concept are refused alike; refusing is what never destroys.
The floor is fail-closed beyond the plan's "no error" wording: an error, an
unrecognised disposition, and a concept the gate returned no verdict for are
all refusals. quarantine_review stays its own bucket, as at Door B.
origin/channel are validated against the guard's pinned vocabulary -- it
derives trust from origin by enum identity, so an unrecognised string would be
silently downgraded rather than caught.
Three primitives promoted for reuse rather than duplicated:
reduce_to_id_grammar and check_filename_length to materialize.py, and
extract.decode_text. Door C slugs the WHOLE concept path, so tables/users.md
and views/users.md stay distinct. Concept discovery folds case explicitly
rather than globbing *.md, whose case-sensitivity follows the filesystem and
would import the same bundle differently on APFS and ext4.
README's "what is gated today" section corrected: it claimed nothing is gated,
which is no longer true, but the honest statement is narrower than "the doors
are gated" -- the library cannot verify that an injected adapter is a real
guard, and a permissive stub is believed.
405 tests green; ruff, ruff format and mypy --strict clean.
