A concept named its source file by basename and, when segmented, carried a
`source_offset` into the text THIS LIBRARY extracted. Following that pointer
needed the corpus directory, the extractor and its exact transitive version --
none of which the bundle carries. Hand-walked on a real K2 concept: six steps,
four of them requiring knowledge from outside the bundle, to learn that a
requirement sits on pages 12-13 of a 20-page document.
The address is spec's: `sources: [{ resource, title }]`, where `resource` is
the dropped file's inbox-relative path (SPEC v0.2 5.1:303-306 -- "an absolute
URL, a bundle-relative path, or a path into a `references/` subdirectory").
The locator is ours, and it has to be: 5.1 has no field for a place within a
resource, and the pinned guard (1.3.0) rejects every route to putting one
inside a `sources` entry -- a non-allowlisted key by name, a nested flow list
as "scalar leaves only", and quoting as an unsupported form. So the locator is
top-level keys shaped like `source_offset`, and a path carrying a flow
terminator is refused fail-fast rather than mangled.
The unit table is built AT EXTRACTION, where the extracted text and the
original's structure are known to agree: pdf -> `source_pages` from
pdfplumber's own page numbers (a page that yielded no text does not renumber
the ones after it), xlsx -> `source_sheet` + `source_rows`, everything else ->
`source_lines`. `source_offset` stays.
Two measurements changed the design before it shipped. A `paragraphs` key for
docx would name a number the document does not have: `<w:p>` counts of
108/27/65/176/57 against converted-markdown lines of 75/33/67/144/63, not one
pair agreeing -- so the key is `source_lines` and says what it indexes. And an
empty spreadsheet row renders exactly like a table separator: the content-based
rule ate 8 empty rows on the K2 price sheet and reported its last row as 92
against a workbook that says 100. The separator is now found by position, and
`tomrad.xlsx` keeps that red.
One profile moves. `provenance` is a policy object, `None` everywhere but
`SEGMENTED_OKF_V0_2`; the other five shipped profiles are byte-identical.
K2 rebuilt from a frozen src copy: 629 concepts, 1108 files, name set identical,
0 ids moved, 479 files byte-identical, 629 changed and 0 lines removed anywhere.
629/629 now carry an address and a locator. New ref
`sha256-tree:665563a2f74423fcbcc8e4f0b0954ee73b73985ac0418de4f6987bd162a1f7c8`;
`2f82fcfe...` is stale. The pre-pass payload does not grow by one byte
(209 092 B before and after, 18 changed lines: the ref and eight per-concept
digests) -- because an excerpt carries the body, not the frontmatter, which is
also why the consumer still cannot cite "file X page 12" from a payload alone.
Report: docs/2026-09-08-proveniens-k2.md. 1339 tests, ruff and mypy clean.
Co-Authored-By: Claude <claude-opus-5>
Arm E, capability only. A profile MAY name a renderer per suffix; no
domain-aware renderer is written here, that stays a Non-Goal, and `_RENDERERS`
is empty on purpose so the emptiness reads as a decision rather than an
omission.
THE LAYERING IS THE DESIGN, not an implementation detail. `extract.py` is the
extraction registry and must not import the contract layer, or the dependency
runs backwards and the registry stops standing on its own. So `extract_text`
gains a keyword-only `renderer: Callable[[str], str] | None`, knowing nothing
about profiles, and `inbox.py` -- which already holds the profile at that call
site -- resolves a NAME to a function. A test asserts extract.py still contains
no reference to the profile layer, because that constraint is the whole reason
the parameter is shaped this way.
The renderer runs AFTER extraction, never instead of it, so it never has to
re-implement a reader and the two cannot drift. The default is identity, which
is what keeps the five byte-pinned goldens byte-pinned -- asserted per suffix
rather than once.
An unknown renderer NAME is refused rather than falling back to identity: a
silent fallback would produce a bundle that looks rendered and is not, which is
the failure mode this arm exists to make visible. That needed a registered code
(`unknown_renderer`) and its test -- slightly beyond the step's named files,
but the capability cannot ship without defining what an unknown name does.
`tests/test_profile.py`'s exact-field-set assertion went red, as the plan's risk
table predicted. Updated deliberately with the reason recorded: that assertion
exists so a field cannot arrive without someone deciding it should, and its red
run is the mechanism working.
Suite 917 -> 926. All five goldens byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The v0.2 profile lands additively: DEFAULT keeps stating commons' §5 layer
byte for byte (the golden suite is the proof, not the claim), and OKF_V0_2
adds the families v0.2 introduces on top of DEFAULT's key order.
Two questions the plan left open were the operator's, and both were decided
before code rather than discovered during it:
- `generated.by` is `process:llm-ingestion-okf` — plan V1's option (d), the
§7 process actor form. The value carries no version, which is what keeps a
byte-compared fixture stable across releases and leaves a shared
cross-implementation fixture possible. A-E3 was written against the
`<producer>/<version>` form and is now STALE in the pilot specification; the
correction is owed to portfolio-optimiser-claude before they run.
- `sources[].resource` is the manifest source's locator verbatim: the file
root, the sql connection_ref (an env-var NAME, never its value), or the http
base_url. `credential_ref` is not a locator and is never emitted.
Ownership becomes a policy on the profile rather than a literal in the gate.
The emitter and `_is_ingest_owned` are coupled through the stamp value, so
OwnershipPolicy is where they meet and can only change together. Recognition
is one-way by decision: OKF_V0_2 owns the v0.1 stamp too, so a DEFAULT-written
bundle re-runs IN PLACE (operator, 2026-07-27), while DEFAULT still refuses a
v0.2 file rather than replacing it — V-A3's fail-safe is preserved.
An unquotable locator is refused rather than emitted. Measured with PyYAML
rather than reasoned: `[{ id: a, resource: data, backup }]` raises nothing and
parses to a mapping with a `backup` key nobody wrote, so the failure mode is a
silently wrong provenance record. Validation, not repair.
Deliberately NOT here: the `okf_version` declaration. §12 makes it a MAY and
none of upstream's four reference bundles exercises it; WHERE it goes is open
between upstream's root-index frontmatter and catalog's body-line convention,
and catalog verifies against upstream first. It is declared once, at D5.
V-A5 is extended to the new profile — the one whose NAME is the place a
version literal would look natural, and it carries none.
542 tests, mypy --strict clean.
Phase 3 step 1. What Phases 1 and 2 hard-coded about a valid bundle now
lives on one frozen `BundleProfile`, and `DEFAULT` states exactly the
ingest-spec v1 + Phase 2 contract. Nothing observable changes: the 425
existing tests are unmodified and green, and `git diff --stat examples/`
is empty, which is assumption C1's whole proof. Both oracles are real
rather than nominal — the golden suite compares `read_bytes()`, and Door
B pins its frontmatter block as an exact string.
Moved onto the profile, each one previously a constant with a reader:
the index name and its managed-link shape (template plus pattern, kept
honest by a round-trip test), the three filename namespaces
(`ingest-`/`inbox-`/`import-` and `.md`), the reserved `verdict` layer
(duplicated in `manifest` and `inbox` before this), the frontmatter key
order, and the `source_query` whitespace collapse.
Two details are worth naming. The DEFAULT key order spans both doors:
Door A's seven keys and Door B's six are subsequences of one nine-key
order, so a single canonical order reproduces both doors byte-for-byte.
And emission follows commons decision D1 — ordered prefix, then any
remaining keys sorted — which is the mechanism the proving consumer's
hash registry needs; under DEFAULT the tail is always empty.
`TypePolicy` refuses the reserved layer at CONSTRUCTION (assumption C3),
so a profile admitting `verdict` cannot be built, let alone passed to a
door. It reports refusals rather than raising them, because Door A
refuses with `ManifestError` and Door B with `MaterializationError` for
the same type — the wording and the stable code come from the policy,
the exception class stays each door's own.
Deliberately NOT moved, each for a stated reason: the required and
allowlisted key sets the proving consumer needs (no code reads them
until the STRICT_V1 validator exists, and a field nothing reads is a
claim nothing tests), the id grammar (a pattern the slugger derives a
separator class from, not a flat name), `NAME_MAX_BYTES` (a filesystem
fact, not a contract choice), and every disposition/origin/channel
vocabulary (guard territory, always). The profile is also not exported
from the package root yet — the optional `profile` argument on the
flows is step 3's plumbing change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
