llm-ingestion-okf/CLAUDE.md

# llm-ingestion-okf

## Context

Shared OKF (Open Knowledge Format) ingestion library. Three entry doors,
one boundary rule:

- **Door A — spec-based ingestion:** implements the normative
  `ingest-spec.md` owned by `portfolio-optimiser-commons` (manifest →
  `file`/`sql`/`http` connector → deterministic materialization of
  `ingest-{id}.md` → index generation; zero model calls). This repo
  IMPLEMENTS the spec; commons keeps authorship. Spec changes the library
  needs go via commons, never edited locally. The library ships the §11
  golden fixtures (byte-exact) for the three door-A source types
  (`ingest-golden-{file,sql,http}/`, shipped in `9dd86b1`).
- **Door B — bundle inbox:** converts dropped files to OKF concepts. The drop
  directory is walked RECURSIVELY, sorted by relative path, and a concept's
  `source_file` is that relative path (`/`-separated) while its NAME still
  comes from the basename — so a nested duplicate hits the §3 collision
  refusal rather than vanishing. Dot-directories and a bundle nested inside
  the inbox are skipped with a code, never silently, because recursion makes
  the door's own output reachable as its own input (operator 2026-09-06; the
  flat listing was not a boundary, it was an absence with no denominator). All
  file-type→text extraction lives HERE (the guard is text-only). v1 core:
  `md`, `txt`, `csv`, `json`, `html` (stdlib). **`html` got a measured
  segmentation row 2026-09-10.** Until then `_HTMLTextExtractor.text()` was
  `" ".join("".join(parts).split())`, and `str.split()` with no argument splits
  on newlines too, so extraction of ANY HTML file returned unconditionally ONE
  line while every boundary grammar in `propose` is line-anchored -- measured
  outside this repo, **828 of 828** real sections gave 0 plans and exit 2 at
  every sample point, and a coarser 145-document cut gave 145 of 145. Block tags
  now open their own lines and `h1`-`h6` carry the ATX marker for their OWN level
  (a flat `#` would hand `_ATX` three top-level boundaries where the document
  declares one section and two subsections). The output grammar is MARKDOWN, the
  same the office rows reach the proposer through, so **no HTML-only heading
  grammar exists**; the fix is in the extractor and **never** the converter,
  because `.html` stays out of `_PANDOC_FORMATS` on CVE-2025-51591. After:
  **828 of 828 plans, exit 0, 3206 concepts / 6015 md -- the markdown path's
  count EXACTLY**, and the same at 414 (1651) and 83 (343). Text preservation is
  an EXACT invariant and not a percentage: strip the added ATX markers and the
  non-whitespace sequence is identical to the old extractor's, **828 of 828
  files**, character ratio **1.000000**. `_SKIP_TAGS` stays `{script, style}`.
  Exposure elsewhere measured rather than argued: **0 of 86** K2 corpus files and
  **0 of 5** smoke-folder files are HTML, and the smoke bundle is byte-identical
  before and after. `_EVIDENCE` gains a `.html` row at `measured` -- and a `.pdf` row at
  `measured` since 2026-09-10, the row with the most measurement behind it and
  no entry in the table at all -- with the limit
  that travels with it -- one product, one format, one publisher, and a
  generator's cut, not 828 documents anyone wrote.
  **`.xml` became a CORE type 2026-09-11** and it is the first row whose
  ceiling is structural rather than recovered. A NISO-STS zip from a
  publisher's own viewer was **110 of 110 unreadable, 0 plans, exit 2** --
  `.xml` was in neither registry -- and the one xml file in it IS the whole
  product: R761 Prosesskoden:2025, the document round 12 met as a 701-page
  PDF, carrying 7 715 `<sec>`, **2 761 with a `<title>`**, 4 954 with a
  `<label>` and no title, 10 `<table-wrap>`, root `<standard>`, 0 `<!DOCTYPE`.
  Its `<sec>`-nesting depths over the titled sections are
  28/118/500/1141/868/97/9, row for row the fasit's own. The reader is stdlib
  (`xml.etree.ElementTree`) and adds NO dependency -- `defusedxml` and `lxml`
  are 0 occurrences in `uv.lock` -- so it is core beside `.html` rather than
  behind `[extract]`, which would make a pure-stdlib type binary-dependent.
  **The output grammar is MARKDOWN**, the same the office rows reach the
  proposer through: `<sec>` with a `<title>`
  becomes one ATX line carrying `<label>` + space + `<title>` at its own
  nesting depth, `<sec>` with only a `<label>` becomes a body line with the
  label in front (never a heading -- 4 954 of 7 715 are lettered points and one
  heading each buries the document's own 2 761), and `<table-wrap>` becomes its
  label plus one markdown table through `render_table`. **`<label>` carries the
  number and `<title>` carries the text** -- 2 of 2 761 titles begin with a
  digit -- so emitting `<title>` alone scores 0 of 2 761 with nothing in the
  code looking wrong. Inline is an ALLOWLIST and block is the default, the
  inverse of the HTML reader, because block-versus-inline is a property of HTML
  and XML has no such universal; the allowlist is load-bearing at 1 701
  `<italic>` and 1 396 `<bold>` inside that document's prose. The ATX ceiling is
  6 and STS nesting reaches 7, so the depth is CLIPPED and not dropped: 9 of
  2 761 sit at depth 7 and `#######` matches nothing. A `<!DOCTYPE` is REFUSED
  unparsed with its own code, a guarantee about the code rather than about the
  machine -- measured on 3.14.0 with pyexpat 2.7.3, an external SYSTEM entity
  is refused by the stdlib but the billion-laughs limit comes from libexpat
  >= 2.4.0 and not from Python, while `pyproject.toml` requires only `>=3.10`.
  XML that is not STS keeps its text in document order and gets NO invented
  structure, and `.xml` never routes through the converter -- a second parser
  that would never see that refusal. **XML that is not STS also gets 0 plans
  and a FAILED build (exit 2), and that is NOT an `.xml` defect**: a folder
  holding one `.txt` of prose with no headings gives the same three lines and
  the same exit, so it is general `okf build` behaviour for any structureless
  document. The gate stays -- a run replaying zero plans would emit a flat
  bundle and call it success -- because separating "0 plans, 0 unreadable" from
  "0 plans because nothing could be read" changes the outcome on **0 of the 4**
  reference corpora.
  **THE READER REACHED ITS CEILING IN ROUND 13 AND THE BUILD DID NOT, AND
  ROUND 14 CLOSED IT AT THE SHIPPED DEFAULTS.** The reader emitted 2 761 of
  2 761 heading lines while the build delivered **23 concepts and 15 of 2 761
  boundaries** -- two steps after the reader, each measured: the orphan check
  took **710 of 2 761** (710 of 710 removed headings are followed immediately
  by another heading, **0 of 2 051** delivered ones are -- they are container
  sections) and Arm F took **2 066** more, 2 089 -> 23. `find_candidates`
  already skipped both for `outline_marks`, which is why the PDF bookmark arm
  reaches 2 762; an STS `<sec><title>` is the same class of declaration and
  only arrived as `rule:heading`. The fix is ONE new rule constant reached from
  ONE row: `extract.xml_outline` reports the marks the reader WROTE ITSELF --
  **no bridge, no tolerance constant, no `unresolved` bucket**, the difference
  from `pdf_outline` whose naive nearest-line rule was wrong on 1 840 of 2 762
  -- `propose.RULE_XML_SECTION` (`rule:xml-section`) is its own name in
  `RULE_NAMES` and `_ORPHAN_EXEMPT`, and `build_plan` chooses the route by the
  **ROW** (`DECLARED_STRUCTURE_IDS`), never by the text: the same markdown from
  a `.md` file is still a guess and still carries `rule:heading`. At shipped
  defaults, no flag: **2 761 concepts**, **2 761 of 2 761** declared sections
  became a concept with the source's own directory AND title, **0** concepts
  matching no declaration, `a)`-points **0 of 4 954**, table blocks **10 of
  10**, hit@1/8/50 **3/6 / 5/6 / 6/6** from 0/6 / 0/6 / 0/6 with the
  known-positive at rank 1, and **2 761 shared concept ids** with the PDF arm
  (100 % of this bundle, 2 761 of 2 762 of that one) against round 13's 2 022.
  **NO other file type changes one byte and it is MEASURED on the bytes**: the
  whole 43-document reference corpus rebuilt is `diff -r`-identical to the
  pinned bundle (865 md), the five-document folder is `diff -r`-identical,
  `okf project` stays byte-equal to `okf build`, and the PDF arm still proposes
  2 762. Two directories of 2 738 still hold two concepts (`11`, `12`) -- the
  publisher reuses a section number, the same 2 the PDF arm has, and 0 is not
  reachable without inventing an id; round 13's 14 such directories were false
  positives of the TEXT route reading the document's own contents listing and
  are gone. Report:
  `docs/2026-09-10-k3-runde14-deklarert-struktur-tar-ruten.md`.
  **Since K3-19 an STS document's own identity names its directory**
  (`extract.declared_identity`, read by `cli._document_prefixes` and the door):
  the directory was the delivery file's stem, a UUID occurring **0 times** in
  the document, while its one `<std-ident>` carried `<doc-number>`. Only the
  stem is replaced, and a declared name two documents in one run claim is
  used by NEITHER -- the `slug_owners` gate would refuse both with "rename
  one", which a name read from inside a document cannot obey. The `sources`
  title is `<doc-number>` + `<year>`, then `<title-wrap>`, then the file name:
  R761's `<full>` carries a COMMA, a flow terminator, so it is never written
  and never cleaned up. A titled section's `description` is its own FIRST
  spec point (first `<p>` of the first DIRECT-child `sec-type="spec"`, whole),
  carried by the plan entry and screened by the gate: 2 026 of 2 761 on R761,
  none invented for the rest. SS 4.1 sets no length, so the one-paragraph
  limit is ours. The registries are COUPLED: a row in
  `_CORE_EXTRACTORS` and not in `segmentation._STDLIB_EXTRACTOR_IDS` refuses
  every proposal for the type, two layers away from the extractor.
  `pdf`/`docx`/`xlsx` only via
  the optional `[extract]` extra; without it those types are rejected
  fail-fast. The extra ships `pdfplumber` for `pdf` (chosen on ONE measured
  property: it keeps a requirement table's label and value on the same line
  where three alternatives do not); `docx`/`xlsx` still ship no parser.
  `pptx` and `md` were measured end to end for the first time 2026-09-10
  (`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md` § 4) on two
  hand-built documents, which is more than zero and is not a fasit: `md`
  recovers 3 of 4 declared headings, and `pptx` segments per slide only where
  the deck's slides carry title placeholders the converter recognises — a deck
  whose slides do not lands as ONE concept. A converter attribute also leaks
  into concept titles (`{#slide-N}`, `{#sheet-1}`), reaching 2 of 810 files on
  the K2 default bundle and 1 of 30 on the operator's test folder; because a
  filename is reduced from its title, fixing it RENAMES concept ids a consumer
  has already cited, so it is an operator question and not a patch.
  Structured table recovery is **out of scope** — two independent parsers
  return the same wrong shape, so the breakage is document geometry, not a
  library choice. PDFs enter as prose, and drawn content (figures) does not
  survive extraction at all, which every `pdf` extraction warns about.
  Under the `STRUCTURED_V1` profile Door B additionally DERIVES structure —
  title (leading heading → `title` key → `path.stem`), document number,
  hierarchy, and cross-references — writes it into the concept frontmatter, and
  projects it into a faceted index entry. Every inferred field is named in a
  `derived` list; an unmarked heuristic is worse than none.
  Under the SEGMENTED v0.2 profile a concept additionally POINTS BACK at the
  original: `sources: [{ resource, title }]` in SPEC §5.1's form (`resource` is
  the inbox-relative path), plus a locator per format — `source_pages`,
  `source_sheet`+`source_rows`, else `source_lines`. The locator keys are OURS
  and must stay top-level: §5.1 has no field for a place within a resource, and
  the pinned guard rejects every route to putting one inside a `sources` entry
  (non-allowlisted key, nested flow list, quoted scalar), so a locator in the
  entry would emit bundles Door C could never read back. The unit table is
  built AT EXTRACTION — a page number cannot be recovered from joined text —
  and `source_offset` stays. `source_lines` indexes the EXTRACTED text, never
  the original's paragraphs: measured, docx `<w:p>` counts and converted-line
  counts do not agree on a single one of five documents. Record:
  `docs/2026-09-08-proveniens-k2.md`. The index is a
  PROJECTION recomputed from the whole bundle each round, which is what makes
  rebuild-from-scratch equal an incremental update byte for byte. `DEFAULT` is
  untouched and byte-identical. Record: `docs/plan/structure-derivation.md`.
- **Door C — external bundle import:** third-party OKF bundles are assessed
  per concept via the guard's `okf.import_bundle`; only concepts clearing the
  guard's non-blocking floor are merged/indexed here. Two invariants, both
  load-bearing: a merged concept is written **verbatim** (this library's
  line-oriented frontmatter parser cannot round-trip the block lists the
  guard's parser accepts, so stamping an external concept would destroy sender
  data and persist bytes the guard never screened), and ownership is therefore
  proven by **content identity** — an occupied target name is re-used only
  when the bytes there are already identical, never overwritten otherwise.

**Boundary rule (non-negotiable, zero overlap):** `llm-ingestion-guard`
(pinned `>=1.2,<2.0`) answers "is this content safe to persist?" —
scan/sanitize/quarantine/fail-secure/provenance-stamp. This library is
plumbing: connect source → materialize deterministic OKF bundle → generate
index. Never reimplement security; call the guard at persist gates
(`prepare_input`/`screen_output`, `okf.import_bundle`). When in doubt which
side of the boundary something belongs on: ask the operator.

**Implementation baseline:** the stricter behaviors from
`portfolio-optimiser` (streaming row caps, utf-8-sig, in-memory staging with
pre-mutation collision gate, validated `ingested_at`, typed `IngestError`)
are the library baseline. First consumer: `portfolio-optimiser-claude`.

### Roadmap (phases 1–3 shipped; what follows is demand-driven)

1. **Phase 1 — Door A (Python).** ingest-spec implementation + the §11
   golden fixtures. Consumers: `portfolio-optimiser-claude` first, then
   `portfolio-optimiser`.
2. **Phase 2 — Doors B/C (Python).** Bundle inbox and external-bundle
   import, guard-gated.
3. **Phase 3 — Configurable bundle contract.** Types, layers, frontmatter
   sets, index shape, and reserved-file policy become config instead of
   constants; proving consumer is `claude-code-llm-wiki` (`strict-v1`
   profile). Two consumers hold opposite postures on whether an index is
   authored or directory-derived, so neither is a library invariant and
   nothing here enumerates a directory unless the profile says derived.
4. **Phase 4 — Node half (`node/`).** Zero-dependency Node/ESM package
   (importable *and* CLI-invokable, vendorable per plugin — matching the
   marketplace precedent) for the second-brain world: bundle check, index
   generation, inbox split/frontmatter/write, and doc conversion
   (docx/pdf/eml/html → md). Covers okr, linkedin-studio, ms-ai-architect,
   and the marketplace catalog.

5. **Phase 5 — MCP as a way to populate a bundle. NOT COMMITTED; needs-based
   (operator 2026-08-02, superseding the 2026-07-27 commitment.)** No MCP work,
   and no data-lake or database source types, are undertaken without a stated
   need. `docs/plan/mcp-bundle-population.md` stays as a design record, not a
   queue. Its open fork — whether we are the MCP **server** (an agent calls our
   doors as tools) or an MCP **client** (a manifest source type pulling from
   someone else's server) — no longer blocks anything, because nothing waits
   behind it. It is a question to answer *if* a need arrives, not before. This
   is also why `sql` staying sqlite-only is not a gap: a Postgres driver would
   be runtime dependency number two, bought for no asked-for use.

The two halves share the OKF contract and fixture suite, **not code**.

**Standing posture (operator 2026-08-02).** Phases 1–3 shipped; the library now
runs on what it has. Work is defect fixes, improvements, and features that a
consumer has actually asked for or that measured feedback shows are needed —
not roadmap completion for its own sake. The upstream version policy below is
the one exception, and it is not a counterexample: "always latest" is a promise
already made to consumers, so an upstream release *is* the stated need.
Phase 4 keeps four named consumers with working implementations to lift, so its
need is real but untriggered — it starts when one of them asks, not on a date.

### Upstream version policy (standing, non-negotiable)

**The library always supports the current latest version of Google OKF.** Set by
the operator 2026-07-26. Phases 1–3 were built against v0.1; v0.2 shipped
2026-07-25, so v0.2 support is committed work — not contingent on a consumer
asking for it. Plan: `docs/plan/okf-v0.2-alignment.md`.

Support is **additive, expressed as a new profile**, never a migration of
existing ones. This is what makes the policy sustainable instead of a recurring
crisis, and it is bounded by three facts that do not yield to it:

- `DEFAULT` states commons' ingest-spec §5 layer — its `generated` shape is
  commons' call, raised there, never patched locally. **This fired 2026-08-09:**
  commons ratified and executed the O2 form, so `DEFAULT` now stamps
  `generated: { by: process:okf-ingest, at: <ingested_at> }` and four goldens
  moved with it. It is not a counterexample to "additive, never a migration" —
  that rule governs *upstream* versions, and commons' spec is a separate axis
  `DEFAULT` tracks by definition. `DEFAULT` stays v0.1 on everything upstream
  owns. Ownership recognition is one-way, so the cost to a consumer stays a
  re-run: a profile carrying an actor still owns the older literal stamp.
- `STRICT_V1` mirrors the proving consumer's ratified contract — changing another
  repo's contract from here violates O2.
- `okf_version`'s *value* belongs to catalog (decision E1).

**Rollout is pilot-first.** A new upstream version reaches a small pilot set on a
pre-release tag and is revised on their feedback before general availability —
consumers testing real data find what fixtures cannot. `OKF_LATEST` means the
latest version supported as *stable*, so flipping that alias is the GA event, not
a merge side effect.

Two invariants fall out: no profile hard-codes an upstream version, and no bundle
declares a version its shape has not earned. The first has a mechanism, not just
an intention: **a profile names a key, a caller owns its value.** `okf_version`
is declared through `materialize_bundle(..., root_frontmatter_values=...)`
because its value tracks the upstream Google version and belongs to catalog
(decision E1) — a constant here would claim a decision we do not own, and would
be the one thing to chase on every upstream release. Where upstream itself defers a
contract — v0.2's attestation receipt and verdict wire formats — the format is
supported and the unspecified runtime is not; it re-enters scope when upstream
specifies it. Because "always latest" decays silently, the release checklist
carries an upstream-version re-check.

**Structured frontmatter values are emitted in YAML *flow* form, never block.**
Both are valid YAML and an upstream reader recovers the same structure from
either, but this library's parser is line-oriented: it round-trips a flow
mapping as an opaque value and cannot read the block form at all — two block
mappings sharing an inner key (§10.2's `executor` and `attester`, both carrying
`resource`) collapse into one namespace and the first is lost silently.
Emitting block would produce bundles we cannot read back. Reading it needs the
structured reader (D1b); until then the constraint binds what we write.

**Every upstream release runs `docs/upstream-okf-upgrade-runbook.md`.** Pin the
commit, enumerate the whole `okf/` tree, **read the shipped example bundles and not
only `SPEC.md`**, classify the diff, measure our exposure and each consumer's, plan
additively, pilot before GA, then inform every OKF-consuming repo. The runbook is
not optional and not a summary of good intentions: each step names the concrete
failure it prevents, and all of them are failures that happened during v0.1 → v0.2.

**This repo is a black box for its consumers.** The target cost of an upstream
release to a consuming repo is **a re-run, nothing more**: support is additive
(a new profile, never a migration), existing profiles stay byte-stable, new public
parameters are keyword-only with defaults so positional call sites stay
source-compatible, and consumer golden fixtures must not churn. The boundary is
stated every time rather than glossed — the library absorbs *shape* changes, not
upstream changes to content a consumer authored (v0.2's `timestamp` and
`# Citations` supersessions). For that class the deliverable is a measured exposure
report per consumer, sent before they ask.

Phase 4 preconditions (coordination, not unilateral moves):
- Lifts okr's reference implementations (`okf-check.mjs`, `okf-index.mjs`,
  innboks libs) in agreement with okr and the marketplace catalog; the
  catalog remains the convention owner and re-pins its shared gate here.
- linkedin-studio's `ingest/published/` provenance-record grammar stays
  plugin-local by design (different lifecycle) — do not normalize it.
- The Node-side persist gate remains security territory: guard-as-contract
  (per okr's adoption doc) until a Node guard exists in the security repo.
  No security reimplementation here, in either runtime.

### Non-goals (all phases)

- Verdict/feedback machinery (method-spec) — stays in consumer repos.
- Embedding/RAG/retrieval layers.
- Security functionality — always the guard's domain.

## Stack

Python 3.10+. Package `llm_ingestion_okf` (src layout, hatchling).
**Exactly one runtime dependency, ever:** `llm-ingestion-guard>=1.2,<2.0`
(itself zero-dep), landed with the Door B/C persist gates. Everything else is
stdlib, and a packaging test enforces it. Only `guard_adapter.py` imports the
guard; importing the package does not. Install channel until the package
index exists (a direct reference is a channel, not the pin):
`pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v1.2.0"`.
Binary extraction parsers live behind the `[extract]` extra only — today
`pdfplumber>=0.11.10,<0.12` for `pdf`. Extracted PDF text is pinned to an
exact transitive parser version (`pdfminer.six==20260107`), so widening that
range is a fixture migration, guarded by a frozen literal in
`tests/test_extract.py`; see `tests/fixtures/README.md`.

Phase 4 adds a `node/` half: Node/ESM with zero npm dependencies
(`node:` builtins only), both importable and CLI-invokable, consumed by
vendoring per plugin rather than npm publishing. The halves share contract
and fixtures, never code.

## Conventions

- Type hints everywhere; `mypy --strict` target.
- Determinism is bit-exact: `ingested_at` is an explicit required argument
  (no wall-clock defaults); LF-only output; golden fixtures compared
  byte-for-byte.
- Filenames and titles are normalized to Unicode NFC before use
  (`materialize.reduce_to_id_grammar`, `inbox.process_inbox`): macOS/APFS
  hands filenames over in decomposed form, so an `é` arrives as `e` +
  combining acute. Without normalizing first, the same visual name (e.g. a
  Norwegian slugger title like "linkedin-studio") reduces differently
  depending on which form it arrived in, splitting one title into two
  generated filenames.
- No model calls anywhere in the run path.
- Credentials only as env-var *references* resolved at runtime; never in
  manifests, logs, or frontmatter.
- Network access requires an explicit per-run opt-in flag; refuse fail-fast
  otherwise.
- Conventional Commits: `type(scope): description`.
- English for all code, docs, and commit messages (public repo).

## Commands

- Test: `pytest`
- Lint: `ruff check .` + `ruff format --check .`
- Type check: `mypy --strict src/`
- Folder to questionable bundle in ONE command: `okf project <folder>` —
  `okf build` with the package default into `<out>/.okf/<id>/` plus `okf skill`
  into `<out>/.claude/skills/<id>-consume/`, `<out>` defaulting to cwd and
  `<id>` to the folder name reduced to `[a-z0-9-]`. It owns NO flag that moves
  a bundle's bytes and a test holds it byte-equal to `okf build`; two build
  paths would leave every measurement report pinned to a bundle nobody
  produces. **That invariant was FALSE from the day those two
  flags became defaults until O6 measured it, and the test could not see it:** `cli.build`'s Python SIGNATURE defaulted
  `keep_table_heading` and `sheet_section_rows` to `False` while argparse
  defaulted both to `True`, and `project.create` calls `build()` as a function,
  so it read the signature. Measured on a five-document folder, `okf project`
  wrote **15 concepts / 30 files** against `okf build`'s **26 / 52**, the whole
  difference in the priced spreadsheet -- the document a question about price
  has to reach. The byte-equality test compared `project.create` against the
  same `build()`, so both sides carried the same wrong value, and its two
  fixture documents had neither a table nor a sheet: **a test and the code
  agreeing over a set where the difference cannot appear.** Two tests now hold
  it -- one comparing the signature's defaults against argparse's for every
  same-typed parameter, one building a document whose concept count actually
  moves with the two flags. `skills/okf-prosjekt/` is the Claude Code skill
  over it.
- **The generated consumption skill states THREE modes and RELATIVE paths**
  (O6, 2026-09-09). Question (the default), hypothesis (decomposed into
  premises and answered PER PREMISE as `confirmed` / `refuted` /
  `undecidable-from-bundle` -- three literals, no fourth; a weak source is
  `[sourced-not-sufficient]` on that PREMISE, because four premises and one
  weak source is three answers and one gap), and a task producing a document
  (source per claim IN the artefact, an ungrounded paragraph written and marked
  rather than dropped, the cut declared inside the document because the
  document travels without the chat). The five markings are untouched -- the
  modes add no sixth. In the layout `okf project` writes, the commands are
  `okf consume .okf/<id>` and `okf check --skill
  .claude/skills/<id>-consume/SKILL.md`, runnable from the project root, which
  is where `okf project`'s own closing line tells the reader to start `claude`;
  a path OUTSIDE that root stays absolute on purpose, since `../../..` is not
  more portable, only harder to read. Two absolute paths to zero -- and
  **O5's published "4 absolute paths -> 0" was measured with `grep -c "^/"`
  against paths indented by two spaces**, a query that could not have found one
  either way, so the zero was never a measurement. Every path assertion here
  runs its pattern against a known-positive first.
- Build a bundle: `okf build <folder> --bundle <dir> --bundle-id <id>
  --okf-version <v>` — the installed console script (`[project.scripts]`),
  the packaged form of what used to be a shell loop over two `tools/`
  scripts. It is orchestration only: the proposer and the corpus harness
  live in `llm_ingestion_okf.propose` and `llm_ingestion_okf.corpus`, and
  the `tools/` scripts are thin entry points to the same functions so the
  published reproduction blocks still run. Path scope for a document's
  proposals is its RELATIVE path minus the extension (the door walks
  recursively, and two same-named documents in different folders must not
  collide); `--ingested-at` and `--proposed-at` default to one shared epoch
  constant rather than the clock, because a wall-clock default takes
  rebuild-equals-incremental away from anyone who omits them.
  **TEN segmentation rules are REACHABLE here, and since 2026-09-09 ALL
  TEN are ON by default** -- the tenth is `--contents-name`, round 9's repair
  of clause 1, which admits a title into a contents run only when a NAME
  survives stripping its page number. Measured: clause 1 discarded 68
  candidates over 11 of 39 readable documents, 19 of them over 5 documents
  rows of a drawing's dimension chain, a schematic's labels, a door schedule,
  a coordinate column and a soil-layer table. The threshold is SWEPT
  (`propose.CONTENTS_NAME_RUN = 2`) and collapses at both ends: at 1 it
  rescues 13 of 19, at 3 the two-letter section name `VA` stops being a name
  and takes four REAL contents entries with it. At 2 it rescues 16 of 19 and 0
  of 49. Corpus 429 -> 447 candidates, K2 436 -> 453 concepts / 832 -> 865 md
  (`21af4a1aa98315cf...`), the 12-position reference label-identical in BOTH
  readings and hit@8 `[1,1,1,1,1,None]` on the new bundle AND Arm B. Opt-out
  `--no-contents-name`.
  **An ELEVENTH flag, `--bold-title`, is OFF** (round 10, 2026-09-09): the rule
  for the type whose container declares nothing. `rtf` measured 0 of 0 declared
  headings, 0 concepts, 1368 of 1368 characters in no segment. The grammar is
  MARKDOWN, not `rtf` -- the converter already writes the author's bold title
  as `**...**` in the same output every office row produces, so no `rtf`-only
  heading grammar exists, the same shape of decision as the PDF font reader's
  ATX form. Three parameters swept over 47 readable documents and ONE carried
  (refusing a line that ends in terminal punctuation: false-positive lines
  9-12 -> 1-2); a maximum title length and a stand-between-blank-lines clause
  are both FLAT and neither is in the rule. The last false positive is closed
  by G1, `_gate_outline`'s own principle, so false positives are **0 of the 31
  declaring documents**. Reach **2 of 39** corpus documents, both `docx`, **0
  of 33 `pdf`**. **BOTH alternatives the order named were measured and
  FELLED**: a hand-laid fixture that DECLARES a heading style has it discarded
  by the converter, and `rtf` -> `docx` -> markdown yields 0 ATX headings on
  that same document, because the loss is in the `rtf` READER before any
  writer.
  The nine below are unchanged -- `--outline-run 3`, `--table-grid` and
  `--unit-fold` since 2026-09-08, `--drop-wrapped-outline` and
  `--outline-gate` since 2026-09-09, `--sheet-section-rows`,
  `--keep-table-heading` and `--first-span-from-zero` since 2026-09-10, and
  `--close-span-gaps` since 2026-09-11, each
  with an explicit opt-out (`--outline-run 0`, `--no-table-grid`,
  `--no-unit-fold`, `--keep-wrapped-outline`, `--no-outline-gate`,
  `--no-sheet-section-rows`, `--no-keep-table-heading`,
  `--no-first-span-from-zero`, `--no-close-span-gaps`) that together reproduce
  the pre-move bytes -- measured, `diff -rq` 0 differences, not asserted. **The 2026-09-09 pair is one
  decision and cannot be split**: the gate takes `pdf` from 2 of 8 to 5 of 8
  and the pair takes it to 7 of 8 (the sheet 5 of 12 -> 10 of 12, `docx`
  unchanged at 3 of 3). **The gate is G1+G2:** Arm D's RECOVERED headings are
  admitted only where the document DECLARES none of its own -- which is
  `fold_units` clause 2's principle moved from voting to admission -- plus any
  one recovered heading covering `propose.OUTLINE_SHARE` (0.20, swept flat
  from 0.10 to 0.30 and collapsing at both ends) of the text. It filters at
  ADMISSION, before spans close, so the text a removed mark opened is carried
  by the mark above; the post-filter form scores identically and loses that
  text, which is why only one of them shipped. **The bar it had to clear is
  now the bar**: reference cells up AND hit@8 holding rank 1 on every row on
  every bundle. `--sheet-section-rows --keep-table-heading` reaches 11 of 12
  and SHIPPED 2026-09-10, after two rounds off. It was held back because on a
  K2 bundle built with it row 1 fell rank 1 -> 2 (the gold document goes 1
  concept -> 12), under both prior exponents. **That was never these rules'
  defect and it is not a segmentation question**: RRF emits a distinct rank
  for every concept in a signal that scored them all EQUALLY, so the gold
  document's own twelve concepts fill the document-prior tie group and the one
  leading the body signal takes position 11 instead of 1. The repair is the
  reading side's `consume.DEFAULT_TIE_SHARED_RANK`, and with it every
  acceptance condition holds at once. **The fusion was punishing fine-graining
  for being fine-grained**, which put the segmentation side and the retrieval
  side in competition over one number for two rounds. Arm E joined a session after the other two, on a number measured
  AFTER the first move: without it Arm F's table clause has no joined table to
  fold, and the shipped D+F default scored 2 of 12 with `docx` 0 of 3 against
  the 5 of 12 the fold was published with. **The proposer's own defaults did NOT move** (`propose.py`'s rules stay
  off): the goldens and every published reproduction block are pinned to them,
  so the two layers disagree on purpose and `cli.DEFAULT_OUTLINE_RUN` /
  `cli.DEFAULT_UNIT_FOLD` say where. The cost to a consumer is a re-run and it
  is not small: the 43-document reference corpus goes 629 concepts / 1108 files
  (the delivered 2026-09-03 tree) to 492 / 944 after the 2026-09-08 move and to
  425 / 810 after the 2026-09-09 one (`bdf4977ca5a443c4...`) and to
  **436 / 832** after the 2026-09-10 one (`8dff8a8e6c15d2f7...`, default flags,
  default epoch stamp, measured on `38104b7` + this round). On the operator's
  own five-document folder the last move is 15 concepts / 30 files -> 26 / 52. Digests published
  before 2026-09-09 were computed with a path-DEPENDENT command and are not
  comparable to this one; the reproducible form is `find . -type f | sort |
  xargs shasum -a 256 | shasum -a 256` from inside the bundle, under which the
  previous default is `862116da16e422f6...`. The pinned artifact lives at
  `~/corpora/okf-telling-20260829/K2-bundle-default-20260910` and
  `tests/test_default_bundle_pin.py` holds its concept count AND its per-row
  hit@8 ranks -- the count alone survived a configuration that lost a rank,
  which is how a previous round's regression hid. Since 2026-09-10 it also
  holds the KNOWN-NEGATIVE on the same bytes: read with
  `--no-tie-shared-rank`, the shipped default bundle reproduces the very fall
  the rules were held back for, so the pin names its own cause instead of
  being green for an unstated reason. **And the number the
  decision cites belongs to another configuration:** Arm F's 5 of 12 was
  measured with `--table-grid` ON; without it the same sample scores 2 of 12
  and `docx` 0 of 3, because the fold's table clause has no joined table to
  fold. The ten: `--contents-name` (round 9), `--outline-run N` (Arm D),
  `--table-grid` (Arm E),
  `--unit-fold` (Arm F), `--keep-table-heading` (D1), `--sheet-section-rows`
  and `--drop-wrapped-outline` (both D3), `--outline-gate` (G1+G2),
  `--first-span-from-zero` and `--close-span-gaps`, each passed to the
  proposer unchanged. **The last two are the same defect at two ends and
  NEITHER is a segmentation rule**: a mark removed after its neighbour's span
  was closed takes that text out of the plan. Round 8 measured the whole
  remainder -- 43 631 characters, 2.51 %, over 8 of 32 documents with a plan --
  down to **0**, with the concept count identical at 436 and every hit@8 row
  holding rank 1 on both K2 bundles and the reference sheet label-identical at
  11 of 12. Three steps leak: the orphan check (18 527 characters over 15 of
  39 documents), `fold_units` clause 1 between entries (7 514) and the same
  clause on the last run (all 17 590 tail characters; with `unit_fold=False`
  the corpus tail gap is 0). **Round 7's own § 5 does not reproduce**: it
  reports `md` at 3 of 4 declared headings and a `rule:table-block` displacing
  `## 3 Prising`, but D1 -- the repair for exactly that -- became the default
  in the same commit, so the number describes the configuration that existed
  before the move. On `a364ef4` the default recovers **4 of 4**, and
  `tests/test_md_declared_headings.py` now holds the cell with its cause as a
  known-negative. Report:
  `docs/2026-09-11-k3-runde8-tabellblokk-og-siste-spenn.md`. That last
  one is ON since 2026-09-10 and is not a segmentation rule at all -- it adds
  no boundary, and the K2 concept count is identical with and without it
  (425 = 425 on the 2026-09-09 default). It repairs a measured loss: **32 of
  the 32** documents that get a plan left the text above their first concept
  in NO segment. **The hole is bigger than that rule, and this is the number
  to carry:** measured 2026-09-10, the pre-move default left **207 435
  characters, 11.92 %** of the corpus in no segment -- 163 804 above the first
  entry, 26 041 BETWEEN entries, 17 590 after the last. The rule closes the
  first part entirely and 79 % of the whole; **43 631 characters, 2.51 %, over
  8 of 32 documents remain**, and the between-part has a named mechanism (a
  `rule:table-block` candidate displacing a DECLARED heading and opening below
  it). Neither remainder is a ceiling; both are in STATE with their numbers. Until that day the build path called the proposer with no
  arm flag at all, so a tender PDF that Arm D splits into nine concepts landed
  as one -- a build path a full arm behind the proposer. Exposing them was not
  the same decision as moving one, and the two were taken a session apart:
  **which arm ships as the default is the operator's**, answered 2026-09-08 as
  above. Adding a flag still leaves the default byte-identical (measured by
  digest before and after, and by Arm E over all 43 corpus documents); MOVING
  the default is the one thing that does not, which is why it took an operator
  decision and carries an opt-out. Arm C
  (`--max-segment-chars`) stays unexposed: no reference has ever been measured
  for its cap. The two D3 rules read grammars nothing else here reads: a table
  row's FIRST CELL (a run of bare numeric labels cuts the block that holds
  them, which is the only way to reach a sheet whose units are rows and the
  opposite direction from Arm E), and whether a RECOVERED heading's line is a
  wrapped sentence (a heading is a complete line; quoted regulation and a
  recovered table row are not). Both remove or add nothing anywhere else: over
  the 43-document corpus they change 1 and 5 of 39 readable documents, and
  **0 of 5 `docx` either way**. Reports:
  `docs/2026-09-08-k3-arm-f-mot-enhetsarket.md`,
  `docs/2026-09-08-k3-runde2-per-filtype.md` and
  `docs/2026-09-08-k3-runde3-per-filtype.md`.
- **`--frontmatter KEY=VALUE` (K3-19, repeatable) is not a segmentation flag**
  and moves no byte unless given: it stamps a key on every concept of the run,
  split on the FIRST `=` and written verbatim on ONE line -- a block-form
  `sources` is invisible to `parse_frontmatter`, so the flow form is the only
  one that survives our own readers. It adds any key and REPLACES only
  `sources` and `description`, the two with a derived layer below them:
  precedence flag > what the document declares > file name. Every other key
  the door writes (`inbox._door_keys`, including Door A's `ingest_manifest`,
  which would make that door claim a Door B file) is refused before anything
  is read. `okf project` does not take it -- it owns no flag that moves a
  bundle's bytes.
- **A TWELFTH flag, `--pdf-outline`, is OFF** (round 12, 2026-09-10) and it is
  the only one here that does not read the extracted text at all: it cuts a PDF
  at the boundaries its own `/Outlines` bookmark tree declares. It is NOT Arm D
  -- `--outline-run`/`--outline-gate` are a TEXT heuristic over numbered lines
  in the extracted text, and this opens a structure index the file already
  carries. Measured on ONE 701-page process code whose publisher also ships a
  NISO-STS structure for it, so the fasit is the publisher's own 2 761 titled
  sections: the shipped default finds **1 967 of 2 761**, **0 of its 28**
  chapters, and **794 of 794** misses have their heading text PRESENT in the
  extracted text -- the line is read, the boundary is never opened. With the
  arm: **2 759 of 2 761 (99.9 %)**, chapter level **28 of 28**, concept titles
  identical to the source after normalisation **2 761 of 2 761** (the bookmark
  title is complete because it does not come from the page), false positives
  **163 of 2 182 -> 3 of 2 762**, directories carrying two concept files
  **132 -> 2** with the 65 contents-copy pairs at **0**, front-matter concepts
  **72 -> 2**. Consumption: fasit present in the bundle **4 of 7 -> 7 of 7**,
  hit@1/8/50 **1/6 - 2/6 - 4/6 -> 3/6 - 5/6 - 6/6**; the known-positive is a
  real concept now and ranks 13, so it is delivered at k=50 and not at k=8 --
  the segmentation half of that row is closed and the ranking half is not.
  **The bridge from (page, `/XYZ` top) to a line index is the whole risk and
  BOTH routes are measured:** `extract_text_lines` splits lines identically to
  `extract_text` on **701 of 701** pages and that check SHIPS per page, the y
  route and the title route disagree on **0 of 2 762**, flat from 0 to 8 pt and
  collapsing at 12, so the rule carries **no tolerance constant**; the naive
  nearest-line rule was wrong on **1 840 of 2 762**, one line early every time.
  The orphan check is NOT applied to a bookmark mark -- it asks whether anything
  stands under a candidate's first line, the right question for a guess and the
  wrong one for a publisher's declaration; **683 of 2 762** marks are container
  sections and applying it scores 2 079. An unresolvable `/Dest` is dropped and
  COUNTED (R761 has 0 of 2 763; one of the eight reference PDFs has 2 of 2). NO
  new dependency: `pdfminer.six` already ships under `pdfplumber` in
  `[extract]`, so `uv.lock` is untouched and `pypdf` stays out. Cost 119.22 s ->
  183.31 s wall, peak RSS 3 252 -> 3 251 MiB, pages parsed 1 -> 1. **The default
  did not move**, and the reach is why: **1 of 8** reference PDFs carries a
  usable tree, and a bookmark tree is the publisher's CLAIM about its own
  structure. On the folder where no PDF has one, `diff -r` is empty against both
  the arm off and the pre-change tree. Report:
  `docs/2026-09-10-k3-runde12-pdf-outlines.md`.
- **Three PDF READER flags, all off, and they sit BEFORE every segmentation
  flag** -- an arm changes how the proposer cuts a text, these change what the
  text says. `--pdf-headings font` infers a heading from typography (dominant
  font size above the document's character-weighted body median AND a bold font
  name -- the CONJUNCTION measured at recall 1.000 / precision 0.846, where
  adding weight as a disjunct took precision 0.786 -> 0.524) and emits it as an
  ATX heading in the SAME markdown the office path produces, so `_ATX` applies
  unchanged and **no PDF-only heading grammar exists**. It is off **by
  measurement, not by caution**: against the operator's unit worksheet it takes
  `pdf` from **2 of 8 to 0 of 8**, losing two exact matches, because on those
  documents the outline rule already recovers the document's own numbered
  chapters and a second heading source can only add. The cost of the ATX form is
  named rather than hidden: a font-inferred heading carries `rule:heading` and is
  indistinguishable in the artifact from one the document declared, which is why
  `RULE_POPPLER_SIZE_AND_BOLD` was deliberately NOT assigned to it -- that name
  records a poppler measurement on a path that cannot ship. `--ocr` reads a page
  as an IMAGE when its own text never arrived (empty, or `(cid:N)` codes at or
  above `OCR_CID_SHARE = 0.10`, a threshold READ OFF the measured per-page
  distribution: 834 pages over 32 files, 818 at exactly 0.0 and 16 at 0.93 or
  above, nothing between). Its engine is the optional `ocr` group
  (`rapidocr`/`onnxruntime`/`pypdfium2`) and **never** a runtime dependency; a
  packaging test pins both halves, and without it every affected file is a coded
  rejection (`extractor_ocr_group_missing`), never a crash. `--ocr` can never
  become a default -- an optional dependency in the default path would make an
  ordinary install fail on the first scanned page. Report:
  `docs/2026-09-08-k3-runde4-pdf-skrift-og-ocr.md`.
  **`--pdf-headings font-reserve`** is the third value on that same option
  (`none`, `font`, `font-reserve` — three answers to one question, so no caller
  can ask for two at once): the same typographic rule applied ONLY where Arm D's
  outline gate admits no run at all, typography as a second heading source where
  there is no first one. The condition lives in ONE function
  (`propose.heading_reserve_applies`) that the proposer and the door both
  consult, the door receiving it as a callable the way it already receives
  `gate` — a plan indexes the exact string it was proposed against, so a reserve
  firing on one side only would turn every document it touches into a coded
  rejection. It reads the gate AS CONFIGURED, so at `--outline-run 0` it is
  unconditional and equals round 4's "font instead of Arm D". **Off, and the
  measurement is that it changes nothing measurable:** on the twelve-position
  reference it alters **not one cell** — the five positions where it fires are
  one PDF whose glyphs carry no ToUnicode mapping and four office documents the
  PDF reader never touches — and the position it was built for has **three**
  outline runs, so the reserve is silent there by construction. Its reach is
  real but unrated: **4 of 39** readable corpus documents, none in the sample.
  Report: `docs/2026-09-08-k3-runde5-hitat8-og-skriftakse.md`, which also
  corrects two of this repository's own published figures — the S7 candidate
  ranks (96 of 629 / 159 of 492 were measured with the cost vocabulary reaching
  only half the ranker; consistently scored they are **10 of 629** and **19 of
  492**) and round 4's attribution of that concept's non-delivery to the default
  move (it is not delivered on the Arm B bundle either, by a different
  mechanism). hit@8 over the six published questions holds at **5 of 6 on both
  K2 bundles**, so the default move cost the retrieval side nothing.
- Consume a bundle: `okf consume <bundle> --question "<q>"
  [--k N] [--limit N] [--out PATH] [--ref IDENTITY]` — the **pre-pass**
  `docs/consumption-contract.md` § 1 defines, and the only reading direction
  this library has. **It moved into the package 2026-09-08 (O5)** and the move
  it was written for is the one that happened: `build_payload(...)` was always
  the entry point with the CLI a thin `main()`, so it was a move and not a
  rewrite. What forced it was the generated skill — from `tools/` it emitted
  `python3 <absolute path>/tools/okf_consume.py`, so the skill could not be
  moved, shared or run by anyone without that clone. `tools/okf_consume.py`
  remains as an ALIAS (`sys.modules[__name__] = _impl`, never a re-export: a
  re-export binds copies, and a caller patching one patches a binding the
  implementation never reads). Deterministic and offline by construction: no model call, no socket,
  no clock, stdlib plus this package only. It **walks the index tree, never a
  directory** — § 9.2 forbids enumerating one unless the named profile says the
  index is derived, and measured, `entries_match_directory` is `True` for
  `STRICT_V1` alone; the walk loses nothing (629 = 629 on the K2 bundle,
  controlled in a test against the very method § 9.2 forbids). `--ref` is an
  **assertion**, never an override: the emitted identity is always the computed
  one, because § 3.3 exists to stop a payload being labelled with an identity
  its bytes do not have. Three exit codes: 0 written, 1 refused, 2 did not run.
  **Every excerpt carries the concept's `title`**, plus `req_number`, the § 5.1
  address `sources`, and **every top-level `source_*` key by PREFIX** — never an
  allowlist, because a list names the producers its author thought of and one
  bundle locates by `source_element_id` on 269 of 274 concepts. A prefix, never
  a substring (`resource_owner` is not a locator). An absent key stays absent
  and an undecodable address is named (`sources_unreadable`). `sources` is READ
  in both YAML forms because the two real bundles disagree (flow 629/629 on one,
  block 270/270 on the other) — reading block is not a licence to write it, the
  emission rule is unchanged. Contract § 8 makes `title` a MUST (checker code
  `excerpt_unnamed`) and the rest SHOULD, because they are conditional on the
  producer. The measurement behind it: rank 1 of 8 on 3 of 3 bundles, correct
  answer on 1 of 3.
- Connect a bundle to Claude Code: `okf skill <bundle> --out
  <dir>` instantiates `skills/okf-consume-template/` for THAT bundle — its id,
  ref, concept count, conditional-field denominators, whole-bundle cost and
  breaking point, all measured, plus a reference payload the checker accepts.
  It was kept in `tools/` until 2026-09-08 because a wheel-installed
  `okf skill` would emit a command pointing at a file the wheel does not carry.
  That objection was about what the GENERATED skill NAMES, and O5 answered it
  by changing that: the emitted commands are `okf consume` and `okf check`,
  names on PATH. The template and `docs/consumption-contract.md` (the § 7.4
  known-positive) are force-included into the wheel from the file they are
  authored in — one authored copy, no committed duplicate.
  The form was chosen on a measurement: the contract checker passed the
  UNFILLED template and passed a skill built for another bundle, so it could not
  tell the two apart — the choice rests on § 5/§ 6.4/§ 7.6 being per-bundle
  numbers a generic skill can only leave as holes or state falsely, and that
  argument NEVER rested on conformance, so closing the measurement leaves it
  standing. **That half is CLOSED 2026-09-10 by `bundle_mismatch`, the checker's
  SIXTEENTH rule** (`RULES` is a tuple and `Report.rules_evaluated` is
  `len(RULES)`, so every published «15 rules» line is now «16 rules» — a
  contract change for anyone quoting it). It compares the identity a skill
  DECLARES against the identity its payload declares, **both `bundle_id` AND
  `ref`**: three distinct builds on this machine carry the one `bundle_id`
  `k2-trinn1-20260903` at three refs, so an id-only rule would pass a stale
  skill, which is the case the generated skill warns about in its own words.
  SS 3.3 is the ground — «a version is the producer's assertion; a ref is a fact
  about bytes» — and SS 3.1 for the same question one level down, an excerpt
  naming a bundle its payload does not. Reproduced on `113b3f8` before any code
  moved, three pairs at `conformant: 15 rules over 8 excerpts and 438 withheld
  entries, 0 findings` and rc 0; after, all three at rc 1 with **one** finding
  over 16 rules — a foreign corpus's payload, the unfilled template, and (the
  arm that separates a whole rule from half of one) a payload sharing the id at
  a foreign ref. **An identity the rule cannot READ is a finding, never a silent
  pass**, or the template passes again. The right pair is untouched at rc 0 / 0
  findings, and `{}` is unchanged at **9** findings because a payload declaring
  no identity stays `ref_missing`'s defect — no rule restates another. NO new
  field was needed: the identity already lived in the generated skill's prose,
  now one authored copy in `skill.identity_line` read back by
  `contract_check.skill_identity`, generated skill bytes byte-identical before
  and after on both tracked bundles. **The rule compares DECLARED against
  DECLARED and never opens the bundle**, so a payload lying about its own `ref`
  still passes — that is `okf consume --ref`'s job and needs a bundle path this
  command deliberately does not take; «closed» means the three measured forms
  now fell, not that no fourth exists. Report:
  `docs/2026-09-10-k3-runde15-bundle-mismatch.md`.
  The first instantiated consumption skill is `skills/okf-consume/`; the
  measurement behind the form, including the control that FAILED, is
  `docs/2026-09-07-okf-konsumskill-maaling.md`. **That skill was the new rule's
  first REAL find**: hand-filled for K2 before `okf skill` existed, it declared
  no bundle identity a reader can act on, so `okf check` refused it against its
  own shipped example payload (rc 1, 1 finding) — **1 of 1** shipped hand-made
  instantiated skill. **Since 2026-09-11 (K3-18) it is GENERATED** by `okf
  skill` from `examples/ingest-golden-segmented-okf-v0-2/expected-bundle`, the
  bundle its payload always came from, with the command in its
  `references/README.md`: `--force` and `--example-question "Hva sier
  veiledningen om krav?"` are both required (the payload test asserts bytes for
  that question), and the checkout prefix is then stripped, because `okf skill`
  writes the bundle root and the skill path ABSOLUTE when `--out` is not under
  `.claude/skills/`. The pair is rc 0, 16 rules, 0 findings, and a test holds
  the shipped bytes to the generator's. Its frontmatter `name` is now
  `b-golden-segmented-okf-v0-2-consume`: Claude Code takes a project skill's
  COMMAND from the directory name and uses `name` only as a display label, and
  nothing here named the skill `okf-consume`. Report:
  `docs/2026-09-11-k3-runde18-konsumskillen-regenerert.md`. **The ranking is this
  repository's own choice** — the contract binds a payload, not a retrieval
  algorithm (§ 10) — and it has FOUR optional widenings. Three are **off by
  default** and keep the default payload byte-identical; the fourth
  (`--tie-shared-rank`) became the default 2026-09-10 and is the one change in
  this repository that alters a payload with NO bundle changing, so a consumer
  pinned to the old excerpt order needs `--no-tie-shared-rank`. `--cost-vocabulary`: a
  declared cost/price/quantity vocabulary family that bridges a question and a
  document naming money with different words, gated on the QUESTION carrying
  such a term, so a question without one is byte-identical either way. It moves
  a measured case from candidate rank 249 to 10 and does **not** deliver it —
  the budget is a second, independent lock. Measured, with two rules falsified
  before building and the `k`-sweep that showed a higher `k` can EVICT a gold
  concept, in `docs/2026-09-08-blindsone-below-k-k2.md`.
  `--reserve-top-rank` is that second lock: the pack is an exact knapsack over a
  SUM, so it has no opinion about rank and out-sums a top-ranked excerpt costing
  a large share of the budget. The flag gives rank one its bytes first, AFTER
  the `over_budget_alone` pre-exclusion and never before, and declares
  `budget.reserved` in the payload. It fixes the eviction and does **not** close
  the mandate-shaped blind spot (that concept ranks 10, not 1); the budget stays
  the caller's decision, because deriving a limit from the corpus was measured
  and falsified — two defensible derivations, 49x apart, one of them breaking
  the known-positive. `docs/2026-09-08-blindsone-laas2-budsjett-k2.md`.
  **A SIXTH flag, `--stem-prefix`, is ON since 2026-09-09** and is the second
  change here that alters a payload with NO bundle changing (opt-out
  `--no-stem-prefix`). `MIN_SHARED_PREFIX = 4` exists for Norwegian
  compounding and also matches four characters that are not a stem: on the
  pinned 453-concept bundle, control first, `under` occurs 79 times by equality
  and matches 172 by prefix, `bilateral` occurs **0** and matched **400 of
  453** through `bilag`, `standhaftig` 0 and 219 through `standard`. The two
  extra known-negatives were FOUND, not chosen -- every 4-character prefix
  ranked by document frequency, then a real word taken from the widest. **Three
  candidates were measured and all three failed on the SAME row**: a longer
  floor (5-8), a coverage share (0.5-0.8) and a long-words-only floor (>= 8).
  Decomposed, row 1's token `prisene` reaches its gold document through
  `pris|sammenstilling` on four characters -- 0.57 of one word and 0.22 of the
  other -- so **the over-match and the wanted match are one mechanism** and no
  threshold on length or coverage separates them. The fourth candidate does:
  the shared prefix must be a WORD the bundle uses. `bilateral` 400 -> 0 and
  512 -> 0, `standhaftig` 219 -> 56 and 235 -> 33, every hit@8 row keeping rank
  1 on BOTH bundles. `undersjoisk` stops at **162** because `under` IS a word
  here -- a genuine Norwegian morpheme, so that residual is a different answer,
  never a ceiling. The vocabulary is the BUNDLE's own, so the rule makes a
  payload corpus-dependent the way `rarity_weights` already is.
  **A SEVENTH flag, `--source-quota N`, is ON at 2 since 2026-09-10** (opt-out
  `--no-source-quota`) and is the third change here that alters a payload with
  NO bundle changing. It caps how many DELIVERED places one `source_file` may
  take, cutting where `shortlist = candidates[:k]` cuts, so the freed place goes
  to the next candidate and `k` is still delivered in full. The defect was
  measured OUTSIDE this repo on a 3206-concept bundle of a published handbook:
  the code's own process overview is **28 of 3206 concepts (0.87 %)** and **8.0 %
  of the source characters** yet took **8 of 8** delivered places on one question
  and **7 of 8** on the known-positive, which was not delivered at all --
  identical at 343 and 1651 concepts, so it is the corpus's COMPOSITION (it holds
  its own table of contents) and not its size, and a split would move it rather
  than remove it. Swept over {2, 3, 4, off} on three bundles with the fasit
  prefixes validated against the bundle FIRST (that control caught a defect in
  the measuring query itself): at 2 and 3 hit@8 goes **5 of 6 to 6 of 6 on BOTH
  K2 bundles** with all five standing rank-1 rows unmoved -- the recovered row
  had missed on every bundle and every configuration measured until now -- and at
  4 and off it stays 5 of 6. On the handbook bundle hit@8 goes **2 of 6 to 4 of
  6** and the dominant document's share **8 of 8 to 2 of 8**. 2 rather than 3 on
  rank. **What the gain is NOT:** hit@8 asks whether the gold DOCUMENT was
  delivered and a document quota raises how many distinct documents a payload
  holds, so that metric is not neutral with respect to this rule; the five rows
  already at rank 1 are, and did not move. **The adverse case is named:** a
  one-document bundle has one `source_file` on every concept, so the quota would
  deliver 2 where `k` were asked -- the shortlist is topped back up from the
  best-ranked over-quota candidates, making such a bundle byte-identical to the
  quota being off. The `WITHHOLDING_RULES` vocabulary goes six to seven
  (`source_quota_exceeded`, a DIVERSITY drop and not a relevance one) and is
  published in the contract SS 5.3 and in the generated SKILL.md, verified by
  reading the generated file. Editing the contract moved the SS 7.4
  known-positive (12 563 -> 13 238 encoded, delta 336 -> 345), which is that
  coupling working. `--rarity-weight` was measured against the same defect and
  does NOT repair it -- it leaves the dominant document at 8 of 8 places on the
  question it floods -- and stays off.
  `--rarity-weight` is the third: each lexical hit weighs `log(N/df)` over the
  bundle's own concepts instead of 1, so an identifier is not worth what a
  common verb is worth. It enters the RANKING and never the GATE — `lexical`
  stays a count, because a word every concept carries weighs exactly 0 and a
  weighted gate is what `54a0bc2` falsified. Off by default BY MEASUREMENT: it
  delivers one of three requirement lookups and takes a priced sheet from
  candidate rank 10 to 2, leaves one gold unmoved and costs another seven rank
  positions. Two limits are decomposed rather than guessed, and both are
  someone else's mechanism: `MIN_SHARED_PREFIX = 4` makes a unique identifier
  read as 135-of-446 common, and RRF consumes RANKS, so no weighting inside a
  signal can move a gold that already leads it.
  `docs/2026-09-08-sjeldenhetsvekt.md`.
  `--tie-shared-rank` is the fourth and **the only one that is now ON**
  (2026-09-10, opt-out `--no-tie-shared-rank`). It is a correction to the
  TIE-BREAK
  rather than a weight: RRF ranks every concept in every signal, including a
  signal that scored them all the same, and the declared `(-score, concept_id)`
  tie-break then orders that group by id. Measured on N500, whose document
  prior has **two** distinct values over 270 concepts, that signal contributed
  alphabetical UUID order and put a concept answering 7 of 7 question tokens at
  fused rank 14 — outside the cut — behind concepts sharing only `tunnel` and
  `vann`. Under shared ranks it is rank 3 and 2 of the 16 covering concepts are
  delivered. **It shipped OFF on a measurement that was CONDITIONAL and stopped
  being true in a commit reported as changing nothing.** The published cost —
  hit@8 falling 5 of 6 to 4 of 6 — is real only at `DOCUMENT_PRIOR_EXPONENT`
  1.0. Round 6 moved that exponent to 0.5 for an unrelated reason and correctly
  reported it moved no hit@8 row; nobody measured the PAIR. Swept 2026-09-10
  over 2 exponents x 3 bundles x 6 rows: at 0.5 the rule holds
  `[1,1,1,1,1,–]` on all three bundles and FIXES the split bundle's row 1
  (2 -> 1), which is what let `--sheet-section-rows --keep-table-heading`
  become a build default. **A flag's "off by measurement" is a measurement of a
  CONFIGURATION, not a property of the flag** — when a constant it interacts
  with moves, its default is unmeasured again, and nothing in the tree says so
  because the two decisions live in different files. The adverse case is
  recorded rather than hidden: on a synthetic 30-concept fixture where one
  signal separates and two do not, shared ranks move a gold from rank 18 to 30
  (`tests/test_okf_consume.py`). Note also that
  `docs/2026-09-08-sjeldenhetsvekt.md`'s figures were measured under the older
  tie-break and are NOT re-measured — on one fixture the change takes the
  weight's gold from fused rank 18 to 1.
  `docs/2026-09-08-rangeringsbom-sammensatte-ord.md` and
  `docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`.
  The other three stay off. A FIFTH flag is not a ranking widening and is
  listed apart: `--withheld-titles`
  gives each `withheld` entry the concept's `title`, so a reader can see WHAT
  was withheld without reading the bundle (§ 2.2 forbids going to look). The
  code is 11 lines; the bytes are the reason it is off. Measured, it grows an
  N500 payload 37.9 % and takes the 629-concept K2 bundle's BOOKKEEPING to
  122 704 B — past the 120 000-byte limit itself — which would have made the
  breaking point then published in the hand-filled K2 copy of
  `skills/okf-consume/SKILL.md` ("~75 KB at 629 concepts … at roughly 8 000
  concepts") false on the day it shipped. That copy was replaced by a generated
  one 2026-09-11; the measurement of the flag stands.

## Workflow

- TDD: no production code without a failing test first.
- This repo is published PUBLICLY (`open/` namespace on Forgejo). `STATE.md`
  and `docs/oppstartsprompt.md` are LOCAL-ONLY (gitignored) — never commit
  session state or internal briefs. No secrets, sober English prose, no
  marketing language.
- **Consumer content stays at form level in public files.** Some consumers we
  read are private (`claude-code-llm-wiki` is, pending an Anthropic ToS
  assessment). Key names, counts, gate names, profile fields and contract shapes
  are publishable; page bodies, full title or path lists from a consumer's
  bundle, and Anthropic-derived prose are not. Findings about a private
  consumer's data go back to them through coord, never as a file here. This
  costs nothing — every question this library asks of a consumer is about shapes
  and key sets — and it is not reversible once pushed.
- After `git commit`: push to Forgejo (`git push origin`) immediately.
  Never GitHub.

## Communication patterns

### Linking to local files

When pointing to local files in responses, always use markdown link syntax
with a descriptive name:

- Use `[Human-friendly name](file:///absolute/path)` — never bare
  `file:///...` URLs or autolinks `<file://...>`.
- Always use absolute paths. Never `~/` or relative paths.
- For multiple files, render as a bullet list of named markdown links.

Why: bare `file://` URLs only render the first as clickable across multiple
lines. Named markdown links make each entry independently clickable and look
cleaner.