llm-ingestion-okf/CHANGELOG.md

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Fixed

- **A block `sources:` sequence no longer loses the address in the flat
  readers (K3-24).** `consume.read_sources` has always read both YAML forms;
  the three copies of this library's line-oriented frontmatter grammar read
  only the flow one and returned the key with an EMPTY value for the block
  form -- not a `KeyError` a consumer can catch, an address that disappears.
  Measured 2026-09-12 over four bundles a producer ships, denominator = files
  carrying a frontmatter block: 2 756 of 2 757, 446 of 447, 1 133 of 1 134 and
  270 of 271 concept files lost it, while PyYAML 6.0.3 and the pinned guard
  1.4.0 both read it on 100 % of the same files. After: **0 of each**, and all
  three flat readers return what BOTH reference readers return on 4 609 of
  4 609 files, plus 2 762 of 2 762 in a flow-form bundle that is unchanged.
  `materialize.parse_frontmatter` is public API, so the external consumer is
  the one this repairs.

  The value type was the choice and it was measured: widening the return type
  from `dict[str, str]` costs 15 `mypy --strict` errors across four modules
  plus a signature every outside caller follows; rendering the entries back
  into the flow form those readers already round-trip costs 0. The rendering
  is a READING projection, not a claim that the value is writable.

  Narrow on purpose: `profiles.STRUCTURED_BLOCK_KEYS` is `{"sources"}`, the
  key `read_sources` already knows how to read, and one grammar now serves all
  four call sites (`profiles.read_block_mappings`). A shipped fixture carrying
  a block `verified:` still reads as an empty value, pinned by a test so the
  next widening is a decision rather than a side effect. The K3-20 guarantee
  is unmoved and asserted per reader copy: a nested key never enters the
  document's namespace.

### Changed

- **No bundle bytes move.** A five-document folder built before and after is
  `diff -r`-identical (52 files), so no ranking measurement is owed. The
  emission rule is untouched: this library still writes flow.

- **Two docstrings and one README paragraph corrected rather than left
  standing.** `materialize._render_sources` gave three measured reasons for
  refusing to emit the block form; reason 1 (a block list round-trips to an
  empty value) and reason 3 (B6's acceptance test cannot pass) FELL with this
  fix and are struck. Reason 2 STANDS and now carries the rule alone,
  re-measured by reading `portfolio-optimiser` at `6eb58e5`: `read_provenance`
  returns `UnreadableProvenance(reason="block-sequence")`. It is not the
  guard's objection -- guard 1.4.0 reads the block form on 4 609 of 4 609
  files. The README said this library "cannot read the block form" where one
  reader could and one could not; it now separates the two by KEY.

## [0.8.4] — 2026-09-11

### Added

- **`parent` reaches the reader (K3-21 A).** `okf consume` resolves a
  concept's `parent:` pointer -- a `segment_id`, unique only inside one
  document's plan -- among the concepts sharing its `source_file`, and an
  excerpt carries `parent: { concept_id, title }`, conditional like
  `req_number`; a pointer that lands nowhere is named `parent_unresolved:
  true`. A heading-only body whose plan entry has a parent gains ONE line,
  `Enclosing section: [<title>](/<bundle-relative path>)` (SPEC § 5.1, § 6.1).
  Measured on the one standard with such sections: 675 of 710 carry exactly
  one link, 0 broken, 72 265 B = 4.49 % of body bytes; hit@1/8/50 6/6 at both
  `k` with the known-positive at rank 1.
- **`okf consume --follow-parent` (K3-21 B, off).** `parent` also carries the
  enclosing concept's `text` with that concept's own `sha256`, placed after the
  cut from the room it left, in rank order -- the delivered set is the same
  with it as without it (16 of 16 payloads); a text that does not fit is cut
  and marked `truncated`.

### Changed

- **`okf check` has seventeen rules** (`parent_unfollowable`): a `parent` that
  is not a `concept_id` and `title`, names its own excerpt, or names a concept
  in neither `excerpts` nor `withheld`. Every "16 rules" line a consumer quotes
  is now "17 rules". Contract § 8 gains point 6, and its sentence "additional
  members are permitted and are not read by the checker" now says the checker
  reads only the members § 8 names. A consumer that does not know `parent`
  has nothing to do: it is conditional and absent on every bundle without it.
- **The § 7.4 known-positive moved** (13 238 / 12 893 / 345 -> 14 721 /
  14 346 / 375), because it IS the contract document: every payload's
  `budget.known_positive` block moves with no bundle changing. Measured on 32
  regression payloads, everything outside that block is byte-identical.
- **`--shell-parent` stays off, on a measurement** rather than on "`okf
  consume` reads no `parent` key", which A made false: the link's absolute
  path repeats the document's directory in 675 bodies and moved the delivered
  set on 2 of 8 questions at the default `k` (3 of 8 at `k` 50), hit@k
  unchanged.
- **The guard pin moves from `v1.3.0` to `v1.4.0`** (`[tool.uv.sources]`, and
  the pip fallback in the README). Installing `@v0.8.4` gives a different
  guard than installing `@v0.8.3`: 1.4.0 parses a flow sequence of plain
  scalars -- `source_offset: [1, 24]`, `derived: [number]` -- where 1.3.0
  raised "a flow sequence admits flow mappings only". Measured with both
  guards' `parse_frontmatter` on one bundle built from the five-document
  project: 1.3.0 refuses 26 of its 28 frontmatter blocks, 1.4.0 refuses 0 of
  28. The dependency range `llm-ingestion-guard>=1.2,<2.0` is unchanged.

### Fixed

- **The index resolves a `parent` naming a segment of its own document
  (K3-21 C).** 675 of 675 such facets rendered `parent: pN?` while the concept
  stood in the bundle; now 0. The two segmented goldens' index files move one
  `?` each (4 lines).
- **A declared section below markdown's sixth level keeps its level in the
  plan (K3-21 D).** The NISO-STS reader clipped the outline mark to 6 along
  with the heading; the mark now carries the declared depth. On one standard
  the plan moves on exactly 2 entries (`--shell-parent` only), and 710 of 710
  shells point at the ancestor the `<sec>` nesting names (708 before).
- **Frontmatter this library writes is YAML a YAML reader reads back the same
  (K3-22).** SPEC § 11 point 1 requires "a parseable YAML frontmatter block" in
  every file. Measured with PyYAML 6.0.3 before the change, the pinned K2
  default bundle failed `safe_load` on 41 of 455 blocks (and a 42nd parsed to
  a truncated title), and each R761 build on 1 -- every one a block scalar
  written verbatim: a title with `": "` or `" #"`, a leading `- `, `*` or `**`,
  a trailing `:`.
  - **Block scalars:** a value K3-19's plain-scalar rule refuses is written
    double-quoted, `\` and `"` escaped; every other value keeps its bytes.
    Rebuilt, the five-document project moves 0 files, each R761 build 1 line
    and the K2 default bundle 42 `title` lines, after which all 454 of its
    frontmatters parse and read back the same.
  - **Flow leaves (`sources`, Door A's list, a run-stated flow value):** the
    pinned guard refuses any quote inside a flow mapping, so a leaf PyYAML
    would refuse (`?`, `,[]{}`, `": "`, `" #"`, a quote, a leading indicator)
    has no form both read and is refused with the door's existing code
    (`inbox_source_file_unaddressable`, `inbox_source_title_unaddressable`,
    `source_reference_unquotable`, `run_frontmatter_invalid`).
  - **Behaviour change:** `okf build --frontmatter 'sources=[{ resource: <URL
    with a query string>, … }]'` now exits 2 and writes nothing. K3-19's own
    flagged R761 build used such a URL and wrote 2 761 of 2 761 frontmatters
    PyYAML refuses.
  - **Readers** (`parse_frontmatter`, the index and structure readers, both
    `read_sources` branches) unquote a `"`-wrapped value; `'`-wrapped values
    are untouched. On 25 273 files of existing bundles the readers return
    exactly what they returned before.
  - The generated `SKILL.md` header goes through the same block rule.
  - PyYAML joins the `dev` dependency group only; `src/` imports no yaml.
  - Report: `docs/2026-09-11-k3-runde22-yaml-lesbar-frontmatter.md`.

## [0.8.3] — 2026-09-11

### Added

- **`okf build --shell-parent` (K3-20), off by default.** A concept whose body
  is its heading alone gets `parent:` naming the `segment_id` of the nearest
  ancestor that holds text: the nearest preceding plan entry at a smaller
  level, passing over an ancestor that is empty too. Nothing is copied and no
  boundary moves. It reads the plan's level and order, never the row. Measured
  on one process code, 710 of 2 761 concepts are heading-only; the route names
  the ancestor the document's own nesting names on 708 of them (two sit at
  depth 7, which a markdown heading clips to 6), where reading section numbers
  gets 686.
  - **Off, by measurement:** `okf consume` reads no `parent` key, so no payload
    ranks differently, while the flag moves the bytes of every bundle holding
    a heading-only section.
  - **Known cost:** the index projects `parent` as a document NUMBER, so a
    segment id always renders unresolved there (`parent: p1?`). The same key
    already carries both meanings for an adjudicator's declared parent.

### Fixed

- **A directory every concept id shares no longer ranks the concepts
  (K3-20).** `okf consume`'s first fusion signal read a concept's title
  together with every segment of its id. On a one-document bundle every id
  starts with the same directory, and since K3-19 an STS document names that
  directory after its own number, so a question naming the document matched
  every concept -- except the one whose title already named it, which gained
  nothing because the overlap counts a question token once. Measured on a
  2 761-concept bundle, the known-positive fell from rank 1 to not delivered
  at the default `k` (13 at `k` = 50). `consume.shared_id_prefix` now keeps
  the leading directories EVERY id shares out of that signal: the
  known-positive is rank 1 at both `k` and S1-S6 stay 6/6.
  - **Consumer cost: payloads move only on a bundle whose ids all share a
    leading directory**, which is what a one-document build produces. Where
    they share none, the signal reads the same string as before, and the
    measured multi-document bundles are byte-identical. The old order is
    reproducible by no flag.
  - **Measured and felled:** dropping each concept's own document directory
    instead took a hit@8 row on the pinned 43-document bundle from rank 5 to
    not delivered.

### Changed

- **A NISO-STS document's own identity names its directory and titles its
  `sources` entry (K3-19).** `okf build` put every concept of an STS delivery
  under a directory named for the delivery file -- measured, a UUID occurring
  0 times in the document -- while the document's one `<std-ident>` carried a
  `<doc-number>`. `extract.declared_identity` reads exactly one `<std-ident>`
  (`<doc-number>`, `<year>`) and exactly one `<title-wrap>`; a value stated
  more than once is not read. The directory is the `<doc-number>` through the
  id grammar, replacing only the file's stem, and `sources[0].title` is
  `<doc-number>` + `<year>`, then the `<title-wrap>` title, then the file name
  -- the first that can be written into the flow mapping verbatim (the
  measured `<full>` carries commas, so it never is). A declared name two
  documents in one run claim is used by neither, and stderr says so.
  - **Consumer cost: a re-run, and an STS document's concept ids move**
    (`<uuid>/...` -> `<doc-number>/...`). Every other file type is untouched:
    the five-document folder rebuilds identical except the `log.md` line that
    records the venv's converter path.
  - **Measured side effect on ranking, reported rather than repaired:** hit@1
    / 8 / 50 over S1-S6 stays 6/6 at both `k`, but the known-positive falls
    from rank 1 to not delivered at the default `k` (13 at `k` = 50).
    `consume`'s first signal reads a concept id's segments, and on a
    one-document bundle every id now carries the document's own name; renaming
    only the directory back restores rank 1. `--rarity-weight` delivers it at
    rank 4 with S1-S6 unmoved, and stays off.

### Added

- **`okf build --frontmatter KEY=VALUE`**, repeatable, stamps a key on every
  concept of a run -- for what an operator knows and a document does not say,
  such as an edition or a publisher's address. SPEC SS 4.1 "Extensions" lets
  a producer add any key; SS 11 forbids a consumer to reject one. Split on the
  FIRST `=`, and the value is written verbatim on ONE line, because this
  package's readers are line-oriented and blind to a block-form `sources`.
  Precedence: a stated value beats what the document declares, which beats the
  file name. A run may add any key and REPLACE only `sources` and
  `description`; every key the door writes itself -- including Door A's
  `ingest_manifest`, which would make that door claim the file -- is refused
  before a proposal is written (`run_frontmatter_invalid`), as is a value that
  would not read back as stated. Reachable as `build(frontmatter=...)` and as a
  keyword-only `concept_frontmatter_values` on `measure`, `process_inbox` and
  `render_inbox_concept`. Without the flag nothing moves. **Note:** a
  `sources` value carrying a URL or `X:Y` in the flow mapping passes `okf
  check` and is refused by PyYAML's `safe_load` -- measured on 2 761 of 2 761
  concepts with such a value -- and it is written verbatim as stated.
- **`description` for an STS section, from its own first spec point.** The
  first `<p>` of the first direct-child `<sec sec-type="spec">`, whole,
  carried by the plan entry, screened by the gate, and written only where a
  YAML reader reads it verbatim (`inbox._yaml_plain`; over 2 024 measured
  values the rule and PyYAML agree on every one). The spec sets no length, so
  the one-paragraph limit is ours. On the measured document: 2 026 of 2 761
  titled sections carry a spec point, 1 807 descriptions are written (2 points
  have no `<p>`, 217 carry `": "`), none is invented, and none is derived from
  a title.

`--ingested-at` alone was confirmed to stamp every concept, on the segmented
route too, and to date `log.md`: 2 761 of 2 761. Report:
[`docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md`](docs/2026-09-11-k3-runde19-dokumentidentitet-og-frontmatter.md).

## [0.8.2] — 2026-09-11

### Added

- **`okf check` refuses a skill and a payload that name different bundles
  (`bundle_mismatch`).** The checker had published this hole about itself since
  2026-09-08 and not closed it: reproduced on this repository's HEAD, it
  reported `conformant: 15 rules over 8 excerpts and 438 withheld entries, 0
  findings` for a skill generated from one corpus against a payload assembled
  from another; the same line for the UNFILLED template against that payload;
  and the same line again for a payload sharing the skill's `bundle_id` at a
  foreign `ref`. All three now exit **1** with one finding. The right pair is
  untouched at exit 0 with 0 findings, and a payload declaring no identity at
  all stays `ref_missing`'s defect at 9 findings -- no rule restates another.
  - **BOTH halves are compared, and the `ref` half is the load-bearing one.**
    Three distinct builds on one machine were measured carrying the same
    `bundle_id`, so an id comparison alone would pass a stale skill -- the case
    the generated skill warns about in its own words ("if the bundle moves, the
    ref moves with it and this file is stale"). SS 3.3: "a version is the
    producer's assertion; a ref is a fact about bytes".
  - **An identity the rule cannot read is a finding, never a silent pass.**
    That is what refuses the unfilled template, whose `<CORPUS>` and `<REF>`
    are not an identity. It also refuses the repository's own hand-made
    `skills/okf-consume/SKILL.md`, which predates `okf skill` and declares no
    bundle identity a reader can act on -- **1 of 1** shipped hand-made skill,
    a real find and not a fixture.
  - **The rule compares a DECLARED identity against a DECLARED identity and
    never opens the bundle**, so a payload misreporting its own `ref` still
    passes. Proving a ref against bytes is `okf consume --ref`'s job and needs
    a bundle path this command deliberately does not take.

### Changed

- **The checker's rule count is 16, not 15**, and `Report.rules_evaluated` is
  the denominator every report line quotes -- so `15 rules` becomes `16 rules`
  in every published line. A consumer citing the old number is citing a number
  that has changed. No payload bytes move: this is the checker, not the
  pre-pass.
- `skill.identity_line` is now the single authored copy of the sentence a
  generated skill declares its bundle in, read back by
  `contract_check.skill_identity` and held to it by a test. Generated skill
  bytes are unchanged -- measured, both tracked bundles byte-identical before
  and after on the same interpreter.

### Fixed

- **The shipped `skills/okf-consume/` is generated, and passes the check it
  tells its reader to run.** The hand-filled copy predated `okf skill`,
  declared no bundle identity, and was refused against the payload shipped
  beside it: `NOT conformant: 16 rules over 3 excerpts and 0 withheld entries,
  1 findings` (`bundle_mismatch`, exit 1). It is now `okf skill`'s output for
  `examples/ingest-golden-segmented-okf-v0-2/expected-bundle` -- the bundle its
  payload always came from -- and the pair is `conformant: 16 rules over 3
  excerpts and 0 withheld entries, 0 findings`, exit 0. The payload's bytes do
  not move. Regenerate with the command in
  `skills/okf-consume/references/README.md`; two tests hold the pair and the
  generator's bytes.
  - **Its frontmatter `name` changed** from `okf-consume` to
    `b-golden-segmented-okf-v0-2-consume`. Claude Code takes a project or
    personal skill's command from its DIRECTORY, which stays `okf-consume`, so
    a copy at `.claude/skills/okf-consume/` is still `/okf-consume`; only the
    display label moves. Nothing in this repository named the skill
    `okf-consume`.
  - **Its prose no longer states K2 numbers.** The 629-concept figures belonged
    to a corpus that cannot ship; the generated numbers describe the
    three-concept golden bundle and nothing larger.
- **`uv.lock` records this package at 0.8.1.** The 0.8.1 version bump never
  reached the lockfile, which still said 0.7.0, so `uv lock --check` exited 1
  on a clean checkout and any non-frozen `uv` command rewrote the file. One
  line; nothing else in the lock moved.
- **`--title-covered` no longer lifts a short title over a title that answers
  more of the question.** 0.8.1's partition read every concept whose WHOLE
  title the question accounts for before everything the fusion ranked above
  it. That is a claim about the covered title's PRECISION, and it overrode the
  fusion even against a title answering MORE of the question: measured on a
  26-concept bundle of five tender documents, a question naming a section by
  three of its title tokens also held a neighbour's whole one-token title, and
  the neighbour took rank 1 from the section the question names. A covered
  concept now RISES through the fusion's order and stops beneath the first
  concept whose title answers more question tokens, by equality, than it
  holds. Same flag, no new parameter, no new constant.
  - **What the rule was built for does not move.** On the 2 761-concept bundle
    of one standard no covered concept had such a title above it, so all 8
    payloads are byte-identical to 0.8.1's at default `k` AND at `--k 50`;
    hit@1/8/50 stays 6/6 · 6/6 · 6/6 with the known-positive at rank 1.
  - **Where the rule never fires nothing moves either**, measured on the bytes:
    the pinned K2 bundle 6 of 6 payloads identical, Arm B 6 of 6, the three N
    bundles 15 of 15.
  - **Four other repairs were measured and not taken**: a minimum title length
    sold hit@1 back to 3 of 6; a share of the question holds only in a band
    set by the question's word count (1/9 < s <= 1/6); an order inside the
    covered group cannot act on a group of one; a closed stop list touches no
    title involved and would be a new vocabulary to maintain.
  - **0.8.1's unbounded order is reproducible by no flag.** It differs from
    this one only where a covered concept has such a title above it -- 1 of
    the 4 questions measured on that bundle, 0 of 8 on the standard, 0 of 27
    elsewhere. `--no-title-covered` still reproduces the pre-0.8.1 order.
  - **One constructed variant still reads the short title first, and no rule
    reading titles alone separates it.** A shortened form of the same question
    shares ONE token with each of the two titles; a form that blocks on any
    token the covered title lacks fixes it and takes one of the standard's
    scored questions, the same shape with the opposite answer, from rank 1 to
    3. Report: `docs/2026-09-11-k3-runde17-dekningen-stopper-ved-en-bredere-tittel.md`.

## [0.8.1] — 2026-09-10

### Added

- **`--title-covered` (ON by default, opt out with `--no-title-covered`): a
  question that accounts for a concept's WHOLE title reads that concept first.**
  On the 2 761-concept bundle of one standard, the answering section was
  delivered at rank 1 on **3 of 6** scored questions and **none of the reading
  side's six flags moved that number** -- the whole sweep sits at 3/6 or worse.
  Measured on that bundle, before and after: hit@1/8/50 **3/6 - 5/6 - 5/6 ->
  6/6 - 6/6 - 6/6** at default `k` and **3/6 - 5/6 - 6/6 -> 6/6 - 6/6 - 6/6** at
  `--k 50`, with the known-positive holding rank 1 at both and the
  known-negative still not a hit. S1 4 -> 1, S5 not delivered -> 1, S6 3 -> 1.
  - **THE DEFECT IS THAT BOTH LEXICAL SIGNALS ARE UNNORMALISED COVERAGE
    COUNTS.** They measure how much of the QUESTION a candidate answers and
    nothing measures how much of the CANDIDATE the question accounts for, so a
    section titled with the question's subject alone scores what a narrower
    section titled with that subject PLUS a qualifier scores, and then loses on
    the body count. Decomposed per miss: S1 turns on `hvordan`, an interrogative
    pronoun; S5 on `hvilke` and `stilles` in a body 7x the gold's, and on
    `betonghvelv ~ betongkonstruksjoner` through the four-character stem
    `betong`; S6 on an exact tie broken by `concept_id`.
  - **A PARTITION, never a fourth RRF signal, and the arithmetic is why.** RRF
    consumes ranks alone, so with shared ranks a rule whose positive group has
    `m` members is worth `1/61 - 1/(61 + m)` -- a rule firing on ONE concept of
    2 761 is worth 0.00026 against a body gap of 0.0029. **A precise rule is
    worth LEAST under this fusion.** Measured as a signal it moves hit@1 not at
    all (3/6, both as a third and as a fourth signal); as a partition it reaches
    6/6. `lookup_hits` is the same shape for the same measured reason, and it
    still wins: the new partition lands below it, with a test and its control.
  - **By EQUALITY, never by shared prefix.** Four shared leading characters take
    the group from 1 to 6 on one question and 9 to 31 on another, with the
    answering section falling to candidate rank 6 and the known-positive to 2.
  - **TWO CANDIDATE REPAIRS WERE MEASURED AND FELLED FIRST.** Pivoted length
    normalisation of the body signal collapses at every value swept
    (b = 0.25/0.5/0.75/1.0 -> hit@8 3/6, 1/6, 1/6, 0/6, and at b = 1.0 the
    known-positive falls to rank 49): the median concept holds 22 tokens against
    a mean of 60, so length normalisation promotes thousands of tiny concepts.
    Title PRECISION as a signal reaches candidate hit@1 5/6 and takes the
    known-positive from 1 to 4 every time it does.
  - **NOTHING ELSE MOVES AND IT IS MEASURED ON THE BYTES.** The pinned K2 bundle
    keeps `(1,1,1,1,1,5)` and its 7 pin tests, Arm B keeps `(1,1,1,1,1,5)`, and
    the payloads on both are **byte-identical on 6 of 6 questions**; n100/n200/
    n500 payloads are byte-identical on 5 questions each; the 828-file HTML
    corpus still gives 828 plans, 0 unreadable and 6 015 md with `diff -rq`
    empty; the five-document folder is `diff -r`-identical at 26 concepts / 52
    md; `okf project` stays byte-equal to `okf build`. hit@k on n100/n200/n500
    is **NOT MEASURED** -- this repository holds no gold set for them, which is
    0 gold sets and not 0 hits.
  - **THIS IS THE FOURTH READING-SIDE CHANGE THAT MOVES A PAYLOAD WITH NO BUNDLE
    CHANGING.** A consumer pinned to the previous excerpt order needs
    `--no-title-covered`. The rule fires on **0 of 21** measured cells outside
    that one bundle, so "no regression" there means it never fires -- not that
    it fires harmlessly. Report:
    `docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md`.
  - **A MEASURED DOWNSIDE, written as a known limitation and not as a fixed
    defect: a SHORT, GENERIC title is covered in full by more questions than a
    long one is.** On a five-document folder (26 concepts) a constructed
    known-negative question demoted the answering section from delivered rank 1
    to rank 2: a neighbouring concept titled with a single common process word
    has its WHOLE title accounted for by that question, while the answering
    section's longer title does not. The other seven delivered places did not
    move. The repair is a later round's; nothing here narrows the rule, and the
    opt-out is `--no-title-covered`.
- `build_payload`'s signature defaults are now held equal to the consume CLI's
  argparse defaults by a test, for every same-named parameter. This is O6's
  defect in the other command: `cli.build` defaulted two flags `False` in the
  signature and `True` in argparse, and a caller reaching it as a function read
  the signature.

## [0.8.0] — 2026-09-10

### Added

- **A section the SOURCE DECLARES now takes the route declared structure takes,
  at the shipped defaults.** `.xml` gained a reader in the entry above and the
  reader reached its ceiling -- **2 761 of 2 761** heading lines -- while the
  build delivered **23 concepts and 15 of 2 761 boundaries**. Everything after
  the reader ate it, and both steps are measured: the **orphan check** removed
  **710 of 2 761** (710 of 710 removed headings are followed immediately by
  another heading and **0 of 2 051** delivered ones are -- they are container
  sections), and **Arm F** folded **2 066** more, 2 089 -> 23.
  - `extract.xml_outline` reports the marks the reader wrote itself. There is
    **no bridge** and therefore no tolerance constant and no `unresolved`
    bucket: the reader appended the line it names. That is the difference from
    `pdf_outline`, whose naive nearest-line rule was wrong on 1 840 of 2 762.
  - `propose.RULE_XML_SECTION` (`rule:xml-section`) is its own name in
    `RULE_NAMES` and in `_ORPHAN_EXEMPT`, so an artifact still distinguishes an
    element the reader transcribed from a bridged bookmark
    (`rule:pdf-outline`) and from a heading somebody guessed (`rule:heading`).
  - The route is chosen by the ROW (`DECLARED_STRUCTURE_IDS = {"xml"}`), never
    by the text: the same markdown arriving from a `.md` file is still a guess.
    **No other file type changes one byte** -- `diff -r` on the five-document
    reference folder is empty (52 md, 26 concepts, 0 of 5 rejected, 0 `.xml`
    files in it), `okf project` is still byte-equal to `okf build`, the pinned
    K2 bundle is unchanged, and the PDF arm still proposes 2 762 segments.
  - Measured at SHIPPED DEFAULTS, not behind a flag: **2 761 concepts**;
    **2 761 of 2 761** declared sections became a concept with the source's own
    directory and title; **0** concepts match no declaration; `a)`-points
    **0 of 4 954**; table blocks **10 of 10**; hit@1/8/50 **3/6 · 5/6 · 6/6**
    (from 0/6 · 0/6 · 0/6) with the known-positive at rank 1. Cross-arm,
    **2 761 shared concept ids** -- 100 % of the XML bundle and 2 761 of 2 762
    of the PDF arm's, up from round 13's 2 022.
  - Two directories of 2 738 still hold two concepts (`11`, `12`): the
    publisher reuses a section number for two distinct sections, and it is the
    same 2 the PDF arm has. Round 13's 14 such directories were false positives
    of the text route reading the document's own contents listing, and they are
    gone.

- **`.xml` is a core file type, NISO-STS aware, with a generic fallback.** A
  publisher's own viewer delivers a zip that holds 0 html, 1 xml and 109
  images; `okf build` on it was **110 of 110 unreadable, 0 plans, exit 2**, and
  the conservation identity `merged + coded rejections == N` was never written
  because the run aborted earlier. The one xml file is the whole product: 7 715
  `<sec>`, **2 761 with a `<title>`**, 4 954 lettered points, 10
  `<table-wrap>`, and a `<sec>`-nesting depth distribution row-for-row
  identical to the publisher's own structure fasit.
  - The output grammar is MARKDOWN, the same the office and HTML rows reach the
    proposer through: `propose.py` is untouched. `<label>` + `<title>` become
    one ATX line at the section's own depth; a `<sec>` with only a `<label>` is
    a body line and never a heading (**0 of 4 954** became concepts);
    `<table-wrap>` becomes one markdown table (**10 of 10**, against 0 of 10 on
    the PDF path).
  - The reader emits **2 761 of 2 761** heading lines and preserves text
    exactly -- 1 283 395 of 1 283 395 non-whitespace characters, ratio
    **1.000000**. The BUILD reaches 2 065 of 2 761 with `--no-unit-fold` and 15
    of 2 761 on the shipped defaults; the whole distance is two proposer rules,
    decomposed with denominators in the report.
  - hit@k over six questions, k=50: **3/6 · 5/6 · 6/6**, matching the PDF arm
    row for row, with the known-positive moving from **rank 13 to rank 1**.
    2 022 concept ids are shared between the two channels -- 96.8 % of the XML
    bundle.
  - **No new dependency:** `xml.etree.ElementTree` is stdlib and `uv.lock` is
    untouched. A `<!DOCTYPE` is REFUSED unparsed with its own code, which is a
    guarantee about this package rather than about the installed libexpat.
  - `.xml` never routes through the converter, and it is measured about 12x
    faster and about 30x smaller in peak memory than the PDF arm on the same
    document and the same machine.
  - **XML that is not STS gives 0 plans and a FAILED build, and that is not an
    `.xml` defect.** The known-positive that decides it: a folder holding one
    `.txt` of prose with no headings gives exactly the same three lines and the
    same exit 2. This is general `okf build` behaviour for any structureless
    document -- extraction works, 0 unreadable, the text is there, and the
    proposer has nothing to propose. The gate that refuses a run with no plans
    stays: a run replaying zero plans would emit a flat bundle and report it as
    a success. Separating "0 plans, 0 unreadable" from "0 plans because nothing
    could be read" would change the outcome on **0 of the 4** reference
    corpora, so it is not separated.
  - Report: `docs/2026-09-11-k3-runde13-xml-sts.md`.

### Fixed

- **A PDF bookmark sharing a line with another left no trace.** `pdf_outline`
  collected marks in a dict keyed on the destination line index, so a second
  bookmark on a line was discarded by `setdefault` in silence: measured on a
  701-page document, **2 763 nodes in, 2 762 marks out, `unresolved` = 0**.
  `PdfOutline` now carries `collided`, and the identity `nodes in == marks +
  unresolved + collided` holds. Keeping both nodes was measured and felled --
  the two candidates then open at one offset and the first closes with an empty
  span the orphan check deletes.

- **`--pdf-outline` (OFF): cut a PDF at the boundaries its own `/Outlines`
  bookmark tree declares.** Measured outside this repository on one 701-page
  process code whose publisher also ships a NISO-STS structure for it: the
  shipped default recovers **1 967 of 2 761** titled sections, **0 of its 28**
  chapters, and **794 of 794** misses have their heading text present in the
  extracted text -- the line was read, the boundary was never opened. The same
  file carries a 2 763-node bookmark tree that matches **2 761 of 2 761** STS
  titles exactly after `re.sub(r"\s+","",s).lower()`. With the arm on:
  **2 759 of 2 761 boundaries (99.9 %)**, depth 1 **28 of 28**, concept titles
  identical to the publisher's own after that normalisation **2 761 of 2 761**,
  false positives **3 of 2 762** (was 163 of 2 182), directories carrying two
  concept files **2** (was 132, of which 65 were a contents copy and a body
  section under one id), front-matter concepts **2 of 2 762** (was 72). Seven
  of seven consumption fasit now exist in the bundle (was four); hit@1/8/50 is
  **3/6 · 5/6 · 6/6** against **1/6 · 2/6 · 4/6**.
  - It is a SEGMENTATION arm, not a reader option: the extracted text is byte
    for byte the same either way, and `--pdf-headings`/`--ocr` stay the only
    two things that change what a PDF says.
  - A PDF with no bookmark tree is **byte-identical with the flag on**;
    `pdfminer`'s `PDFNoOutlines` is "this file has no index", never an error.
    Measured on the five-document smoke folder: `diff -r` empty against both
    the arm off and the pre-change tree.
  - No new dependency and no second parse of the file's pages: the tree is read
    through `pdfminer.six`'s `PDFDocument.get_outlines()`, which
    `pdfplumber` already ships under the existing `[extract]` extra. Cost on
    the 701-page document: 119.22 s -> 183.31 s wall, peak RSS 3 252 -> 3 251
    MiB.
  - An unresolvable `/Dest` is dropped and COUNTED, never fabricated into a
    boundary and never a refusal of the file. That document has 0 of 2 763;
    one of the eight reference PDFs in this repository's own sample has 2 of 2.
  - **The default does not move in this release.** Reach measured: 1 of the 8
    reference PDFs carries a usable tree at all.

- **`.pdf` has a row in `extract._EVIDENCE`, as `measured`.** It was the row
  with the most measurement behind it and no entry in the table, which is the
  one way a table like that misleads while every entry in it is true.


## [0.7.0] — 2026-09-09

The first screen an agent reads, the three shapes of request the skill answers,
and one defect that made the command the first screen recommends build a worse
bundle than the command it claims to be.

> **Note for path importers, added 2026-09-09 after a report from a consumer.**
> `tools/okf_consume.py` ends by replacing its own `sys.modules` entry with the
> packaged `llm_ingestion_okf.consume`. A caller importing it with
> `importlib.util.spec_from_file_location` holds the object `module_from_spec`
> returned, which that line does not reach: read the module back out of
> `sys.modules[<name>]` after `exec_module`, or import `llm_ingestion_okf.consume`
> directly. From the commit that adds this note the file also copies the
> implementation's public names into its own globals, so a path-imported object
> carries them -- but that restores attribute ACCESS only, never patch-through.

### Fixed

- **`okf project` built a bundle two rules behind `okf build`.** `cli.build`'s
  Python signature defaulted `keep_table_heading` and `sheet_section_rows` to
  `False` while argparse defaulted both to `True`; `project.create` calls
  `build()` as a function and passes no flag list, so it read the signature.
  Measured on a five-document folder: `okf project` wrote **15 concepts / 30
  files** where `okf build` on the same folder wrote **26 / 52**, and the whole
  difference was in the priced spreadsheet — the document a question about price
  has to reach. The invariant test that was supposed to catch this could not:
  it compared `project.create` against the same `build()` function, so both
  sides carried the same wrong value, and its two fixture documents had neither
  a table nor a sheet. Both gaps are now tests: one compares the signature's
  defaults against argparse's, the other builds a document whose concept count
  actually moves with the two flags. After the fix the two paths are byte-equal
  on that folder (`diff -rq`, 0 differences).

### Added

- **A first screen for a reader who has not used this before**, agent or human:
  what it is in one sentence, one install line, two commands, and the three
  shapes of request. The phase-status paragraph that used to open the README
  moved down to `## What this library is`; nothing was deleted.
- **Three modes in the consumption skill**, stated in the template, the
  instantiated skill and the generator:
  - **Question** — as before, the default.
  - **Hypothesis** — decomposed into premises and answered **per premise** as
    `confirmed` / `refuted` / `undecidable-from-bundle`, three literals with no
    fourth value. A premise whose excerpt is real but does not carry the
    conclusion is `[sourced-not-sufficient]` on **that premise**, not on the
    whole answer: four premises and one weak source is three answers and one
    gap, and reporting it as one refusal throws the three away.
  - **Task that produces a document or a paragraph** — every claim in the
    written artefact carries `(bundle_id, concept_id)`, the excerpt's `sha256`,
    its `title` and whichever `source_*` keys it has; an ungrounded paragraph is
    **written and marked**, never dropped; and the cut (`considered`,
    `withheld`, `delivered`) is declared inside the document, because the
    document travels without the chat.
  The five markings are untouched — the modes add no sixth.

### Changed

- **The generated skill states relative paths where it can.** In the layout
  `okf project` writes, the commands are now `okf consume .okf/<id>` and
  `okf check --skill .claude/skills/<id>-consume/SKILL.md`, runnable from the
  project root — which is where `okf project`'s own closing line tells the
  reader to start `claude`. A path outside the project root stays absolute on
  purpose: `../../..` is not more portable, only harder to read. The two
  absolute paths a generated skill carried are now zero, measured with a query
  shown capable of finding first — O5's published "4 → 0" used `grep -c "^/"`
  against paths indented by two spaces, which could not have matched either way.
- **One tag is pinned everywhere.** `README.md` pinned `v0.4.0` on its install
  lines and `v0.6.0` further down, and `llms.txt` pinned `v0.4.0`; an agent
  reading from the top installed a tag without `okf project`. All install lines
  now name `v0.7.0`, and the earlier tags are kept as a labelled history
  section rather than as commands. `llms.txt` gained the `okf project` form and
  a pointer to the Claude Code section.

## [0.6.0] — 2026-09-08

The first tag since `v0.5.0a2`, so everything that had accumulated as
Unreleased is in it — those sections are kept below, under their own heading,
rather than folded together. What follows first is what O5 added, and why the
release needed a minor of its own: the installed command grew from one
subcommand to five, and one build default moved.

### Added

- **`okf consume`, `okf check`, `okf skill` and `okf project` are subcommands
  of the installed `okf` command.** Until now the pre-pass, the contract
  checker and the skill generator lived in `tools/` and were reachable only
  from a clone; a consumer who installed this library could build a bundle and
  had no way to read one back. Measured before the move: a consumption skill
  generated from a checkout carried **four** lines naming that checkout by
  absolute path, two of them the commands the skill tells a reader to run, so
  the skill could not be moved, shared, or run by anyone else. The generated
  skill now names `okf consume` and `okf check` — names on PATH — and a test
  asserts the repository appears in it nowhere, with a known-positive so the
  zero is a measurement rather than a search that could not find.
  `tools/okf_consume.py`, `tools/okf_contract_check.py` and `tools/okf_skill.py`
  remain as thin aliases, so every published reproduction block still runs.
- **`okf project <folder>`: a folder of documents to a bundle you can ask a
  question of, in one command.** It runs `okf build` with this package's
  default into `<out>/.okf/<id>/`, generates the skill into
  `<out>/.claude/skills/<id>-consume/`, and prints what it read, what it wrote,
  which documents are in the folder but not in the bundle, and which landed
  whole as a single concept — the two cases a question can only be answered
  `[sourced-not-sufficient]` in. `<out>` defaults to the current directory and
  `<id>` to the folder's name reduced to `[a-z0-9-]`. It owns no flag that
  changes a bundle's bytes, and a test holds the project bundle byte-equal to
  the `okf build` bundle of the same folder.
- **`skills/okf-prosjekt/`**, a Claude Code skill (Norwegian) that wraps
  `okf project` and reads its summary back.
- **The template and the contract document travel in the wheel.** `okf skill`
  instantiates `skills/okf-consume-template/SKILL.md` and `okf consume`
  measures `docs/consumption-contract.md` as its section 7.4 known-positive;
  neither was installable before. Both are force-included from the file they
  are authored in, so there is still exactly one copy of each.

### Changed

- **`okf build`'s default now includes Arm E (`--table-grid`), with
  `--no-table-grid` as its opt-out.** The default moved to Arm D plus Arm F
  earlier in the same day; measured afterwards, that combination is Arm F with
  nothing to fold. The fold's table clause folds a table back into the heading
  that introduces it, and with Arm E off a grid table is not one block but one
  block per rule line. On the operator's twelve-document reference the shipped
  default scored **2 of 12** and `docx` **0 of 3**, against the **5 of 12** the
  fold was published with — which had been measured with Arm E on. Three arms
  are now on by default, each with an explicit opt-out; `--outline-run 0
  --no-table-grid --no-unit-fold` reproduces the pre-2026-09-08 bytes.
- **This tag does not make OKF v0.2 generally available.** `OKF_LATEST` is
  unchanged.

### Also in this release: everything that had accumulated since v0.5.0a2


- **The corpus harness (`tools/okf_corpus_run.py`) can replay segmentation
  plans, and it writes the bundle's `log.md`.** `--plans-dir` names the
  proposals to replay and the profile follows from it; `--bundle-id` and
  `--okf-version` are arguments, never constants, because a profile names a key
  and the caller owns its value. `log.md` is written in SPEC section 9 form and
  dated from `ingested_at`, so `merged + sum(coded rejections) == N` is
  checkable from the bundle alone rather than only from a report that does not
  travel with it. Without `--plans-dir` a run is unchanged.
- **The K2 rebuild that measured all of this.** The harness had passed
  `STRUCTURED_V1` and no plans, so a 43-document corpus arrived as 39 flat
  concepts with no `adjudication` key anywhere. Rebuilt, the same corpus yields
  629 concepts, 618 of them `adjudication: proposed`. Record:
  `docs/2026-09-03-k2-bundle-rebuild.md`.

- **`tools/okf_propose_segments.py` takes `--path-prefix`.** Section numbering
  is document-local, so across 39 documents 618 proposed entries claimed only
  601 distinct paths -- 17 collisions that Door B's gate refuses per document.
  Scoping each document's entries under a caller-supplied prefix removes all
  17. Without the flag every artifact already produced is byte-identical.

### Changed

- **A spreadsheet's tables are written as PIPE tables, so a row survives as a
  row.** The converter's default markdown writer emits SIMPLE tables, which pad
  every cell out to the width of the widest cell in its column. Measured on a
  real 43-document corpus: one 594-character prose cell turned the sheet holding
  the tender's prices into a 67 244-character whitespace carpet with runs of up
  to **887 characters between a label and its amount**, and the header row named
  a single column because only the first cell of the source's row 1 is filled.
  The bytes reached a live consumer's model in two of eleven prompts and
  appeared in none of its eleven answers. The same sheet through the pipe writer
  is **11 048 characters with no whitespace run longer than two**, one row per
  line, each source column its own cell. `--columns=1` is part of the fix and
  not cosmetic: the pipe writer pads to a width computed from that setting, so
  at the default a NARROW table gains runs of up to 45. Spreadsheet-only: the
  other four office rows have the same defect available to the same one-line
  change, but a spreadsheet is a grid with no prose fallback, while moving the
  prose rows would move a corpus denominator nothing has measured.
  `tests/test_extract.py` pins that scoping with three digests. Record:
  `docs/2026-09-08-prisform-og-loggen-k2.md`.

- **An integral spreadsheet number loses the converter's trailing `.0`, and the
  shared string table is what makes that safe.** The converter renders a numeric
  cell as a double, so `5647500` arrives as `5647500.0` -- and a TEXT cell
  reading `92.0` arrives as `92.0` too, which the output alone cannot tell
  apart. The rewrite is bounded to a table cell whose entire content is such a
  number, by unescaped pipes on both sides, and it is skipped whenever the same
  literal is in the workbook's shared string table. Any workbook this cannot
  read keeps its converter decimals rather than being guessed at.

- **The bundle's root `index.md` no longer links `log.md`.** The harness added
  that link (`95eb271`) so a reader entering at `index.md` could reach the one
  file carrying `N`; it was a LOCAL choice and said so. Consumption contract
  SS 9.2 forbids a consumer from enumerating the bundle directory unless the
  named profile says the index is derived, which makes the index tree the entire
  map a consumer may use -- so everything it links is a document. Measured: a
  consumer walking a 629-concept bundle that way returned **630**, and a corpus
  run's own log became readable and citable as content. `5a0c879` excluded
  `log.md` from OUR walk, which fixed the count on one side of a disagreement
  produced on the other. The log is still written to the bundle root, which is
  where SPEC section 9 puts it, and `tools/okf_consume.py` still excludes a
  linked `log.md` -- every bundle built between `95eb271` and this change
  carries the link.

- **The pinned `llm-ingestion-guard` moves to `v1.3.0`** (`[tool.uv.sources]`
  and `uv.lock`; the `>=1.2,<2.0` range in `[project.dependencies]` already
  covered it and is unchanged). What this fixes is that the guard could not
  read back what this library WRITES: at `1.2.0` the flow-form `sources` in the
  OKF v0.2 golden was refused outright, and flow is the only form this library
  is able to emit, because its own line-oriented parser cannot round-trip the
  block form. Measured before the bump so the test discriminates rather than
  merely passes, and pinned by
  `tests/test_guard_adapter.py::test_the_guard_parses_the_flow_form_sources_our_goldens_emit`.
  Two rows of the gate table in `docs/okf-nokkelinventar.md` moved, not one:
  the BLOCK form of `sources` now passes too, which retires G30 -- though it
  changes nothing about what we emit, since our own parser is still the binding
  constraint. `resource` is allowlisted only inside a `sources` entry, so
  section 10.2's `executor`/`attester` resource stays rejected through every
  carrier and the Door C boundary is unmoved. No public API of this library
  changes; a consumer's cost is a re-run.
- **`uv.lock` also picks up `pypandoc-binary==1.17`**, which is a stale lockfile
  being corrected rather than a new dependency: the package was already declared
  in the `[extract]` extra, and `uv lock --check` reports the lockfile as out of
  date on the untouched tree. Core still has exactly one runtime dependency.
- **`tools/okf_propose_segments.py` writes no artifact when it has nothing to
  propose**, exiting `1` (distinct from `2`, "could not do the job") instead of
  `0` with an empty plan. An empty plan cannot be replayed -- `process_inbox`
  refuses one, because a plan naming no entry would persist nothing for a
  document that was dropped -- so the file's only possible use was to fail a
  run later, and it did. Measured: 11 of 39 documents in the K2 corpus propose
  zero segments.

- **A concept can now record more than one source.** `sources` renders a flow
  sequence of N flow mappings on one line, so a v0.2 profile can express
  multi-source provenance instead of the single entry that was the measured
  ceiling on SPEC 5.1 coverage. **A single source is byte-identical to before**,
  so every golden is unmoved and no existing bundle changes.

  The form is flow, not the block list PM decision B6 prescribed, and the
  reason is measured: this library's frontmatter parser is line-oriented and
  skips indented lines, so a block list round-trips to an EMPTY value with every
  entry silently gone -- and the consumer the decision was written for accepts
  the multi-entry flow sequence while classifying a block sequence as unreadable
  provenance. Emitting block would have produced records neither side can read.
  A negative-control test pins the block form's data loss so the reason stays
  measurable rather than remembered.

  New error code `sources_empty`: an empty list is refused, because
  `sources: []` reads as a measured absence when it is the absence of a
  measurement.

- **The consumption contract, stated normatively** in
  `docs/consumption-contract.md`, with a copyable skill template
  (`skills/okf-consume-template/`) and a checker (`tools/okf_contract_check.py`)
  that reads its mechanically checkable half: payload shape, source marking per
  excerpt, the closed `adjudication` and `trust_tier` state sets, denominator
  identity, and the budget gate with its validated known-positive. The checker
  ships outside `src/`, so no consumer's install surface changes.

- **Five office formats through a vendored converter**, behind the same
  optional `[extract]` extra: `docx`, `xlsx`, `pptx`, `odt`, `rtf`. The
  converter binary travels inside the wheel and is resolved by path rather than
  found on `PATH`, with its version asserted against a pin -- `pypandoc`
  searches `PATH` first and takes the highest version it finds, so a vendored
  binary buys nothing until something resolves it explicitly. A host carrying a
  different converter is refused rather than silently used.

  **Two of the five rows are measured; three are not.** The corpus this work
  was measured on contains zero `pptx`, `odt` and `rtf` files, so those rows
  work by construction and have never met a document anyone wrote. The
  distinction is asserted in the suite, not left in a comment.

  `.doc` (Word 97) stays out -- the converter does not read it. Drawn content
  does not survive extraction in any format, and every conversion warns about
  it.

- **Four error codes** for the converter path: `extractor_binary_missing`,
  `extractor_binary_version`, `extractor_convert_error` and
  `extractor_empty_conversion`.

- **`tests/test_docs_promises.py`**, which asserts the README's published
  format list against the registries it describes. This exists because the
  README's previous promise -- that `docx` and `xlsx` always fail fast -- went
  false silently when the converter landed. A published guarantee is a test
  obligation.


- **Door B extracts `pdf` behind the optional `[extract]` extra.** The extra is
  populated for the first time, with one parser: `pdfplumber>=0.11.10,<0.12`
  (MIT). The default install is unchanged — still exactly one runtime
  dependency, still stdlib otherwise — and a packaging test enforces that.

  **The parser choice was forced by a measurement, not by preference**
  (`docs/2026-08-21-g2-pdf-extraction-measurement.md`). On a real Vegnormalene
  requirement table, `pdfplumber` keeps 4 of 4 rows with label and value on the
  same line; `pypdf`, `pdfminer.six` and `pymupdf` each keep 0 of 4, emitting
  all labels and then all values. A downstream reader can only re-pair those by
  guessing, and in a requirements document a wrong pairing looks right.
  `pymupdf` was additionally excluded on licence (AGPL-3.0 or commercial):
  this package is MIT and an optional extra must not hand a consumer copyleft
  they did not choose.

  **`docx` and `xlsx` were unchanged at this release**, shipping no parser and
  failing fast with `extractor_extra_missing`. That is no longer true: a later
  release added a vendored converter reaching five office formats. This entry
  is left as written — a changelog records what a release did — and the
  correction is stated here so a reader arriving at this line is not misled.

- **`ExtractionWarning`**, exported from the package. Every `pdf` extraction
  emits one. Text extraction recovers text; anything a PDF *draws* — figures,
  diagrams, images — has no text to recover, so only captions survive and a
  bundle built from drawn documents is **incomplete by construction**. That is
  categorically true rather than document-specific, so it is stated rather than
  detected: deciding "is there a figure on this page" is a layout heuristic this
  library does not own. A named class so it can be filtered deliberately.

- **Two error codes**, both mirroring existing patterns rather than inventing
  behaviour: `extractor_empty_pdf` (a PDF yielded no text on any page — a
  scanned or image-only document; refused rather than persisted as an empty
  concept, which would be the silent skip this registry exists to prevent) and
  `extractor_pdf_error` (the parser failed on the bytes; the third-party
  exception is wrapped, never leaked).

### Changed

- **The `[extract]` gate for `pdf` is now an import probe rather than a
  membership test.** The rejection did not change: without the extra installed,
  `pdf` still raises `ExtractionError` with code `extractor_extra_missing` and
  the same message naming the remedy. A consumer that has not installed the
  extra sees no difference at all. That behaviour is asserted unconditionally,
  including on machines where the parser *is* installed, so it cannot rot into
  a skipped test.

  **Known limitation, stated rather than worked around:** structured table
  recovery is out of scope. `pdfplumber.extract_tables()` and
  `PyMuPDF.find_tables()` — two independent implementations — return the *same*
  wrong shape for the measured requirement table, and across the whole handbook
  only 45 of 196 detected table objects are clean enough to hand to
  `render_table` unchanged. The breakage is in the documents' ruling geometry,
  not in either library. PDFs therefore enter this library as **prose**, with
  table lines correctly paired.

  **Extracted PDF text is pinned to an exact parser version.** `pdfplumber`
  pins `pdfminer.six==20260107` exactly, and `pdfminer.six` ships date-stamped
  releases with no stability contract. Extraction is deterministic within a
  parser version (measured across five configurations) and not guaranteed
  across one, so any golden fixture built on extracted PDF text is a fixture
  migration away from a parser upgrade. `tests/test_extract.py` freezes the
  expected text of a committed fixture so that upgrade breaks something visible
  instead of drifting silently; see `tests/fixtures/README.md`.

- **`DEFAULT` now stamps `generated: { by: process:okf-ingest, at: <ingested_at> }`
  instead of `generated: true`.** This is a byte change in every concept file Door A
  writes under `DEFAULT`, so a consumer's own golden fixtures will show one changed
  line per generated file.

  The trigger is not an upstream Google release. `DEFAULT` states the ingest-spec
  layer owned by `portfolio-optimiser-commons`, and they ratified this shape
  (2026-08-02) and executed it in their spec on 2026-08-09. §7 defines the value:
  the actor is the fixed `process:okf-ingest`, `at` repeats `ingested_at` verbatim,
  and it is unquoted because frontmatter is parsed line-oriented — a quote would be
  a character in the value rather than syntax a reader strips.

  **What it costs a consumer is a re-run, and nothing else.** Ownership recognition
  is one-way: a profile carrying an actor still owns the older literal stamp, so a
  bundle written by an earlier version re-runs in place rather than tripping the §3
  collision gate. No call signature changed and no key was added or removed.

  Two things it is NOT. It is not a migration of `DEFAULT` onto OKF v0.2 — the
  profile remains v0.1 on every axis upstream owns, and still emits no `sources`.
  And it does not make `DEFAULT` and `OKF_V0_2` the same profile; they now agree on
  the stamp and continue to differ in index root frontmatter, type-conditional
  requirements, and `sources` derivation.

  Door B (`process_inbox`) is deliberately unchanged: its marker is `generated`
  plus `source_file`, it is not governed by the ingest-spec, and it stays disjoint
  from Door A's `ingest_manifest`.

- **The v0.5.0 pilot set gained a fourth member: `portfolio-optimiser`.** Admitted
  2026-08-09 on their request. `v0.5.0a1`'s entry below says "do not pin this tag
  outside the pilot set" and names three repos; that entry is left as written,
  because it records what was true when the tag shipped. This is the amendment,
  and the sentence still binds — the set is now those three plus
  `portfolio-optimiser`.

  The reason is the producer axis, not the count. `portfolio-optimiser-claude` is
  parked, and with it parked no original member could *emit* a v0.2 bundle at all:
  `claude-code-llm-wiki` is read-only in the pilot and `catalog` is gate-side. The
  new member consumes the same Door A. Nothing about the provisional surface
  changes: `OKF_LATEST` still points at `DEFAULT`, and the v0.2 surface may still
  move on pilot feedback without a deprecation cycle.

  **If you pin `v0.5.0a2`, its guard pin is `>=0.2,<0.3`** — that tag resolves
  against guard `v0.2.0`, *not* the `v0.3.4` this repo's `main` now uses. `main`
  moved to `>=0.3,<0.4` after the tag. Also note `tool.uv.sources` is not
  inherited transitively: a consumer supplies the guard's source itself.

- **The guard pin moved to `>=0.3,<0.4`, resolved against `v0.3.4`.** The
  window is widened only after measurement, never before: the 19-fixture
  guard-surface suite was run against `v0.3.4` in a scratch venv first, and
  reproduced exactly the three deltas measured against `v0.3.3` — no new ones.
  `v0.3.4`'s own fixes are regex-complexity repairs, one of them
  (`okf._MD_LINK_RE`) on Door C's call path, with no disposition changes.

- **Door C now passes `allow_reserved=False` to `okf.import_bundle`.** The
  guard added the keyword in the `0.3.x` line and defaults it `True` for the
  received-bundle path, which would merge a sender's `index.md` / `log.md`
  instead of rejecting them. Door C overrides it, keeping the unconditional
  reserved-name refusal committed to before the keyword existed. The reason is
  structural rather than a second opinion on the guard's scan: Door C generates
  the merged bundle's `index.md` from what it merged and writes every merged
  concept verbatim, so a sender's `index.md` would be a second, irreconcilable
  claim on one path.

  **This is not a behaviour change for anyone on the previous pin.** Under
  `v0.2.0` the keyword did not exist and reserved names were refused by
  construction; the explicit argument preserves that outcome across the bump.
  A consumer sees the same rejections, with the same reasons, before and after.

### Fixed

- **`tools/okf_adjudicate.py` exits 2 on a malformed plan instead of raising.**
  A `SegmentationError` from the plan grammar used to escape `main()` as a
  traceback and exit 1, while every other malformed-plan case in the same file
  already returned 2. Exit codes are the interface a caller scripts against,
  and exit 1 with a traceback says "this command broke" where the truth is
  "this file is not a plan" -- the two are the same code an unhandled bug would
  produce. The refusal itself is unchanged: nothing is written either way, and
  the grammar is untouched. Now one line on stderr naming the error code, and
  exit 2. A caller that treated a non-zero exit as failure sees no difference;
  one that distinguished 1 from 2 sees a malformed plan move into the class it
  belonged to. Pre-existing since the empty-verdict branch landed, and pinned
  until now by a test that recorded it as a finding rather than fixing it.

- **A verdict `okf_adjudicate` cannot read back is no longer reported as a
  malformed plan.** The command parses the plan before building anything and
  parses its own verdict after, so only the first failure is the operator's
  file. The second is reachable -- an empty `--adjudicator` produces a verdict
  the grammar refuses -- and before the change above it surfaced as a
  traceback. Catching the grammar error at the top would have turned it into a
  clean, confident, wrong diagnosis naming the plan file, which is worse than
  the traceback: it sends the operator to fix the one artifact that was fine.
  The verdict parse now raises `AdjudicationError`, so the message names what
  actually failed. Exit 2 either way; nothing is written either way.

- **`okf build --ingested-at` alone now stamps every concept, not just the
  unsegmented ones.** `--proposed-at` defaulted independently to
  `DEFAULT_STAMP`, so a caller who set only `--ingested-at` stamped the 11 of
  629 concepts that read the call's value directly, while the 618 segmented
  concepts -- which read `segment.ingested_at`, the plan's `proposed_at` --
  stayed on `1970-01-01T00:00:00Z`. Measured on K2 rebuilt at `fbaac6d`.
  `proposed_at` now defaults to `ingested_at` when omitted; a caller who wants
  the proposal and the replay dated differently still passes both explicitly.
  Neither flag passed still yields `DEFAULT_STAMP` for both, byte-identical to
  before.

- **The consumption pre-pass (`tools/okf_consume.py`) no longer counts a
  linked `log.md` as a concept.** `link_log_in_root_index` (`corpus.py`,
  `95eb271`) links a run's own log from the root index for bundle navigation;
  the index walk that enumerates concepts followed that link like any other
  and counted the log as one, inflating a 629-concept K2 rebuild to 630 and
  letting the log rank and get cut like real content. The link stays --
  `docs/consumption-contract.md` is silent on `log.md`, and `95eb271` already
  named the link a LOCAL choice rather than conformance -- but the walk now
  recognises `LOG_NAME` the same way it recognises the index itself: reachable
  for navigation, never a concept. A bundle whose index does not link its log
  (every bundle built before `95eb271`, including the delivered
  `K2-bundle-20260903`) computes the same `sha256-tree` ref before and after.

## [0.5.0a2] — 2026-07-31

**This is the pre-release the pilots pin. `v0.5.0a1` was tagged and abandoned
unused — do not pin it.** It carried a `generated.by` actor that the spec owner
had already excluded, and it was caught before any pilot was notified.

### Fixed

- **`OKF_V0_2`'s `generated.by` actor is `process:okf-ingest`**, not
  `process:llm-ingestion-okf`. Commons decided the fixed id on this repo's own
  proposal 2026-07-31, superseding option (d) chosen here 2026-07-27. The
  exclusion is `ingest-spec.md:7-8`, frozen on the spec being framework-neutral:
  normalising *our* repo name into the id would force every other conformant
  implementation to write it into its own output.

  Nothing in the wild carried the excluded value — `OKF_V0_2` did not exist at
  `v0.4.0`, and no pilot had been notified — so this costs a tag rather than a
  migration. It is recorded rather than quietly folded in because the failure it
  avoids is specific: `actor` is both the stamp written and the value owned back
  (`OwnershipPolicy`), and recognition is one-way, so a pilot holding bundles
  stamped with the excluded id would have hit `collision_unstamped` on its own
  files the moment the id was corrected.

## [0.5.0a1] — 2026-07-31 — ABANDONED, do not pin

**Pre-release. PROVISIONAL surface.** Shipped to a named pilot set —
`portfolio-optimiser-claude`, the plugin marketplace catalog, and
`claude-code-llm-wiki` — so that real data can find what fixtures cannot. The
v0.2 surface may change on their feedback **without a deprecation cycle**.
Saying so up front is what buys the freedom to act on the feedback; discovering
it later is what would make a pilot a de-facto release. Do not pin this tag
outside the pilot set.

Support for a new upstream version is **additive — a new profile, never a
migration**. `OKF_LATEST` still points at `DEFAULT`; flipping it is the GA
event, not a side effect of this tag.

### Added

- **`OKF_V0_2` profile.** Google OKF v0.2 as a profile alongside `DEFAULT` and
  `STRICT_V1`. It closes nothing: §14 forbids a conformant consumer to reject on
  an unknown `type` value or on unknown additional keys, so `type` is the only
  required key. It NAMES `okf_version` but never carries its value — that value
  tracks the upstream Google version and belongs to catalog (decision E1), so
  the caller supplies it via `materialize_bundle(..., root_frontmatter_values=…)`
  and the profile fixes only the key and its position.
- **`root_frontmatter_values`** on `materialize_bundle`, keyword-only: the
  mechanism behind "a profile names a key, a caller owns its value".
- **`sources` emitted as an inline flow sequence**, populated from the manifest.
- **The v0.2 golden fixture**, compared byte-for-byte like the others.

### Changed

- **`profile` on `materialize_bundle` is keyword-only**, so existing positional
  call sites stay source-compatible.
- **`_is_ingest_owned` is profile-aware.** Ownership is a policy on the profile,
  not a literal; recognition stays one-way.

### Fixed

- **`root_frontmatter` permitted a key without requiring it — the two had been
  the same profile field.** Upstream says MAY where the field said MUST, so a
  root index that legally omitted an optional key was rejected. `root_frontmatter`
  now permits and orders; the new `root_frontmatter_required` requires.
  `STRICT_V1` keeps requiring its three (unchanged for its consumer), `OKF_V0_2`
  requires none, `DEFAULT` is untouched. Measured against the eight root indexes
  available locally: 7 of 8 failed before, 0 of 8 after.

### Notes

- **`DEFAULT` and `STRICT_V1` are byte-stable across this release.** The emit
  path is byte-identical; the golden suite is the check.
- **Known, deliberately unfixed:** `TypePolicy`'s closed branch does not strip
  quotes from a declared type. All three call sites pass `DEFAULT.types`, and
  both `DEFAULT` and `OKF_V0_2` set `allowed=None`, so the branch is
  unreachable in shipped code — it fires only for a caller constructing
  `STRICT_V1` and calling `rejection` directly. Fixing it would repair a write
  path that never sees quotes, and would fix the meaning of a quote character
  without a value model that can express one, in a line-oriented format.

## [0.4.0] — 2026-07-25

Phase 2. The bundle inbox (Door B) and external-bundle import (Door C) ship,
and with them this library's first — and only ever — runtime dependency.

### Added

- **Door B — bundle inbox (`process_inbox`).** Per dropped file: bytes →
  extraction → persist gate → render → collision gate → write → index link.
  Returns an `InboxResult` whose four buckets (`persisted`, `quarantined`,
  `rejected`, `failed`) are disjoint and complete, so a file that vanished
  mid-run surfaces as a missing entry rather than as nothing. One bad file
  never aborts the run: only an invalid `ingested_at`, a reserved `okf_type`,
  and a missing inbox directory fail the whole run, each being wrong for every
  file at once. Concept files are named `inbox-{slug}.md`, disjoint from
  `index.md`, Door A's `ingest-*`, and `promoted-verdict-*`; `source_sha256`
  is taken over the original dropped bytes, so provenance stays re-verifiable
  against the operator's file.
- **Door C — external bundle import (`import_bundle`).** Reads an external OKF
  bundle and hands it *whole* to an injected gate over the guard's
  `okf.import_bundle` — a bundle-level call, because it resolves the
  cross-link graph across concepts — merging only concepts that clear the
  non-blocking floor. Two invariants, both load-bearing: a merged concept is
  written **verbatim** (stamping it would require round-tripping frontmatter
  through this library's line-oriented parser, which cannot represent the
  block lists the guard's parser accepts, and would persist bytes the guard
  never screened), and ownership is therefore proven by **content identity** —
  an occupied target name is re-used only when the bytes there are already
  identical, never overwritten. Re-importing an unchanged bundle is a no-op.
- **`extract_text` — the Door B extraction registry.** All file-type → text
  extraction lives in this library, because the guard is text-only. `md`/`txt`
  pass through, `csv` renders the markdown table, `json` is fenced verbatim,
  and `html`/`htm` reduce to text with `html.parser` — stdlib throughout.
  `pdf`, `docx`, and `xlsx` are gated behind the `[extract]` extra, which
  ships no parser yet: those types fail fast with a typed error naming the
  extra, never a silent skip.
- **`llm_ingestion_okf.guard_adapter`** — the shipped gate over the real
  guard (`inbox_gate`, `import_gate`), and the only module here that imports
  it. Door B screens the exact bytes it persists: the guard's `prepare_input`
  bookend prepares text for a model call this library never makes, so
  `screen_output` alone is used and the screened string is the written string.
  It follows that the gate **refuses rather than repairs** — a file carrying
  an invisible carrier is rejected, not stripped and persisted. The policy is
  `PRESET_USER_UPLOAD`, so any finding at all is held back.
- **15 new stable error codes**, each registered in the `errors.py` docstrings
  (the stability contract) and pinned by the error-code conformance suite:
  `extractor_decode_error`, `extractor_empty_csv`, `extractor_extra_missing`,
  `extractor_unknown`, `import_label_invalid`, `import_path_empty`,
  `import_path_too_long`, `import_provenance_invalid`,
  `import_slug_collision`, `inbox_slug_collision`, `inbox_slug_empty`,
  `inbox_slug_too_long`, `inbox_source_file_invalid`, `inbox_title_invalid`,
  `okf_type_reserved`.

### Changed

- **`llm-ingestion-guard>=0.2,<0.3` is now a mandatory runtime dependency.**
  Installing this package installs the guard. Importing it does not: only
  `guard_adapter` imports the guard, so a Door A consumer keeps working
  whatever state the dependency is in, and the doors themselves still take an
  *injected* gate. Until the guard is published to a package index, install it
  from its tag — see the README. A packaging test enforces that this stays the
  only runtime dependency.
- **An extraction `title` containing `[` or `]` is rejected at manifest load**
  (ingest-spec §4, ratified D1). Previously only single-line was validated. A
  manifest that loaded before and carries a bracket in a title now fails fast
  with code `manifest_schema`: the title renders verbatim into the index link
  label `- [title](target)`, where a bracket breaks index-link and navigation
  parsing downstream.

### Fixed

- **Materializing one manifest no longer deletes the files another manifest
  stamped into the same bundle.** The §3 ownership scan classified every
  ingest-stamped file as replaceable, so a second manifest sharing a bundle
  removed the first one's concept files and their index links. Ownership is
  now narrowed to files whose stamp names the running manifest by stem — the
  stem is stable across content edits, so an edited manifest still reclaims
  what a prior run of itself wrote. The operator-copy restriction (a generated
  file copied into curated content) remains documented, not enforced.

### Notes

- Phase 2's binary extraction is **not** in this release: the `[extract]`
  extra is declared but empty, and `pdf`/`docx`/`xlsx` therefore fail fast.
  That is the one outstanding item from the phase, and it ships separately.
- **The ownership change above is an extension point, not a spec fix.** Filed
  under "Fixed" because it stops silent data loss, but the spec owner
  (`portfolio-optimiser-commons`) has since pointed out that removing *every*
  ingest-stamped file is verbatim what ingest-spec v1 §5 mandates: v1 assumes
  one manifest per bundle and defers multiple manifests feeding one bundle as
  a named extension point. So this release implements that extension point and
  outruns the frozen text rather than conforming to it. The mechanic itself is
  not at risk — matching by manifest stem follows from the spec's own
  `{stem}@{h}` stamp definition, since `{h}` changes on every content edit —
  but the surrounding prose is queued for amendment in commons and is not
  ratified. Treat multi-manifest bundles as ahead of the spec until it is.
- Exception `__cause__` preservation is now pinned by a conformance suite, one
  test per fail-fast wrap site, alongside the existing `.code` suite.

## [0.3.2] — 2026-07-23

### Fixed

- **Frontmatter and index-label values are emitted verbatim; only
  `source_query` is whitespace-collapsed.** Earlier releases collapsed every
  whitespace run in every frontmatter value and index link label to a single
  space. §5 of `ingest-spec.md` mandates that collapse for `source_query`
  alone — where a multi-line SQL `SELECT` must render on one line — while
  every other value is validated single-line at manifest load and passed
  through unchanged: validation, not repair. A `title` carrying an internal
  whitespace run now survives byte-for-byte at both the `title` frontmatter
  and the index link label, instead of being silently altered. Output bytes
  change only for values that contained a collapsible whitespace run; the
  shipped golden fixtures and both consumers are unaffected.

## [0.3.1] — 2026-07-19

### Fixed

- **Documentation corrected a security claim that did not hold.** The module
  docstring and README stated that this library "calls the guard at every persist
  gate". That described the intended end state in the present tense. Door A — the
  only door shipped — is ungated: the package has zero runtime dependencies and
  calls no guard function before writing to disk. Both places now say so plainly,
  and state that gating external or untrusted content is the caller's
  responsibility (`okf.import_bundle`, or `prepare_input`/`screen_output`) until
  the persist gates land with Doors B and C.

  No behavior changed in this release. The correction is published because a
  consumer read the earlier wording as safe-by-default and would have persisted
  ungated content on that basis.

## [0.3.0] — 2026-07-17

### Added

- **Stable machine-readable error codes.** `IngestError` gained a keyword-only
  `code` attribute (default `"unspecified"`). Roughly 24 codes are documented in
  the `errors.py` docstrings, and those docstrings are the stability contract.
  Consumers should assert on `exc.value.code`, not on message text.

### Changed

- **Exception message text is explicitly declared unstable.** It may change in any
  release. Tests matching on message strings (`pytest.raises(match=...)`) should
  migrate to code comparisons.

### Notes

- Generic schema shape violations deliberately share the single code
  `manifest_schema`. One code per validation rule would have frozen an
  unnecessarily large surface. Finer resolution is a separate decision, not an
  assumed requirement.

## [0.2.0] — 2026-07-17

### Added

- **PEP 561 support.** The `py.typed` marker ships with the package, so consumers
  get the inline type hints without a mypy override.
- **`IngestResult.stamp`** exposes the spec §5 provenance stamp on the result
  object.

## [0.1.0] — 2026-07-16

Phase 1 (Door A) implemented against the normative `ingest-spec.md` owned by
`portfolio-optimiser-commons`. Never tagged; consumers pinned the commit
`dae0bd1a` directly.

### Added

- Fail-fast manifest validation (spec §3–§4).
- Spec §5 body renderers as pure functions.
- The `file` connector (CSV, fail-closed path boundary), the `sql` connector
  (read-only sqlite, env-resolved credentials), and the `http` connector behind an
  explicit per-run network opt-in.
- Spec §5 materialization with an in-memory staging collision gate.
- Index maintenance on re-materialization (spec §6).
- The spec §11 golden fixtures, compared byte-for-byte.
- The Door A public surface: `materialize_bundle` plus the typed error hierarchy
  rooted in `IngestError`.

[0.6.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.5.0a2...v0.6.0
[0.5.0a2]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.5.0a1...v0.5.0a2
[0.5.0a1]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.4.0...v0.5.0a1
[0.4.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.2...v0.4.0
[0.3.2]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.1...v0.3.2
[0.3.1]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.0...v0.3.1
[0.3.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.2.0...v0.3.0
[0.2.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/dae0bd1a...v0.2.0
[0.1.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/src/commit/dae0bd1a