llm-ingestion-okf/docs
Kjell Tore Guttormsen 38104b7df5 feat(propose,consume,profiles,importer): recovery yields to declaration, and 9 % of the corpus that was in no segment
One rule explains every remaining `pdf` miss on the twelve-position reference:
where a document DECLARES headings, Arm D's RECOVERED headings are the whole of
the excess, and every declared one is a unit the reference wants. `--outline-gate`
admits recovery only where the document declares none of its own, plus any one
recovered heading covering OUTLINE_SHARE (0.20) of the text. It is `fold_units`
clause 2's own principle moved from voting to admission, and it filters at
ADMISSION so the text a removed mark opened is carried by the mark above it --
the post-filter form scores identically on all twelve positions and loses that
text, which is why only one of them shipped.

`--outline-gate` and `--drop-wrapped-outline` become the package default, one
decision because neither carries the reference alone: `pdf` 2 of 8 -> 5 of 8
alone, 7 of 8 together; the sheet 5 of 12 -> 10 of 12; `docx` unchanged at 3 of
3. Each keeps an explicit opt-out. The bar the move had to clear was not the
reference: hit@8 on a K2 bundle built with it holds 5 of 6 at ranks 1,1,1,1,1,-,
no row losing rank 1. `--sheet-section-rows --keep-table-heading` reaches 11 of
12 and does NOT ship, because on a bundle built with it row 1 falls rank 1 -> 2.
Cost to a consumer is a re-run: 492 concepts / 944 files -> 425 / 810.

DOCUMENT_PRIOR_EXPONENT makes the document prior sublinear (total/n**0.5). A sum
measures size and a density is diluted by every unit carrying none of the
question, so a document split 1 -> 12 lost its prior by 12. Swept over five
values on 18 rows it is at least as good as the delivered density everywhere and
strictly better on three. Stated plainly: end to end it moved NOT ONE hit@8 row
on any of four bundles, so it did not solve the knot it was adopted for -- what
did is that the `pdf` gain never needed `--sheet-section-rows`.

`--first-span-from-zero` is off and repairs a measured loss found while chasing
one position's 940 characters: 32 of the 32 documents that get a plan leave the
text above their first concept in no segment -- 159 704 characters, 9.18 % of
the corpus, 45 841 from one document. It changes nothing on the reference. Off
because it moves the first span of essentially every bundle with no hit@8 number
behind it yet.

vegnormal-okf FUNN 2: SPEC section 8's own star row parsed as prose, so every
concept behind one was unreachable to the section 9.2 walk. `IndexPolicy.also_reads`
carries it for the SEGMENTED profiles, read-only, after the emitted pattern
misses -- the asymmetry `sources` already has. DEFAULT and STRICT_V1 untouched (O2).

vegnormal-okf FUNN 1: Door C's own outcome was refused at exit 1,
`bundle_id_missing`. `import_bundle` now takes `root_frontmatter_values`,
keyword-only, rendered before any disk mutation, written only when the index is
created -- Door B's mechanism and ordering.

Report: docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md.
Suite 1478 passed (1449 before), ruff and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-09 14:17:00 +02:00
..
plan fix(corpus): the root index links the bundle's own log 2026-09-03 20:23:56 +02:00
2026-08-21-g2-pdf-extraction-measurement.md docs(extract): measure one Vegnormalene PDF page against the extraction registry 2026-08-21 19:16:15 +02:00
2026-08-29-konverteringsarmen-arm-a.md docs(measure): run the conversion arm on K2 - pandoc wins both office formats 2026-08-30 00:41:48 +02:00
2026-08-29-telling-offentlig-prosjektkorpus.md docs(measure): count the public project corpus - K2 measured, K1 blocked 2026-08-29 07:03:50 +02:00
2026-09-02-k3-k4-k5-baseline.md docs(measurement): K3, K4 and K5 numbers against the committed method 2026-09-02 15:55:19 +02:00
2026-09-02-k3-k4-k5-metode.md docs(measurement): K3, K4 and K5 method with n declared before any verdict 2026-09-02 15:46:03 +02:00
2026-09-02-korpuskjoring-k1-k2.md docs(measurement): K1 and K2 over the K2 corpus 2026-09-02 15:43:45 +02:00
2026-09-03-k2-bundle-rebuild.md test(corpus): a rebuild with log.md present reproduces the bundle exactly 2026-09-03 05:16:25 +02:00
2026-09-04-cid-andel-k2.md docs(measure): CID-glyph share across K2 -- Bilag 9.1 alone, not a K1 arm 2026-09-04 20:23:41 +02:00
2026-09-04-k2-pptx-odt-rtf.md docs(measure): K2 on pptx/odt/rtf -- denominator 0 in both named corpora 2026-09-04 21:12:41 +02:00
2026-09-04-k3-arm-c.md docs(measure): K3 with Arm C beside the baseline, two numbers, no threshold 2026-09-04 17:58:50 +02:00
2026-09-07-k2-pptx-odt-rtf-fixtures.md test(fidelity): K2 on pptx, odt and rtf -- 59/59 and 20/20, on a set we wrote 2026-09-07 05:22:12 +02:00
2026-09-07-k3-arm-d.md docs(measure): K3 Arm D beside a re-rated Arm B -- reach and no threshold 2026-09-07 03:04:01 +02:00
2026-09-07-k3-arm-e.md docs(measure): K3 Arm E beside Arm B and Arm D -- three rows, no threshold 2026-09-07 11:48:58 +02:00
2026-09-07-k3-enhetsark-metode.md docs(measure): K3 unit worksheet delivered for operator, no code changed 2026-09-07 16:36:16 +02:00
2026-09-07-okf-konsumskill-maaling.md docs(consume): refutation sweep — smoothed-IDF rarity weighting cannot zero both known-negatives 2026-09-07 09:56:36 +02:00
2026-09-08-blindsone-below-k-k2.md feat(consume): measure the below-k blind spot, add one flag-gated vocabulary bridge 2026-09-08 04:55:45 +02:00
2026-09-08-blindsone-laas2-budsjett-k2.md feat(consume): measure the budget lock, add one flag-gated top-rank reservation 2026-09-08 05:18:30 +02:00
2026-09-08-claude-code-skill-vilkaarlig-bundle.md docs(consume): the Claude Code recipe, measured end to end on two bundles 2026-09-08 15:32:32 +02:00
2026-09-08-eksakt-oppslag.md fix(consume): match an identifier by equality, deliver the concept a question names 2026-09-08 13:27:59 +02:00
2026-09-08-k3-arm-f-mot-enhetsarket.md feat(propose): Arm F, one unit fold behind a flag, measured against the operator's worksheet [skip-docs] 2026-09-08 16:28:41 +02:00
2026-09-08-k3-runde2-per-filtype.md feat(cli,propose): reach the arms from okf build, keep a sheet heading behind a flag, fix two PDF contents mechanisms 2026-09-08 18:21:25 +02:00
2026-09-08-k3-runde3-per-filtype.md docs(cli): the new default's K2 digest, measured on the bundle it produces 2026-09-08 21:14:51 +02:00
2026-09-08-k3-runde4-pdf-skrift-og-ocr.md feat(extract,cli): typography as a PDF heading source and OCR behind an optional group, both off 2026-09-08 23:10:47 +02:00
2026-09-08-k3-runde5-hitat8-og-skriftakse.md feat(propose,cli): typography as a reserve, and the two of our own numbers it took to measure it 2026-09-09 00:25:51 +02:00
2026-09-08-kravnummer-tokenisering.md feat(propose,cli): the sheet's rows, the wrapped sentence, and a default that moved 2026-09-08 20:34:18 +02:00
2026-09-08-o5-okf-project.md feat(cli): okf project/consume/check/skill, and a generated skill with no path into a checkout 2026-09-08 21:47:38 +02:00
2026-09-08-prisform-og-loggen-k2.md fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index 2026-09-08 10:06:58 +02:00
2026-09-08-proveniens-k2.md feat(inbox): point every concept at the document it came from, with a locator per format 2026-09-08 14:39:24 +02:00
2026-09-08-rangeringsbom-sammensatte-ord.md fix(consume): the compound-word miss is a degenerate signal's tie-break, behind a flag 2026-09-08 18:53:18 +02:00
2026-09-08-sjeldenhetsvekt.md feat(propose,cli): the sheet's rows, the wrapped sentence, and a default that moved 2026-09-08 20:34:18 +02:00
2026-09-09-k3-runde6-outline-gaten-og-prioren.md feat(propose,consume,profiles,importer): recovery yields to declaration, and 9 % of the corpus that was in no segment 2026-09-09 14:17:00 +02:00
consumption-contract.md feat(consume): carry every source_* key by prefix, and generate a skill per bundle 2026-09-08 15:13:22 +02:00
okf-nokkelinventar.md build(deps): pin llm-ingestion-guard v1.3.0 so the gate reads our own goldens 2026-09-03 20:41:03 +02:00
phase-3-split-table.md docs(phase-3): keep the index-entry grammar on one line 2026-08-04 12:37:02 +02:00
upstream-okf-upgrade-runbook.md feat(watch): weekly OKF upstream watch that can prove it found nothing 2026-08-23 20:38:37 +02:00