feat(propose,consume,profiles,importer): recovery yields to declaration, and 9 % of the corpus that was in no segment

One rule explains every remaining `pdf` miss on the twelve-position reference:
where a document DECLARES headings, Arm D's RECOVERED headings are the whole of
the excess, and every declared one is a unit the reference wants. `--outline-gate`
admits recovery only where the document declares none of its own, plus any one
recovered heading covering OUTLINE_SHARE (0.20) of the text. It is `fold_units`
clause 2's own principle moved from voting to admission, and it filters at
ADMISSION so the text a removed mark opened is carried by the mark above it --
the post-filter form scores identically on all twelve positions and loses that
text, which is why only one of them shipped.

`--outline-gate` and `--drop-wrapped-outline` become the package default, one
decision because neither carries the reference alone: `pdf` 2 of 8 -> 5 of 8
alone, 7 of 8 together; the sheet 5 of 12 -> 10 of 12; `docx` unchanged at 3 of
3. Each keeps an explicit opt-out. The bar the move had to clear was not the
reference: hit@8 on a K2 bundle built with it holds 5 of 6 at ranks 1,1,1,1,1,-,
no row losing rank 1. `--sheet-section-rows --keep-table-heading` reaches 11 of
12 and does NOT ship, because on a bundle built with it row 1 falls rank 1 -> 2.
Cost to a consumer is a re-run: 492 concepts / 944 files -> 425 / 810.

DOCUMENT_PRIOR_EXPONENT makes the document prior sublinear (total/n**0.5). A sum
measures size and a density is diluted by every unit carrying none of the
question, so a document split 1 -> 12 lost its prior by 12. Swept over five
values on 18 rows it is at least as good as the delivered density everywhere and
strictly better on three. Stated plainly: end to end it moved NOT ONE hit@8 row
on any of four bundles, so it did not solve the knot it was adopted for -- what
did is that the `pdf` gain never needed `--sheet-section-rows`.

`--first-span-from-zero` is off and repairs a measured loss found while chasing
one position's 940 characters: 32 of the 32 documents that get a plan leave the
text above their first concept in no segment -- 159 704 characters, 9.18 % of
the corpus, 45 841 from one document. It changes nothing on the reference. Off
because it moves the first span of essentially every bundle with no hit@8 number
behind it yet.

vegnormal-okf FUNN 2: SPEC section 8's own star row parsed as prose, so every
concept behind one was unreachable to the section 9.2 walk. `IndexPolicy.also_reads`
carries it for the SEGMENTED profiles, read-only, after the emitted pattern
misses -- the asymmetry `sources` already has. DEFAULT and STRICT_V1 untouched (O2).

vegnormal-okf FUNN 1: Door C's own outcome was refused at exit 1,
`bundle_id_missing`. `import_bundle` now takes `root_frontmatter_values`,
keyword-only, rendered before any disk mutation, written only when the index is
created -- Door B's mechanism and ordering.

Report: docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md.
Suite 1478 passed (1449 before), ruff and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 14:17:00 +02:00
commit 38104b7df5
16 changed files with 1301 additions and 42 deletions

View file

@ -0,0 +1,66 @@
"""SPEC section 8's star row is READ, and still never written.
vegnormal-okf, 2026-09-08 (FUNN 2): measured against
`SEGMENTED_OKF_V0_2.index.parse_entry`, the row form OKF SPEC section 8 shows
in its own example -- `* [Title](id-x.md) - description` -- returns `None`,
so the walk in section 9.2 reads it as curated prose and every concept behind
such a row is unreachable. Google's own generator writes that form. A bundle
this library cannot walk is exactly the silent loss the "arbitrary bundle"
direction forbids.
The fix is asymmetric on purpose, and the asymmetry is this repository's
existing posture rather than a new one: `sources` is likewise READ in both
YAML forms and WRITTEN in one. Reading a form is not a licence to emit it --
every emitted byte still comes from `link_template`, so no golden moves.
The known-negative is the reason the widening is bounded: a curated prose line
must still survive verbatim, and a star row is admitted only where its whole
line is an entry.
"""
from __future__ import annotations
from llm_ingestion_okf.profiles import DEFAULT, SEGMENTED_OKF_V0_2, SEGMENTED_V1, STRICT_V1
SPEC_STAR = "* [Tittel](id-x.md) - beskrivelse"
SPEC_STAR_BARE = "* [Tittel](id-x.md)"
OWN_FORM = "- [Tittel](id-x.md)"
def test_the_spec_star_row_is_read_by_the_segmented_profiles() -> None:
for profile in (SEGMENTED_OKF_V0_2, SEGMENTED_V1):
entry = profile.index.parse_entry(SPEC_STAR)
assert entry is not None, (
f"{profile.index.name}: SPEC section 8's own row form reads as prose"
)
assert entry.label == "Tittel"
assert entry.target == "id-x.md"
assert entry.description == "beskrivelse"
bare = profile.index.parse_entry(SPEC_STAR_BARE)
assert bare is not None and bare.target == "id-x.md"
def test_this_library_still_writes_only_its_own_form() -> None:
"""No emitted byte moves: the template is untouched and renders the hyphen."""
line = SEGMENTED_OKF_V0_2.index.render_link("Tittel", "id-x.md")
assert line == OWN_FORM
assert SEGMENTED_OKF_V0_2.index.parse_entry(line) is not None
def test_curated_prose_still_survives_verbatim() -> None:
"""The known-negative. A widening that swallowed prose would destroy it."""
for prose in (
"* En kulepunktlinje som ikke er en oppforing",
"* [Tittel](id-x.md) etterfulgt av mer tekst som ikke er en beskrivelse etter bindestrek",
"Se * [Tittel](id-x.md) - beskrivelse midt i en setning",
"*[Tittel](id-x.md)",
):
assert SEGMENTED_OKF_V0_2.index.parse_entry(prose) is None, prose
def test_the_profiles_that_state_another_repositorys_contract_do_not_move() -> None:
"""O2: `DEFAULT` states commons' spec and `STRICT_V1` the wiki's contract."""
assert DEFAULT.index.parse_entry(SPEC_STAR) is None
assert DEFAULT.index.parse_entry(OWN_FORM) is not None
assert STRICT_V1.index.parse_entry(SPEC_STAR) is not None
assert STRICT_V1.index.parse_entry(OWN_FORM) is None