feat(propose,consume,profiles,importer): recovery yields to declaration, and 9 % of the corpus that was in no segment
One rule explains every remaining `pdf` miss on the twelve-position reference: where a document DECLARES headings, Arm D's RECOVERED headings are the whole of the excess, and every declared one is a unit the reference wants. `--outline-gate` admits recovery only where the document declares none of its own, plus any one recovered heading covering OUTLINE_SHARE (0.20) of the text. It is `fold_units` clause 2's own principle moved from voting to admission, and it filters at ADMISSION so the text a removed mark opened is carried by the mark above it -- the post-filter form scores identically on all twelve positions and loses that text, which is why only one of them shipped. `--outline-gate` and `--drop-wrapped-outline` become the package default, one decision because neither carries the reference alone: `pdf` 2 of 8 -> 5 of 8 alone, 7 of 8 together; the sheet 5 of 12 -> 10 of 12; `docx` unchanged at 3 of 3. Each keeps an explicit opt-out. The bar the move had to clear was not the reference: hit@8 on a K2 bundle built with it holds 5 of 6 at ranks 1,1,1,1,1,-, no row losing rank 1. `--sheet-section-rows --keep-table-heading` reaches 11 of 12 and does NOT ship, because on a bundle built with it row 1 falls rank 1 -> 2. Cost to a consumer is a re-run: 492 concepts / 944 files -> 425 / 810. DOCUMENT_PRIOR_EXPONENT makes the document prior sublinear (total/n**0.5). A sum measures size and a density is diluted by every unit carrying none of the question, so a document split 1 -> 12 lost its prior by 12. Swept over five values on 18 rows it is at least as good as the delivered density everywhere and strictly better on three. Stated plainly: end to end it moved NOT ONE hit@8 row on any of four bundles, so it did not solve the knot it was adopted for -- what did is that the `pdf` gain never needed `--sheet-section-rows`. `--first-span-from-zero` is off and repairs a measured loss found while chasing one position's 940 characters: 32 of the 32 documents that get a plan leave the text above their first concept in no segment -- 159 704 characters, 9.18 % of the corpus, 45 841 from one document. It changes nothing on the reference. Off because it moves the first span of essentially every bundle with no hit@8 number behind it yet. vegnormal-okf FUNN 2: SPEC section 8's own star row parsed as prose, so every concept behind one was unreachable to the section 9.2 walk. `IndexPolicy.also_reads` carries it for the SEGMENTED profiles, read-only, after the emitted pattern misses -- the asymmetry `sources` already has. DEFAULT and STRICT_V1 untouched (O2). vegnormal-okf FUNN 1: Door C's own outcome was refused at exit 1, `bundle_id_missing`. `import_bundle` now takes `root_frontmatter_values`, keyword-only, rendered before any disk mutation, written only when the index is created -- Door B's mechanism and ordering. Report: docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md. Suite 1478 passed (1449 before), ruff and mypy clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
b01492b7f5
commit
38104b7df5
16 changed files with 1301 additions and 42 deletions
89
tests/test_default_bundle_pin.py
Normal file
89
tests/test_default_bundle_pin.py
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
"""The bundle the DEFAULT build produces, pinned where a regression goes red.
|
||||
|
||||
`tests/test_okf_consume.py` pinned hit@8 against the Arm B bundle alone -- the
|
||||
configuration `okf build` stopped emitting on 2026-09-08. A published number
|
||||
measured on a bundle nobody produces is a number that cannot regress, so the
|
||||
guarantee it looks like was never held by anything.
|
||||
|
||||
This file pins the CURRENT default: `--outline-run 3 --table-grid --unit-fold
|
||||
--drop-wrapped-outline --outline-gate`, the configuration round 6 moved the
|
||||
default to on 2026-09-09 after measuring hit@8 on exactly this bundle.
|
||||
|
||||
The gold set is LOCAL-ONLY and stays that way: no question and no
|
||||
`gold_document` is reproduced here, and a row is named by its INDEX, the way
|
||||
`docs/2026-09-07-okf-konsumskill-maaling.md` already names them.
|
||||
|
||||
The bundle itself is a build artefact, not a fixture: it is 810 files of a
|
||||
consumer's corpus and this repository is public. Absent, these tests SKIP with
|
||||
the command that rebuilds it -- "not measured", never zero.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(PROJECT_ROOT / "tools"))
|
||||
|
||||
import okf_consume # noqa: E402
|
||||
import okf_consume_measure # noqa: E402
|
||||
|
||||
#: Built by:
|
||||
#: okf build <corpus>/K2/trinn1 \
|
||||
#: --bundle ~/corpora/okf-telling-20260829/K2-bundle-default-20260909 \
|
||||
#: --bundle-id k2-trinn1-20260903 --okf-version 0.2
|
||||
#: with no arm flag at all -- the package default, which is the point.
|
||||
DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260909"
|
||||
GOLD_SET = PROJECT_ROOT / ".claude/projects/2026-09-07-okf-consume-prepass/hit-at-k-questions.json"
|
||||
|
||||
requires_default_bundle = pytest.mark.skipif(
|
||||
not DEFAULT_BUNDLE.is_dir() or not GOLD_SET.is_file(),
|
||||
reason=(
|
||||
f"the default-configuration K2 bundle is not present at {DEFAULT_BUNDLE}. "
|
||||
"NOT MEASURED, not zero: rebuild it with `okf build <corpus>/K2/trinn1 "
|
||||
"--bundle <that path> --bundle-id k2-trinn1-20260903 --okf-version 0.2`"
|
||||
),
|
||||
)
|
||||
|
||||
#: Measured 2026-09-09 on the bundle above.
|
||||
EXPECTED_CONCEPTS = 425
|
||||
EXPECTED_HITS = 5
|
||||
#: Rank per question INDEX, `None` for the row that misses on every bundle and
|
||||
#: every configuration measured so far. The identity is the index; the question
|
||||
#: stays in the local-only gold set.
|
||||
EXPECTED_RANKS = (1, 1, 1, 1, 1, None)
|
||||
|
||||
|
||||
@requires_default_bundle
|
||||
def test_the_default_bundle_holds_its_concept_count() -> None:
|
||||
assert len(list(okf_consume.enumerate_concepts(DEFAULT_BUNDLE))) == EXPECTED_CONCEPTS
|
||||
|
||||
|
||||
@requires_default_bundle
|
||||
def test_hit_at_eight_holds_rank_one_on_every_row_it_held() -> None:
|
||||
"""The acceptance criterion round 6's default move had to clear.
|
||||
|
||||
Not the hit COUNT alone: the count survived a configuration that lost a
|
||||
row from rank 1 to rank 2, which is exactly how the previous round's
|
||||
regression hid. The rank per row is the pin.
|
||||
"""
|
||||
questions = json.loads(GOLD_SET.read_text(encoding="utf-8"))["questions"]
|
||||
assert len(questions) == len(EXPECTED_RANKS), "the gold set changed shape"
|
||||
ranks = []
|
||||
for entry in questions:
|
||||
payload = okf_consume.build_payload(DEFAULT_BUNDLE, question=entry["question"])
|
||||
excerpts = payload["excerpts"]
|
||||
assert isinstance(excerpts, list)
|
||||
ranks.append(okf_consume_measure.hit_rank(excerpts, entry["gold_document"]))
|
||||
assert tuple(ranks) == EXPECTED_RANKS, f"hit@8 ranks moved: {ranks}"
|
||||
assert sum(rank is not None for rank in ranks) == EXPECTED_HITS
|
||||
|
||||
|
||||
@requires_default_bundle
|
||||
def test_the_bundle_declares_the_identity_the_reader_needs() -> None:
|
||||
"""Whatever else moves, the bundle stays one the reading direction opens."""
|
||||
assert okf_consume.root_bundle_id_of(DEFAULT_BUNDLE) == "k2-trinn1-20260903"
|
||||
Loading…
Add table
Add a link
Reference in a new issue