One rule explains every remaining `pdf` miss on the twelve-position reference: where a document DECLARES headings, Arm D's RECOVERED headings are the whole of the excess, and every declared one is a unit the reference wants. `--outline-gate` admits recovery only where the document declares none of its own, plus any one recovered heading covering OUTLINE_SHARE (0.20) of the text. It is `fold_units` clause 2's own principle moved from voting to admission, and it filters at ADMISSION so the text a removed mark opened is carried by the mark above it -- the post-filter form scores identically on all twelve positions and loses that text, which is why only one of them shipped. `--outline-gate` and `--drop-wrapped-outline` become the package default, one decision because neither carries the reference alone: `pdf` 2 of 8 -> 5 of 8 alone, 7 of 8 together; the sheet 5 of 12 -> 10 of 12; `docx` unchanged at 3 of 3. Each keeps an explicit opt-out. The bar the move had to clear was not the reference: hit@8 on a K2 bundle built with it holds 5 of 6 at ranks 1,1,1,1,1,-, no row losing rank 1. `--sheet-section-rows --keep-table-heading` reaches 11 of 12 and does NOT ship, because on a bundle built with it row 1 falls rank 1 -> 2. Cost to a consumer is a re-run: 492 concepts / 944 files -> 425 / 810. DOCUMENT_PRIOR_EXPONENT makes the document prior sublinear (total/n**0.5). A sum measures size and a density is diluted by every unit carrying none of the question, so a document split 1 -> 12 lost its prior by 12. Swept over five values on 18 rows it is at least as good as the delivered density everywhere and strictly better on three. Stated plainly: end to end it moved NOT ONE hit@8 row on any of four bundles, so it did not solve the knot it was adopted for -- what did is that the `pdf` gain never needed `--sheet-section-rows`. `--first-span-from-zero` is off and repairs a measured loss found while chasing one position's 940 characters: 32 of the 32 documents that get a plan leave the text above their first concept in no segment -- 159 704 characters, 9.18 % of the corpus, 45 841 from one document. It changes nothing on the reference. Off because it moves the first span of essentially every bundle with no hit@8 number behind it yet. vegnormal-okf FUNN 2: SPEC section 8's own star row parsed as prose, so every concept behind one was unreachable to the section 9.2 walk. `IndexPolicy.also_reads` carries it for the SEGMENTED profiles, read-only, after the emitted pattern misses -- the asymmetry `sources` already has. DEFAULT and STRICT_V1 untouched (O2). vegnormal-okf FUNN 1: Door C's own outcome was refused at exit 1, `bundle_id_missing`. `import_bundle` now takes `root_frontmatter_values`, keyword-only, rendered before any disk mutation, written only when the index is created -- Door B's mechanism and ordering. Report: docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md. Suite 1478 passed (1449 before), ruff and mypy clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
127 lines
4.8 KiB
Python
127 lines
4.8 KiB
Python
"""Door C's own outcome is a bundle the reading direction can open.
|
|
|
|
vegnormal-okf, 2026-09-08 (FUNN 1): `import_bundle` wrote the root index with
|
|
no frontmatter and took no `root_frontmatter_values`, so it could not declare
|
|
`bundle_id`. `okf consume` then refused the result with exit 1,
|
|
`bundle_id_missing` -- section 3.1's identity tuple is `(bundle_id,
|
|
concept_id)` and half of it was absent. The consumer's workaround was to use
|
|
Door C as a GATE and write the consumable tree themselves.
|
|
|
|
The fix is the mechanism Door B already has and Door C did not: a profile
|
|
names a key, the CALLER owns its value (decision E1). It is keyword-only with
|
|
a default of `None`, so every existing call site emits the bytes it always did
|
|
-- a consumer with branch bases built through this door is not asked to
|
|
rebuild them, which is the boundary this repository states for its own
|
|
consumers.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
from llm_ingestion_okf.importer import import_bundle
|
|
from llm_ingestion_okf.materialize import parse_frontmatter
|
|
from llm_ingestion_okf.profiles import DEFAULT, SEGMENTED_V1
|
|
|
|
from test_import_flow import CONCEPT, StubImportGate, place
|
|
|
|
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
|
TOOL = PROJECT_ROOT / "tools" / "okf_consume.py"
|
|
KNOWN_POSITIVE = PROJECT_ROOT / "examples" / "ingest-golden-segmented-okf-v0-2" / "expected-bundle"
|
|
|
|
|
|
def _consume(bundle: Path) -> subprocess.CompletedProcess[str]:
|
|
return subprocess.run(
|
|
[sys.executable, str(TOOL), str(bundle), "--question", "users"],
|
|
capture_output=True,
|
|
text=True,
|
|
check=False,
|
|
)
|
|
|
|
|
|
def test_the_known_positive_is_readable_first() -> None:
|
|
"""Before an exit 1 counts as a finding, the chain must be shown able to pass."""
|
|
assert _consume(KNOWN_POSITIVE).returncode == 0
|
|
|
|
|
|
def test_door_c_without_a_bundle_id_is_still_refused(tmp_path: Path) -> None:
|
|
"""The defect, kept as a test: silence is not the fix, a named value is."""
|
|
place(tmp_path / "source", "tables/users.md", CONCEPT)
|
|
bundle = tmp_path / "bundle"
|
|
import_bundle(
|
|
tmp_path / "source",
|
|
bundle,
|
|
"1970-01-01T00:00:00Z",
|
|
origin="external",
|
|
channel="manual",
|
|
gate=StubImportGate(),
|
|
)
|
|
assert (bundle / "index.md").is_file()
|
|
assert "bundle_id" not in parse_frontmatter(bundle / "index.md")
|
|
assert _consume(bundle).returncode == 1
|
|
|
|
|
|
def test_door_c_with_a_bundle_id_produces_a_consumable_bundle(tmp_path: Path) -> None:
|
|
place(tmp_path / "source", "tables/users.md", CONCEPT)
|
|
bundle = tmp_path / "bundle"
|
|
import_bundle(
|
|
tmp_path / "source",
|
|
bundle,
|
|
"1970-01-01T00:00:00Z",
|
|
origin="external",
|
|
channel="manual",
|
|
gate=StubImportGate(),
|
|
profile=SEGMENTED_V1,
|
|
root_frontmatter_values={"bundle_id": "imported-2026-09-09"},
|
|
)
|
|
declared = parse_frontmatter(bundle / "index.md")
|
|
assert declared["bundle_id"] == "imported-2026-09-09"
|
|
done = _consume(bundle)
|
|
assert done.returncode == 0, done.stderr
|
|
payload = json.loads(done.stdout)
|
|
assert payload["bundle"]["bundle_id"] == "imported-2026-09-09"
|
|
|
|
|
|
def test_a_second_run_does_not_duplicate_the_declaration(tmp_path: Path) -> None:
|
|
"""The index is APPENDED to across runs; the frontmatter must not be."""
|
|
place(tmp_path / "source", "tables/users.md", CONCEPT)
|
|
bundle = tmp_path / "bundle"
|
|
for _ in range(2):
|
|
import_bundle(
|
|
tmp_path / "source",
|
|
bundle,
|
|
"1970-01-01T00:00:00Z",
|
|
origin="external",
|
|
channel="manual",
|
|
gate=StubImportGate(),
|
|
profile=SEGMENTED_V1,
|
|
root_frontmatter_values={"bundle_id": "imported-2026-09-09"},
|
|
)
|
|
text = (bundle / "index.md").read_text(encoding="utf-8")
|
|
assert text.count("bundle_id:") == 1
|
|
assert parse_frontmatter(bundle / "index.md")["bundle_id"] == "imported-2026-09-09"
|
|
|
|
|
|
def test_the_default_profile_names_no_root_key_and_says_so(tmp_path: Path) -> None:
|
|
"""Naming a key the profile does not carry is refused BEFORE any write."""
|
|
place(tmp_path / "source", "tables/users.md", CONCEPT)
|
|
bundle = tmp_path / "bundle"
|
|
assert DEFAULT.index.root_frontmatter == ()
|
|
try:
|
|
import_bundle(
|
|
tmp_path / "source",
|
|
bundle,
|
|
"1970-01-01T00:00:00Z",
|
|
origin="external",
|
|
channel="manual",
|
|
gate=StubImportGate(),
|
|
root_frontmatter_values={"bundle_id": "x"},
|
|
)
|
|
except Exception as exc: # noqa: BLE001 - the code is the assertion
|
|
assert getattr(exc, "code", "") == "index_root_frontmatter_unexpected"
|
|
else: # pragma: no cover - a pass here is the defect
|
|
raise AssertionError("an unnamed root key was written")
|
|
assert not bundle.exists(), "a refused call left a partially written bundle"
|