feat(propose,cli): the coverage gap had one cause, and round 7's own decomposition did not reproduce

Round 7 named two open items: a table-block candidate displacing a declared
heading (26 041 characters between entries, `md` at 3 of 4 declared headings)
and 17 590 characters after the last entry, never examined. Measured on
`a364ef4`, the first premise does not reproduce and the second is not a
separate mechanism.

`md` recovers 4 of 4, not 3 of 4. D1 (`--keep-table-heading`) is the repair
for exactly the mechanism round 7 § 5 describes, and round 7 moved it into
the default in the same commit -- so § 5 is a pre-move measurement presented
as the post-move state. That is round 7's own trap, stated in its own report:
a number about a rule is a measurement of a configuration. Nothing held the
cell, which is why it could be wrong and stay wrong.

The remaining gap has ONE cause. Every rule closes a span against the NEXT
MARK; three steps then remove a mark after its neighbour's `end` was fixed
against it. The orphan check leaks 18 527 characters over 15 of 39 documents,
`fold_units` clause 1 leaks 7 514 between entries, and the same clause on the
last run leaks all 17 590 tail characters -- with `unit_fold=False` the corpus
tail gap is 0. Round 6 already established the principle (filter at admission,
let the mark above carry the text) and it was never applied post-filter.

`--close-span-gaps` states it once, after the fold, where every removal has
happened: a span runs to the next SURVIVOR, the last to the end of the text.
It adds no boundary. Measured: 43 631 characters (2.51 %) -> 0, entries
429 = 429, K2 concepts 436 = 436, the operator's folder 52 md = 52.

All four acceptance conditions hold at once. The 12-position reference is
label-identical -- 11 of 12 under |F|[3]=12 and 10 of 12 under |F|[3]=11,
both readings reported, not one position moved. hit@8 holds [1,1,1,1,1,-] on
the round 8 bundle (436), the round 7 default (436) and Arm B (629), and the
known-negative reproduces on the new bytes: `--no-tie-shared-rank` gives
[2,1,1,1,1,-].

The pin is rebuilt against the default that actually ships
(`K2-bundle-default-20260911`, 832 files, digest 8c93e5e3...); two independent
builds of it differ in nothing, `log.md` included.

Two measurement failures are recorded rather than hidden. The corpus identity
check first used `xargs` without `-print0`, every filename split on its
spaces, `shasum` read nothing, and both sides agreed on a digest of the
failure; a control against the digest of empty input separated them. And the
smoke build cannot prove the flag arrives -- the operator's folder has a
coverage gap of zero already, so `diff -rq` is 0 either way; the plumbing has
its own test on a document that has the defect.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 17:43:56 +02:00
commit 956714594d
8 changed files with 592 additions and 22 deletions

View file

@ -7,10 +7,10 @@ guarantee it looks like was never held by anything.
This file pins the CURRENT default: `--outline-run 3 --table-grid --unit-fold
--drop-wrapped-outline --outline-gate --first-span-from-zero
--sheet-section-rows --keep-table-heading`, plus the reading side's
`tie_shared_rank`. Round 6 moved the first five on 2026-09-09 and round 7 moved
the last four on 2026-09-10, each after measuring hit@8 on exactly the bundle
its own default produces.
--sheet-section-rows --keep-table-heading --close-span-gaps`, plus the reading
side's `tie_shared_rank`. Round 6 moved the first five on 2026-09-09, round 7
moved four more on 2026-09-10 and round 8 moved the last on 2026-09-11, each
after measuring hit@8 on exactly the bundle its own default produces.
The gold set is LOCAL-ONLY and stays that way: no question and no
`gold_document` is reproduced here, and a row is named by its INDEX, the way
@ -37,10 +37,16 @@ import okf_consume_measure # noqa: E402
#: Built by:
#: okf build <corpus>/K2/trinn1 \
#: --bundle ~/corpora/okf-telling-20260829/K2-bundle-default-20260910 \
#: --bundle ~/corpora/okf-telling-20260829/K2-bundle-default-20260911 \
#: --bundle-id k2-trinn1-20260903 --okf-version 0.2
#: with no arm flag at all -- the package default, which is the point.
DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260910"
#:
#: Rebuilt 2026-09-11 for `--close-span-gaps`. Digest, from inside the bundle:
#: find . -type f -print0 | sort -z | xargs -0 shasum -a 256 | shasum -a 256
#: -> 8c93e5e3222577a2b3352ca83af980e403d3a571c3a467b83c3d8170b1df2b69
#: Two independent builds of it differ in NOTHING (`diff -rq`), including
#: `log.md`, which carries the corpus path and never the bundle's own.
DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260911"
GOLD_SET = PROJECT_ROOT / ".claude/projects/2026-09-07-okf-consume-prepass/hit-at-k-questions.json"
requires_default_bundle = pytest.mark.skipif(
@ -52,10 +58,12 @@ requires_default_bundle = pytest.mark.skipif(
),
)
#: Measured 2026-09-10 on the bundle above. The count moved 425 -> 436 with
#: `--sheet-section-rows --keep-table-heading`; `--first-span-from-zero` moved
#: it by NOTHING, which is the point of that rule -- it adds no boundary and
#: only moves one span's start.
#: Measured 2026-09-11 on the bundle above. The count moved 425 -> 436 with
#: `--sheet-section-rows --keep-table-heading`; `--first-span-from-zero` and
#: `--close-span-gaps` each moved it by NOTHING, which is the point of both --
#: they add no boundary, they only move a span's start or its end. Round 8's
#: rule closed 43 631 characters (2.51 % of the corpus) that were in no
#: segment, and the count is byte-for-byte the same 436.
EXPECTED_CONCEPTS = 436
EXPECTED_HITS = 5
#: Rank per question INDEX, `None` for the row that misses on every bundle and

View file

@ -0,0 +1,100 @@
"""`md` recovers every heading it declares, and D1 is what makes that true.
Round 7 reported this cell as **3 of 4**, with the mechanism decomposed: a
`rule:table-block` candidate opens BELOW `## 3 Prising` instead of a
`rule:heading` opening ON it, so the heading is bodiless, the orphan check
drops it, and the fold then merges the nameless block into the section above.
Re-measured on `a364ef4` (round 7's own commit), the default recovers **4 of
4**. The report's § 5 is a measurement of the configuration that existed
BEFORE the same round moved `--keep-table-heading` into the default -- D1 is
precisely the repair for that mechanism, and it shipped in the commit the
report describes. Nothing in the tree said so, because the decomposition and
the default move live in different files.
That is the round-7 trap restated: a number about a rule is a measurement of a
CONFIGURATION. This file is the missing pin. The cell is asserted here, and so
is its cause -- turn D1 off and the third heading becomes a table block again.
"""
from __future__ import annotations
from llm_ingestion_okf import cli
from llm_ingestion_okf.propose import RULE_HEADING, RULE_TABLE_BLOCK, find_candidates
DEFAULT = dict(
outline_run=cli.DEFAULT_OUTLINE_RUN,
table_grid=cli.DEFAULT_TABLE_GRID,
unit_fold=cli.DEFAULT_UNIT_FOLD,
keep_table_heading=cli.DEFAULT_KEEP_TABLE_HEADING,
sheet_section_rows=cli.DEFAULT_SHEET_SECTION_ROWS,
drop_wrapped_outline=cli.DEFAULT_DROP_WRAPPED_OUTLINE,
outline_gate=cli.DEFAULT_OUTLINE_GATE,
first_span_from_zero=cli.DEFAULT_FIRST_SPAN_FROM_ZERO,
close_span_gaps=cli.DEFAULT_CLOSE_SPAN_GAPS,
)
#: The same structure round 7 built: three chapters, the last one a heading
#: with a table directly under it and a closing line after the table.
DOCUMENT = """# Prosjektbeskrivelse Testdokument
Dette dokumentet har en kjent struktur: tre kapitler paa niva 1, hvert med
en kort brodtekst, samt en innledning over det forste kapitlet.
## 1 Omfang
Leveransen omfatter tre delytelser. Hver delytelse har egen frist og egen
akseptansetest. Teksten her er brodtekst og skal ikke bli en overskrift.
## 2 Krav til dokumentasjon
Dokumentasjon leveres i PDF. Tegninger leveres i DWG. Modeller leveres i IFC.
Krav nummer 2.1 gjelder alle tre formatene.
## 3 Prising
| Post | Beskrivelse | Enhet | Mengde |
| --- | --- | --- | --- |
| 1 | Riving av eksisterende dekke | m2 | 420 |
| 2 | Ny baerekonstruksjon | tonn | 38 |
| 3 | Tekniske installasjoner | RS | 1 |
Prisene fylles ut av tilbyder ved tilbudsfrist.
"""
DECLARED = (
"Prosjektbeskrivelse Testdokument",
"1 Omfang",
"2 Krav til dokumentasjon",
"3 Prising",
)
def _declared_headings(**kwargs: object) -> list[str]:
candidates = find_candidates(DOCUMENT, **{**DEFAULT, **kwargs}) # type: ignore[arg-type]
return [c.title for c in candidates if c.rule == RULE_HEADING]
def test_the_default_recovers_all_four_declared_headings() -> None:
assert _declared_headings() == list(DECLARED)
def test_the_priced_section_is_a_heading_and_not_a_table_block() -> None:
"""The specific cell: `## 3 Prising` keeps its own line and its own name."""
candidates = find_candidates(DOCUMENT, **DEFAULT)
prising = [c for c in candidates if c.title == "3 Prising"]
assert len(prising) == 1
assert prising[0].rule == RULE_HEADING
assert DOCUMENT[prising[0].start : prising[0].end].startswith("## 3 Prising")
def test_without_d1_the_cell_falls_back_to_three_of_four() -> None:
"""The known-negative: the pin names its own cause instead of being green
for an unstated reason. Without `--keep-table-heading` the heading is
orphaned and a table block carries its name from a line below it."""
assert _declared_headings(keep_table_heading=False) == list(DECLARED[:3])
candidates = find_candidates(DOCUMENT, **{**DEFAULT, "keep_table_heading": False})
carrier = [c for c in candidates if c.title == "3 Prising"]
assert len(carrier) == 1
assert carrier[0].rule == RULE_TABLE_BLOCK
assert not DOCUMENT[carrier[0].start : carrier[0].end].startswith("## 3 Prising")

153
tests/test_span_gaps.py Normal file
View file

@ -0,0 +1,153 @@
"""No character stands BETWEEN two concepts, or after the last one.
Round 7 closed the text ABOVE the first concept (`--first-span-from-zero`,
163 804 characters, 79 % of the whole gap) and named the remainder without
opening it: 43 631 characters, 2.51 % of the corpus, over 8 of the 32
documents that get a plan -- 26 041 between entries and 17 590 after the last.
Measured here, the remainder has ONE cause, not the two the shape suggests.
Every rule in `find_candidates` closes a span against the NEXT MARK, and three
separate steps then remove a mark after its neighbour's `end` was already
fixed against it:
* the **orphan check** drops a heading with no body, and the heading line
itself -- `## 3 Prising\n`, 14 characters -- is what the next candidate no
longer starts at. Measured with the fold off: **18 527 characters over 15 of
39 documents**;
* `fold_units` **clause 1** discards a contents run, and the text those
entries opened goes with them. It accounts for the remaining **7 514**
characters between entries;
* the same clause discarding the LAST run takes the tail with it. Measured:
with `unit_fold=False` the tail gap over the corpus is **0**, so all
**17 590** tail characters are clause 1's.
That is round 6's own principle -- the outline gate filters at ADMISSION,
"before spans close, so the text a removed mark opened is carried by the mark
above" -- not applied to the two steps that remove marks AFTER spans close.
This rule states it once, at the end, where every removal has already
happened: a candidate's span runs to the next candidate's start, and the last
one runs to the end of the text.
It adds NO boundary and removes none. A plan's entry count is identical either
way, which is the same property `--first-span-from-zero` has and the reason
both can be measured by coverage rather than by count.
"""
from __future__ import annotations
from pathlib import Path
from llm_ingestion_okf import cli
from llm_ingestion_okf.propose import find_candidates
ARMS = dict(outline_run=3, table_grid=True, unit_fold=True, first_span_from_zero=True)
#: The orphan check's gap: `## Tom` has no body, so it is dropped -- and the
#: seven characters of its own heading line belong to no segment.
ORPHAN = "## Forste\nInnhold under.\n## Tom\n## Andre\nInnhold under.\n"
#: Clause 1's gap: three page-numbered siblings are read as a contents list
#: and discarded, and the body under them goes too. 130 of 159 characters.
CONTENTS_TAIL = (
"## Innledning\nBrodtekst her.\n"
"## Kapittel en 3\nNoe innhold.\n"
"## Kapittel to 5\nMer innhold.\n"
"## Kapittel tre 9\nEn avsluttende brodtekst som ingen kandidat dekker.\n"
)
#: The known-negative: nothing is removed, so nothing is carried.
NO_GAP = "## Forste\nInnhold under.\n## Andre\nInnhold under.\n"
def _uncovered(text: str, **kwargs: object) -> int:
candidates = find_candidates(text, **ARMS, **kwargs) # type: ignore[arg-type]
if not candidates:
return len(text)
between = sum(
max(0, candidates[i].start - candidates[i - 1].end) for i in range(1, len(candidates))
)
return candidates[0].start + between + max(0, len(text) - candidates[-1].end)
def test_the_default_still_loses_the_orphaned_heading_line() -> None:
"""The defect, kept visible: the dropped mark's own line goes nowhere."""
assert _uncovered(ORPHAN) == len("## Tom\n")
def test_the_default_still_loses_the_tail_under_a_discarded_contents_run() -> None:
"""82 % of this document is in no segment, and none of it is a listing."""
assert _uncovered(CONTENTS_TAIL) == 130
def test_no_character_lies_between_two_concepts() -> None:
assert _uncovered(ORPHAN, close_span_gaps=True) == 0
candidates = find_candidates(ORPHAN, **ARMS, close_span_gaps=True)
carried = ORPHAN[candidates[0].start : candidates[0].end]
assert "## Tom" in carried, "the removed mark's text is carried by the mark ABOVE"
def test_no_character_lies_after_the_last_concept() -> None:
assert _uncovered(CONTENTS_TAIL, close_span_gaps=True) == 0
candidates = find_candidates(CONTENTS_TAIL, **ARMS, close_span_gaps=True)
assert CONTENTS_TAIL[candidates[-1].start : candidates[-1].end].endswith(
"En avsluttende brodtekst som ingen kandidat dekker.\n"
)
def test_the_rule_adds_and_removes_no_boundary() -> None:
"""Only `end` moves. Titles, rules and starts are identical."""
for text in (ORPHAN, CONTENTS_TAIL, NO_GAP):
plain = find_candidates(text, **ARMS)
closed = find_candidates(text, **ARMS, close_span_gaps=True)
assert [c.title for c in plain] == [c.title for c in closed]
assert [c.rule for c in plain] == [c.rule for c in closed]
assert [c.start for c in plain] == [c.start for c in closed]
def test_a_document_with_no_gap_is_untouched() -> None:
"""The known-negative, as IDENTICAL objects rather than an equal count."""
before = find_candidates(NO_GAP, **ARMS)
after = find_candidates(NO_GAP, **ARMS, close_span_gaps=True)
assert before == after
assert _uncovered(NO_GAP) == 0
def test_it_is_on_by_default_in_the_build_command() -> None:
assert cli.DEFAULT_CLOSE_SPAN_GAPS is True
def test_the_parser_carries_the_default_and_an_opt_out() -> None:
base = ["build", "src", "--bundle", "out", "--bundle-id", "x"]
assert cli.parse_args(base).close_span_gaps is True
assert cli.parse_args([*base, "--no-close-span-gaps"]).close_span_gaps is False
def test_the_flag_reaches_the_proposer_from_the_build_command(tmp_path: Path) -> None:
"""The plumbing, and it needs its own test rather than the smoke build.
Measured 2026-09-11: the operator's five-document folder has a coverage gap
of ZERO under round 7's default already, so building it with the rule on
and off gives `diff -rq` 0 differences. That is the rule behaving correctly
on an input with nothing to carry -- and it means that build proves nothing
about whether the flag arrives. This one uses a document that HAS a gap.
"""
inbox = tmp_path / "in"
inbox.mkdir()
(inbox / "d.md").write_text(CONTENTS_TAIL, encoding="utf-8")
def concepts(bundle: Path, **kwargs: object) -> str:
report = cli.build(
inbox,
bundle=bundle,
bundle_id="t",
okf_version="0.2",
ingested_at="1970-01-01T00:00:00Z",
**kwargs, # type: ignore[arg-type]
)
assert report.codes == ()
return "".join(path.read_text(encoding="utf-8") for path in sorted(bundle.rglob("*.md")))
on = concepts(tmp_path / "on")
off = concepts(tmp_path / "off", close_span_gaps=False)
assert "En avsluttende brodtekst som ingen kandidat dekker." in on
assert "En avsluttende brodtekst som ingen kandidat dekker." not in off