Kjell Tore Guttormsen
a364ef4c7d
Round 6 left two things behind: a measured coverage hole with the repair built
but not the number to move it, and a retrieval regression that kept two
spreadsheet rules off the default. Both are measured; four defaults move.
THE HOLE IS BIGGER THAN THE RULE BUILT FOR IT. Measured over the 39-document
corpus, the pre-move default left 207 435 characters -- 11.92 % -- in no
segment: 163 804 above the first entry, 26 041 between entries, 17 590 after
the last. `--first-span-from-zero` closes the first part entirely, 79 % of the
whole, leaving 43 631 (2.51 %) over 8 of 32 documents. It adds no boundary and
the K2 concept count is identical with and without it (425 = 425); hit@8 holds
[1,1,1,1,1,-] under both tie-breaks and the 12-position reference does not move
one cell.
THE RANKER WAS PUNISHING FINE-GRAINING FOR BEING FINE-GRAINED. Round 6 held
`--sheet-section-rows --keep-table-heading` back because on a bundle built with
them row 1 fell rank 1 -> 2, and ruled out the document prior because its rank
over 39 DOCUMENTS was 1 on both bundles. RRF reads a rank over CONCEPTS:
splitting the gold document 1 -> 12 puts its own twelve concepts in that
signal's whole top tie group, so the one leading the body signal takes position
11, contributing 1/71 where the undivided concept contributed 1/61. A signal
that scored them all EQUALLY still emitted twelve ranks, ordered by concept_id,
and the fusion read alphabetical order as a measurement.
AND THE REPAIR WAS ALREADY IN THE TREE, OFF FOR A COST THAT HAD BEEN REMOVED.
`--tie-shared-rank` shipped off 2026-09-08 because hit@8 fell 5 of 6 to 4 of 6.
Swept over 2 prior exponents x 3 bundles x 6 rows: that fall exists only at
DOCUMENT_PRIOR_EXPONENT 1.0. Round 6 moved the exponent to 0.5 for an unrelated
reason and correctly reported it moved no hit@8 row -- nobody measured the
pair. A flag's "off by measurement" is a measurement of a CONFIGURATION, not a
property of the flag.
Shipped, each with an explicit opt-out and both directions measured rather than
asserted: --no-first-span-from-zero --no-sheet-section-rows
--no-keep-table-heading reproduces the previous bundle byte for byte (diff -rq,
0 differences), and the no-flag build equals the explicit-flag build.
--no-tie-shared-rank reproduces the previous excerpt order -- the one change
here that alters a payload with no bundle changing.
K2 default: 436 concepts / 832 files, digest 8dff8a8e6c15d2f7..., hit@8 5 of 6
ranks [1,1,1,1,1,-]. The pin now holds its own known-negative on those bytes:
read with --no-tie-shared-rank the same bundle gives [2,1,1,1,1,-], so the
green assertion names its cause. Consumer cost is a re-run: 425/810 -> 436/832
on the reference corpus, 15/30 -> 26/52 on a five-document folder.
pptx and md measured end to end for the first time, on two hand-built
documents: md recovers 3 of 4 declared headings, pptx segments per slide only
where the converter recognised a title placeholder. A converter attribute leaks
into titles ({#slide-N}, {#sheet-1}) on 2 of 810 K2 files and 1 of 30 on a
five-document folder -- not fixed, because a filename is reduced from its title
and the fix renames concept ids a consumer has cited.
Suite 1486 (1478 before), ruff and mypy --strict clean over 21 files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
77 lines
3.3 KiB
Python
77 lines
3.3 KiB
Python
"""No character stands before the first concept.
|
|
|
|
Round 3 recorded it as a footnote about ONE position: under
|
|
`--drop-wrapped-outline` that document's surviving concept starts at line 18 of
|
|
302, so 940 characters of the note's opening sit in no segment. Measured across
|
|
the whole 39-document corpus it is not one position -- it is **32 of the 32
|
|
documents that get a plan at all, 159 704 characters, 9.18 % of the corpus**,
|
|
with 45 841 characters lost from a single document. A concept boundary is a
|
|
statement about where a unit BEGINS; it was never a statement that the text
|
|
above the first one belongs nowhere.
|
|
|
|
The rule: when nothing precedes the first concept, the first concept starts at
|
|
0. It adds no boundary and removes none -- only the first span's `start` moves
|
|
-- so a plan's entry count is identical either way.
|
|
|
|
OFF by default, and the reason is the round's own acceptance criterion rather
|
|
than doubt about the defect: it moves the first span of essentially every
|
|
segmented bundle, and the round budget held one K2 build, which measured the
|
|
outline gate. A default that changes every bundle's first concept without a
|
|
hit@8 measurement behind it would be exactly the move this round refused to
|
|
make elsewhere. The measured cost and the outstanding measurement are in
|
|
`docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md`.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from llm_ingestion_okf import cli
|
|
from llm_ingestion_okf.propose import find_candidates
|
|
|
|
ARMS = dict(outline_run=3, table_grid=True, unit_fold=True)
|
|
|
|
WITH_PREAMBLE = (
|
|
"En tittelside og et forord som ingen overskrift star foran.\n"
|
|
"Enda en linje av det samme.\n"
|
|
"\n"
|
|
"## Forste avsnitt\nInnhold under.\n"
|
|
"## Andre avsnitt\nInnhold under.\n"
|
|
)
|
|
|
|
NO_PREAMBLE = "## Forste avsnitt\nInnhold under.\n## Andre avsnitt\nInnhold under.\n"
|
|
|
|
|
|
def test_the_default_still_loses_the_preamble() -> None:
|
|
"""The defect, kept as a test so the flag's reason stays visible."""
|
|
first = find_candidates(WITH_PREAMBLE, **ARMS)[0]
|
|
assert first.start > 0
|
|
|
|
|
|
def test_the_first_concept_starts_at_zero() -> None:
|
|
candidates = find_candidates(WITH_PREAMBLE, **ARMS, first_span_from_zero=True)
|
|
assert candidates[0].start == 0
|
|
assert WITH_PREAMBLE[candidates[0].start : candidates[0].end].startswith("En tittelside")
|
|
|
|
|
|
def test_a_document_with_no_preamble_is_untouched() -> None:
|
|
"""The known-negative, as IDENTICAL objects rather than an equal count."""
|
|
before = find_candidates(NO_PREAMBLE, **ARMS)
|
|
after = find_candidates(NO_PREAMBLE, **ARMS, first_span_from_zero=True)
|
|
assert before == after
|
|
assert before[0].start == 0
|
|
|
|
|
|
def test_the_rule_adds_and_removes_no_boundary() -> None:
|
|
plain = find_candidates(WITH_PREAMBLE, **ARMS)
|
|
moved = find_candidates(WITH_PREAMBLE, **ARMS, first_span_from_zero=True)
|
|
assert [c.title for c in plain] == [c.title for c in moved]
|
|
assert [c.end for c in plain] == [c.end for c in moved]
|
|
assert [c.start for c in plain][1:] == [c.start for c in moved][1:]
|
|
|
|
|
|
def test_it_is_on_by_default_in_the_build_command() -> None:
|
|
"""Moved 2026-09-10, once the hit@8 number round 6 lacked was measured.
|
|
|
|
The opt-out is asserted in `tests/test_round7_defaults.py`; here the point
|
|
is only that the rule below is what a caller gets without saying anything.
|
|
"""
|
|
assert cli.DEFAULT_FIRST_SPAN_FROM_ZERO is True
|