open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(cli,consume): the first span, and the fusion that punished fine-graining

Browse source 
Round 6 left two things behind: a measured coverage hole with the repair built
but not the number to move it, and a retrieval regression that kept two
spreadsheet rules off the default. Both are measured; four defaults move.

THE HOLE IS BIGGER THAN THE RULE BUILT FOR IT. Measured over the 39-document
corpus, the pre-move default left 207 435 characters -- 11.92 % -- in no
segment: 163 804 above the first entry, 26 041 between entries, 17 590 after
the last. `--first-span-from-zero` closes the first part entirely, 79 % of the
whole, leaving 43 631 (2.51 %) over 8 of 32 documents. It adds no boundary and
the K2 concept count is identical with and without it (425 = 425); hit@8 holds
[1,1,1,1,1,-] under both tie-breaks and the 12-position reference does not move
one cell.

THE RANKER WAS PUNISHING FINE-GRAINING FOR BEING FINE-GRAINED. Round 6 held
`--sheet-section-rows --keep-table-heading` back because on a bundle built with
them row 1 fell rank 1 -> 2, and ruled out the document prior because its rank
over 39 DOCUMENTS was 1 on both bundles. RRF reads a rank over CONCEPTS:
splitting the gold document 1 -> 12 puts its own twelve concepts in that
signal's whole top tie group, so the one leading the body signal takes position
11, contributing 1/71 where the undivided concept contributed 1/61. A signal
that scored them all EQUALLY still emitted twelve ranks, ordered by concept_id,
and the fusion read alphabetical order as a measurement.

AND THE REPAIR WAS ALREADY IN THE TREE, OFF FOR A COST THAT HAD BEEN REMOVED.
`--tie-shared-rank` shipped off 2026-09-08 because hit@8 fell 5 of 6 to 4 of 6.
Swept over 2 prior exponents x 3 bundles x 6 rows: that fall exists only at
DOCUMENT_PRIOR_EXPONENT 1.0. Round 6 moved the exponent to 0.5 for an unrelated
reason and correctly reported it moved no hit@8 row -- nobody measured the
pair. A flag's "off by measurement" is a measurement of a CONFIGURATION, not a
property of the flag.

Shipped, each with an explicit opt-out and both directions measured rather than
asserted: --no-first-span-from-zero --no-sheet-section-rows
--no-keep-table-heading reproduces the previous bundle byte for byte (diff -rq,
0 differences), and the no-flag build equals the explicit-flag build.
--no-tie-shared-rank reproduces the previous excerpt order -- the one change
here that alters a payload with no bundle changing.

K2 default: 436 concepts / 832 files, digest 8dff8a8e6c15d2f7..., hit@8 5 of 6
ranks [1,1,1,1,1,-]. The pin now holds its own known-negative on those bytes:
read with --no-tie-shared-rank the same bundle gives [2,1,1,1,1,-], so the
green assertion names its cause. Consumer cost is a re-run: 425/810 -> 436/832
on the reference corpus, 15/30 -> 26/52 on a five-document folder.

pptx and md measured end to end for the first time, on two hand-built
documents: md recovers 3 of 4 declared headings, pptx segments per slide only
where the converter recognised a title placeholder. A converter attribute leaks
into titles ({#slide-N}, {#sheet-1}) on 2 of 810 K2 files and 1 of 30 on a
five-document folder -- not fixed, because a filename is reduced from its title
and the fix renames concept ids a consumer has cited.

Suite 1486 (1478 before), ruff and mypy --strict clean over 21 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 15:40:01 +02:00
commit a364ef4c7d
11 changed files with 768 additions and 99 deletions
Download patch file Download diff file Expand all files Collapse all files

112
CLAUDE.md
View file

@ -27,6 +27,16 @@ one boundary rule:
fail-fast. The extra ships `pdfplumber` for `pdf` (chosen on ONE measured
property: it keeps a requirement table's label and value on the same line
where three alternatives do not); `docx`/`xlsx` still ship no parser.
`pptx` and `md` were measured end to end for the first time 2026-09-10
(`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md` § 4) on two
hand-built documents, which is more than zero and is not a fasit: `md`
recovers 3 of 4 declared headings, and `pptx` segments per slide only where
the deck's slides carry title placeholders the converter recognises — a deck
whose slides do not lands as ONE concept. A converter attribute also leaks
into concept titles (`{#slide-N}`, `{#sheet-1}`), reaching 2 of 810 files on
the K2 default bundle and 1 of 30 on the operator's test folder; because a
filename is reduced from its title, fixing it RENAMES concept ids a consumer
has already cited, so it is an operator question and not a patch.
Structured table recovery is **out of scope** — two independent parsers
return the same wrong shape, so the breakage is document geometry, not a
library choice. PDFs enter as prose, and drawn content (figures) does not
@ -266,12 +276,16 @@ and fixtures, never code.
collide); `--ingested-at` and `--proposed-at` default to one shared epoch
constant rather than the clock, because a wall-clock default takes
rebuild-equals-incremental away from anyone who omits them.
**Eight segmentation rules are REACHABLE here, and FIVE of them are ON by
default** -- `--outline-run 3`, `--table-grid` and `--unit-fold` since
2026-09-08, `--drop-wrapped-outline` and `--outline-gate` since 2026-09-09,
each with an explicit opt-out (`--outline-run 0`, `--no-table-grid`,
`--no-unit-fold`, `--keep-wrapped-outline`, `--no-outline-gate`) that
together reproduce the pre-move bytes. **The 2026-09-09 pair is one
**Eight segmentation rules are REACHABLE here, and since 2026-09-10 ALL
EIGHT are ON by default** -- `--outline-run 3`, `--table-grid` and
`--unit-fold` since 2026-09-08, `--drop-wrapped-outline` and
`--outline-gate` since 2026-09-09, `--sheet-section-rows`,
`--keep-table-heading` and `--first-span-from-zero` since 2026-09-10, each
with an explicit opt-out (`--outline-run 0`, `--no-table-grid`,
`--no-unit-fold`, `--keep-wrapped-outline`, `--no-outline-gate`,
`--no-sheet-section-rows`, `--no-keep-table-heading`,
`--no-first-span-from-zero`) that together reproduce the pre-move bytes --
measured, `diff -rq` 0 differences, not asserted. **The 2026-09-09 pair is one
decision and cannot be split**: the gate takes `pdf` from 2 of 8 to 5 of 8
and the pair takes it to 7 of 8 (the sheet 5 of 12 -> 10 of 12, `docx`
unchanged at 3 of 3). **The gate is G1+G2:** Arm D's RECOVERED headings are
@ -284,9 +298,17 @@ and fixtures, never code.
text, which is why only one of them shipped. **The bar it had to clear is
now the bar**: reference cells up AND hit@8 holding rank 1 on every row on
every bundle. `--sheet-section-rows --keep-table-heading` reaches 11 of 12
and does NOT ship, because on a K2 bundle built with it row 1 falls rank
1 -> 2 (the gold document goes 1 concept -> 12) -- measured under BOTH prior
exponents, so the sublinear prior does not buy it. Arm E joined a session after the other two, on a number measured
and SHIPPED 2026-09-10, after two rounds off. It was held back because on a
K2 bundle built with it row 1 fell rank 1 -> 2 (the gold document goes 1
concept -> 12), under both prior exponents. **That was never these rules'
defect and it is not a segmentation question**: RRF emits a distinct rank
for every concept in a signal that scored them all EQUALLY, so the gold
document's own twelve concepts fill the document-prior tie group and the one
leading the body signal takes position 11 instead of 1. The repair is the
reading side's `consume.DEFAULT_TIE_SHARED_RANK`, and with it every
acceptance condition holds at once. **The fusion was punishing fine-graining
for being fine-grained**, which put the segmentation side and the retrieval
side in competition over one number for two rounds. Arm E joined a session after the other two, on a number measured
AFTER the first move: without it Arm F's table clause has no joined table to
fold, and the shipped D+F default scored 2 of 12 with `docx` 0 of 3 against
the 5 of 12 the fold was published with. **The proposer's own defaults did NOT move** (`propose.py`'s rules stay
@ -295,16 +317,22 @@ and fixtures, never code.
`cli.DEFAULT_UNIT_FOLD` say where. The cost to a consumer is a re-run and it
is not small: the 43-document reference corpus goes 629 concepts / 1108 files
(the delivered 2026-09-03 tree) to 492 / 944 after the 2026-09-08 move and to
**425 / 810** after the 2026-09-09 one (`bdf4977ca5a443c4...`, default flags,
default epoch stamp, measured on `b01492b` + this round). Digests published
425 / 810 after the 2026-09-09 one (`bdf4977ca5a443c4...`) and to
**436 / 832** after the 2026-09-10 one (`8dff8a8e6c15d2f7...`, default flags,
default epoch stamp, measured on `38104b7` + this round). On the operator's
own five-document folder the last move is 15 concepts / 30 files -> 26 / 52. Digests published
before 2026-09-09 were computed with a path-DEPENDENT command and are not
comparable to this one; the reproducible form is `find . -type f | sort |
xargs shasum -a 256 | shasum -a 256` from inside the bundle, under which the
previous default is `862116da16e422f6...`. The pinned artifact lives at
`~/corpora/okf-telling-20260829/K2-bundle-default-20260909` and
`~/corpora/okf-telling-20260829/K2-bundle-default-20260910` and
`tests/test_default_bundle_pin.py` holds its concept count AND its per-row
hit@8 ranks -- the count alone survived a configuration that lost a rank,
which is how the previous round's regression hid. **And the number the
which is how a previous round's regression hid. Since 2026-09-10 it also
holds the KNOWN-NEGATIVE on the same bytes: read with
`--no-tie-shared-rank`, the shipped default bundle reproduces the very fall
the rules were held back for, so the pin names its own cause instead of
being green for an unstated reason. **And the number the
decision cites belongs to another configuration:** Arm F's 5 of 12 was
measured with `--table-grid` ON; without it the same sample scores 2 of 12
and `docx` 0 of 3, because the fold's table clause has no joined table to
@ -312,12 +340,18 @@ and fixtures, never code.
`--unit-fold` (Arm F), `--keep-table-heading` (D1), `--sheet-section-rows`
and `--drop-wrapped-outline` (both D3), `--outline-gate` (G1+G2) and
`--first-span-from-zero`, each passed to the proposer unchanged. That last
one is OFF and is not a segmentation rule at all -- it adds no boundary. It
repairs a measured loss the round found while chasing one position's 940
characters: **32 of the 32** documents that get a plan leave the text above
their first concept in NO segment, 159 704 characters, **9.18 %** of the
corpus, 45 841 from one document. It is off only because moving it changes
every bundle's first span and no hit@8 number stands behind it yet. Until that day the build path called the proposer with no
one is ON since 2026-09-10 and is not a segmentation rule at all -- it adds
no boundary, and the K2 concept count is identical with and without it
(425 = 425 on the 2026-09-09 default). It repairs a measured loss: **32 of
the 32** documents that get a plan left the text above their first concept
in NO segment. **The hole is bigger than that rule, and this is the number
to carry:** measured 2026-09-10, the pre-move default left **207 435
characters, 11.92 %** of the corpus in no segment -- 163 804 above the first
entry, 26 041 BETWEEN entries, 17 590 after the last. The rule closes the
first part entirely and 79 % of the whole; **43 631 characters, 2.51 %, over
8 of 32 documents remain**, and the between-part has a named mechanism (a
`rule:table-block` candidate displacing a DECLARED heading and opening below
it). Neither remainder is a ceiling; both are in STATE with their numbers. Until that day the build path called the proposer with no
arm flag at all, so a tender PDF that Arm D splits into nine concepts landed
as one -- a build path a full arm behind the proposer. Exposing them was not
the same decision as moving one, and the two were taken a session apart:
@ -440,8 +474,11 @@ and fixtures, never code.
measurement behind it, including the control that FAILED, is
`docs/2026-09-07-okf-konsumskill-maaling.md`. **The ranking is this
repository's own choice** — the contract binds a payload, not a retrieval
algorithm (§ 10) — and it has FOUR optional widenings, all **off by default**
and all keeping the default payload byte-identical. `--cost-vocabulary`: a
algorithm (§ 10) — and it has FOUR optional widenings. Three are **off by
default** and keep the default payload byte-identical; the fourth
(`--tie-shared-rank`) became the default 2026-09-10 and is the one change in
this repository that alters a payload with NO bundle changing, so a consumer
pinned to the old excerpt order needs `--no-tie-shared-rank`. `--cost-vocabulary`: a
declared cost/price/quantity vocabulary family that bridges a question and a
document naming money with different words, gated on the QUESTION carrying
such a term, so a question without one is byte-identical either way. It moves
@ -470,7 +507,9 @@ and fixtures, never code.
read as 135-of-446 common, and RRF consumes RANKS, so no weighting inside a
signal can move a gold that already leads it.
`docs/2026-09-08-sjeldenhetsvekt.md`.
`--tie-shared-rank` is the fourth, and it is a correction to the TIE-BREAK
`--tie-shared-rank` is the fourth and **the only one that is now ON**
(2026-09-10, opt-out `--no-tie-shared-rank`). It is a correction to the
TIE-BREAK
rather than a weight: RRF ranks every concept in every signal, including a
signal that scored them all the same, and the declared `(-score, concept_id)`
tie-break then orders that group by id. Measured on N500, whose document
@ -478,13 +517,28 @@ and fixtures, never code.
alphabetical UUID order and put a concept answering 7 of 7 question tokens at
fused rank 14 — outside the cut — behind concepts sharing only `tunnel` and
`vann`. Under shared ranks it is rank 3 and 2 of the 16 covering concepts are
delivered. Off by default BY MEASUREMENT: the three requirement lookups hold
at rank 1 and the K2 digest holds, but hit@8 over the six published questions
falls 5 of 6 to 4 of 6, because K2's prior is coarse rather than degenerate
(6 values over 39 documents) and one gold sat early in its tie group — a
benefit that was never a measurement, but a published row all the same.
`docs/2026-09-08-rangeringsbom-sammensatte-ord.md`.
A FIFTH flag is not a ranking widening and is listed apart: `--withheld-titles`
delivered. **It shipped OFF on a measurement that was CONDITIONAL and stopped
being true in a commit reported as changing nothing.** The published cost —
hit@8 falling 5 of 6 to 4 of 6 — is real only at `DOCUMENT_PRIOR_EXPONENT`
1.0. Round 6 moved that exponent to 0.5 for an unrelated reason and correctly
reported it moved no hit@8 row; nobody measured the PAIR. Swept 2026-09-10
over 2 exponents x 3 bundles x 6 rows: at 0.5 the rule holds
`[1,1,1,1,1,–]` on all three bundles and FIXES the split bundle's row 1
(2 -> 1), which is what let `--sheet-section-rows --keep-table-heading`
become a build default. **A flag's "off by measurement" is a measurement of a
CONFIGURATION, not a property of the flag** — when a constant it interacts
with moves, its default is unmeasured again, and nothing in the tree says so
because the two decisions live in different files. The adverse case is
recorded rather than hidden: on a synthetic 30-concept fixture where one
signal separates and two do not, shared ranks move a gold from rank 18 to 30
(`tests/test_okf_consume.py`). Note also that
`docs/2026-09-08-sjeldenhetsvekt.md`'s figures were measured under the older
tie-break and are NOT re-measured — on one fixture the change takes the
weight's gold from fused rank 18 to 1.
`docs/2026-09-08-rangeringsbom-sammensatte-ord.md` and
`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`.
The other three stay off. A FIFTH flag is not a ranking widening and is
listed apart: `--withheld-titles`
gives each `withheld` entry the concept's `title`, so a reader can see WHAT
was withheld without reading the bundle (§ 2.2 forbids going to look). The
code is 11 lines; the bytes are the reason it is off. Measured, it grows an