feat(cli,consume): the first span, and the fusion that punished fine-graining

Round 6 left two things behind: a measured coverage hole with the repair built
but not the number to move it, and a retrieval regression that kept two
spreadsheet rules off the default. Both are measured; four defaults move.

THE HOLE IS BIGGER THAN THE RULE BUILT FOR IT. Measured over the 39-document
corpus, the pre-move default left 207 435 characters -- 11.92 % -- in no
segment: 163 804 above the first entry, 26 041 between entries, 17 590 after
the last. `--first-span-from-zero` closes the first part entirely, 79 % of the
whole, leaving 43 631 (2.51 %) over 8 of 32 documents. It adds no boundary and
the K2 concept count is identical with and without it (425 = 425); hit@8 holds
[1,1,1,1,1,-] under both tie-breaks and the 12-position reference does not move
one cell.

THE RANKER WAS PUNISHING FINE-GRAINING FOR BEING FINE-GRAINED. Round 6 held
`--sheet-section-rows --keep-table-heading` back because on a bundle built with
them row 1 fell rank 1 -> 2, and ruled out the document prior because its rank
over 39 DOCUMENTS was 1 on both bundles. RRF reads a rank over CONCEPTS:
splitting the gold document 1 -> 12 puts its own twelve concepts in that
signal's whole top tie group, so the one leading the body signal takes position
11, contributing 1/71 where the undivided concept contributed 1/61. A signal
that scored them all EQUALLY still emitted twelve ranks, ordered by concept_id,
and the fusion read alphabetical order as a measurement.

AND THE REPAIR WAS ALREADY IN THE TREE, OFF FOR A COST THAT HAD BEEN REMOVED.
`--tie-shared-rank` shipped off 2026-09-08 because hit@8 fell 5 of 6 to 4 of 6.
Swept over 2 prior exponents x 3 bundles x 6 rows: that fall exists only at
DOCUMENT_PRIOR_EXPONENT 1.0. Round 6 moved the exponent to 0.5 for an unrelated
reason and correctly reported it moved no hit@8 row -- nobody measured the
pair. A flag's "off by measurement" is a measurement of a CONFIGURATION, not a
property of the flag.

Shipped, each with an explicit opt-out and both directions measured rather than
asserted: --no-first-span-from-zero --no-sheet-section-rows
--no-keep-table-heading reproduces the previous bundle byte for byte (diff -rq,
0 differences), and the no-flag build equals the explicit-flag build.
--no-tie-shared-rank reproduces the previous excerpt order -- the one change
here that alters a payload with no bundle changing.

K2 default: 436 concepts / 832 files, digest 8dff8a8e6c15d2f7..., hit@8 5 of 6
ranks [1,1,1,1,1,-]. The pin now holds its own known-negative on those bytes:
read with --no-tie-shared-rank the same bundle gives [2,1,1,1,1,-], so the
green assertion names its cause. Consumer cost is a re-run: 425/810 -> 436/832
on the reference corpus, 15/30 -> 26/52 on a five-document folder.

pptx and md measured end to end for the first time, on two hand-built
documents: md recovers 3 of 4 declared headings, pptx segments per slide only
where the converter recognised a title placeholder. A converter attribute leaks
into titles ({#slide-N}, {#sheet-1}) on 2 of 810 K2 files and 1 of 30 on a
five-document folder -- not fixed, because a filename is reduced from its title
and the fix renames concept ids a consumer has cited.

Suite 1486 (1478 before), ruff and mypy --strict clean over 21 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 15:40:01 +02:00
commit a364ef4c7d
11 changed files with 768 additions and 99 deletions

112
CLAUDE.md
View file

@ -27,6 +27,16 @@ one boundary rule:
fail-fast. The extra ships `pdfplumber` for `pdf` (chosen on ONE measured fail-fast. The extra ships `pdfplumber` for `pdf` (chosen on ONE measured
property: it keeps a requirement table's label and value on the same line property: it keeps a requirement table's label and value on the same line
where three alternatives do not); `docx`/`xlsx` still ship no parser. where three alternatives do not); `docx`/`xlsx` still ship no parser.
`pptx` and `md` were measured end to end for the first time 2026-09-10
(`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md` § 4) on two
hand-built documents, which is more than zero and is not a fasit: `md`
recovers 3 of 4 declared headings, and `pptx` segments per slide only where
the deck's slides carry title placeholders the converter recognises — a deck
whose slides do not lands as ONE concept. A converter attribute also leaks
into concept titles (`{#slide-N}`, `{#sheet-1}`), reaching 2 of 810 files on
the K2 default bundle and 1 of 30 on the operator's test folder; because a
filename is reduced from its title, fixing it RENAMES concept ids a consumer
has already cited, so it is an operator question and not a patch.
Structured table recovery is **out of scope** — two independent parsers Structured table recovery is **out of scope** — two independent parsers
return the same wrong shape, so the breakage is document geometry, not a return the same wrong shape, so the breakage is document geometry, not a
library choice. PDFs enter as prose, and drawn content (figures) does not library choice. PDFs enter as prose, and drawn content (figures) does not
@ -266,12 +276,16 @@ and fixtures, never code.
collide); `--ingested-at` and `--proposed-at` default to one shared epoch collide); `--ingested-at` and `--proposed-at` default to one shared epoch
constant rather than the clock, because a wall-clock default takes constant rather than the clock, because a wall-clock default takes
rebuild-equals-incremental away from anyone who omits them. rebuild-equals-incremental away from anyone who omits them.
**Eight segmentation rules are REACHABLE here, and FIVE of them are ON by **Eight segmentation rules are REACHABLE here, and since 2026-09-10 ALL
default** -- `--outline-run 3`, `--table-grid` and `--unit-fold` since EIGHT are ON by default** -- `--outline-run 3`, `--table-grid` and
2026-09-08, `--drop-wrapped-outline` and `--outline-gate` since 2026-09-09, `--unit-fold` since 2026-09-08, `--drop-wrapped-outline` and
each with an explicit opt-out (`--outline-run 0`, `--no-table-grid`, `--outline-gate` since 2026-09-09, `--sheet-section-rows`,
`--no-unit-fold`, `--keep-wrapped-outline`, `--no-outline-gate`) that `--keep-table-heading` and `--first-span-from-zero` since 2026-09-10, each
together reproduce the pre-move bytes. **The 2026-09-09 pair is one with an explicit opt-out (`--outline-run 0`, `--no-table-grid`,
`--no-unit-fold`, `--keep-wrapped-outline`, `--no-outline-gate`,
`--no-sheet-section-rows`, `--no-keep-table-heading`,
`--no-first-span-from-zero`) that together reproduce the pre-move bytes --
measured, `diff -rq` 0 differences, not asserted. **The 2026-09-09 pair is one
decision and cannot be split**: the gate takes `pdf` from 2 of 8 to 5 of 8 decision and cannot be split**: the gate takes `pdf` from 2 of 8 to 5 of 8
and the pair takes it to 7 of 8 (the sheet 5 of 12 -> 10 of 12, `docx` and the pair takes it to 7 of 8 (the sheet 5 of 12 -> 10 of 12, `docx`
unchanged at 3 of 3). **The gate is G1+G2:** Arm D's RECOVERED headings are unchanged at 3 of 3). **The gate is G1+G2:** Arm D's RECOVERED headings are
@ -284,9 +298,17 @@ and fixtures, never code.
text, which is why only one of them shipped. **The bar it had to clear is text, which is why only one of them shipped. **The bar it had to clear is
now the bar**: reference cells up AND hit@8 holding rank 1 on every row on now the bar**: reference cells up AND hit@8 holding rank 1 on every row on
every bundle. `--sheet-section-rows --keep-table-heading` reaches 11 of 12 every bundle. `--sheet-section-rows --keep-table-heading` reaches 11 of 12
and does NOT ship, because on a K2 bundle built with it row 1 falls rank and SHIPPED 2026-09-10, after two rounds off. It was held back because on a
1 -> 2 (the gold document goes 1 concept -> 12) -- measured under BOTH prior K2 bundle built with it row 1 fell rank 1 -> 2 (the gold document goes 1
exponents, so the sublinear prior does not buy it. Arm E joined a session after the other two, on a number measured concept -> 12), under both prior exponents. **That was never these rules'
defect and it is not a segmentation question**: RRF emits a distinct rank
for every concept in a signal that scored them all EQUALLY, so the gold
document's own twelve concepts fill the document-prior tie group and the one
leading the body signal takes position 11 instead of 1. The repair is the
reading side's `consume.DEFAULT_TIE_SHARED_RANK`, and with it every
acceptance condition holds at once. **The fusion was punishing fine-graining
for being fine-grained**, which put the segmentation side and the retrieval
side in competition over one number for two rounds. Arm E joined a session after the other two, on a number measured
AFTER the first move: without it Arm F's table clause has no joined table to AFTER the first move: without it Arm F's table clause has no joined table to
fold, and the shipped D+F default scored 2 of 12 with `docx` 0 of 3 against fold, and the shipped D+F default scored 2 of 12 with `docx` 0 of 3 against
the 5 of 12 the fold was published with. **The proposer's own defaults did NOT move** (`propose.py`'s rules stay the 5 of 12 the fold was published with. **The proposer's own defaults did NOT move** (`propose.py`'s rules stay
@ -295,16 +317,22 @@ and fixtures, never code.
`cli.DEFAULT_UNIT_FOLD` say where. The cost to a consumer is a re-run and it `cli.DEFAULT_UNIT_FOLD` say where. The cost to a consumer is a re-run and it
is not small: the 43-document reference corpus goes 629 concepts / 1108 files is not small: the 43-document reference corpus goes 629 concepts / 1108 files
(the delivered 2026-09-03 tree) to 492 / 944 after the 2026-09-08 move and to (the delivered 2026-09-03 tree) to 492 / 944 after the 2026-09-08 move and to
**425 / 810** after the 2026-09-09 one (`bdf4977ca5a443c4...`, default flags, 425 / 810 after the 2026-09-09 one (`bdf4977ca5a443c4...`) and to
default epoch stamp, measured on `b01492b` + this round). Digests published **436 / 832** after the 2026-09-10 one (`8dff8a8e6c15d2f7...`, default flags,
default epoch stamp, measured on `38104b7` + this round). On the operator's
own five-document folder the last move is 15 concepts / 30 files -> 26 / 52. Digests published
before 2026-09-09 were computed with a path-DEPENDENT command and are not before 2026-09-09 were computed with a path-DEPENDENT command and are not
comparable to this one; the reproducible form is `find . -type f | sort | comparable to this one; the reproducible form is `find . -type f | sort |
xargs shasum -a 256 | shasum -a 256` from inside the bundle, under which the xargs shasum -a 256 | shasum -a 256` from inside the bundle, under which the
previous default is `862116da16e422f6...`. The pinned artifact lives at previous default is `862116da16e422f6...`. The pinned artifact lives at
`~/corpora/okf-telling-20260829/K2-bundle-default-20260909` and `~/corpora/okf-telling-20260829/K2-bundle-default-20260910` and
`tests/test_default_bundle_pin.py` holds its concept count AND its per-row `tests/test_default_bundle_pin.py` holds its concept count AND its per-row
hit@8 ranks -- the count alone survived a configuration that lost a rank, hit@8 ranks -- the count alone survived a configuration that lost a rank,
which is how the previous round's regression hid. **And the number the which is how a previous round's regression hid. Since 2026-09-10 it also
holds the KNOWN-NEGATIVE on the same bytes: read with
`--no-tie-shared-rank`, the shipped default bundle reproduces the very fall
the rules were held back for, so the pin names its own cause instead of
being green for an unstated reason. **And the number the
decision cites belongs to another configuration:** Arm F's 5 of 12 was decision cites belongs to another configuration:** Arm F's 5 of 12 was
measured with `--table-grid` ON; without it the same sample scores 2 of 12 measured with `--table-grid` ON; without it the same sample scores 2 of 12
and `docx` 0 of 3, because the fold's table clause has no joined table to and `docx` 0 of 3, because the fold's table clause has no joined table to
@ -312,12 +340,18 @@ and fixtures, never code.
`--unit-fold` (Arm F), `--keep-table-heading` (D1), `--sheet-section-rows` `--unit-fold` (Arm F), `--keep-table-heading` (D1), `--sheet-section-rows`
and `--drop-wrapped-outline` (both D3), `--outline-gate` (G1+G2) and and `--drop-wrapped-outline` (both D3), `--outline-gate` (G1+G2) and
`--first-span-from-zero`, each passed to the proposer unchanged. That last `--first-span-from-zero`, each passed to the proposer unchanged. That last
one is OFF and is not a segmentation rule at all -- it adds no boundary. It one is ON since 2026-09-10 and is not a segmentation rule at all -- it adds
repairs a measured loss the round found while chasing one position's 940 no boundary, and the K2 concept count is identical with and without it
characters: **32 of the 32** documents that get a plan leave the text above (425 = 425 on the 2026-09-09 default). It repairs a measured loss: **32 of
their first concept in NO segment, 159 704 characters, **9.18 %** of the the 32** documents that get a plan left the text above their first concept
corpus, 45 841 from one document. It is off only because moving it changes in NO segment. **The hole is bigger than that rule, and this is the number
every bundle's first span and no hit@8 number stands behind it yet. Until that day the build path called the proposer with no to carry:** measured 2026-09-10, the pre-move default left **207 435
characters, 11.92 %** of the corpus in no segment -- 163 804 above the first
entry, 26 041 BETWEEN entries, 17 590 after the last. The rule closes the
first part entirely and 79 % of the whole; **43 631 characters, 2.51 %, over
8 of 32 documents remain**, and the between-part has a named mechanism (a
`rule:table-block` candidate displacing a DECLARED heading and opening below
it). Neither remainder is a ceiling; both are in STATE with their numbers. Until that day the build path called the proposer with no
arm flag at all, so a tender PDF that Arm D splits into nine concepts landed arm flag at all, so a tender PDF that Arm D splits into nine concepts landed
as one -- a build path a full arm behind the proposer. Exposing them was not as one -- a build path a full arm behind the proposer. Exposing them was not
the same decision as moving one, and the two were taken a session apart: the same decision as moving one, and the two were taken a session apart:
@ -440,8 +474,11 @@ and fixtures, never code.
measurement behind it, including the control that FAILED, is measurement behind it, including the control that FAILED, is
`docs/2026-09-07-okf-konsumskill-maaling.md`. **The ranking is this `docs/2026-09-07-okf-konsumskill-maaling.md`. **The ranking is this
repository's own choice** — the contract binds a payload, not a retrieval repository's own choice** — the contract binds a payload, not a retrieval
algorithm (§ 10) — and it has FOUR optional widenings, all **off by default** algorithm (§ 10) — and it has FOUR optional widenings. Three are **off by
and all keeping the default payload byte-identical. `--cost-vocabulary`: a default** and keep the default payload byte-identical; the fourth
(`--tie-shared-rank`) became the default 2026-09-10 and is the one change in
this repository that alters a payload with NO bundle changing, so a consumer
pinned to the old excerpt order needs `--no-tie-shared-rank`. `--cost-vocabulary`: a
declared cost/price/quantity vocabulary family that bridges a question and a declared cost/price/quantity vocabulary family that bridges a question and a
document naming money with different words, gated on the QUESTION carrying document naming money with different words, gated on the QUESTION carrying
such a term, so a question without one is byte-identical either way. It moves such a term, so a question without one is byte-identical either way. It moves
@ -470,7 +507,9 @@ and fixtures, never code.
read as 135-of-446 common, and RRF consumes RANKS, so no weighting inside a read as 135-of-446 common, and RRF consumes RANKS, so no weighting inside a
signal can move a gold that already leads it. signal can move a gold that already leads it.
`docs/2026-09-08-sjeldenhetsvekt.md`. `docs/2026-09-08-sjeldenhetsvekt.md`.
`--tie-shared-rank` is the fourth, and it is a correction to the TIE-BREAK `--tie-shared-rank` is the fourth and **the only one that is now ON**
(2026-09-10, opt-out `--no-tie-shared-rank`). It is a correction to the
TIE-BREAK
rather than a weight: RRF ranks every concept in every signal, including a rather than a weight: RRF ranks every concept in every signal, including a
signal that scored them all the same, and the declared `(-score, concept_id)` signal that scored them all the same, and the declared `(-score, concept_id)`
tie-break then orders that group by id. Measured on N500, whose document tie-break then orders that group by id. Measured on N500, whose document
@ -478,13 +517,28 @@ and fixtures, never code.
alphabetical UUID order and put a concept answering 7 of 7 question tokens at alphabetical UUID order and put a concept answering 7 of 7 question tokens at
fused rank 14 — outside the cut — behind concepts sharing only `tunnel` and fused rank 14 — outside the cut — behind concepts sharing only `tunnel` and
`vann`. Under shared ranks it is rank 3 and 2 of the 16 covering concepts are `vann`. Under shared ranks it is rank 3 and 2 of the 16 covering concepts are
delivered. Off by default BY MEASUREMENT: the three requirement lookups hold delivered. **It shipped OFF on a measurement that was CONDITIONAL and stopped
at rank 1 and the K2 digest holds, but hit@8 over the six published questions being true in a commit reported as changing nothing.** The published cost —
falls 5 of 6 to 4 of 6, because K2's prior is coarse rather than degenerate hit@8 falling 5 of 6 to 4 of 6 — is real only at `DOCUMENT_PRIOR_EXPONENT`
(6 values over 39 documents) and one gold sat early in its tie group — a 1.0. Round 6 moved that exponent to 0.5 for an unrelated reason and correctly
benefit that was never a measurement, but a published row all the same. reported it moved no hit@8 row; nobody measured the PAIR. Swept 2026-09-10
`docs/2026-09-08-rangeringsbom-sammensatte-ord.md`. over 2 exponents x 3 bundles x 6 rows: at 0.5 the rule holds
A FIFTH flag is not a ranking widening and is listed apart: `--withheld-titles` `[1,1,1,1,1,]` on all three bundles and FIXES the split bundle's row 1
(2 -> 1), which is what let `--sheet-section-rows --keep-table-heading`
become a build default. **A flag's "off by measurement" is a measurement of a
CONFIGURATION, not a property of the flag** — when a constant it interacts
with moves, its default is unmeasured again, and nothing in the tree says so
because the two decisions live in different files. The adverse case is
recorded rather than hidden: on a synthetic 30-concept fixture where one
signal separates and two do not, shared ranks move a gold from rank 18 to 30
(`tests/test_okf_consume.py`). Note also that
`docs/2026-09-08-sjeldenhetsvekt.md`'s figures were measured under the older
tie-break and are NOT re-measured — on one fixture the change takes the
weight's gold from fused rank 18 to 1.
`docs/2026-09-08-rangeringsbom-sammensatte-ord.md` and
`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`.
The other three stay off. A FIFTH flag is not a ranking widening and is
listed apart: `--withheld-titles`
gives each `withheld` entry the concept's `title`, so a reader can see WHAT gives each `withheld` entry the concept's `title`, so a reader can see WHAT
was withheld without reading the bundle (§ 2.2 forbids going to look). The was withheld without reading the bundle (§ 2.2 forbids going to look). The
code is 11 lines; the bytes are the reason it is off. Measured, it grows an code is 11 lines; the bytes are the reason it is off. Measured, it grows an

View file

@ -88,19 +88,25 @@ did not pass them.
### The segmentation flags ### The segmentation flags
Eight rules are reachable from `okf build`. **Five of them are ON by default** Eight rules are reachable from `okf build`, and since 2026-09-10 **all eight
`--outline-run 3`, `--table-grid` and `--unit-fold` since 2026-09-08, and are ON by default** — `--outline-run 3`, `--table-grid` and `--unit-fold` since
`--drop-wrapped-outline` and `--outline-gate` since 2026-09-09 — each an 2026-09-08, `--drop-wrapped-outline` and `--outline-gate` since 2026-09-09, and
operator decision, and each with an explicit opt-out: `--outline-run 0`, `--sheet-section-rows`, `--keep-table-heading` and `--first-span-from-zero`
`--no-table-grid`, `--no-unit-fold`, `--keep-wrapped-outline`, since 2026-09-10 — each an operator decision, and each with an explicit
`--no-outline-gate`. Passing all five reproduces the pre-2026-09-08 bytes opt-out: `--outline-run 0`, `--no-table-grid`, `--no-unit-fold`,
exactly. The other three are off. Each line below carries the number it was `--keep-wrapped-outline`, `--no-outline-gate`, `--no-sheet-section-rows`,
measured at, and nothing beyond it. `--no-keep-table-heading`, `--no-first-span-from-zero`. Passing all eight
reproduces the pre-2026-09-08 bytes exactly, and the last three reproduce the
pre-2026-09-10 bundle byte for byte — measured with `diff -rq`, 0 differences,
not asserted. Each line below carries the number it was measured at, and
nothing beyond it.
**A re-run is what this costs a consumer, and it is not a small one:** on the **A re-run is what this costs a consumer, and it is not a small one:** on the
43-document reference corpus the default bundle goes from **629 concepts in 43-document reference corpus the default bundle goes from **629 concepts in
1108 files** (the 2026-09-03 tree) to **492 in 944** after the 2026-09-08 move 1108 files** (the 2026-09-03 tree) to **492 in 944** after the 2026-09-08 move,
and to **425 in 810** after the 2026-09-09 one. The proposer's own defaults to **425 in 810** after the 2026-09-09 one and to **436 in 832** after the
2026-09-10 one (digest `8dff8a8e6c15d2f7…`). On a five-document folder the last
move is **15 concepts in 30 files → 26 in 52**. The proposer's own defaults
(`tools/okf_propose_segments.py`) did NOT move, so every published reproduction (`tools/okf_propose_segments.py`) did NOT move, so every published reproduction
block still runs as written. block still runs as written.
@ -117,11 +123,11 @@ rank was measured in the same session and did NOT ship; see
| `--outline-run N` (default **3**) | also propose a boundary where the document's own bare-integer numbering sustains an ascending run of at least `N`; `0` is this arm's opt-out | a tender PDF whose headings are bare integers: **no boundary** at `0`, **9 concepts** at `3`, against a reference of 9 | | `--outline-run N` (default **3**) | also propose a boundary where the document's own bare-integer numbering sustains an ascending run of at least `N`; `0` is this arm's opt-out | a tender PDF whose headings are bare integers: **no boundary** at `0`, **9 concepts** at `3`, against a reference of 9 |
| `--table-grid` (**on** by default; opt out with `--no-table-grid`) | a pandoc grid-table rule line no longer closes an open table block, so one grid table is one concept | a `.docx` experience list: **21 → 6** concepts | | `--table-grid` (**on** by default; opt out with `--no-table-grid`) | a pandoc grid-table rule line no longer closes an open table block, so one grid table is one concept | a `.docx` experience list: **21 → 6** concepts |
| `--unit-fold` (**on** by default; opt out with `--no-unit-fold`) | discard a contents-list run, fold a deeper heading into its parent, fold a table into the shorter heading that introduces it. Adds no boundary, so it can only reduce a plan | on a 12-document sample scored against an operator's unit worksheet: **5 of 12** match — but that figure was measured with `--table-grid` ON, and the shipped default does not include it. Measured without it the same sample scores **2 of 12**, `docx` **0 of 3**, because the fold's table clause has no joined table to fold | | `--unit-fold` (**on** by default; opt out with `--no-unit-fold`) | discard a contents-list run, fold a deeper heading into its parent, fold a table into the shorter heading that introduces it. Adds no boundary, so it can only reduce a plan | on a 12-document sample scored against an operator's unit worksheet: **5 of 12** match — but that figure was measured with `--table-grid` ON, and the shipped default does not include it. Measured without it the same sample scores **2 of 12**, `docx` **0 of 3**, because the fold's table clause has no joined table to fold |
| `--keep-table-heading` | keep a heading whose body is empty only because a table opens under it, and absorb that table into its span | the two spreadsheets in that corpus, and **0 of 32 `pdf` and 0 of 5 `docx`**: the concept count does not move (1 → 1), its first byte does — the concept gains the heading line it was missing | | `--keep-table-heading` (**on** by default since 2026-09-10; opt out with `--no-keep-table-heading`) | keep a heading whose body is empty only because a table opens under it, and absorb that table into its span | the two spreadsheets in that corpus, and **0 of 32 `pdf` and 0 of 5 `docx`**: the concept count does not move (1 → 1), its first byte does — the concept gains the heading line it was missing |
| `--sheet-section-rows` | cut an open table block at the rows that label its sections — a run of at least three rows whose first cell is a bare numeric label. The opposite direction from `--table-grid`, which decides how far a block extends | a tender price sheet whose whole body is one table block: **1 → 12 concepts**, against a reference of 11 cost groups plus the sheet's preamble. Whole corpus: **1 of 39** readable documents changes, **0 of 32 `pdf`, 0 of 5 `docx`, 1 of 2 `xlsx`** | | `--sheet-section-rows` (**on** by default since 2026-09-10; opt out with `--no-sheet-section-rows`) | cut an open table block at the rows that label its sections — a run of at least three rows whose first cell is a bare numeric label. The opposite direction from `--table-grid`, which decides how far a block extends | a tender price sheet whose whole body is one table block: **1 → 12 concepts**, against a reference of 11 cost groups plus the sheet's preamble. Whole corpus: **1 of 39** readable documents changes, **0 of 32 `pdf`, 0 of 5 `docx`, 1 of 2 `xlsx`**. It reached 11 of 12 on the reference two rounds before it shipped, and was held back both times by a RETRIEVAL cost that turned out not to be its own: on a bundle built with it the gold document splits 1 → 12 concepts and row 1 of the hit@8 set fell rank 1 → 2. The repair is on the reading side (`--tie-shared-rank`, now the default), and with it in place the sheet reaches 11 of 12 with hit@8 holding **5 of 6 at ranks 1,1,1,1,1,** |
| `--drop-wrapped-outline` (**on** by default since 2026-09-09; opt out with `--keep-wrapped-outline`) | do not admit an `--outline-run` candidate whose line continues onto the next one. Judges recovered candidates only, never a heading the document declares | quoted regulation text, whose numbered paragraphs match the outline grammar exactly: **4 → 1 concepts**, the reference. Whole corpus: **5 of 39**, all `pdf`; on the 12-document sample **8 of 34** outline candidates wrap, and none of the 26 the operator kept. On the reference it carries `pdf` from **5 of 8 to 6 of 8** together with the gate below, and neither reaches 7 of 8 without the other | | `--drop-wrapped-outline` (**on** by default since 2026-09-09; opt out with `--keep-wrapped-outline`) | do not admit an `--outline-run` candidate whose line continues onto the next one. Judges recovered candidates only, never a heading the document declares | quoted regulation text, whose numbered paragraphs match the outline grammar exactly: **4 → 1 concepts**, the reference. Whole corpus: **5 of 39**, all `pdf`; on the 12-document sample **8 of 34** outline candidates wrap, and none of the 26 the operator kept. On the reference it carries `pdf` from **5 of 8 to 6 of 8** together with the gate below, and neither reaches 7 of 8 without the other |
| `--outline-gate` (**on** by default since 2026-09-09; opt out with `--no-outline-gate`) | admit `--outline-run`'s RECOVERED headings only where the document declares none of its own, plus any one recovered heading whose span covers `OUTLINE_SHARE` (0.20) of the text. Applied at admission, before spans are closed, so the text a removed mark opened is carried by the mark above it rather than lost | on the 12-document sample: `pdf` **2 of 8 → 5 of 8** alone and **7 of 8** with the rule above, `docx` unchanged at **3 of 3**. Whole corpus: it fires on **25 of 39** readable documents, changes the plan in **15 of 39**, and removes **64 of 485** proposed entries. No plan disappears (32 → 32) | | `--outline-gate` (**on** by default since 2026-09-09; opt out with `--no-outline-gate`) | admit `--outline-run`'s RECOVERED headings only where the document declares none of its own, plus any one recovered heading whose span covers `OUTLINE_SHARE` (0.20) of the text. Applied at admission, before spans are closed, so the text a removed mark opened is carried by the mark above it rather than lost | on the 12-document sample: `pdf` **2 of 8 → 5 of 8** alone and **7 of 8** with the rule above, `docx` unchanged at **3 of 3**. Whole corpus: it fires on **25 of 39** readable documents, changes the plan in **15 of 39**, and removes **64 of 485** proposed entries. No plan disappears (32 → 32) |
| `--first-span-from-zero` | start the first concept at character 0, so the text above it belongs to a segment instead of to none. Adds no boundary and removes none | **off by default.** Measured over the 39-document corpus: **32 of the 32** documents that get a plan lose their opening today — **159 704 characters, 9.18 %** of the corpus, 45 841 from one document. On the 12-position reference it changes nothing (10 of 12 either way). It is off because it moves the first span of essentially every bundle and no hit@8 number has been measured behind it yet | | `--first-span-from-zero` (**on** by default since 2026-09-10; opt out with `--no-first-span-from-zero`) | start the first concept at character 0, so the text above it belongs to a segment instead of to none. Adds no boundary and removes none | Measured over the 39-document corpus, the default before this rule left **207 435 characters — 11.92 %** — in no segment at all: **163 804 above the first entry** (in **32 of the 32** documents that get a plan), 26 041 *between* entries and 17 590 after the last. This rule closes the first part entirely, 79 % of the whole, leaving **43 631 characters (2.51 %) over 8 of 32 documents** with two named mechanisms of their own. It adds no boundary and the K2 concept count is identical with and without it (**425 = 425**); on the 12-position reference it changes **not one cell**, and hit@8 on a K2 bundle built with it holds **5 of 6 at ranks 1,1,1,1,1,** under both tie-breaks |
They compose, and the order above is the order they apply in. Measured on a They compose, and the order above is the order they apply in. Measured on a
five-document tender folder (2 `pdf`, 2 `docx`, 1 `xlsx`), concepts per five-document tender folder (2 `pdf`, 2 `docx`, 1 `xlsx`), concepts per
@ -235,7 +241,32 @@ concept from withheld to delivered and a priced sheet from candidate rank 10 to
the four-character prefix matcher makes a unique identifier read as the four-character prefix matcher makes a unique identifier read as
135-of-446 common on that bundle. Where it cannot help is decomposed rather 135-of-446 common on that bundle. Where it cannot help is decomposed rather
than guessed: RRF fuses RANKS, so a weight moves nothing on a signal the gold than guessed: RRF fuses RANKS, so a weight moves nothing on a signal the gold
already leads. `docs/2026-09-08-sjeldenhetsvekt.md`. already leads. `docs/2026-09-08-sjeldenhetsvekt.md`. Its published figures were
measured under the pre-2026-09-10 tie-break and are not re-measured.
`--tie-shared-rank` is **on** by default since 2026-09-10 (opt out with
`--no-tie-shared-rank`), and it is the one change in this library that alters a
payload with no bundle changing — a consumer pinned to the previous excerpt
order needs the opt-out. RRF emits a rank for every concept in every signal,
including a signal that scored them all the same, and the declared tie-break
then orders that group by `concept_id`; the fusion reads alphabetical order as
if it were a measurement. Shared ranks make a signal that separates nothing
contribute the same constant to each concept in the group. What it buys is
general rather than cosmetic: a document the segmenter splits from 1 concept
into 12 fills that signal's whole top tie group with its own concepts, so the
one that leads the body signal takes position 11 instead of 1 and the document
loses fused rank 1 to a single-concept competitor leading nothing — **the
fusion was punishing fine-graining for being fine-grained**, which put the
segmentation side and the retrieval side in competition over one number.
It shipped OFF on 2026-09-08 because hit@8 fell 5 of 6 to 4 of 6, and that
figure is real and **conditional**: swept over 2 document-prior exponents x 3
bundles x 6 rows, the lost row is lost only at exponent 1.0. The exponent moved
to 0.5 on 2026-09-09 for an unrelated reason, correctly reported as moving no
hit@8 row, and nobody measured the pair — so a rule sat behind a published
number that had stopped being true in the same commit. A flag's "off by
measurement" is a measurement of a *configuration*, not a property of the flag.
`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`.
It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a
`sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a `sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a

View file

@ -0,0 +1,304 @@
# K3 round 7: the first span, and the fusion that punished fine-graining
Round 6 shipped the outline gate and left two things behind: a measured
coverage hole it had built the repair for but not the number to move it, and a
retrieval regression that kept two spreadsheet rules off the default. This
round measured both. Three defaults moved and one file type turned out never to
have been measured at all.
Everything here is measured on this repository at `38104b7` plus this round's
changes, with `~/repos/llm-ingestion-okf/.venv/bin/python`, against the cached
rendering `/tmp/k3r3/text43` (39 readable documents) and the criterion in
`/tmp/k3r5/score5.py` imported unchanged. Bundle digests are
`find . -type f | sort | xargs shasum -a 256 | shasum -a 256` from inside the
bundle, so they are path-independent.
## 0. One divergence from the numbers this round was handed
The order states the coverage loss as **159 704 characters, 9.18 % of the
corpus**. Measured here with the shipped default over the same 39 documents,
the text above every plan's first entry is **163 804 characters, 9.42 %** — a
difference of 4 100 characters, 2.5 % of the figure. Both agree on the
denominator (39 documents, 32 with a plan) and on the shape of the finding.
The method used here is in § 1 and the 4 100 are not chased further; where the
two disagree the number in this report is the one measured today.
The same measurement also found more of the hole than the figure it was handed
covers, which is § 1.
## 1. Coverage: the hole is bigger than the first span, and the first span was 79 % of it
Characters lying in NO segment, over the 39-document corpus (1 739 511
characters, 32 documents with a plan):
| configuration | in no segment | share | documents | above first | between | after last |
|---|---|---|---|---|---|---|
| pre-round-7 default | 207 435 | **11.92 %** | 32 of 32 | 163 804 | 26 041 | 17 590 |
| + `--first-span-from-zero` | 43 631 | 2.51 % | 8 of 32 | **0** | 26 041 | 17 590 |
| round 7 default (all three) | 43 631 | **2.51 %** | 8 of 32 | 0 | 26 041 | 17 590 |
`--first-span-from-zero` closes **163 804 of 207 435**, 79 % of the hole, and
it closes the whole of the part that hits every document. What remains is
**43 631 characters, 2.51 % of the corpus, over 8 of 32 documents**, in two
named parts that are NOT the same defect and are not fixed here:
* **26 041 characters between one entry's end and the next one's start.** The
mechanism is § 4's: a `rule:table-block` candidate opens BELOW the heading
line that introduces it, so the heading's own line falls between two spans.
* **17 590 characters after the last entry ends.** Not investigated this round.
Both are written into STATE with their mechanism. Neither is a ceiling; each is
a measurement with a denominator and a named cause.
### 1.1 What it cost to move it
`--first-span-from-zero` adds no boundary, and the measurement says so twice
over. On the 12-position reference it changes **not one cell**`pdf` 7 of 8,
`docx` 3 of 3, `xlsx` 0 of 1, sheet 10 of 12, identical to the default without
it, per position and per label. On a K2 bundle built with it: **425 concepts /
810 files**, the same count as the default without it, and hit@8 `[1, 1, 1, 1,
1, None]` = 5 of 6 — **under both tie-breaks**, so its acceptance does not
borrow from § 2's move.
## 2. The fusion punished fine-graining, and the repair was already in the tree
### 2.1 The decomposition
Round 6 measured that a K2 bundle built with `--sheet-section-rows
--keep-table-heading` takes row 1 of the hit@8 set from rank 1 to rank 2, and
held the two rules back for it. It also measured that the DOCUMENT-prior rank
is 1 on both bundles, which ruled the prior out — correctly, and that is why
the cause stayed unfound: the prior's rank over 39 DOCUMENTS is not the prior
signal's rank over 425 CONCEPTS, and RRF reads the second.
Per-signal, row 1, gold concept against the concept that takes fused rank 1:
| bundle | gold doc's units | gold: title / body / prior rank | competitor: title / body / prior | fused |
|---|---|---|---|---|
| default (425) | 1 | 6 / 4 / **1** | — (gold IS rank 1) | gold 1 |
| D1+D3 (436) | 12 | 16 / 4 / **11** | 5 / 9 / 13 | gold **2** |
The prior signal on the split bundle scores all twelve of the gold document's
concepts identically (2.8735) and they take positions **1 through 12**. The
one that leads the body signal lands at position 11 and contributes `1/71`
where the undivided concept contributed `1/61`. The competitor is a
one-concept document that leads nothing and is beaten in every signal it can
be compared on — it wins because its evidence is in one unit and the gold's is
spread over twelve, and because the gold's own siblings fill the tie group
ahead of it.
So the order's hypothesis is right in its conclusion and needed one correction
in its mechanism: it is not that twelve concepts are each individually weaker.
It is that a signal which scores them all EQUALLY still emits twelve distinct
ranks, ordered by `concept_id`, and the fusion reads that alphabetical order as
if it were a measurement.
### 2.2 That rule already existed and was off for a cost that had been removed
`--tie-shared-rank` — shipped 2026-09-08, off by default — is exactly this
repair. Its published reason for being off is that hit@8 fell 5 of 6 to 4 of 6.
**That figure is real and it is conditional.** Swept over the prior exponent,
3 bundles x 6 rows:
| exponent | bundle | `tie_shared_rank` off | on |
|---|---|---|---|
| 1.0 | Arm B (629) | `1,1,1,1,1,` 5/6 | `1,,1,1,1,` **4/6** |
| 1.0 | default (425) | `1,1,1,1,1,` 5/6 | `1,1,1,1,1,` 5/6 |
| 1.0 | D1+D3 (436) | `2,1,1,1,1,` 5/6 | `1,1,1,1,1,` 5/6 |
| **0.5** | Arm B (629) | `1,1,1,1,1,` 5/6 | `1,1,1,1,1,` **5/6** |
| **0.5** | default (425) | `1,1,1,1,1,` 5/6 | `1,1,1,1,1,` 5/6 |
| **0.5** | D1+D3 (436) | `2,1,1,1,1,` 5/6 | `1,1,1,1,1,` 5/6 |
Round 6 moved `DOCUMENT_PRIOR_EXPONENT` from 1.0 to 0.5 for an unrelated
reason and reported, correctly, that it moved no hit@8 row end to end. What
nobody measured is the PAIR: the exponent move removed the only cost keeping
`--tie-shared-rank` off. A rule sat behind a published number that had stopped
being true in the same commit that was reported as changing nothing.
**The lesson is the durable one from this round**: a flag's "off by
measurement" is a measurement of a configuration, not a property of the flag.
When a constant that flag interacts with moves, its default is unmeasured
again — and nothing in the tree said so, because the two decisions live in
different files.
### 2.3 What shipped, and the control in the same measurement
`consume.DEFAULT_TIE_SHARED_RANK = True`, opt-out `--no-tie-shared-rank`. With
it, all three of the round's acceptance conditions hold at once, and the
build-side rules could move:
| bundle | flags over the pre-round-7 default | concepts | files | digest | hit@8 | ranks |
|---|---|---|---|---|---|---|
| pre-round-7 default | — | 425 | 810 | `bdf4977ca5a443c4…` | 5 of 6 | 1,1,1,1,1, |
| + first span | `--first-span-from-zero` | 425 | 810 | — | 5 of 6 | 1,1,1,1,1, |
| **round 7 default** | the above `--sheet-section-rows --keep-table-heading` | **436** | **832** | `8dff8a8e6c15d2f7…` | **5 of 6** | **1,1,1,1,1,** |
| round 7 default, `--no-tie-shared-rank` | (same bytes, read differently) | 436 | 832 | same | 5 of 6 | **2**,1,1,1,1, |
The last row is the known-negative and it is on the SHIPPED bytes rather than a
fixture: the regression round 6 refused to ship is reproduced on the bundle
that now ships, by turning off the one thing that removes it.
`tests/test_default_bundle_pin.py` holds both rows.
## 3. The reference sheet, and the two readings of `xlsx`
Reproduced before any code moved, criterion unchanged:
| configuration | `pdf` | `docx` | `xlsx` | sheet |
|---|---|---|---|---|
| pre-round-7 default | 7 of 8 | 3 of 3 | 0 of 1 | 10 of 12 |
| + `--first-span-from-zero` | 7 of 8 | 3 of 3 | 0 of 1 | 10 of 12 (not one cell moves) |
| + `--sheet-section-rows --keep-table-heading`, `|F|` = 11 | 7 of 8 | 3 of 3 | 0 of 1 | 10 of 12 |
| + the same, `|F|` = 12 | 7 of 8 | 3 of 3 | **1 of 1** | **11 of 12** |
Per position on the default, all at distance 0: pos 1 `treffer`/15, pos 2 /9,
pos 4 /1, pos 6 /6, pos 7 /30, pos 8 /8, pos 9 /4, pos 5, 10, 11 `docx`
`treffer`. Pos 0 is `for grovt` with U=1 and `|F|` undefined. Pos 3 is
`for grovt` with U=0, S=1.
**D-2 is unanswered and this report does not answer it.** `F[3]` was not
touched in the criterion. What the measurement adds is that the choice between
the two readings does not decide whether the rules ship: under `|F|` = 11 the
sheet rules leave the sheet at 10 of 12 and take position 3's distance from
**12 to 1** (U=12 against 11, `for fint`); under `|F|` = 12 they take it to
`treffer`. Neither reading is worse off with them on.
## 4. `pptx` and `md`: the mandate's weakest axis, now measured
`extract.py:76` marks `.pptx` `unmeasured` and the 12-position reference is 8
`pdf`, 3 `docx`, 1 `xlsx` — no `pptx`, no `md`. Two documents with a
hand-known structure were built for this (a markdown file with 1 + 3 declared
headings, and the same content written to `.pptx` through the converter). The
fasit is NOT invented: each document's structure is what its author wrote, and
the question asked is only whether the default recovers it.
**`pptx` extracts, and its segmentation depends on a property of the deck.**
Two decks, both extracted with rc 0 and no rejection code:
| deck | structure | concepts | matches |
|---|---|---|---|
| section slide + 3 content slides | 4 slides | **1** | no |
| 4 title slides | 4 slides | **4** | yes |
The mechanism: the converter emits one ATX heading per slide **that carries a
title placeholder it recognised**, and plain body lines for the rest. In deck 1
only slide 1 got a heading, so the document DECLARES one heading — and the
outline gate, correctly by its own rule, then admits none of Arm D's recovered
ones. The whole deck lands as one concept. This is round 6's principle working
exactly as designed on an input its measurement never contained.
**`md` extracts and loses one of four declared headings**: 3 concepts against
the 4 the document declares. Decomposed below.
**A converter attribute leaks into concept titles, and it is not a `pptx`
curiosity.** Every `pptx` concept measured carries `{#slide-N}` in its `title`,
and on real corpora the same leak appears as `{#sheet-1}`: **2 of 810** files
on the previous default K2 bundle, **2 of 1108** on Arm B, and **1 of 30** on
the operator's own test folder. Because a concept's FILENAME is reduced from
its title, the leak is in a concept id a consumer has already cited by name.
Not fixed here — see § 6.
## 5. § 4's `md` defect is § 1's remaining 26 041 characters
The markdown document declares `# Prosjektbeskrivelse` and `## 1`, `## 2`,
`## 3`. Candidates, decomposed:
* with `--no-unit-fold`: four candidates, and the fourth is
`rule:table-block` starting at 503 — **not** `rule:heading` at 489. The
heading line `## 3 Prising` is 14 characters that belong to no segment.
* with the fold on (the default): three candidates. The table block, having no
heading of its own, is folded into section 2, whose span now runs to the end
of the file. The boundary is gone.
The root cause is in the first step, not the fold: **a table-block candidate
displaces a heading the document DECLARED**, and opens below it. That is the
same principle round 6 established for Arm D against declared headings —
recovery yields to declaration — not yet applied between the table rule and the
heading rule.
Reach over the 39-document corpus, so this is a measurement and not an
anecdote: **5 of 39** documents carry a table-block candidate at all; the fold
changes the entry count on **17 of 39**, removing **198** entries; and with the
fold off, **18 527 characters over 15 of 39 documents** sit between one
candidate's end and the next one's start. Under the shipped default that
between-gap is **26 041 characters**.
Not fixed this round: it changes what the proposer emits on 5 of 39 documents,
so it needs its own K2 build and its own hit@8 read, which is one full arm's
worth of acceptance. It is in STATE with this mechanism and these numbers.
## 6. What did NOT move, and why
* **The `{#…}` leak.** The fix is small; the consequence is not. A concept's
filename is reduced from its title, so stripping the attribute RENAMES the
affected concepts — including one an external consumer has already cited by
its full id in writing. Renaming a concept id another repository names is not
this round's call to make; it is a question, with the exposure measured (2 of
810, 2 of 1108, 1 of 30).
* **`MIN_SHARED_PREFIX` and the prefix over-match.** Untouched, and
deliberately: § 2 changes the FUSION and the rarity weight changes a signal's
scores. Moving both would measure two changes with one number. The df figure
it rests on (159 of 270) is round 5's and is still not re-measured.
* **The remaining 17 590 characters after the last entry.** Named, not
investigated.
* **`--pdf-headings`, `--ocr`, `--rarity-weight`, `--cost-vocabulary`,
`--reserve-top-rank`, `--withheld-titles`, `--max-segment-chars`.** All
unchanged and all still off.
## 7. Cost to a consumer
A re-run, and it is not small. The 43-document reference corpus:
* **629 concepts / 1 108 files** (Arm B, delivered 2026-09-03)
* **492 / 944** after the 2026-09-08 move
* **425 / 810** after the 2026-09-09 move
* **436 / 832** after this one — digest `8dff8a8e6c15d2f7…`, default flags,
default epoch stamp.
On the operator's own five-document test folder the same move is **15 concepts
/ 30 files → 26 / 52**.
Every moved default has an explicit opt-out, and the opt-outs were measured
rather than asserted: `--no-first-span-from-zero --no-sheet-section-rows
--no-keep-table-heading` reproduces the pre-round-7 bundle **byte for byte**
(`diff -rq`, 0 differences), and `--no-tie-shared-rank` reproduces the previous
excerpt order.
The reading-side change is the one to flag hardest, because it has no bundle to
diff: `okf consume` now returns a different excerpt ORDER on any bundle with a
tied signal, without any bundle changing. `--no-tie-shared-rank` is the way
back.
## 8. Honesty limits
* **n = 12 positions on the reference sheet**, stratified 8 `pdf`, 3 `docx`,
1 `xlsx`. Every per-type fraction here is a fraction of 8, 3 or 1. A single
`xlsx` position carries the entire `xlsx` claim.
* **n = 6 questions on the retrieval side, one rater, one gold set.** Row 6
misses on every bundle and every configuration measured so far, so "5 of 6"
has always been "5 of 5 reachable rows plus one that no configuration has
reached".
* **Every bundle here is the SAME 43-document corpus read again.** "General"
in this report means "holds across configurations of one corpus", never
"holds across corpora". The `pptx`/`md` documents in § 4 are the only inputs
measured this round that are not from it, and they were written for it.
* **`pptx`, `md`, `odt` and `rtf` remain outside the reference sheet.** § 4
measures two hand-built documents, which is more than zero and is not a
fasit. `odt` and `rtf` are still `unmeasured` with no document behind them
at all.
* **`|F|` for position 0 is undefined and position 3's is unanswered.** Both
are operator decisions with the numbers already delivered; neither is
assumed here.
* **"Delivered" is not "answered".** hit@8 measures whether a concept from the
gold document reaches the payload, not whether the payload answers the
question. The last measurement of that, on 3 bundles, was rank 1 of 8 on 3 of
3 and a correct answer on 1 of 3.
* **The synthetic adverse case for § 2 is recorded, not hidden.** On a
30-concept fixture where one signal separates and two do not, shared ranks
move a gold from rank 18 to rank 30. It is in
`tests/test_okf_consume.py` with that number. Nothing like it appears in the
three real bundles, which is an absence over three corpora, not a proof.
* **`docs/2026-09-08-sjeldenhetsvekt.md`'s figures were measured under the
pre-2026-09-10 tie-break** and are not re-measured here. On one fixture the
change takes the weight's gold from fused rank 18 to rank 1, so at least one
of that report's numbers no longer describes the default.
* **The 4 100-character divergence in § 0 is unexplained.**

View file

@ -150,16 +150,35 @@ DEFAULT_UNIT_FOLD = True
DEFAULT_OUTLINE_GATE = True DEFAULT_OUTLINE_GATE = True
DEFAULT_DROP_WRAPPED_OUTLINE = True DEFAULT_DROP_WRAPPED_OUTLINE = True
#: Round 6's third rule, and the only one whose default this round did NOT #: Round 6 built it and left it off for want of one number; round 7 measured
#: decide. Measured over the 39-document corpus: every one of the 32 documents #: that number and moved it (2026-09-10). Measured over the 39-document corpus:
#: that gets a plan loses the text above its first concept -- 159 704 #: every one of the 32 documents that gets a plan lost the text above its first
#: characters, 9.18 % of the corpus, 45 841 from a single document. The repair #: concept -- 159 704 characters, 9.18 % of the corpus, 45 841 from a single
#: is one line and it adds no boundary. It is off because it moves the first #: document. It adds NO boundary; it moves one span's start, so it is a repair
#: span of essentially every segmented bundle and this round's budget held one #: to coverage rather than a segmentation rule, which is why it sits apart from
#: K2 build, which measured the outline gate; a default that changed every #: the arms above.
#: bundle's first concept with no hit@8 number behind it would be the move this #:
#: round refused to make elsewhere. The number it needs is named in STATE. #: The acceptance it had to clear was the same one round 6 set: the 12-position
DEFAULT_FIRST_SPAN_FROM_ZERO = False #: reference does not move (it changes not one cell, 7 of 8 `pdf`, 3 of 3
#: `docx`, 10 of 12) AND hit@8 holds rank 1 on every row it held, measured end
#: to end on a K2 bundle built with it. Opt-out `--no-first-span-from-zero`.
DEFAULT_FIRST_SPAN_FROM_ZERO = True
#: Round 3's two spreadsheet rules (D1 and D3), held back through rounds 5 and
#: 6 by a RETRIEVAL regression rather than by the reference: they take the
#: sheet to 11 of 12 and `xlsx` to its fasit, and on a K2 bundle built with
#: them row 1 of the hit@8 set fell from rank 1 to rank 2.
#:
#: Round 7 decomposed that fall and it was never these rules' fault. Splitting
#: the gold document from 1 concept into 12 makes its own 12 concepts occupy
#: the document-prior signal's entire top tie group, so the concept leading the
#: body signal takes prior position 11 instead of 1 -- RRF reading a tie-break
#: by `concept_id` as if it were a measurement. `consume.DEFAULT_TIE_SHARED_RANK`
#: is the repair, and with it in place all three of round 7's acceptance
#: conditions hold at once. Opt-outs `--no-sheet-section-rows`,
#: `--no-keep-table-heading`.
DEFAULT_SHEET_SECTION_ROWS = True
DEFAULT_KEEP_TABLE_HEADING = True
#: Round 4's two PDF READERS, both off. They are not arms and they are not in #: Round 4's two PDF READERS, both off. They are not arms and they are not in
#: the paragraph above: an arm changes how the proposer cuts a text, and these #: the paragraph above: an arm changes how the proposer cuts a text, and these
@ -530,27 +549,42 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
build_parser.add_argument( build_parser.add_argument(
"--keep-table-heading", "--keep-table-heading",
action="store_true", action="store_true",
default=DEFAULT_KEEP_TABLE_HEADING,
help=( help=(
"D1, passed to the proposer unchanged: keep a heading whose body is " "D1, passed to the proposer unchanged: keep a heading whose body is "
"empty only because a table opens under it, and absorb that table " "empty only because a table opens under it, and absorb that table "
"into its span. Absent (the default) is OFF. Measured on a tender " "into its span. ON since 2026-09-10. Measured on a tender price "
"price sheet: the concept count does not move (1 -> 1) and the " "sheet: the concept count does not move (1 -> 1) and the concept "
"concept gains the heading line it was missing" "gains the heading line it was missing"
), ),
) )
build_parser.add_argument(
"--no-keep-table-heading",
action="store_false",
dest="keep_table_heading",
help="D1's explicit opt-out. A default a caller cannot turn off is not a default",
)
build_parser.add_argument( build_parser.add_argument(
"--sheet-section-rows", "--sheet-section-rows",
action="store_true", action="store_true",
default=DEFAULT_SHEET_SECTION_ROWS,
help=( help=(
"D3, passed to the proposer unchanged: cut an open table block at " "D3, passed to the proposer unchanged: cut an open table block at "
"the rows that label its sections -- a RUN of rows whose first cell " "the rows that label its sections -- a RUN of rows whose first cell "
"is a bare numeric label. The opposite direction from Arm E, which " "is a bare numeric label. The opposite direction from Arm E, which "
"decides how far a block extends; this decides where it is cut " "decides how far a block extends; this decides where it is cut "
"inside. Absent (the default) is OFF. Measured on a tender price " "inside. ON since 2026-09-10, once the fusion stopped punishing the "
"sheet whose whole body is one table block: 1 concept -> 12, " "split it produces. Measured on a tender price sheet whose whole "
"against a reference of 11 cost groups plus the sheet's preamble" "body is one table block: 1 concept -> 12, against a reference of "
"11 cost groups plus the sheet's preamble"
), ),
) )
build_parser.add_argument(
"--no-sheet-section-rows",
action="store_false",
dest="sheet_section_rows",
help="D3's explicit opt-out. With --no-keep-table-heading it reproduces the pre-2026-09-10 bytes",
)
build_parser.add_argument( build_parser.add_argument(
"--drop-wrapped-outline", "--drop-wrapped-outline",
action="store_true", action="store_true",
@ -599,10 +633,20 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
default=DEFAULT_FIRST_SPAN_FROM_ZERO, default=DEFAULT_FIRST_SPAN_FROM_ZERO,
help=( help=(
"Start the first concept at character 0, so the text above it is " "Start the first concept at character 0, so the text above it is "
"carried by a segment instead of by none. Absent (the default) is " "carried by a segment instead of by none. Measured over the "
"OFF. Measured over the 39-document corpus: 32 of the 32 documents " "39-document corpus: 32 of the 32 documents with a plan lost "
"with a plan lose their opening today -- 159 704 characters, 9.18 " "their opening before this moved -- 159 704 characters, 9.18 "
"%% of the corpus. It adds no boundary and moves one span's start" "%% of the corpus. It adds no boundary and moves one span's start. "
"ON since 2026-09-10"
),
)
build_parser.add_argument(
"--no-first-span-from-zero",
action="store_false",
dest="first_span_from_zero",
help=(
"The repair's explicit opt-out, for a consumer who needs the "
"pre-2026-09-10 first span byte for byte"
), ),
) )
build_parser.add_argument( build_parser.add_argument(

View file

@ -1063,6 +1063,29 @@ def document_scores(
#: fusion is invariant to any monotone transform of the individual signals. #: fusion is invariant to any monotone transform of the individual signals.
RRF_K = 60 RRF_K = 60
#: Whether concepts a signal scores EQUALLY share that group's first rank
#: instead of being ordered inside it by `concept_id`. ON since 2026-09-10.
#:
#: WHY IT MOVED. It shipped OFF on 2026-09-08 on a measurement -- hit@8 over
#: the six published questions fell 5 of 6 to 4 of 6 on the 629-concept bundle.
#: Round 7 re-measured that fall and it is CONDITIONAL on the document prior
#: being a sum: swept over `DOCUMENT_PRIOR_EXPONENT` x 3 bundles x 6 rows, the
#: lost row is lost at exponent 1.0 and held at 0.5. Round 6 moved that
#: exponent to 0.5 for an unrelated reason and nobody re-measured the pair, so
#: a rule was left off by a cost that had already been removed.
#:
#: WHAT IT BUYS. It is the repair for the defect that kept `--sheet-section-rows
#: --keep-table-heading` off the build default: a document split from 1 concept
#: into 12 puts its own 12 concepts in the prior signal's whole top tie group,
#: so the concept leading the body signal takes prior position 11 rather than
#: 1 and the document loses fused rank 1 to a single-concept competitor that
#: leads nothing. Fine-graining was being punished for being fine-grained.
#:
#: Measured at exponent 0.5 on three bundles, ranks per row: `[1, 1, 1, 1, 1,
#: None]` on all three with it on, against `[2, 1, 1, 1, 1, None]` on the split
#: bundle with it off. Opt-out `--no-tie-shared-rank`.
DEFAULT_TIE_SHARED_RANK = True
def concept_scores( def concept_scores(
concepts: Sequence[Concept], concepts: Sequence[Concept],
@ -1072,7 +1095,7 @@ def concept_scores(
cost_vocabulary: bool = False, cost_vocabulary: bool = False,
weights: Mapping[str, float] | None = None, weights: Mapping[str, float] | None = None,
lookup: bool = True, lookup: bool = True,
tie_shared_rank: bool = False, tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
) -> list[tuple[Concept, float, int]]: ) -> list[tuple[Concept, float, int]]:
"""Every concept, ordered best first, fused from three signals by RRF. """Every concept, ordered best first, fused from three signals by RRF.
@ -1464,7 +1487,7 @@ def build_payload(
cost_vocabulary: bool = False, cost_vocabulary: bool = False,
reserve_top_rank: bool = False, reserve_top_rank: bool = False,
rarity_weight: bool = False, rarity_weight: bool = False,
tie_shared_rank: bool = False, tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
withheld_titles: bool = False, withheld_titles: bool = False,
) -> dict[str, object]: ) -> dict[str, object]:
"""One bundle plus one question, cut to one contract-conformant payload. """One bundle plus one question, cut to one contract-conformant payload.
@ -1666,13 +1689,22 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser.add_argument( parser.add_argument(
"--tie-shared-rank", "--tie-shared-rank",
action="store_true", action="store_true",
default=DEFAULT_TIE_SHARED_RANK,
help=( help=(
"let concepts a signal scores EQUALLY share that score group's " "let concepts a signal scores EQUALLY share that score group's "
"first rank, so a signal that separates nothing contributes the " "first rank, so a signal that separates nothing contributes the "
"same constant to each of them instead of ordering them by id. OFF " "same constant to each of them instead of ordering them by id. ON "
"by default. See docs/2026-09-08-rangeringsbom-sammensatte-ord.md" "since 2026-09-10. See docs/2026-09-08-rangeringsbom-sammensatte-ord.md "
"for the rule and docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md "
"for why its published cost no longer holds"
), ),
) )
parser.add_argument(
"--no-tie-shared-rank",
action="store_false",
dest="tie_shared_rank",
help="The rule's explicit opt-out, reproducing the pre-2026-09-10 order",
)
parser.add_argument( parser.add_argument(
"--withheld-titles", "--withheld-titles",
action="store_true", action="store_true",

View file

@ -638,10 +638,13 @@ def test_sheet_section_rows_reaches_the_proposer_from_the_build_command(tmp_path
def test_the_build_default_leaves_a_sheet_as_one_concept(tmp_path: Path) -> None: def test_the_build_default_leaves_a_sheet_as_one_concept(tmp_path: Path) -> None:
"""The control the test above rests on: same inbox, no flag, one concept. """The control the test above rests on: same inbox, D3 OFF, one concept.
Without it, a change that turned D3 on by default would leave the assertion D3 became the default on 2026-09-10, so the control now names the opt-out
above green while cutting every table a consumer has ever ingested. rather than saying nothing. It is still a control and still load-bearing:
it is what makes the assertion above a statement about D3 rather than a
statement about this fixture, and it is the test that goes red if
`--no-sheet-section-rows` ever stops reproducing the pre-2026-09-10 cut.
""" """
inbox = inbox_for_sheet_sections(tmp_path) inbox = inbox_for_sheet_sections(tmp_path)
plans = tmp_path / "plans-sheet-plain" plans = tmp_path / "plans-sheet-plain"
@ -653,6 +656,8 @@ def test_the_build_default_leaves_a_sheet_as_one_concept(tmp_path: Path) -> None
str(plans), str(plans),
"--proposed-at", "--proposed-at",
PROPOSED_AT, PROPOSED_AT,
"--no-sheet-section-rows",
"--no-keep-table-heading",
) )
== 0 == 0
) )

View file

@ -6,14 +6,17 @@ measured on a bundle nobody produces is a number that cannot regress, so the
guarantee it looks like was never held by anything. guarantee it looks like was never held by anything.
This file pins the CURRENT default: `--outline-run 3 --table-grid --unit-fold This file pins the CURRENT default: `--outline-run 3 --table-grid --unit-fold
--drop-wrapped-outline --outline-gate`, the configuration round 6 moved the --drop-wrapped-outline --outline-gate --first-span-from-zero
default to on 2026-09-09 after measuring hit@8 on exactly this bundle. --sheet-section-rows --keep-table-heading`, plus the reading side's
`tie_shared_rank`. Round 6 moved the first five on 2026-09-09 and round 7 moved
the last four on 2026-09-10, each after measuring hit@8 on exactly the bundle
its own default produces.
The gold set is LOCAL-ONLY and stays that way: no question and no The gold set is LOCAL-ONLY and stays that way: no question and no
`gold_document` is reproduced here, and a row is named by its INDEX, the way `gold_document` is reproduced here, and a row is named by its INDEX, the way
`docs/2026-09-07-okf-konsumskill-maaling.md` already names them. `docs/2026-09-07-okf-konsumskill-maaling.md` already names them.
The bundle itself is a build artefact, not a fixture: it is 810 files of a The bundle itself is a build artefact, not a fixture: it is 832 files of a
consumer's corpus and this repository is public. Absent, these tests SKIP with consumer's corpus and this repository is public. Absent, these tests SKIP with
the command that rebuilds it -- "not measured", never zero. the command that rebuilds it -- "not measured", never zero.
""" """
@ -34,10 +37,10 @@ import okf_consume_measure # noqa: E402
#: Built by: #: Built by:
#: okf build <corpus>/K2/trinn1 \ #: okf build <corpus>/K2/trinn1 \
#: --bundle ~/corpora/okf-telling-20260829/K2-bundle-default-20260909 \ #: --bundle ~/corpora/okf-telling-20260829/K2-bundle-default-20260910 \
#: --bundle-id k2-trinn1-20260903 --okf-version 0.2 #: --bundle-id k2-trinn1-20260903 --okf-version 0.2
#: with no arm flag at all -- the package default, which is the point. #: with no arm flag at all -- the package default, which is the point.
DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260909" DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260910"
GOLD_SET = PROJECT_ROOT / ".claude/projects/2026-09-07-okf-consume-prepass/hit-at-k-questions.json" GOLD_SET = PROJECT_ROOT / ".claude/projects/2026-09-07-okf-consume-prepass/hit-at-k-questions.json"
requires_default_bundle = pytest.mark.skipif( requires_default_bundle = pytest.mark.skipif(
@ -49,8 +52,11 @@ requires_default_bundle = pytest.mark.skipif(
), ),
) )
#: Measured 2026-09-09 on the bundle above. #: Measured 2026-09-10 on the bundle above. The count moved 425 -> 436 with
EXPECTED_CONCEPTS = 425 #: `--sheet-section-rows --keep-table-heading`; `--first-span-from-zero` moved
#: it by NOTHING, which is the point of that rule -- it adds no boundary and
#: only moves one span's start.
EXPECTED_CONCEPTS = 436
EXPECTED_HITS = 5 EXPECTED_HITS = 5
#: Rank per question INDEX, `None` for the row that misses on every bundle and #: Rank per question INDEX, `None` for the row that misses on every bundle and
#: every configuration measured so far. The identity is the index; the question #: every configuration measured so far. The identity is the index; the question
@ -70,6 +76,10 @@ def test_hit_at_eight_holds_rank_one_on_every_row_it_held() -> None:
Not the hit COUNT alone: the count survived a configuration that lost a Not the hit COUNT alone: the count survived a configuration that lost a
row from rank 1 to rank 2, which is exactly how the previous round's row from rank 1 to rank 2, which is exactly how the previous round's
regression hid. The rank per row is the pin. regression hid. The rank per row is the pin.
On THIS bundle that is not a hypothetical -- see the test below, which
reproduces the fall on these exact bytes by turning the reading-side
default off.
""" """
questions = json.loads(GOLD_SET.read_text(encoding="utf-8"))["questions"] questions = json.loads(GOLD_SET.read_text(encoding="utf-8"))["questions"]
assert len(questions) == len(EXPECTED_RANKS), "the gold set changed shape" assert len(questions) == len(EXPECTED_RANKS), "the gold set changed shape"
@ -87,3 +97,31 @@ def test_hit_at_eight_holds_rank_one_on_every_row_it_held() -> None:
def test_the_bundle_declares_the_identity_the_reader_needs() -> None: def test_the_bundle_declares_the_identity_the_reader_needs() -> None:
"""Whatever else moves, the bundle stays one the reading direction opens.""" """Whatever else moves, the bundle stays one the reading direction opens."""
assert okf_consume.root_bundle_id_of(DEFAULT_BUNDLE) == "k2-trinn1-20260903" assert okf_consume.root_bundle_id_of(DEFAULT_BUNDLE) == "k2-trinn1-20260903"
@requires_default_bundle
def test_the_reading_default_is_what_holds_row_one_on_these_bytes() -> None:
"""The known-negative, on the shipped bundle rather than a fixture.
Round 7 moved `--sheet-section-rows --keep-table-heading` into the build
default, which splits row 1's gold document from 1 concept into 12. Round 6
measured that exact split costing row 1 its rank, and held the two rules
back for it. What removed the cost is `consume.DEFAULT_TIE_SHARED_RANK`,
and this test is the proof that it is still what removes it: turn it off
on these bytes and the fall comes back.
Without this, `EXPECTED_RANKS` above would be a green assertion with no
stated cause, and a later change to the fusion could take the cause away
while the pin stayed green on some other accident.
"""
questions = json.loads(GOLD_SET.read_text(encoding="utf-8"))["questions"]
ranks = []
for entry in questions:
payload = okf_consume.build_payload(
DEFAULT_BUNDLE, question=entry["question"], tie_shared_rank=False
)
excerpts = payload["excerpts"]
assert isinstance(excerpts, list)
ranks.append(okf_consume_measure.hit_rank(excerpts, entry["gold_document"]))
assert ranks[0] == 2, "the known-negative stopped being negative"
assert tuple(ranks[1:]) == EXPECTED_RANKS[1:]

View file

@ -68,5 +68,10 @@ def test_the_rule_adds_and_removes_no_boundary() -> None:
assert [c.start for c in plain][1:] == [c.start for c in moved][1:] assert [c.start for c in plain][1:] == [c.start for c in moved][1:]
def test_it_is_off_by_default_in_the_build_command() -> None: def test_it_is_on_by_default_in_the_build_command() -> None:
assert cli.DEFAULT_FIRST_SPAN_FROM_ZERO is False """Moved 2026-09-10, once the hit@8 number round 6 lacked was measured.
The opt-out is asserted in `tests/test_round7_defaults.py`; here the point
is only that the rule below is what a caller gets without saying anything.
"""
assert cli.DEFAULT_FIRST_SPAN_FROM_ZERO is True

View file

@ -522,10 +522,19 @@ def test_reversing_the_input_order_does_not_change_the_output_order() -> None:
def test_a_concept_in_a_high_scoring_document_outranks_an_equally_lexical_one() -> None: def test_a_concept_in_a_high_scoring_document_outranks_an_equally_lexical_one() -> None:
# `tie_shared_rank=False` for the same reason `lookup=False` appears
# elsewhere in this file: the claim is about the DOCUMENT PRIOR, and the
# default tie-break (shared since 2026-09-10) puts this fixture's two
# concepts in the same prior tie group, which makes the assertion true
# in both directions and so measures nothing. Isolate the stage under test.
concepts = _fixture_concepts() concepts = _fixture_concepts()
question = "Hvordan skal prisene fylles ut?" question = "Hvordan skal prisene fylles ut?"
lifted = okf_consume.concept_scores(concepts, question, {"krav": 10.0, "dyp": 0.0}) lifted = okf_consume.concept_scores(
dropped = okf_consume.concept_scores(concepts, question, {"krav": 0.0, "dyp": 10.0}) concepts, question, {"krav": 10.0, "dyp": 0.0}, tie_shared_rank=False
)
dropped = okf_consume.concept_scores(
concepts, question, {"krav": 0.0, "dyp": 10.0}, tie_shared_rank=False
)
krav_first = [c.concept_id for c, _, _ in lifted].index("krav/pristabell") krav_first = [c.concept_id for c, _, _ in lifted].index("krav/pristabell")
krav_later = [c.concept_id for c, _, _ in dropped].index("krav/pristabell") krav_later = [c.concept_id for c, _, _ in dropped].index("krav/pristabell")
assert krav_first < krav_later assert krav_first < krav_later
@ -1771,7 +1780,9 @@ def test_counting_leaves_the_identifier_worth_no_more_than_the_common_word() ->
# The claim here is about the FUSION, so the fusion is what is read. # The claim here is about the FUSION, so the fusion is what is read.
for identifier_in_body in (False, True): for identifier_in_body in (False, True):
concepts = _rarity_corpus(identifier_in_body=identifier_in_body) concepts = _rarity_corpus(identifier_in_body=identifier_in_body)
ranked = okf_consume.concept_scores(concepts, RARITY_QUESTION, {}, lookup=False) ranked = okf_consume.concept_scores(
concepts, RARITY_QUESTION, {}, lookup=False, tie_shared_rank=False
)
assert _rank_of(ranked, RARITY_GOLD) == 18 assert _rank_of(ranked, RARITY_GOLD) == 18
lexical = {concept.concept_id: value for concept, _, value in ranked} lexical = {concept.concept_id: value for concept, _, value in ranked}
assert lexical["krav/c-00"] == 6 assert lexical["krav/c-00"] == 6
@ -1786,7 +1797,9 @@ def test_weighting_a_hit_by_its_rarity_lifts_the_identifier_into_the_cut() -> No
weights = okf_consume.rarity_weights( weights = okf_consume.rarity_weights(
okf_consume.normalise(RARITY_QUESTION), okf_consume.searchable_text(concepts) okf_consume.normalise(RARITY_QUESTION), okf_consume.searchable_text(concepts)
) )
ranked = okf_consume.concept_scores(concepts, RARITY_QUESTION, {}, weights=weights) ranked = okf_consume.concept_scores(
concepts, RARITY_QUESTION, {}, weights=weights, tie_shared_rank=False
)
assert _rank_of(ranked, RARITY_GOLD) <= 8 assert _rank_of(ranked, RARITY_GOLD) <= 8
@ -1817,9 +1830,28 @@ def test_a_rarity_weight_cannot_move_a_signal_the_gold_already_leads() -> None:
assert _signal_rank(concepts, RARITY_QUESTION, RARITY_GOLD, None) == 1 assert _signal_rank(concepts, RARITY_QUESTION, RARITY_GOLD, None) == 1
assert _signal_rank(concepts, RARITY_QUESTION, RARITY_GOLD, weights) == 1 assert _signal_rank(concepts, RARITY_QUESTION, RARITY_GOLD, weights) == 1
ranked = okf_consume.concept_scores( ranked = okf_consume.concept_scores(
concepts, RARITY_QUESTION, {}, weights=weights, lookup=False concepts, RARITY_QUESTION, {}, weights=weights, lookup=False, tie_shared_rank=False
) )
assert _rank_of(ranked, RARITY_GOLD) == 18 assert _rank_of(ranked, RARITY_GOLD) == 18
# AND the falsification under the 2026-09-10 tie-break, measured rather
# than assumed -- with the COST it carries on this fixture written down.
# The claim survives and is stronger (30 -> 30 with and without the
# weight), but the gold's absolute rank is WORSE than the 18 above: the id
# order that had put it at 18 was luck inside a tie group, and shared ranks
# take that luck away in both directions. This fixture is the adverse
# shape for the rule -- one signal separates, two do not, and the one that
# separates is the one the gold loses. It is recorded here because it is
# the honest limit of the end-to-end measurement that moved the default:
# ranks held at 1 on every row of three real bundles, and a synthetic
# corpus of 30 near-identical concepts is not one of them.
under_default = okf_consume.concept_scores(
concepts, RARITY_QUESTION, {}, weights=weights, lookup=False
)
unweighted_under_default = okf_consume.concept_scores(
concepts, RARITY_QUESTION, {}, lookup=False
)
assert _rank_of(under_default, RARITY_GOLD) == 30
assert _rank_of(unweighted_under_default, RARITY_GOLD) == 30
def test_the_weight_leaves_the_lexical_count_a_count_so_the_cut_is_untouched() -> None: def test_the_weight_leaves_the_lexical_count_a_count_so_the_cut_is_untouched() -> None:
@ -1836,7 +1868,9 @@ def test_the_weight_leaves_the_lexical_count_a_count_so_the_cut_is_untouched() -
} }
weighted = { weighted = {
c.concept_id: v c.concept_id: v
for c, _, v in okf_consume.concept_scores(concepts, RARITY_QUESTION, {}, weights=weights) for c, _, v in okf_consume.concept_scores(
concepts, RARITY_QUESTION, {}, weights=weights, tie_shared_rank=False
)
} }
assert plain == weighted assert plain == weighted
@ -1989,14 +2023,29 @@ def test_the_weight_reaches_the_title_signal_and_reorders_it() -> None:
assert _signal_rank(concepts, question, "krav/t-29", weights) == 1 assert _signal_rank(concepts, question, "krav/t-29", weights) == 1
# `lookup=False`: the gold's title bears the identifier, so the lookup # `lookup=False`: the gold's title bears the identifier, so the lookup
# partition answers this question at rank one. The fusion is the claim. # partition answers this question at rank one. The fusion is the claim.
plain = okf_consume.concept_scores(concepts, question, {}, lookup=False) plain = okf_consume.concept_scores(concepts, question, {}, lookup=False, tie_shared_rank=False)
weighted = okf_consume.concept_scores(concepts, question, {}, weights=weights, lookup=False) weighted = okf_consume.concept_scores(
concepts, question, {}, weights=weights, lookup=False, tie_shared_rank=False
)
# Fused: 30 -> 18. The title signal is reordered from last to first and the # Fused: 30 -> 18. The title signal is reordered from last to first and the
# fused rank moves by twelve, not to one -- the other two signals still # fused rank moves by twelve, not to one -- the other two signals still
# cannot see the identifier. That gap IS the finding of this session, and # cannot see the identifier. That gap IS the finding of this session, and
# the numbers are here so a change to either half shows up as a diff. # the numbers are here so a change to either half shows up as a diff.
assert _rank_of(plain, "krav/t-29") == 30 assert _rank_of(plain, "krav/t-29") == 30
assert _rank_of(weighted, "krav/t-29") == 18 assert _rank_of(weighted, "krav/t-29") == 18
# AND WHAT THE 2026-09-10 TIE-BREAK DID TO THAT GAP, measured rather than
# left implicit. This fixture is the shape shared ranks were built for:
# every body is identical and the prior is empty, so both of the signals
# that "cannot see the identifier" separate NOTHING, and under shared ranks
# they stop ordering by id and contribute one constant each. The title
# signal is then alone in deciding and the gold goes to rank 1 -- the gap
# this test named does not survive the fusion change, and a reader of
# `docs/2026-09-08-sjeldenhetsvekt.md` needs to know the figure there was
# measured under the older tie-break.
under_default = okf_consume.concept_scores(
concepts, question, {}, weights=weights, lookup=False
)
assert _rank_of(under_default, "krav/t-29") == 1
# --- Step 14: exact identifier matching and the lookup signal (O2c) ----------- # --- Step 14: exact identifier matching and the lookup signal (O2c) -----------

View file

@ -0,0 +1,84 @@
"""What round 7 moved to the default, and the opt-out that reproduces the old bytes.
Three moves, each with its own acceptance measurement in
`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`:
* `--first-span-from-zero`, because 32 of the 32 corpus documents with a plan
left their opening text in NO segment -- 9.18 % of the corpus.
* `--sheet-section-rows --keep-table-heading`, which were held back in round 6
by a retrieval regression the third move removes.
* `tie_shared_rank` on the reading side, which is what removed it.
Every move keeps an explicit opt-out: a default a caller cannot turn off is
not a default, and a consumer needing the previous bytes needs a way to say so.
"""
from __future__ import annotations
from pathlib import Path
from llm_ingestion_okf import cli, consume
def test_the_build_defaults_carry_the_three_rules_round_seven_moved() -> None:
assert cli.DEFAULT_FIRST_SPAN_FROM_ZERO is True
assert cli.DEFAULT_SHEET_SECTION_ROWS is True
assert cli.DEFAULT_KEEP_TABLE_HEADING is True
def test_the_reading_side_shares_a_rank_a_signal_did_not_earn_by_default() -> None:
"""`tie_shared_rank` is what let the two build rules above ship.
Measured on the D1+D3 bundle: the gold document splits from 1 concept into
12, its own 12 concepts occupy the document-prior signal's whole top tie
group, and the concept leading the body signal lands at prior position 11
instead of 1. Row 1 falls from fused rank 1 to 2. Shared ranks put it back.
"""
assert consume.DEFAULT_TIE_SHARED_RANK is True
def test_every_moved_default_has_an_opt_out_that_the_parser_accepts() -> None:
args = cli.parse_args(
[
"build",
"src",
"--bundle",
"out",
"--bundle-id",
"x",
"--no-first-span-from-zero",
"--no-sheet-section-rows",
"--no-keep-table-heading",
]
)
assert args.first_span_from_zero is False
assert args.sheet_section_rows is False
assert args.keep_table_heading is False
def test_the_build_defaults_reach_the_parser_without_a_flag() -> None:
args = cli.parse_args(["build", "src", "--bundle", "out", "--bundle-id", "x"])
assert args.first_span_from_zero is True
assert args.sheet_section_rows is True
assert args.keep_table_heading is True
def test_the_consume_parser_offers_an_opt_out_for_the_shared_rank() -> None:
assert consume.parse_args(["b", "--question", "q"]).tie_shared_rank is True
assert (
consume.parse_args(["b", "--question", "q", "--no-tie-shared-rank"]).tie_shared_rank
is False
)
def test_build_payload_defaults_to_the_shared_rank(tmp_path: Path) -> None:
"""The default reaches the API, not only the CLI.
`okf skill` emits `okf consume`, but `tests/test_default_bundle_pin.py` and
every measurement harness call `build_payload` directly. A default living
in `argparse` alone would make the two disagree.
"""
import inspect
signature = inspect.signature(consume.build_payload)
assert signature.parameters["tie_shared_rank"].default is True

View file

@ -137,31 +137,54 @@ def test_shared_rank_lifts_the_concept_the_measuring_signals_lead(tmp_path: Path
assert _gold_rank(root, tie_shared_rank=True) == 1 assert _gold_rank(root, tie_shared_rank=True) == 1
def test_the_payload_is_byte_identical_with_the_flag_off(tmp_path: Path) -> None: def test_the_payload_is_byte_identical_with_the_flag_on(tmp_path: Path) -> None:
"""The rule became the default 2026-09-10; saying so explicitly changes nothing.
The assertion is unchanged in kind -- the implicit and the explicit value
must produce the same bytes -- only the value it names moved.
"""
root = _tie_bundle(tmp_path / "bundle") root = _tie_bundle(tmp_path / "bundle")
without = okf_consume.serialise(okf_consume.build_payload(root, question=QUESTION)) without = okf_consume.serialise(okf_consume.build_payload(root, question=QUESTION))
explicit_off = okf_consume.serialise( explicit_on = okf_consume.serialise(
okf_consume.build_payload(root, question=QUESTION, tie_shared_rank=False) okf_consume.build_payload(root, question=QUESTION, tie_shared_rank=True)
) )
assert without == explicit_off assert without == explicit_on
def test_the_opt_out_reproduces_the_order_the_default_used_to_give(tmp_path: Path) -> None:
"""The other half: a consumer needing the pre-2026-09-10 order can have it.
Load-bearing rather than symmetric. `--no-tie-shared-rank` is the only
thing standing between a consumer pinned to the old excerpt order and a
silent reordering, so the opt-out needs a test that goes red if it stops
being a real alternative -- which it would be if it produced the same
bytes as the default on the very fixture built to separate them.
"""
root = _tie_bundle(tmp_path / "bundle")
default = okf_consume.serialise(okf_consume.build_payload(root, question=QUESTION, k=3))
opted_out = okf_consume.serialise(
okf_consume.build_payload(root, question=QUESTION, k=3, tie_shared_rank=False)
)
assert default != opted_out
def test_the_flag_changes_the_payload_it_is_meant_to_change(tmp_path: Path) -> None: def test_the_flag_changes_the_payload_it_is_meant_to_change(tmp_path: Path) -> None:
root = _tie_bundle(tmp_path / "bundle") root = _tie_bundle(tmp_path / "bundle")
off = okf_consume.build_payload(root, question=QUESTION, k=3) off = okf_consume.build_payload(root, question=QUESTION, k=3, tie_shared_rank=False)
on = okf_consume.build_payload(root, question=QUESTION, k=3, tie_shared_rank=True) on = okf_consume.build_payload(root, question=QUESTION, k=3)
delivered_off = [excerpt["concept_id"] for excerpt in off["excerpts"]] # type: ignore[index] delivered_off = [excerpt["concept_id"] for excerpt in off["excerpts"]] # type: ignore[index]
delivered_on = [excerpt["concept_id"] for excerpt in on["excerpts"]] # type: ignore[index] delivered_on = [excerpt["concept_id"] for excerpt in on["excerpts"]] # type: ignore[index]
assert not any(str(cid).endswith("zz-gull") for cid in delivered_off) assert not any(str(cid).endswith("zz-gull") for cid in delivered_off)
assert str(delivered_on[0]).endswith("zz-gull") assert str(delivered_on[0]).endswith("zz-gull")
def test_the_cli_exposes_the_flag_and_defaults_it_off(tmp_path: Path) -> None: def test_the_cli_exposes_the_flag_and_defaults_it_on(tmp_path: Path) -> None:
root = _tie_bundle(tmp_path / "bundle") root = _tie_bundle(tmp_path / "bundle")
parsed = okf_consume.parse_args([str(root), "--question", QUESTION]) parsed = okf_consume.parse_args([str(root), "--question", QUESTION])
assert parsed.tie_shared_rank is False assert parsed.tie_shared_rank is True
parsed_on = okf_consume.parse_args([str(root), "--question", QUESTION, "--tie-shared-rank"]) parsed_off = okf_consume.parse_args([str(root), "--question", QUESTION, "--no-tie-shared-rank"])
assert parsed_on.tie_shared_rank is True assert parsed_off.tie_shared_rank is False
# The other flag this session added, asserted here so "both default off" # `--withheld-titles` did NOT move with it, asserted here so the two are
# is one measurement rather than two files' worth of trust. # one measurement rather than two files' worth of trust: it is off for a
# reason of BYTES, which nothing this round touched.
assert parsed.withheld_titles is False assert parsed.withheld_titles is False