feat(cli,consume): the first span, and the fusion that punished fine-graining
Round 6 left two things behind: a measured coverage hole with the repair built
but not the number to move it, and a retrieval regression that kept two
spreadsheet rules off the default. Both are measured; four defaults move.
THE HOLE IS BIGGER THAN THE RULE BUILT FOR IT. Measured over the 39-document
corpus, the pre-move default left 207 435 characters -- 11.92 % -- in no
segment: 163 804 above the first entry, 26 041 between entries, 17 590 after
the last. `--first-span-from-zero` closes the first part entirely, 79 % of the
whole, leaving 43 631 (2.51 %) over 8 of 32 documents. It adds no boundary and
the K2 concept count is identical with and without it (425 = 425); hit@8 holds
[1,1,1,1,1,-] under both tie-breaks and the 12-position reference does not move
one cell.
THE RANKER WAS PUNISHING FINE-GRAINING FOR BEING FINE-GRAINED. Round 6 held
`--sheet-section-rows --keep-table-heading` back because on a bundle built with
them row 1 fell rank 1 -> 2, and ruled out the document prior because its rank
over 39 DOCUMENTS was 1 on both bundles. RRF reads a rank over CONCEPTS:
splitting the gold document 1 -> 12 puts its own twelve concepts in that
signal's whole top tie group, so the one leading the body signal takes position
11, contributing 1/71 where the undivided concept contributed 1/61. A signal
that scored them all EQUALLY still emitted twelve ranks, ordered by concept_id,
and the fusion read alphabetical order as a measurement.
AND THE REPAIR WAS ALREADY IN THE TREE, OFF FOR A COST THAT HAD BEEN REMOVED.
`--tie-shared-rank` shipped off 2026-09-08 because hit@8 fell 5 of 6 to 4 of 6.
Swept over 2 prior exponents x 3 bundles x 6 rows: that fall exists only at
DOCUMENT_PRIOR_EXPONENT 1.0. Round 6 moved the exponent to 0.5 for an unrelated
reason and correctly reported it moved no hit@8 row -- nobody measured the
pair. A flag's "off by measurement" is a measurement of a CONFIGURATION, not a
property of the flag.
Shipped, each with an explicit opt-out and both directions measured rather than
asserted: --no-first-span-from-zero --no-sheet-section-rows
--no-keep-table-heading reproduces the previous bundle byte for byte (diff -rq,
0 differences), and the no-flag build equals the explicit-flag build.
--no-tie-shared-rank reproduces the previous excerpt order -- the one change
here that alters a payload with no bundle changing.
K2 default: 436 concepts / 832 files, digest 8dff8a8e6c15d2f7..., hit@8 5 of 6
ranks [1,1,1,1,1,-]. The pin now holds its own known-negative on those bytes:
read with --no-tie-shared-rank the same bundle gives [2,1,1,1,1,-], so the
green assertion names its cause. Consumer cost is a re-run: 425/810 -> 436/832
on the reference corpus, 15/30 -> 26/52 on a five-document folder.
pptx and md measured end to end for the first time, on two hand-built
documents: md recovers 3 of 4 declared headings, pptx segments per slide only
where the converter recognised a title placeholder. A converter attribute leaks
into titles ({#slide-N}, {#sheet-1}) on 2 of 810 K2 files and 1 of 30 on a
five-document folder -- not fixed, because a filename is reduced from its title
and the fix renames concept ids a consumer has cited.
Suite 1486 (1478 before), ruff and mypy --strict clean over 21 files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
38104b7df5
commit
a364ef4c7d
11 changed files with 768 additions and 99 deletions
59
README.md
59
README.md
|
|
@ -88,19 +88,25 @@ did not pass them.
|
|||
|
||||
### The segmentation flags
|
||||
|
||||
Eight rules are reachable from `okf build`. **Five of them are ON by default**
|
||||
— `--outline-run 3`, `--table-grid` and `--unit-fold` since 2026-09-08, and
|
||||
`--drop-wrapped-outline` and `--outline-gate` since 2026-09-09 — each an
|
||||
operator decision, and each with an explicit opt-out: `--outline-run 0`,
|
||||
`--no-table-grid`, `--no-unit-fold`, `--keep-wrapped-outline`,
|
||||
`--no-outline-gate`. Passing all five reproduces the pre-2026-09-08 bytes
|
||||
exactly. The other three are off. Each line below carries the number it was
|
||||
measured at, and nothing beyond it.
|
||||
Eight rules are reachable from `okf build`, and since 2026-09-10 **all eight
|
||||
are ON by default** — `--outline-run 3`, `--table-grid` and `--unit-fold` since
|
||||
2026-09-08, `--drop-wrapped-outline` and `--outline-gate` since 2026-09-09, and
|
||||
`--sheet-section-rows`, `--keep-table-heading` and `--first-span-from-zero`
|
||||
since 2026-09-10 — each an operator decision, and each with an explicit
|
||||
opt-out: `--outline-run 0`, `--no-table-grid`, `--no-unit-fold`,
|
||||
`--keep-wrapped-outline`, `--no-outline-gate`, `--no-sheet-section-rows`,
|
||||
`--no-keep-table-heading`, `--no-first-span-from-zero`. Passing all eight
|
||||
reproduces the pre-2026-09-08 bytes exactly, and the last three reproduce the
|
||||
pre-2026-09-10 bundle byte for byte — measured with `diff -rq`, 0 differences,
|
||||
not asserted. Each line below carries the number it was measured at, and
|
||||
nothing beyond it.
|
||||
|
||||
**A re-run is what this costs a consumer, and it is not a small one:** on the
|
||||
43-document reference corpus the default bundle goes from **629 concepts in
|
||||
1108 files** (the 2026-09-03 tree) to **492 in 944** after the 2026-09-08 move
|
||||
and to **425 in 810** after the 2026-09-09 one. The proposer's own defaults
|
||||
1108 files** (the 2026-09-03 tree) to **492 in 944** after the 2026-09-08 move,
|
||||
to **425 in 810** after the 2026-09-09 one and to **436 in 832** after the
|
||||
2026-09-10 one (digest `8dff8a8e6c15d2f7…`). On a five-document folder the last
|
||||
move is **15 concepts in 30 files → 26 in 52**. The proposer's own defaults
|
||||
(`tools/okf_propose_segments.py`) did NOT move, so every published reproduction
|
||||
block still runs as written.
|
||||
|
||||
|
|
@ -117,11 +123,11 @@ rank was measured in the same session and did NOT ship; see
|
|||
| `--outline-run N` (default **3**) | also propose a boundary where the document's own bare-integer numbering sustains an ascending run of at least `N`; `0` is this arm's opt-out | a tender PDF whose headings are bare integers: **no boundary** at `0`, **9 concepts** at `3`, against a reference of 9 |
|
||||
| `--table-grid` (**on** by default; opt out with `--no-table-grid`) | a pandoc grid-table rule line no longer closes an open table block, so one grid table is one concept | a `.docx` experience list: **21 → 6** concepts |
|
||||
| `--unit-fold` (**on** by default; opt out with `--no-unit-fold`) | discard a contents-list run, fold a deeper heading into its parent, fold a table into the shorter heading that introduces it. Adds no boundary, so it can only reduce a plan | on a 12-document sample scored against an operator's unit worksheet: **5 of 12** match — but that figure was measured with `--table-grid` ON, and the shipped default does not include it. Measured without it the same sample scores **2 of 12**, `docx` **0 of 3**, because the fold's table clause has no joined table to fold |
|
||||
| `--keep-table-heading` | keep a heading whose body is empty only because a table opens under it, and absorb that table into its span | the two spreadsheets in that corpus, and **0 of 32 `pdf` and 0 of 5 `docx`**: the concept count does not move (1 → 1), its first byte does — the concept gains the heading line it was missing |
|
||||
| `--sheet-section-rows` | cut an open table block at the rows that label its sections — a run of at least three rows whose first cell is a bare numeric label. The opposite direction from `--table-grid`, which decides how far a block extends | a tender price sheet whose whole body is one table block: **1 → 12 concepts**, against a reference of 11 cost groups plus the sheet's preamble. Whole corpus: **1 of 39** readable documents changes, **0 of 32 `pdf`, 0 of 5 `docx`, 1 of 2 `xlsx`** |
|
||||
| `--keep-table-heading` (**on** by default since 2026-09-10; opt out with `--no-keep-table-heading`) | keep a heading whose body is empty only because a table opens under it, and absorb that table into its span | the two spreadsheets in that corpus, and **0 of 32 `pdf` and 0 of 5 `docx`**: the concept count does not move (1 → 1), its first byte does — the concept gains the heading line it was missing |
|
||||
| `--sheet-section-rows` (**on** by default since 2026-09-10; opt out with `--no-sheet-section-rows`) | cut an open table block at the rows that label its sections — a run of at least three rows whose first cell is a bare numeric label. The opposite direction from `--table-grid`, which decides how far a block extends | a tender price sheet whose whole body is one table block: **1 → 12 concepts**, against a reference of 11 cost groups plus the sheet's preamble. Whole corpus: **1 of 39** readable documents changes, **0 of 32 `pdf`, 0 of 5 `docx`, 1 of 2 `xlsx`**. It reached 11 of 12 on the reference two rounds before it shipped, and was held back both times by a RETRIEVAL cost that turned out not to be its own: on a bundle built with it the gold document splits 1 → 12 concepts and row 1 of the hit@8 set fell rank 1 → 2. The repair is on the reading side (`--tie-shared-rank`, now the default), and with it in place the sheet reaches 11 of 12 with hit@8 holding **5 of 6 at ranks 1,1,1,1,1,–** |
|
||||
| `--drop-wrapped-outline` (**on** by default since 2026-09-09; opt out with `--keep-wrapped-outline`) | do not admit an `--outline-run` candidate whose line continues onto the next one. Judges recovered candidates only, never a heading the document declares | quoted regulation text, whose numbered paragraphs match the outline grammar exactly: **4 → 1 concepts**, the reference. Whole corpus: **5 of 39**, all `pdf`; on the 12-document sample **8 of 34** outline candidates wrap, and none of the 26 the operator kept. On the reference it carries `pdf` from **5 of 8 to 6 of 8** together with the gate below, and neither reaches 7 of 8 without the other |
|
||||
| `--outline-gate` (**on** by default since 2026-09-09; opt out with `--no-outline-gate`) | admit `--outline-run`'s RECOVERED headings only where the document declares none of its own, plus any one recovered heading whose span covers `OUTLINE_SHARE` (0.20) of the text. Applied at admission, before spans are closed, so the text a removed mark opened is carried by the mark above it rather than lost | on the 12-document sample: `pdf` **2 of 8 → 5 of 8** alone and **7 of 8** with the rule above, `docx` unchanged at **3 of 3**. Whole corpus: it fires on **25 of 39** readable documents, changes the plan in **15 of 39**, and removes **64 of 485** proposed entries. No plan disappears (32 → 32) |
|
||||
| `--first-span-from-zero` | start the first concept at character 0, so the text above it belongs to a segment instead of to none. Adds no boundary and removes none | **off by default.** Measured over the 39-document corpus: **32 of the 32** documents that get a plan lose their opening today — **159 704 characters, 9.18 %** of the corpus, 45 841 from one document. On the 12-position reference it changes nothing (10 of 12 either way). It is off because it moves the first span of essentially every bundle and no hit@8 number has been measured behind it yet |
|
||||
| `--first-span-from-zero` (**on** by default since 2026-09-10; opt out with `--no-first-span-from-zero`) | start the first concept at character 0, so the text above it belongs to a segment instead of to none. Adds no boundary and removes none | Measured over the 39-document corpus, the default before this rule left **207 435 characters — 11.92 %** — in no segment at all: **163 804 above the first entry** (in **32 of the 32** documents that get a plan), 26 041 *between* entries and 17 590 after the last. This rule closes the first part entirely, 79 % of the whole, leaving **43 631 characters (2.51 %) over 8 of 32 documents** with two named mechanisms of their own. It adds no boundary and the K2 concept count is identical with and without it (**425 = 425**); on the 12-position reference it changes **not one cell**, and hit@8 on a K2 bundle built with it holds **5 of 6 at ranks 1,1,1,1,1,–** under both tie-breaks |
|
||||
|
||||
They compose, and the order above is the order they apply in. Measured on a
|
||||
five-document tender folder (2 `pdf`, 2 `docx`, 1 `xlsx`), concepts per
|
||||
|
|
@ -235,7 +241,32 @@ concept from withheld to delivered and a priced sheet from candidate rank 10 to
|
|||
the four-character prefix matcher makes a unique identifier read as
|
||||
135-of-446 common on that bundle. Where it cannot help is decomposed rather
|
||||
than guessed: RRF fuses RANKS, so a weight moves nothing on a signal the gold
|
||||
already leads. `docs/2026-09-08-sjeldenhetsvekt.md`.
|
||||
already leads. `docs/2026-09-08-sjeldenhetsvekt.md`. Its published figures were
|
||||
measured under the pre-2026-09-10 tie-break and are not re-measured.
|
||||
|
||||
`--tie-shared-rank` is **on** by default since 2026-09-10 (opt out with
|
||||
`--no-tie-shared-rank`), and it is the one change in this library that alters a
|
||||
payload with no bundle changing — a consumer pinned to the previous excerpt
|
||||
order needs the opt-out. RRF emits a rank for every concept in every signal,
|
||||
including a signal that scored them all the same, and the declared tie-break
|
||||
then orders that group by `concept_id`; the fusion reads alphabetical order as
|
||||
if it were a measurement. Shared ranks make a signal that separates nothing
|
||||
contribute the same constant to each concept in the group. What it buys is
|
||||
general rather than cosmetic: a document the segmenter splits from 1 concept
|
||||
into 12 fills that signal's whole top tie group with its own concepts, so the
|
||||
one that leads the body signal takes position 11 instead of 1 and the document
|
||||
loses fused rank 1 to a single-concept competitor leading nothing — **the
|
||||
fusion was punishing fine-graining for being fine-grained**, which put the
|
||||
segmentation side and the retrieval side in competition over one number.
|
||||
|
||||
It shipped OFF on 2026-09-08 because hit@8 fell 5 of 6 to 4 of 6, and that
|
||||
figure is real and **conditional**: swept over 2 document-prior exponents x 3
|
||||
bundles x 6 rows, the lost row is lost only at exponent 1.0. The exponent moved
|
||||
to 0.5 on 2026-09-09 for an unrelated reason, correctly reported as moving no
|
||||
hit@8 row, and nobody measured the pair — so a rule sat behind a published
|
||||
number that had stopped being true in the same commit. A flag's "off by
|
||||
measurement" is a measurement of a *configuration*, not a property of the flag.
|
||||
`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`.
|
||||
|
||||
It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a
|
||||
`sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue