feat(propose,cli): the coverage gap had one cause, and round 7's own decomposition did not reproduce
Round 7 named two open items: a table-block candidate displacing a declared
heading (26 041 characters between entries, `md` at 3 of 4 declared headings)
and 17 590 characters after the last entry, never examined. Measured on
`a364ef4`, the first premise does not reproduce and the second is not a
separate mechanism.
`md` recovers 4 of 4, not 3 of 4. D1 (`--keep-table-heading`) is the repair
for exactly the mechanism round 7 § 5 describes, and round 7 moved it into
the default in the same commit -- so § 5 is a pre-move measurement presented
as the post-move state. That is round 7's own trap, stated in its own report:
a number about a rule is a measurement of a configuration. Nothing held the
cell, which is why it could be wrong and stay wrong.
The remaining gap has ONE cause. Every rule closes a span against the NEXT
MARK; three steps then remove a mark after its neighbour's `end` was fixed
against it. The orphan check leaks 18 527 characters over 15 of 39 documents,
`fold_units` clause 1 leaks 7 514 between entries, and the same clause on the
last run leaks all 17 590 tail characters -- with `unit_fold=False` the corpus
tail gap is 0. Round 6 already established the principle (filter at admission,
let the mark above carry the text) and it was never applied post-filter.
`--close-span-gaps` states it once, after the fold, where every removal has
happened: a span runs to the next SURVIVOR, the last to the end of the text.
It adds no boundary. Measured: 43 631 characters (2.51 %) -> 0, entries
429 = 429, K2 concepts 436 = 436, the operator's folder 52 md = 52.
All four acceptance conditions hold at once. The 12-position reference is
label-identical -- 11 of 12 under |F|[3]=12 and 10 of 12 under |F|[3]=11,
both readings reported, not one position moved. hit@8 holds [1,1,1,1,1,-] on
the round 8 bundle (436), the round 7 default (436) and Arm B (629), and the
known-negative reproduces on the new bytes: `--no-tie-shared-rank` gives
[2,1,1,1,1,-].
The pin is rebuilt against the default that actually ships
(`K2-bundle-default-20260911`, 832 files, digest 8c93e5e3...); two independent
builds of it differ in nothing, `log.md` included.
Two measurement failures are recorded rather than hidden. The corpus identity
check first used `xargs` without `-print0`, every filename split on its
spaces, `shasum` read nothing, and both sides agreed on a digest of the
failure; a control against the digest of empty input separated them. And the
smoke build cannot prove the flag arrives -- the operator's folder has a
coverage gap of zero already, so `diff -rq` is 0 either way; the plumbing has
its own test on a document that has the defect.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
a364ef4c7d
commit
956714594d
8 changed files with 592 additions and 22 deletions
11
README.md
11
README.md
|
|
@ -88,14 +88,16 @@ did not pass them.
|
|||
|
||||
### The segmentation flags
|
||||
|
||||
Eight rules are reachable from `okf build`, and since 2026-09-10 **all eight
|
||||
Nine rules are reachable from `okf build`, and since 2026-09-11 **all nine
|
||||
are ON by default** — `--outline-run 3`, `--table-grid` and `--unit-fold` since
|
||||
2026-09-08, `--drop-wrapped-outline` and `--outline-gate` since 2026-09-09, and
|
||||
2026-09-08, `--drop-wrapped-outline` and `--outline-gate` since 2026-09-09,
|
||||
`--sheet-section-rows`, `--keep-table-heading` and `--first-span-from-zero`
|
||||
since 2026-09-10 — each an operator decision, and each with an explicit
|
||||
since 2026-09-10, and `--close-span-gaps` since 2026-09-11 — each an operator
|
||||
decision, and each with an explicit
|
||||
opt-out: `--outline-run 0`, `--no-table-grid`, `--no-unit-fold`,
|
||||
`--keep-wrapped-outline`, `--no-outline-gate`, `--no-sheet-section-rows`,
|
||||
`--no-keep-table-heading`, `--no-first-span-from-zero`. Passing all eight
|
||||
`--no-keep-table-heading`, `--no-first-span-from-zero`,
|
||||
`--no-close-span-gaps`. Passing all nine
|
||||
reproduces the pre-2026-09-08 bytes exactly, and the last three reproduce the
|
||||
pre-2026-09-10 bundle byte for byte — measured with `diff -rq`, 0 differences,
|
||||
not asserted. Each line below carries the number it was measured at, and
|
||||
|
|
@ -128,6 +130,7 @@ rank was measured in the same session and did NOT ship; see
|
|||
| `--drop-wrapped-outline` (**on** by default since 2026-09-09; opt out with `--keep-wrapped-outline`) | do not admit an `--outline-run` candidate whose line continues onto the next one. Judges recovered candidates only, never a heading the document declares | quoted regulation text, whose numbered paragraphs match the outline grammar exactly: **4 → 1 concepts**, the reference. Whole corpus: **5 of 39**, all `pdf`; on the 12-document sample **8 of 34** outline candidates wrap, and none of the 26 the operator kept. On the reference it carries `pdf` from **5 of 8 to 6 of 8** together with the gate below, and neither reaches 7 of 8 without the other |
|
||||
| `--outline-gate` (**on** by default since 2026-09-09; opt out with `--no-outline-gate`) | admit `--outline-run`'s RECOVERED headings only where the document declares none of its own, plus any one recovered heading whose span covers `OUTLINE_SHARE` (0.20) of the text. Applied at admission, before spans are closed, so the text a removed mark opened is carried by the mark above it rather than lost | on the 12-document sample: `pdf` **2 of 8 → 5 of 8** alone and **7 of 8** with the rule above, `docx` unchanged at **3 of 3**. Whole corpus: it fires on **25 of 39** readable documents, changes the plan in **15 of 39**, and removes **64 of 485** proposed entries. No plan disappears (32 → 32) |
|
||||
| `--first-span-from-zero` (**on** by default since 2026-09-10; opt out with `--no-first-span-from-zero`) | start the first concept at character 0, so the text above it belongs to a segment instead of to none. Adds no boundary and removes none | Measured over the 39-document corpus, the default before this rule left **207 435 characters — 11.92 %** — in no segment at all: **163 804 above the first entry** (in **32 of the 32** documents that get a plan), 26 041 *between* entries and 17 590 after the last. This rule closes the first part entirely, 79 % of the whole, leaving **43 631 characters (2.51 %) over 8 of 32 documents** with two named mechanisms of their own. It adds no boundary and the K2 concept count is identical with and without it (**425 = 425**); on the 12-position reference it changes **not one cell**, and hit@8 on a K2 bundle built with it holds **5 of 6 at ranks 1,1,1,1,1,–** under both tie-breaks |
|
||||
| `--close-span-gaps` (**on** by default since 2026-09-11; opt out with `--no-close-span-gaps`) | close a concept's span against the next SURVIVING concept, and the last against the end of the text. Three steps remove a candidate AFTER its neighbour's span was already closed against it — the orphan check, and `fold_units` clause 1 both between entries and on the last run — and the removed mark's text then belongs to no segment. Adds no boundary and removes none; only spans' ends move | It closes the whole remainder the rule above left: **43 631 characters, 2.51 % of the corpus over 8 of the 32 documents with a plan, to 0** — both the 26 041 between entries and the 17 590 after the last. Decomposed: orphan check **18 527** over 15 of 39 documents, clause 1 **7 514** between entries, clause 1 on the last run **all 17 590** of the tail (with `unit_fold=False` the corpus tail gap is 0). The entry count is identical (**429 = 429** on the corpus, **436 = 436** concepts on K2, **52 = 52** md on a five-document folder); on the 12-position reference it changes **not one cell** (11 of 12 under `|F|`[3]=12, 10 of 12 under `|F|`[3]=11), and hit@8 holds **5 of 6 at ranks 1,1,1,1,1,–** on the new bundle, the previous default and Arm B alike |
|
||||
|
||||
They compose, and the order above is the order they apply in. Measured on a
|
||||
five-document tender folder (2 `pdf`, 2 `docx`, 1 `xlsx`), concepts per
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue