docs(cli): the new default's K2 digest, measured on the bundle it produces
The default move landed in the previous commit; this is the number it costs a
consumer, measured rather than predicted. `okf build` with no flag over the
43-document reference corpus now yields 517 concepts in 969 files, content
digest `42c8646d...`, `merged + coded rejections = 43 = N`, 817.6 s. The
delivered 2026-09-03 bundle (629 concepts, 1108 files, `9cd74519...`) is
unchanged on disk and reproducing it needs `--outline-run 0 --no-unit-fold`
AND the stamp it was built with -- "the opt-out flags" alone is true and
incomplete, so the report says both.
README and CLAUDE.md now state which two rules are on, what each opt-out is,
and that the proposer's own defaults did not move. One CLAUDE.md sentence was
false as of the move ("a no-flag build stays byte-identical") and is corrected
in place rather than deleted: adding a flag still leaves the default
byte-identical; moving it does not, which is why it took an operator decision.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
47f6619e6f
commit
efa92ce170
3 changed files with 63 additions and 13 deletions
28
CLAUDE.md
28
CLAUDE.md
|
|
@ -259,16 +259,32 @@ and fixtures, never code.
|
||||||
collide); `--ingested-at` and `--proposed-at` default to one shared epoch
|
collide); `--ingested-at` and `--proposed-at` default to one shared epoch
|
||||||
constant rather than the clock, because a wall-clock default takes
|
constant rather than the clock, because a wall-clock default takes
|
||||||
rebuild-equals-incremental away from anyone who omits them.
|
rebuild-equals-incremental away from anyone who omits them.
|
||||||
**Six segmentation rules are REACHABLE here and all six are OFF by
|
**Six segmentation rules are REACHABLE here, and since 2026-09-08 TWO of
|
||||||
default** (2026-09-08): `--outline-run N` (Arm D), `--table-grid` (Arm E),
|
them are ON by default** -- `--outline-run 3` and `--unit-fold`, the
|
||||||
|
operator's D3 answer (alternative b), each with an explicit opt-out
|
||||||
|
(`--outline-run 0`, `--no-unit-fold`) that together reproduce the pre-move
|
||||||
|
bytes. **The proposer's own defaults did NOT move** (`propose.py`'s rules stay
|
||||||
|
off): the goldens and every published reproduction block are pinned to them,
|
||||||
|
so the two layers disagree on purpose and `cli.DEFAULT_OUTLINE_RUN` /
|
||||||
|
`cli.DEFAULT_UNIT_FOLD` say where. The cost to a consumer is a re-run and it
|
||||||
|
is not small: the 43-document reference corpus goes 629 concepts / 1108 files
|
||||||
|
(`9cd74519...`, the delivered 2026-09-03 tree) to **517 / 969**
|
||||||
|
(`42c8646d...`, default flags, default epoch stamp). **And the number the
|
||||||
|
decision cites belongs to another configuration:** Arm F's 5 of 12 was
|
||||||
|
measured with `--table-grid` ON; without it the same sample scores 2 of 12
|
||||||
|
and `docx` 0 of 3, because the fold's table clause has no joined table to
|
||||||
|
fold. The six (2026-09-08): `--outline-run N` (Arm D), `--table-grid` (Arm E),
|
||||||
`--unit-fold` (Arm F), `--keep-table-heading` (D1), `--sheet-section-rows`
|
`--unit-fold` (Arm F), `--keep-table-heading` (D1), `--sheet-section-rows`
|
||||||
and `--drop-wrapped-outline` (both D3), each passed to the proposer
|
and `--drop-wrapped-outline` (both D3), each passed to the proposer
|
||||||
unchanged. Until that day the build path called the proposer with no
|
unchanged. Until that day the build path called the proposer with no
|
||||||
arm flag at all, so a tender PDF that Arm D splits into nine concepts landed
|
arm flag at all, so a tender PDF that Arm D splits into nine concepts landed
|
||||||
as one -- a build path a full arm behind the proposer. Exposing them is not
|
as one -- a build path a full arm behind the proposer. Exposing them was not
|
||||||
the same decision as moving one: **which arm should ship as the default is
|
the same decision as moving one, and the two were taken a session apart:
|
||||||
the operator's, and a no-flag build stays byte-identical** (measured by
|
**which arm ships as the default is the operator's**, answered 2026-09-08 as
|
||||||
digest before and after, and by Arm E over all 43 corpus documents). Arm C
|
above. Adding a flag still leaves the default byte-identical (measured by
|
||||||
|
digest before and after, and by Arm E over all 43 corpus documents); MOVING
|
||||||
|
the default is the one thing that does not, which is why it took an operator
|
||||||
|
decision and carries an opt-out. Arm C
|
||||||
(`--max-segment-chars`) stays unexposed: no reference has ever been measured
|
(`--max-segment-chars`) stays unexposed: no reference has ever been measured
|
||||||
for its cap. The two D3 rules read grammars nothing else here reads: a table
|
for its cap. The two D3 rules read grammars nothing else here reads: a table
|
||||||
row's FIRST CELL (a run of bare numeric labels cuts the block that holds
|
row's FIRST CELL (a run of bare numeric labels cuts the block that holds
|
||||||
|
|
|
||||||
21
README.md
21
README.md
|
|
@ -82,17 +82,24 @@ did not pass them.
|
||||||
|
|
||||||
### The segmentation flags
|
### The segmentation flags
|
||||||
|
|
||||||
Six rules are reachable from `okf build` and **all six are off by default**;
|
Six rules are reachable from `okf build`. **Two of them are ON by default since
|
||||||
a build with none of them is byte-identical to every bundle published here. Each
|
2026-09-08** — `--outline-run 3` and `--unit-fold`, an operator decision — and
|
||||||
line below carries the number it was measured at, and nothing beyond it — none
|
each has an explicit opt-out, `--outline-run 0` and `--no-unit-fold`. Passing
|
||||||
of them is a recommendation, because which one should become the default has
|
both opt-outs reproduces the pre-2026-09-08 bytes exactly. The other four are
|
||||||
not been decided.
|
off. Each line below carries the number it was measured at, and nothing beyond
|
||||||
|
it.
|
||||||
|
|
||||||
|
**A re-run is what this costs a consumer, and it is not a small one:** on the
|
||||||
|
43-document reference corpus the default bundle goes from **629 concepts in
|
||||||
|
1108 files** to **517 in 969**. The proposer's own defaults
|
||||||
|
(`tools/okf_propose_segments.py`) did NOT move, so every published reproduction
|
||||||
|
block still runs as written.
|
||||||
|
|
||||||
| flag | what it does | measured |
|
| flag | what it does | measured |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `--outline-run N` | also propose a boundary where the document's own bare-integer numbering sustains an ascending run of at least `N` | a tender PDF whose headings are bare integers: **no boundary** at `0`, **9 concepts** at `3`, against a reference of 9 |
|
| `--outline-run N` (default **3**) | also propose a boundary where the document's own bare-integer numbering sustains an ascending run of at least `N`; `0` is this arm's opt-out | a tender PDF whose headings are bare integers: **no boundary** at `0`, **9 concepts** at `3`, against a reference of 9 |
|
||||||
| `--table-grid` | a pandoc grid-table rule line no longer closes an open table block, so one grid table is one concept | a `.docx` experience list: **21 → 6** concepts |
|
| `--table-grid` | a pandoc grid-table rule line no longer closes an open table block, so one grid table is one concept | a `.docx` experience list: **21 → 6** concepts |
|
||||||
| `--unit-fold` | discard a contents-list run, fold a deeper heading into its parent, fold a table into the shorter heading that introduces it. Adds no boundary, so it can only reduce a plan | on a 12-document sample scored against an operator's unit worksheet: **5 of 12** match, against **2** for the default, no cell worse. Whole corpus: 679 → 485 proposed segments over 39 readable documents |
|
| `--unit-fold` (**on** by default; opt out with `--no-unit-fold`) | discard a contents-list run, fold a deeper heading into its parent, fold a table into the shorter heading that introduces it. Adds no boundary, so it can only reduce a plan | on a 12-document sample scored against an operator's unit worksheet: **5 of 12** match — but that figure was measured with `--table-grid` ON, and the shipped default does not include it. Measured without it the same sample scores **2 of 12**, `docx` **0 of 3**, because the fold's table clause has no joined table to fold |
|
||||||
| `--keep-table-heading` | keep a heading whose body is empty only because a table opens under it, and absorb that table into its span | the two spreadsheets in that corpus, and **0 of 32 `pdf` and 0 of 5 `docx`**: the concept count does not move (1 → 1), its first byte does — the concept gains the heading line it was missing |
|
| `--keep-table-heading` | keep a heading whose body is empty only because a table opens under it, and absorb that table into its span | the two spreadsheets in that corpus, and **0 of 32 `pdf` and 0 of 5 `docx`**: the concept count does not move (1 → 1), its first byte does — the concept gains the heading line it was missing |
|
||||||
| `--sheet-section-rows` | cut an open table block at the rows that label its sections — a run of at least three rows whose first cell is a bare numeric label. The opposite direction from `--table-grid`, which decides how far a block extends | a tender price sheet whose whole body is one table block: **1 → 12 concepts**, against a reference of 11 cost groups plus the sheet's preamble. Whole corpus: **1 of 39** readable documents changes, **0 of 32 `pdf`, 0 of 5 `docx`, 1 of 2 `xlsx`** |
|
| `--sheet-section-rows` | cut an open table block at the rows that label its sections — a run of at least three rows whose first cell is a bare numeric label. The opposite direction from `--table-grid`, which decides how far a block extends | a tender price sheet whose whole body is one table block: **1 → 12 concepts**, against a reference of 11 cost groups plus the sheet's preamble. Whole corpus: **1 of 39** readable documents changes, **0 of 32 `pdf`, 0 of 5 `docx`, 1 of 2 `xlsx`** |
|
||||||
| `--drop-wrapped-outline` | do not admit an `--outline-run` candidate whose line continues onto the next one. Judges recovered candidates only, never a heading the document declares | quoted regulation text, whose numbered paragraphs match the outline grammar exactly: **4 → 1 concepts**, the reference. Whole corpus: **5 of 39**, all `pdf`; on the 12-document sample **8 of 34** outline candidates wrap, and none of the 26 the operator kept |
|
| `--drop-wrapped-outline` | do not admit an `--outline-run` candidate whose line continues onto the next one. Judges recovered candidates only, never a heading the document declares | quoted regulation text, whose numbered paragraphs match the outline grammar exactly: **4 → 1 concepts**, the reference. Whole corpus: **5 of 39**, all `pdf`; on the 12-document sample **8 of 34** outline candidates wrap, and none of the 26 the operator kept |
|
||||||
|
|
|
||||||
|
|
@ -363,6 +363,33 @@ tender PDF is 9 either way, which was the other half of that check.
|
||||||
The default is the operator's decision and it is implemented as given. The
|
The default is the operator's decision and it is implemented as given. The
|
||||||
measurement above is the report back, not an argument for reversing it.
|
measurement above is the report back, not an argument for reversing it.
|
||||||
|
|
||||||
|
### The K2 bundle on the new default, and what `9cd74519…` still names
|
||||||
|
|
||||||
|
The injection asks for the K2 digest re-measured on the new default and written
|
||||||
|
down as the current one. Three bundles were built this session, all from the
|
||||||
|
same 43-document corpus, all with `--bundle-id k2-trinn1-20260903 --okf-version
|
||||||
|
0.2` and the **default epoch stamp** (`1970-01-01T00:00:00Z`), which is why
|
||||||
|
none of them can be compared to the delivered bundle's digest directly:
|
||||||
|
|
||||||
|
| build | flags | concepts | files | content digest |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| the delivered bundle, 2026-09-03 | two-script path, `--ingested-at 2026-09-03T00:00:00Z` | 629 | 1108 | `9cd74519…` |
|
||||||
|
| Arm B control | `--outline-run 0 --no-unit-fold` equivalent | 629 | 1108 | `f09035d9…` |
|
||||||
|
| the D1 pair's other half | `--keep-table-heading` | 629 | 1108 | `432ca424…` |
|
||||||
|
| **the new default** | none | **517** | **969** | **`42c8646d…`** |
|
||||||
|
|
||||||
|
`merged + coded rejections = 43 = N` on all three, exit 0.
|
||||||
|
|
||||||
|
**`9cd74519…` names a bundle, not a configuration.** It is the delivered
|
||||||
|
2026-09-03 tree and it is unchanged on disk, which is the control this session
|
||||||
|
re-ran. Reproducing it needs the stamp it was built with as well as the
|
||||||
|
opt-out flags, so "it now holds only under `--outline-run 0 --no-unit-fold`" is
|
||||||
|
true and incomplete: it also needs `--ingested-at 2026-09-03T00:00:00Z`. The
|
||||||
|
digest a consumer should compare against from today is `42c8646d…`, for the
|
||||||
|
command in the table above, and the 629 → 517 concept move is what a re-run
|
||||||
|
costs them.
|
||||||
|
|
||||||
|
|
||||||
## 9. Tests
|
## 9. Tests
|
||||||
|
|
||||||
`1397 → 1414`. Seventeen new, and which of them were red first is stated rather
|
`1397 → 1414`. Seventeen new, and which of them were red first is stated rather
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue