open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(cli,propose): reach the arms from okf build, keep a sheet heading behind a flag, fix two PDF contents mechanisms

Browse source 
K3 round 2, per file type. Order 20260908T143513Z-6327528123-from-.claude, carrying two operator decisions taken beforehand: D1 the orphan-gate variant goes behind a flag, D2 the arms become reachable from `okf build`. No default moved. Report: docs/2026-09-08-k3-runde2-per-filtype.md.

THE REPRODUCTION HELD, all three numbers, before any edit: `okf build` on the five-document tender folder gives 31 markdown files with both PDFs flat and 5/5 merged; the tender PDF gives no boundary without a flag and 9 with `--outline-run 3` (reference 9); the price sheet gives 1 on HEAD. Both proposer runs had to go through `bash -c` -- zsh does not word-split an unquoted `$flags`, so a sweep hands `--outline-run 3` to argparse as one token and every row comes back exit 2.

D2 -- `cli.py:_propose_plans` called the proposer with no arm argument, so the build path ran Arm B while `tools/okf_propose_segments.py` could run D, E and F. It now passes `--outline-run`, `--table-grid`, `--unit-fold` and `--keep-table-heading` through unchanged. THE DEFAULT DOES NOT MOVE and that is measured, not asserted: same folder, no flags, before and after the change, digest 3af10770...8fbbe2 both times and `diff -rq` clean. The "before" bundle was built before the first edit, because the editable install reads src/ live. Red test on the PLANS and on titles rather than a count, with the same fixture and no flags as its control. Per-document table for B/D/E/F/F2 is in the README and the report; the tender PDF is 1 under the default and 9 under every arm above it, and the reference is 9.

D1 -- a sheet heading with a table opening under it has an empty body, so the orphan check drops it: the NAME survives (carried onto the table block), the LINE does not. `--keep-table-heading` lets the heading survive and absorb the table instead. Price sheet 1 -> 1 concepts, `source_offset` [34, 11048] -> [0, 11048], body now starting at the heading. ELEVEN IS NOT REACHABLE THIS WAY and the number says why: the sheet is one heading and one continuous pipe-table block, and the eleven cost groups are eleven ROWS inside it (lines 10-20 of 103). What is missing is a section-row rule inside a sheet -- the opposite of `--table-grid`. Corpus: the flag changes 2 of 39 documents, both `.xlsx`, under arms B, E and F alike; known-negative 0 of 32 `pdf` and 0 of 5 `docx`. With it off, Arm E over all 43 is byte-identical to session 109's tree (33 plans, 43 `.err`, 4 FAILED, diff exit 0, counts asserted first).

THE PDF REMAINDER, one at a time. Position 9: clause 1 read the list AFTER the orphan check, and a contents list without dot leaders is a run of bodiless headings, so all but the last entry were already gone and the run was one. The run is now measured on the pre-orphan list, predicate written once and read in both places. 11 -> 10. Position 7: the same clause required siblings, and a numbered report's contents list interleaves 1.1/1.1.1/2.1 -- its 34 entries are one block that the level condition cut into runs of 9, 1, 1, 1, 5, 2, 10, 2 and 3, so the short runs survived. The level condition is dropped; the run LENGTH, which is what the CONTENTS_RUN sweep bought, is unchanged. Measured outward: the relaxation changes 1 document of 39 and removes exactly the leftover line. 34 -> 33.

TWO REMAINDERS ARE DECLINED WITH NUMBERS RATHER THAN FIXED. Position 1: the three level-1 candidates are 3 of 3 `rule:outline`, same level, same grammar, and the operator keeps one of them by prose alone -- there is no property to read. Position 4: a title-length rule was measured on paper and falsified -- a real chapter is 56 characters and a real heading in a document the arms already score correct is 88, sitting between position 4's 86 and 91, so no threshold separates the classes. Position 0 stays an extraction failure.

ONE SHIPPED EXPECTATION MOVED and is stated rather than quietly updated: `Innhold 1` is now discarded with the contents list it heads. Its body is in no segment afterwards, which is a real cost on a fixture where that heading has one.

Nine new tests: five red before the implementation, four green by construction and named as such. Three mutations, three red, unmutated control green each time -- restoring the level condition, computing the run post-orphan, absorbing a table unconditionally. 1379 -> 1388 tests. ruff clean, mypy --strict clean on 17 files. K2 bundle untouched (1108 files, 9cd74519...). The K2 ranking control is NOT measured: no bundle was rebuilt with the flag, so the rank is a prediction.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 18:21:25 +02:00
commit ff79cfa19b
7 changed files with 989 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

388
docs/2026-09-08-k3-runde2-per-filtype.md Normal file
View file

@ -0,0 +1,388 @@
# K3 round 2, per file type: the build command, the spreadsheet, the PDF remainder
Second iteration of the per-file-type directive of 2026-09-08 13:05Z. Order
`20260908T143513Z-6327528123-from-.claude`, which carries two decisions already
taken by the operator and asks for neither to be widened: **D1** — the orphan
gate variant goes behind a flag, off by default; **D2** — the measurement arms
become reachable from `okf build`, with the default unchanged.
Everything below is measured on the same 43-document corpus and the same
12-document sample as `docs/2026-09-08-k3-arm-f-mot-enhetsarket.md`, against the
same reference: the operator's filled unit worksheet, which is local-only and
gitignored. No document title or extracted text from it is reproduced here.
## Section 0: what was measured and what was not
**Measured.** The three reproduction numbers the order names, before any edit.
Four flags reaching the proposer from `okf build`, on the artifact. A
byte-identical no-flag bundle, by digest and by `diff -r`. Two rule adjustments,
each red-first, each with a known-negative and a mutation control. The whole
corpus under six configurations, from ONE cached extraction so every variant
reads the same bytes. The four PDF remainders, each decomposed to a named
mechanism.
**Not measured.** No default moved. No K2 bundle was rebuilt, so the K2 ranking
control (the priced concept at rank ≤ 12 with `--cost-vocabulary --k 12`) is
**not measured this round** — it is a property of a bundle built WITH the flag,
and no such bundle exists. The K2 bundle on disk is unchanged, which proves
nothing was overwritten and nothing more. Two of the four PDF remainders are
diagnosed and declined rather than fixed, with the numbers that decline them.
## 1. The reproduction, before any edit
| # | control | command | result |
|---|---|---|---|
| a | `okf build` on the five-document tender folder | `okf build ~/okf-test/dokumenter --bundle … --bundle-id … --okf-version 0.2` | **31** markdown files, both PDFs as one flat `inbox-*.md` each, merged **5/5**, 0 rejections |
| b | the tender PDF through the proposer | `tools/okf_propose_segments.py "<tender>.pdf" --out …` | **no boundary** (exit 1); with `--outline-run 3`: **9** entries |
| c | the price sheet through the proposer | same, on the `.xlsx` | **1** entry on HEAD, against 3 in the 2026-09-07 archive; the reference is **11** cost groups |
All three reproduce. (b) and (c) had to be run under `bash -c`: zsh does not
word-split an unquoted `$flags`, so a sweep loop hands `--outline-run 3` to
`argparse` as ONE token and every row comes back `exit 2`. That is the trap
`docs/2026-09-08-k3-arm-f-mot-enhetsarket.md` recorded, hit again.
## 2. D2: `okf build` reaches the arms, and the default does not move
`cli.py`'s `_propose_plans` called `propose.run` with no arm argument, so the
build path ran Arm B while `tools/okf_propose_segments.py` could run D, E and F.
It now passes `--outline-run`, `--table-grid`, `--unit-fold` and
`--keep-table-heading` through unchanged, same names and same semantics.
**The red test came first** and asserts on the PLANS, on titles rather than on a
count: a fold that kept the right number of concepts by discarding the wrong
ones passes a count assertion. Its control is the same fixture with no flags.
**The byte control.** Same folder, no flags, before and after the change:
```
BEFORE 3af107707b66090e7ab4e4372c5976ca2c448ac0d771c7e9a9d74746d08fbbe2
AFTER 3af107707b66090e7ab4e4372c5976ca2c448ac0d771c7e9a9d74746d08fbbe2
diff -rq: identical, 31 markdown files either way
```
The "before" bundle was built at the start of the session, from the editable
install, before the first edit — a fresh build afterwards would have read the
changed `src/` and compared a tree against itself.
### Concepts per document, per configuration
Five documents, 2 `pdf` / 2 `docx` / 1 `xlsx`. Every column merged 5 of 5 with
0 coded rejections.
| document | B (default) | D | E | F | F2 |
|---|---|---|---|---|---|
| tender PDF, technical requirements | no boundary → 1 | **9** | 9 | 9 | 9 |
| tender PDF, technical layout | no boundary → 1 | no boundary | no boundary | no boundary | no boundary |
| price sheet `.xlsx` | 1 | 1 | 1 | 1 | 1 |
| experience list `.docx` | 21 | 21 | 6 | **3** | 3 |
| agreement `.docx` | 2 | 2 | 1 | **1** | 1 |
| markdown files in the bundle | 31 | 49 | 33 | 30 | 30 |
`B` = no flags. `D` = `--outline-run 3`. `E` = `+ --table-grid`.
`F` = `+ --unit-fold`. `F2` = `+ --keep-table-heading`.
The reference for the first row is 9. **What the measurement says about moving
the default is therefore one sentence: on this folder the default is a full arm
behind the proposer on the one document with a reference.** It is five
documents, one folder, and the decision is the operator's.
## 3. D1: the spreadsheet, behind `--keep-table-heading`
**The mechanism, named.** `propose.py`'s orphan check drops a heading whose body
is empty, and a sheet heading with a pipe table opening two lines below it has
an empty body. The heading's NAME survives (it is carried onto the table block),
its LINE does not: the concept starts at the first table row.
**The variant.** When a heading is orphaned *only* because the very next mark is
a table block, the heading survives and the table is absorbed into its span
rather than emitted. The count does not move; the first byte does. Measured on
the price sheet:
| | default | `--keep-table-heading` |
|---|---|---|
| concepts | 1 | 1 |
| `source_offset` | `[34, 11048]` | `[0, 11048]` |
| first body line | `\| Prisskjema \| …` | `## Prissammenstilling {#sheet-1}` |
**The reference wants eleven, and eleven is not reachable this way.** The sheet
extracts to 103 lines: ONE heading (line 1) and one continuous pipe-table block
(lines 3–103). The eleven numbered cost groups are eleven ROWS inside that
block (`01`, `11+12`, `02` … `92`, lines 10–20). Reaching them needs a rule that
splits a table on its section rows — the exact opposite of `--table-grid`, which
exists to stop a table becoming one concept per row group. **What is missing is
a section-row rule inside a sheet, and no heading-and-table rule can supply it.**
### The corpus numbers for D1
From one cached extraction, so every configuration reads identical bytes.
39 of 43 documents are extractable (4 `FAILED`, unchanged).
| control | result |
|---|---|
| documents changed by `--keep-table-heading`, arm B | **2 of 39** — both `.xlsx` |
| documents changed, arm E | **2 of 39** — the same two |
| documents changed, arm F | **2 of 39** — the same two |
| **known-negative: `pdf` changed** | **0 of 32** |
| **known-negative: `docx` changed** | **0 of 5** |
| both spreadsheets in the corpus | **2 of 2** changed, `[20,1921] → [0,1921]` and `[34,11048] → [0,11048]` |
**With the flag off, nothing moves.** Arm E re-run over all 43 documents on the
round-2 code, against session 109's Arm E tree: 33 plans either way, 43 `.err`
either way, 4 `FAILED` either way, `diff -rq … -x '*.err' -x '_index.txt'` exit
`0`. Counts asserted before the diff, because a diff over two trees where every
document failed compares nothing and exits `0`.
**The K2 ranking control is not measured.** `56ae274` fixed a ranking regression
by restoring the concept's title, and this flag restores the title's LINE to the
body — which is the same direction, and a prediction, not a measurement. No K2
bundle was rebuilt with the flag, so nothing here says where the priced concept
ranks in one. It is round 3's.
## 4. The PDF remainder, one at a time
The reference row was `pdf` **2 of 8**. Each of the four is decomposed below.
Two get a rule adjustment behind the SAME flag as Arm F (`--unit-fold`); two are
declined, with the numbers that decline them.
### Position 9 — one leftover contents line. FIXED.
**Mechanism.** Clause 1 discards a RUN of at least `CONTENTS_RUN` page-numbered
headings. It reads the list `fold_units` is given, which is the list AFTER the
orphan check — and a contents list without dot leaders is precisely a run of
bodiless headings, so the orphan check deletes all of it but the last entry. The
run clause 1 looks for is gone before clause 1 runs; what is left is a run of
one, which is below the threshold by design.
**Adjustment (one).** The run is measured on the PRE-orphan list and carried
forward on the candidate. The predicate is written once and read in both places,
so there cannot be two definitions of a contents list.
**Result.** Position 9: 11 → **10**. Still `too fine` — the reference is 4, and
the remaining 6 are Arm D recovering a risk table's rows as an integer outline,
which the operator ruled out in free text ("radene er ikke overskrifter"). That
is Arm D's recovery, not the fold's, and `--unit-fold` adds no boundary it could
withdraw.
### Position 7 — a contents list that interleaves levels. FIXED.
**Mechanism.** The same clause also required the run's members to be SIBLINGS.
A numbered report's contents list interleaves `1.1`, `1.1.1`, `2.1`, so the
level condition cuts one block into many runs. Measured on the document: its 34
contents entries are one continuous block of page-numbered headings at levels 2
and 3, and the level condition split it into runs of 9, 1, 1, 1, 5, 2, 10, 2 and
3. The runs of one and two survived, and one of them
(`6.2.2 Tverrfaglig kontroll ....`) had a body, so it was emitted as a concept
while its neighbours were discarded.
**Adjustment (one).** The level condition is dropped. What still bounds the rule
is the run LENGTH — the number `CONTENTS_RUN`'s sweep bought — and that is
unchanged.
**Generalisation, measured outward.** Across all 39 readable documents the
relaxation changes **1 document**, and there it removes exactly that one line:
| | value |
|---|---|
| documents whose plan changes | **1 of 39** |
| entries removed | 1 (position 7: 34 → **33**) |
| other documents touched | 0 |
**Result.** Position 7: 34 → **33**. Still `too fine` against a reference of 30,
and the operator had already marked that document "none". The remaining 3 are
the heading rule proposing units the operator does not want; that is not the
fold's to withdraw either.
**One shipped expectation moved with it, and it is stated rather than quietly
updated.** In `test_a_contents_run_is_discarded_and_the_body_survives` the
level-1 heading `Innhold 1` used to survive, because it is not a sibling of the
three level-2 contents lines after it. It is now part of the run and is
discarded — which is what "innholdsfortegnelsen er ikke konsepter" says about a
contents heading. The cost is real and bounded: in that fixture the heading has
a body of its own, and a discarded candidate leaves its body in no segment.
### Position 1 — two recovered chapters. DECLINED, with numbers.
**Mechanism.** 17 concepts against a reference of 15. The three level-1
candidates are `Innledning` (§1), `Referanser` (§6) and `Vedlegg` (§7), and all
three are `rule:outline` — Arm D's recovery of the document's bare-integer
numbering. The operator's free text keeps ONE of them ("vedlegget er eget
oppslag") and the criterion's reference set keeps only that one.
**Why no rule.** **3 of 3** are the same rule, the same level, the same grammar
and the same shape; the worksheet showed the operator dotted headings only, so
the reference set contains `Vedlegg` because the operator named it in prose, not
because anything in the document distinguishes it. A rule that dropped two of
the three would have to read the title, and there is no property here to read.
One document is not a rule, and this one does not even offer a property.
### Position 4 — quoted regulation text. DECLINED, with numbers.
**Mechanism.** The reference is 1 (the whole note as one argument); Arm B gives
exactly that and every arm above it gives 4. The four are `rule:outline`
candidates recovered from numbered legal paragraphs — full sentences, truncated
at the line break, which is why the operator wrote "sitert forskriftstekst er
ikke overskrifter".
**The rule that was tried on paper and falsified.** A heading is a heading and a
sentence is not, so: discard an outline candidate whose TITLE is longer than
some threshold. Every outline title in the 12-document sample, sorted:
| class | lengths |
|---|---|
| real headings | 7, 7, 7, 7, 8, 8, 9, 9, 10, 10, 10, 10, 12, 12, 13, 14, 18, 18, 20, 24, 24, 24, 31, 45, **56**, **88** |
| position 4's sentences | 86, 91, 92, 100 |
**The classes overlap and no threshold separates them.** A real chapter at
position 9 is 56 characters (`Rutiner for behandling av endringer av betydning
for SHA`) and a real heading at position 2 — a document the arms already score
`correct` — is 88. A threshold at 90 leaves one of position 4's four and costs
nothing; a threshold low enough to take all four deletes a chapter at a document
the arms currently get right. Declined.
### Position 0 — an extraction failure. REPORTED, not repaired.
95.1 % `(cid:N)` glyphs. No segmentation rule can repair a document whose text
never arrived. Unchanged, and it is the one position where Arm F proposes
nothing at all because there is no candidate to fold.
## 5. Per file type, before and after
The 12-document sample, concepts per arm. `Arm F` is session 109's number;
`F2` is this round's code with all four flags on.
| pos | type | reference | Arm B | Arm D | Arm E | Arm F (109) | F2 |
|---|---|---|---|---|---|---|---|
| 0 | pdf | — | 0 | 0 | 0 | 0 | 0 |
| 1 | pdf | 15 | 20 | 23 | 23 | 17 | 17 |
| 2 | pdf | 9 | 0 | 9 | 9 | 9 | 9 |
| 3 | xlsx | 11 | 1 | 1 | 1 | 1 | 1 |
| 4 | pdf | 1 | 0 | 4 | 4 | 4 | 4 |
| 5 | docx | 3 | 21 | 21 | 6 | 3 | 3 |
| 6 | pdf | 6 | 6 | 7 | 7 | 7 | 7 |
| 7 | pdf | 30 | 45 | 48 | 48 | 34 | **33** |
| 8 | pdf | 8 | 0 | 8 | 8 | 8 | 8 |
| 9 | pdf | 4 | 5 | 11 | 11 | 11 | **10** |
| 10 | docx | 2 | 15 | 15 | 3 | 2 | 2 |
| 11 | docx | 1 | 2 | 2 | 1 | 1 | 1 |
**The label row does not move.** Using session 109's criterion under its
dominance gate — which is the reading that survived, because column (c) is
relative and (a)/(b) are absolute:
| type | n | Arm B | Arm D | Arm E | Arm F | **F2** |
|---|---|---|---|---|---|---|
| `pdf` | 8 | 2 | 2 | 2 | 2 | **2** |
| `docx` | 3 | 0 | 0 | 1 | 3 | **3** |
| `xlsx` | 1 | 0 | 0 | 0 | 0 | **0** |
**Two cells moved closer and neither crossed.** Position 7 loses a concept the
reference does not want and is still 3 too many; position 9 loses one and is
still 6 too many. **`pdf` is 2 of 8 for the third round running**, and this
round says why with a mechanism per position rather than with one number:
two of the four are fixed as far as the fold can reach, and two are declined
on measurements that are written down above.
**Nothing regressed.** No count moved away from its reference in any cell.
## 6. Whole corpus, 43 documents
| figure | Arm E | Arm F (109) | **F2** | denominator |
|---|---|---|---|---|
| plans written | 33 | 32 | **32** | 43 |
| proposed entries | 679 | 491 | **485** | — |
| `.err` files | 43 | 43 | **43** | 43 |
| `.err` recording `FAILED` | 4 | 4 | **4** | so extractable **39/43** |
| plans differing from Arm F | — | — | **6 of 32** | — |
The six: two `.xlsx` where only the span start moves (D1), three `.pdf` where a
leftover contents line goes (the pre-orphan run), and one `.pdf` where the
level relaxation removes one more (position 7). Cross-checked: the entry count
of every one of the 39 written plans equals the count computed from the cached
extraction, **39 of 39**.
`K2-bundle-20260903` is unchanged: 1108 files,
`9cd745194346cda0c70eab9c7136fa44506203bbe85bc17d7eff2766c6e9b4d1`. That is a
statement that nothing overwrote it, not a rebuild.
## 7. Tests
Nine new tests, plus one shipped expectation moved and stated. **Five were red
before the implementation** (the bodiless run, the level-crossing run, the two
halves of D1, and the arms reaching `okf build`). **Four were green by
construction and are named as such rather than counted as evidence**: two
known-negatives (a single page-numbered heading survives; a heading with its own
body absorbs nothing — the latter was red only as a `TypeError` before the
parameter existed, and its value is the mutation below), a default control, and
a byte control. `--keep-table-heading` taking no argument is green either way,
because `argparse` rejects an unknown flag with the same exit code; it is a
guard, not a measurement.
Three mutations, three red, with the unmutated control green each time:
| mutation | expected to kill | result |
|---|---|---|
| restore the same-level condition in the contents run | the level test | **red** (2 tests) |
| compute the contents run after the orphan check | the bodiless-run test | **red** (3 tests) |
| absorb a table unconditionally | the D1 known-negative | **red** (1 test) |
| (control) unmutated | — | green, 93/93 |
`1379 → 1388` tests. `ruff check src tests tools` clean, `ruff format --check`
clean, `mypy --strict src/` clean on 17 files.
## Reproducing
```
# 1. The three numbers, in bash -- zsh does not split an unquoted $flags.
okf build ~/okf-test/dokumenter --bundle /tmp/b --bundle-id r --okf-version 0.2
find /tmp/b -name '*.md' | wc -l # 31
python3 tools/okf_propose_segments.py "<tender>.pdf" --out /tmp/t.json
python3 tools/okf_propose_segments.py "<tender>.pdf" --out /tmp/t.json --outline-run 3
# 2. The corpus, once, cached -- every variant then reads identical bytes.
# Ascending chunks, each a foreground call under 600 s.
C=~/corpora/okf-telling-20260829
for lo_hi in "1 12" "13 18" "19 26" "27 43"; do
set -- $lo_hi
arm_run "$C/K2-plans-r2-armFoff-20260908" 2026-09-07T00:00:00Z "$1" "$2" \
--outline-run 3 --table-grid
arm_run "$C/K2-plans-r2-armF2-20260908" 2026-09-08T00:00:00Z "$1" "$2" \
--outline-run 3 --table-grid --unit-fold --keep-table-heading
done
# 3. Counts BEFORE the diff.
ls "$C"/K2-plans-r2-armFoff-20260908/*.json | wc -l # 33
ls "$C"/K2-plans-r2-armF2-20260908/*.json | wc -l # 32
grep -l FAILED "$C"/K2-plans-r2-armF2-20260908/*.err | wc -l # 4 -> 39/43
# 4. The flag-off identity, and the flag-on delta.
diff -rq "$C/K2-plans-armE-check-20260908" "$C/K2-plans-r2-armFoff-20260908" \
-x '*.err' -x '_index.txt' # exit 0
diff -rq "$C/K2-plans-armF-20260908" "$C/K2-plans-r2-armF2-20260908" \
-x '*.err' -x '_index.txt' # 6 files
# 5. The consumer bundle, untouched.
cd "$C" && LC_ALL=C find K2-bundle-20260903 -type f | LC_ALL=C sort \
| xargs shasum -a 256 | shasum -a 256
# 1108 files, 9cd745194346cda0c70eab9c7136fa44506203bbe85bc17d7eff2766c6e9b4d1
```
`arm_run` is the function published in `docs/2026-09-07-k3-arm-e.md`
§ Reproducing, unchanged.
## Honesty limits
- **n = 12, one rater, one reference.** Per file type the denominators are 8, 3
and 1. A `1/1` is not a rate, and the `xlsx` row is that.
- **Two of the four PDF remainders are declined, not solved.** The declines rest
on measurements written above; they are still declines.
- **The two adjustments were derived from the two documents they move**, and
screened outward on 39. The outward screen is what bounds the fitting; it does
not remove it.
- **The rules were measured on plans, never on a bundle.** No bundle was built
with `--unit-fold` or `--keep-table-heading` beyond the five-document folder,
so nothing here says what these concepts look like after materialisation at
corpus scale.
- **The K2 ranking control is not measured**, for the reason § 3 gives.
- **No default moved, and no threshold is set.** Which arm should ship as the
default is the operator's decision and the measurement above is one folder,
one corpus and one worksheet wide.