open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(propose,extract,cli): a title that ends in a number, and a converter's own anchor in a concept id

Browse source 
Round 9: the four rests in STATE's NESTE that needed no operator decision.

CLAUSE 1 CLASSIFIED BY THE NUMBER, NOT THE TITLE. `_TRAILING_PAGE_NUMBER`
admitted a candidate into a contents run by asking whether the title ended in
an integer -- a question about the number. A drawing's dimension chain, a
schematic's labels, a door schedule, a coordinate column and a soil-layer
table all end in integers and name nothing. Measured over the 43-document
corpus: 68 candidates discarded over 11 of 39 readable documents, of which
19 over 5 documents are data rows.

That corrects round 8's own decomposition. Its "four misclassified numeric
tables and seven real contents listings" needs each document on one side, and
two of the eleven are both. Read across all 68 titles rather than the
three-title sample: 5 documents carry a data row, 8 carry a real entry.

`--contents-name` requires a NAME to survive stripping the page number. The
threshold is SWEPT, not chosen, and collapses at both ends: at an alphabetic
run of 1 a door schedule keeps a stray `V` and 13 of 19 are rescued; at 3 the
two-letter section name `VA` stops being a name, falls out of run membership,
and takes `RIB`, `MMI` and `Tittelfelt` below `CONTENTS_RUN` with it -- one
acronym costing four REAL entries. At 2: 16 of 19 rescued, 0 of 49 regressed.
The three not rescued carry a real word and are named rather than rounded off.

THE CONVERTER'S ANCHOR WAS IN THE CONCEPT ID. Pandoc writes a sheet as
`## <name> {#sheet-N}` and a titled slide as `## <title> {#slide-N}`. Because
a filename is reduced FROM the title, the anchor reached both. Operator
authorised the strip 2026-09-09 after the exposure was counted: 2 of 810
concepts on the previous default bundle, 2 of 1108 on Arm B, 1 of 26 on the
operator's folder. Two ids renamed, one of which `portfolio-optimiser` has
cited in writing; both are in the report so that message can be sent.

One rule in one function, read by BOTH title-forming sites -- a rule in only
one would leave the id and the title naming the same concept differently. The
known-negative is the point: `Mal for {kundenavn}` is a title an author wrote.

odt/rtf/pptx MEASURED END TO END FOR THE FIRST TIME, on hand-built documents,
because the corpus denominator is genuinely zero (86 files: 66 pdf, 10 docx,
4 xlsx, 2 zip, 2 smc, 2 doc). `_EVIDENCE` gains a third class rather than
stretching an existing one: `constructed` means the row has met a document,
but not one anyone wrote for their own purposes. odt 1 of 1 declared headings;
pptx 2 of 2 on a deck that declares slide titles and 0 of 2 on one that does
not -- round 7's reading of pptx was a fixture property, not the format; rtf
0 segments, because the container has no heading style and the author's title
is bold text. rtf is the one open finding.

ACCEPTANCE, all four. The 12-position reference is label-identical in BOTH
readings (pdf 7/8, docx 3/3, xlsx 0/1 or 1/1, sheet 10/12 or 11/12). One K2
bundle carrying both changes: 453 concepts / 865 md, hit@8 [1,1,1,1,1,None]
on it AND on Arm B, with the known-negative still reproducing on the new
bytes. `okf project` byte-equal to `okf build`, `diff -r` empty. Consumer
cost is a re-run: 436/832 -> 453/865, digest 21af4a1aa98315cf.

Three published numbers corrected: README's 596 tests (1515), README's "15
concepts out" for `okf project` (that was the O6 defect; it is 26), and O6's
print-mode method, which does not reproduce without --allowedTools.

Report: docs/2026-09-09-k3-runde9-restene.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 21:45:08 +02:00
commit 6ff18fd703
15 changed files with 964 additions and 67 deletions
Download patch file Download diff file Expand all files Collapse all files

19
CLAUDE.md
View file

@ -309,8 +309,20 @@ and fixtures, never code.
collide); `--ingested-at` and `--proposed-at` default to one shared epoch
constant rather than the clock, because a wall-clock default takes
rebuild-equals-incremental away from anyone who omits them.
**Nine segmentation rules are REACHABLE here, and since 2026-09-11 ALL
NINE are ON by default** -- `--outline-run 3`, `--table-grid` and
**TEN segmentation rules are REACHABLE here, and since 2026-09-09 ALL
TEN are ON by default** -- the tenth is `--contents-name`, round 9's repair
of clause 1, which admits a title into a contents run only when a NAME
survives stripping its page number. Measured: clause 1 discarded 68
candidates over 11 of 39 readable documents, 19 of them over 5 documents
rows of a drawing's dimension chain, a schematic's labels, a door schedule,
a coordinate column and a soil-layer table. The threshold is SWEPT
(`propose.CONTENTS_NAME_RUN = 2`) and collapses at both ends: at 1 it
rescues 13 of 19, at 3 the two-letter section name `VA` stops being a name
and takes four REAL contents entries with it. At 2 it rescues 16 of 19 and 0
of 49. Corpus 429 -> 447 candidates, K2 436 -> 453 concepts / 832 -> 865 md
(`21af4a1aa98315cf...`), the 12-position reference label-identical in BOTH
readings and hit@8 `[1,1,1,1,1,None]` on the new bundle AND Arm B. Opt-out
`--no-contents-name`. The nine below are unchanged -- `--outline-run 3`, `--table-grid` and
`--unit-fold` since 2026-09-08, `--drop-wrapped-outline` and
`--outline-gate` since 2026-09-09, `--sheet-section-rows`,
`--keep-table-heading` and `--first-span-from-zero` since 2026-09-10, and
@ -370,7 +382,8 @@ and fixtures, never code.
decision cites belongs to another configuration:** Arm F's 5 of 12 was
measured with `--table-grid` ON; without it the same sample scores 2 of 12
and `docx` 0 of 3, because the fold's table clause has no joined table to
fold. The nine: `--outline-run N` (Arm D), `--table-grid` (Arm E),
fold. The ten: `--contents-name` (round 9), `--outline-run N` (Arm D),
`--table-grid` (Arm E),
`--unit-fold` (Arm F), `--keep-table-heading` (D1), `--sheet-section-rows`
and `--drop-wrapped-outline` (both D3), `--outline-gate` (G1+G2),
`--first-span-from-zero` and `--close-span-gaps`, each passed to the

80
README.md
View file

@ -49,7 +49,8 @@ a profile rather than by constants (see [Upstream OKF
versions](#upstream-okf-versions)). Binary extraction runs behind the
optional `[extract]` extra: `pdf` through a PDF parser, and five office
formats through a vendored document converter. Three of those five office
rows are **unmeasured** — see [Binary extraction](#binary-extraction). Phase 4
rows are **constructed** rather than measured — see
[Binary extraction](#binary-extraction). Phase 4
(the Node half) is planned (see `docs/plan/`).
## Install in detail
@ -370,15 +371,31 @@ read, what it wrote, and which documents a question cannot reach. Start `claude`
in that directory and ask in plain language; the generated skill runs the
pre-pass and the contract check itself and marks every claim with its source.
**Running non-interactively.** In print mode the skill needs its tools named
explicitly, or the model answers without ever reading the bundle and marks
every premise `undecidable-from-bundle`:
```sh
claude -p --allowedTools=Bash,Read,Grep,Glob "<the question>"
```
Add `Write,Edit` for the mode that produces a document. `--permission-mode
acceptEdits` alone does **not** do it — measured 2026-09-09 over four runs, the
`okf consume` call is refused without the explicit tool list. The interactive
`claude` above needs none of this.
`<id>` is the folder's name reduced to `[a-z0-9-]`. Run it once per folder with
`--id <name>` to have several bundles reachable at once — each skill carries its
own `bundle_id`, which is what lets a model pick between them. `--out <dir>`
puts the project somewhere other than the current directory.
Measured 2026-09-08 from a fresh `uv tool install` with this repository nowhere
on the path: 5 documents in, 15 concepts out, a skill carrying **0** paths into
any checkout, and `okf check` conformant on its own payload (15 rules, 0
findings). Before that day the same result took a `PYTHONPATH`, a snapshot of a
Measured 2026-09-09 from a fresh `uv tool install` with this repository nowhere
on the path: 5 documents in, **26** concepts out, a skill carrying **0** paths
into any checkout, and `okf check` conformant on its own payload (15 rules, 0
findings). The 2026-09-08 run of the same measurement reported 15 concepts, and
that number was the defect rather than the result: `okf project` was calling
`build()` as a function and reading its signature's defaults, which disagreed
with argparse's on two flags. Two tests now hold the two default sets equal. Before that day the same result took a `PYTHONPATH`, a snapshot of a
clone, and a generated skill that named that clone by absolute path on four
lines — so it could not be moved, shared, or run by anyone else.
@ -646,14 +663,23 @@ From a checkout, the test suite runs with:
.venv/bin/python -m pytest
```
The suite is the verification surface for everything above: 596 tests, run on
2026-08-21 against this branch with the `[extract]` extra installed. Without
the extra the same suite is 589 passed and 7 skipped, measured the same day:
the seven cover the parser path, and the tests holding the fail-fast rejection
for an uninstalled extra run in both. It is not shipped in an installed
The suite is the verification surface for everything above: **1515 tests**,
run on 2026-09-09 against this branch with the `[extract]` extra installed.
(The figure published here until that day was 596, measured 2026-08-21 and
never updated as the suite grew — a count is a measurement with a date on it.)
Without the extra the same suite skips the tests covering the parser path;
that split was last counted on 2026-08-21 as 589 passed and 7 skipped and has
**not** been re-measured since. The tests holding the fail-fast rejection for
an uninstalled extra run in both. The suite is not shipped in an installed
distribution — `tests/` lives at the repository root, so this command needs a
clone rather than a `pip install`.
**Lint is measured against the lockfile.** `uv sync --frozen` resolves ruff
0.15.22, under which `ruff check src tests tools` is clean. A loose install
resolves ruff 0.16.6, under which the same tree reports 144 lint errors and 2
files that would be reformatted — new rules, not new defects. Add
`--extra extract` to that sync or `mypy src` cannot find `pdfplumber`.
A git URL is a PEP 508 direct reference and pins one exact tag, so it is an
install-time *channel*, not the pin: the range above stays the declared
dependency — a wheel built from this branch carries `Requires-Dist:
@ -678,15 +704,33 @@ deterministic within a converter version and not across one.
| `pdf` | `pdfplumber` | measured |
| `docx` | converter | measured |
| `xlsx` | converter | measured |
| `pptx` | converter | **unmeasured** |
| `odt` | converter | **unmeasured** |
| `rtf` | converter | **unmeasured** |
| `pptx` | converter | **constructed** |
| `odt` | converter | **constructed** |
| `rtf` | converter | **constructed** |
**`unmeasured` means what it says.** The corpus this work was measured on
contains **zero** `pptx`, `odt` and `rtf` files, so those three rows work by
construction and have never been checked against a document anyone wrote.
They are not known to be broken; they are not known to be right either, and
the distinction is the point.
**`constructed` means what it says, and it is a weaker word than `measured`
on purpose.** The corpus this work was measured on contains **zero** `pptx`,
`odt` and `rtf` files. Until 2026-09-09 those three rows were `unmeasured` —
they worked by construction and had never been checked against a document
anyone wrote. They have now each been put through end to end on a hand-built
document with a hand-written fasit, which is more than nothing and is not a
corpus:
- `odt` — **1 of 1** declared headings recovered, 1 concept, 0 characters in
no segment. N = 1 document.
- `pptx` — **2 of 2** declared slide titles recovered on a deck that declares
them (a real `<p:ph type="title"/>` placeholder); **0 of 2** on a deck that
does not, where the converter writes `Slide 1` / `Slide 2` because it has no
title to use. That is the converter naming an unnamed slide, not a
segmentation failure. N = 2 decks.
- `rtf` — **0** declared headings, because the container has no heading style
and the author's title is bold text. The proposer therefore proposes
nothing, and the document reaches the bundle inbox as one whole concept:
content preserved, structure zero. N = 1 document. This is the one open
finding of the three.
They are not known to be broken; a single constructed document is not a
denominator, and the distinction is the point.
**What stays out.** `.doc` (Word 97) is not supported — the converter does not
read it. Rastered or scanned PDFs are refused rather than persisted as empty

327
docs/2026-09-09-k3-runde9-restene.md Normal file
View file

@ -0,0 +1,327 @@
# K3 round 9: the rests that did not need the operator
Four things stood in `STATE.md`'s NESTE that no operator decision blocked:
clause 1 misclassifying numeric tables, the `{#…}` converter attribute leaking
into concept titles, the first end-to-end measurement of `odt`/`rtf`/`pptx`,
and the prefix overmatch. This round is those four, plus two documentation
findings PM's re-measurement of O6 produced.
Baseline: HEAD `fbad5b0`, tree clean, tag `v0.7.0` on `1260fac` (two commits
back, deliberately — O5/O6's install measurement is of those bytes), remote
`main` at `b01492b`, so five commits unpushed plus the tag. All four
reproduced before any code was touched.
**Every fraction below carries its denominator. Where a published number did
not reproduce, this report says so before it says anything else.**
## 0 What did not reproduce
**Round 8's decomposition of clause 1 splits the eleven documents into "four
misclassified numeric tables and seven real contents listings". Two of the
eleven are BOTH.** Re-measured with round 8's own instrument
(`/tmp/k3r8/clause1.py`, unchanged), the reach reproduces exactly — **68
candidates over 11 of 39 readable documents** — but reading all 68 dropped
titles rather than the three-title sample changes the split:
| | documents | candidates |
|---|---|---|
| carry at least one misclassified data row | **5** of 11 | **19** of 68 |
| carry at least one real contents entry | **8** of 11 | **49** of 68 |
Documents 21 and 22 appear in both rows. The 4 + 7 = 11 reading is only
available if each document is assigned to one side, and two of them cannot be.
The four purely-misclassified documents are a drawing's dimension chain
(`477 3 025 1`), a P&ID's schematic labels (`40.23`), a door schedule
(`3000 V 1`) and a borehole log's coordinate column (`619649.097`); document 22
additionally carries three soil-layer rows (`- 8.225m Fjell 41`) inside an
otherwise real contents listing.
**The README published 596 tests, measured 2026-08-21 and never updated.** The
suite is 1515. A count is a measurement with a date on it.
**The README published "5 documents in, 15 concepts out" for `okf project`.**
15 was the O6 defect, not the result; the same command writes 26. Corrected.
## 1 Clause 1: a title that ends in a number is not thereby a contents entry
`_TRAILING_PAGE_NUMBER` admits a candidate into a contents run by asking
whether the title ENDS in an integer. That is a question about the number, not
about the title. A drawing's dimension chain, a schematic's labels, a door
schedule and a coordinate column all end in integers and name nothing.
The damage was already gone: round 8's `--close-span-gaps` carries a removed
mark's text on the mark above, so the corpus coverage gap is 0 either way
(re-measured: **0 characters of 1 739 511**, both before and after this round).
What remained was classification.
### 1.1 The rule, and where the threshold sits
A title is admitted only if a NAME survives stripping the page number. What
counts as a name is **swept, not chosen** — an alphabetic run of at least *n*
characters, over the 43-document corpus:
| n | candidates dropped | documents | data rows rescued | REAL entries wrongly rescued |
|---|---|---|---|---|
| — (shipped) | 68 | 11 | 0 of 19 | 0 of 49 |
| 1 | 55 | 9 | **13** of 19 | 0 of 49 |
| **2** | **52** | **8** | **16** of 19 | **0** of 49 |
| 3 | 48 | 8 | 16 of 19 | **4** of 49 |
It collapses at both ends. At n = 1 a door schedule keeps a stray single letter
(`3000 V`) and stays misclassified. At n = 3 the two-letter section name `VA`
stops being a name, falls out of run membership, and the run around it drops
below `CONTENTS_RUN` — so `RIB`, `MMI` and `Tittelfelt` are emitted as concepts
too. One two-letter acronym costs four real entries. **n = 2.**
**The three it does not rescue are named rather than rounded off.** Document
22's `- 8.225m Fjell 41` rows carry a real word, so this predicate cannot
separate them from a contents entry. 16 of 19, not 19 of 19.
### 1.2 What it costs
Corpus, extracted text, default flags: **429 → 447 candidates**, documents with
a plan **32 → 33** (one document had all three of its candidates discarded and
so had no plan at all), characters in no segment **0 → 0**.
On the operator's five-document folder the flag changes **nothing** — `diff -r`
between a build with `--no-contents-name` and one without is empty. Those five
documents contain no run of data rows. A flag whose measured effect on one
folder is zero is still the right default; it is measured where the shape
occurs.
### 1.3 The acceptance criteria, all four
**(i) The red test is green on both sides.** A run of five coordinate rows is
no longer discarded; a contents listing whose shortest entry is the two-letter
`VA` still is. `tests/test_contents_name_part.py`.
**(ii) The twelve-position reference loses no cell, in EITHER reading.**
Scored with round 8's instrument imported unchanged (which imports round 5's
criterion unchanged), round 8's shipped default against round 9's:
| reading | `pdf` | `docx` | `xlsx` | sheet | labels identical |
|---|---|---|---|---|---|
| fasit as authored, `|F|`[3] = 11 | 7 of 8 | 3 of 3 | 0 of 1 | 10 of 12 | **yes** |
| `xlsx` fasit read as 12 (PM's working answer) | 7 of 8 | 3 of 3 | 1 of 1 | 11 of 12 | **yes** |
Not one of the 12 positions changes label. Position 0 stays `for grovt` with
an undefined `|F|`, which is the operator decision this round does not touch.
**(iii) No fasit row loses rank 1, on either bundle.** One K2 bundle built
carrying both of this round's shipped changes
(`--bundle-id k2-trinn1-20260903 --okf-version 0.2`, default flags, default
epoch stamp), 825 s wall time, `merged + coded rejections = 43`, 39 of 43
substantive, 4 coded rejections (1 `extractor_empty_pdf`, 3
`extractor_unknown`):
| bundle | concepts | md | `tie_shared_rank` ON | OFF (known-negative) |
|---|---|---|---|---|
| round 9 (new default) | **453** | 865 | **`[1,1,1,1,1,None]`** | `[2,1,1,1,1,–]` |
| previous default `…-20260911` | 436 | 832 | `[1,1,1,1,1,None]` | `[2,1,1,1,1,–]` |
| Arm B `…-20260903` | 629 | 1108 | `[1,1,1,1,1,None]` | `[1,1,1,1,1,–]` |
hit@8 is **5 of 6** on all three. The known-negative still reproduces on the
NEW bytes, so the pin continues to name its own cause rather than being green
for an unstated reason.
**(iv) `okf project` is still byte-equal to `okf build`.** `diff -r` over the
operator's five-document folder after every change: **0 differences**. The new
flag's `cli.build` signature default and its argparse default are both `True`,
which is the check O6's defect made mandatory.
### 1.4 What it costs a consumer
A re-run, and it is not small. The reference corpus goes 629 concepts / 1108
files (2026-09-03) → 492 / 944 → 425 / 810 → 436 / 832 → **453 / 865**.
Digest, from inside the bundle, `find . -type f -print0 | sort -z | xargs -0
shasum -a 256 | shasum -a 256`: **`21af4a1aa98315cf…`** (the previous default
was `8c93e5e3222577a2…`). On the operator's five-document folder the count is
unchanged at 26 concepts / 52 md.
The pinned artefact is `~/corpora/okf-telling-20260829/K2-bundle-default-20260912`.
It was **copied** from the build scratch rather than rebuilt a second time: the
pin's own comment records that two independent builds differ in nothing,
including `log.md`, which carries the corpus path and never the bundle's own.
`--contents-name`, ON by default since 2026-09-09, opt-out `--no-contents-name`.
## 2 The converter attribute in concept titles
Pandoc writes a spreadsheet's sheet as `## <name> {#sheet-<n>}` and a slide
with a title placeholder as `## <title> {#slide-<n>}`. The brace block is
pandoc's own anchor. It reached the artifact twice: into `title`, and from
there into the concept id, because a filename is reduced FROM the title.
**Exposure, measured 2026-09-09 with a known-positive control run first:**
| bundle / folder | concepts | concepts carrying the attribute |
|---|---|---|
| `K2-bundle-default-20260911` | 436 | **2** |
| `K2-bundle-20260903` (Arm B) | 629 | **2** |
| `~/okf-test/dokumenter` | 26 | **1** |
The "2 of 52" figure carried into this round counted FILES containing `{#`
anywhere, one of which was an `index.md`. At concept level it is 1 of 26.
**The renamed ids, before and after, in full:**
| before | after |
|---|---|
| `del-ii-bilag-7-prisskjema/prissammenstilling-sheet-1` | `del-ii-bilag-7-prisskjema/prissammenstilling` |
| `del-ii-bilag-0-dokumentliste-del-ii/ark1-sheet-1` | `del-ii-bilag-0-dokumentliste-del-ii/ark1` |
**The first of those two is the id `portfolio-optimiser` has cited in
writing.** This round does not send that message: the rename is reported here
so PM can.
The strip lives in one function (`extract.strip_converter_attribute`) read by
both title-forming sites — `propose` names a segment from an ATX heading,
`structure` derives a document title from its leading heading — because a rule
in only one of them would leave the id and the title naming the same concept
differently.
**Provenance holds, measured rather than assumed.** The renamed concept keeps
`source_file`, `source_sha256`, `sources`, `source_sheet: Prissammenstilling`
(the extractor's unit table still reads the anchor, which is why the strip is
applied to a title downstream and never to the extracted text), `references`,
and the parent index entry points at the new filename. `okf check`:
**conformant, 15 rules over 8 excerpts and 18 withheld entries, 0 findings.**
The known-negative is the point: `Mal for {kundenavn}` and
`Feltet {"id": 4} i nyttelasten` are titles an author wrote and are untouched.
## 3 `odt`, `rtf` and `pptx`, measured end to end for the first time
The corpus denominator is genuinely zero, re-counted: `K2/trinn1` holds 86
files — **66 `pdf`, 10 `docx`, 4 `xlsx`, 2 `zip`, 2 `smc`, 2 `doc`** — and not
one `odt`, `rtf` or `pptx`. `K1` is empty. So these rows are measured on
hand-built documents, and the report says so in every row.
| type | declared headings recovered | concepts | chars in no segment | N |
|---|---|---|---|---|
| `odt` | **1 of 1** | 1 | 0 of 1366 | 1 document |
| `pptx` (deck declaring slide titles) | **2 of 2** | 2 | 0 | 1 deck |
| `pptx` (deck declaring none) | **0 of 2** | 2 | 0 of 1285 | 1 deck |
| `rtf` | 0 of **0 declared** | **0** | 1368 of 1368 | 1 document |
End to end through Door B, all three containers, **0 of 3 rejected**.
**`pptx` is not broken, and round 7's reading of it was a fixture property.**
The committed fixture's title shapes carry no `<p:ph type="title"/>`
placeholder, so the converter has no title to name a slide with and writes
`Slide 1` / `Slide 2`. Given a deck that declares one, it writes the authored
title plus `{#slide-N}` — and § 2's strip removes the anchor, so the concept is
named by the title the author wrote. Pinned in
`tests/test_converter_attribute_titles.py`, with the anchor's presence asserted
first so the test cannot be green over an empty set.
**`rtf` is the one open finding.** The container has no heading style; the
author's title is bold text. The proposer proposes nothing, and the document
reaches the bundle inbox as one whole concept — content preserved, structure
zero. That is not data loss and it is not segmentation either.
`extract._EVIDENCE` gains a third class rather than stretching an existing one.
`measured` means real corpus files and a hand-counted fasit; `unmeasured` means
no document has ever been through the row; **`constructed`** means the row has
met a document end to end, but not one anyone wrote for their own purposes. All
three office rows move `unmeasured` → `constructed`. The suite asserts the new
class, and asserts the `unmeasured` set is now empty rather than dropping it.
## 4 The prefix overmatch, re-measured
Round 5's `df 159 of 270` was measured on the N500 bundle and has not been
re-measured since. It is not reproduced here — that bundle is a different one —
but the mechanism is, on both K2 bundles, **with a known-positive control run
first because a query that cannot find would report zero either way**:
| bundle | N | `df(undersjøisk)` prefix rule | `df(undersjøisk)` equality | control `df(under)` |
|---|---|---|---|---|
| `K2-bundle-default-20260911` | 436 | **171** | 0 | 79 |
| `K2-bundle-20260903` | 629 | **174** | 0 | 76 |
A token that occurs **zero** times in either bundle matches 39.2 % and 27.7 %
of their concepts through `MIN_SHARED_PREFIX = 4`. The first attempt at this
measurement returned 0 for the control as well, which is how the broken query
was caught rather than published.
Not fixed this round, and not because it is small: the acceptance bar is zero
regression on hit@8 across six rows and two bundles plus the twelve-position
reference under both readings, and `MIN_SHARED_PREFIX` has a known-positive in
the suite that Norwegian compounding is the reason for. It stands, with this
number.
## 5 Two documentation findings from PM's O6 re-measurement
**(a) The print-mode method as published does not reproduce.** O6 § 6 gave the
method as `cd <project> && claude -p "<the question>"`. Run that way — including
with `--permission-mode acceptEdits` — the `okf consume` call is refused, the
model answers without opening the bundle, and a hypothesis run marks both
premises `undecidable-from-bundle`. The four runs reported in fact used
`claude -p --allowedTools=Bash,Read,Grep,Glob "<question>"` (plus `Write,Edit`
for the document row). The numbers are unaffected; the recipe for getting them
was wrong. Corrected in § 6 of that report and added to the README as its own
short paragraph for agents running non-interactively.
**(b) Green lint hangs on the lockfile.** `uv sync --frozen` resolves ruff
0.15.22 and the tree is clean. A loose install resolves ruff 0.16.6, under
which the same tree reports 144 lint errors and 2 files that would be
reformatted — new rules, not new defects. Not upgraded here. One sentence added
to the README's developer section. **One thing to add to PM's finding:**
`uv sync --frozen` alone also drops the `[extract]` extra, after which
`mypy src` fails with `Cannot find implementation or library stub for module
named "pdfplumber"`. The green command is
`uv sync --frozen --extra extract`.
## 6 A finding reported, not fixed: the `tools/okf_consume.py` shim
`vegnormal-okf` reported that the v0.7.0 shim broke a caller using
`importlib.util.spec_from_file_location`. Reproduced here, with numbers: such a
caller holds a module object carrying **4** public names against the
`sys.modules` entry's **90**, and `build_payload` is absent. `sys.modules[__name__]
= _impl` replaces the registry entry; it cannot reach a module object the
caller already holds.
A backward-compatible repair is one line placed before the alias:
```python
globals().update({k: v for k, v in vars(_impl).items() if not k.startswith("__")})
```
Verified to restore all 90 names on the caller-held object. **The dunder filter
is load-bearing**: an unfiltered `vars(_impl)` overwrites `__name__` with
`llm_ingestion_okf.consume` before the next line reads it, aliasing the module
under the wrong key. Note the repair restores attribute ACCESS and not
patch-through — a caller monkeypatching the copy patches a binding the
implementation does not read, which is the reason the alias exists.
**Not applied.** This round's order fences `tools/okf_consume*.py` as run, not
edited. The mechanism and the line are here so a session whose order authorises
it can apply them in one step.
## Honesty limits
- **The twelve-position reference is n = 12, of which 8 are `pdf`, 3 `docx`
and 1 `xlsx`.** Every per-type fraction is a fraction of 8, 3 or 1. A single
`xlsx` position means that row moves between 0/1 and 1/1 and nothing else.
- **Both sheet readings are reported, never one.** The sheet is 11 of 12 under
`|F|` = 12 and 10 of 12 under `|F|` = 11; the difference is entirely the
`xlsx` cell.
- **The retrieval side is n = 6 questions, one rater, one fasit**, and row 6
misses on every bundle measured so far. `hit@8 5 of 6` is that.
- **"General" rests on ONE corpus of 43 documents**, plus the operator's five.
`K2/trinn2` is byte-identical to `K2/trinn1` and `K1` is empty, so the 86
files are 43 documents.
- **Each new file-type row rests on ONE hand-built document** (`pptx` on two
decks). They are constructed documents, not corpus files, and the evidence
class says so.
- **`delivered` is not `answered`.** Every count here is of concepts and
excerpts a payload carries, never of correct answers.
- **Position 0 is blocked on an operator decision, not on code.** So are the
`xlsx` fasit's `|F|` = 12, the `pptx` fasit, and whether routing between
bundles may rest on `--id`. None of the four moved this round.
- **The renamed concept ids break a citation `portfolio-optimiser` made in
writing.** Both ids are in § 2 so that message can be sent; it was not sent
from here.
- **The threshold sweep in § 1.1 is over one corpus.** n = 2 is where it
collapses on these 43 documents; a corpus with one-letter section names would
put it elsewhere.

16
docs/2026-09-09-o6-forsteskjerm-og-skillmodi.md
View file

@ -233,9 +233,19 @@ was run, because the measurement is of the tag.
## 6 The three end-to-end runs (C1's instrument)
Skills installed in a project directory outside this repository;
`cd <project> && claude -p "<the question>"`; **one run per row, no re-run for a
better answer**. Sources grep-verified against the bundle tree with the
denominator stated.
`cd <project> && claude -p --allowedTools=Bash,Read,Grep,Glob "<the question>"`
(row 3, which produces a document, additionally `Write,Edit`); **one run per
row, no re-run for a better answer**. Sources grep-verified against the bundle
tree with the denominator stated.
**CORRECTED 2026-09-09 (round 9).** This section first stated the method as
`cd <project> && claude -p "<the question>"`, without the tool list. Re-run
that way -- including with `--permission-mode acceptEdits` -- the `okf consume`
call is refused, the model answers without ever opening the bundle, and a
hypothesis run marks both premises `undecidable-from-bundle`. All four runs
reported below in fact used the explicit `--allowedTools` list above; the
method line omitted it, so as written the section did not reproduce. The
NUMBERS are unaffected -- what was wrong was the recipe for getting them.
| # | Mode | Asked | Sources verified | Denominator |
|---|---|---|---|---|

42
src/llm_ingestion_okf/cli.py
View file

@ -170,6 +170,19 @@ DEFAULT_FIRST_SPAN_FROM_ZERO = True
#: rule closes each span against the next SURVIVOR. It adds no boundary, so the
#: concept count is unchanged and the cost to a consumer is a re-run.
DEFAULT_CLOSE_SPAN_GAPS = True
#: Round 9. Clause 1 asked whether a title ENDED in a number, which is a
#: question about the number rather than about the title: a drawing's dimension
#: chain, a P&ID's schematic labels, a door schedule and a borehole log's
#: coordinate column all end in integers and name nothing. Measured over the
#: 43-document corpus, clause 1 discarded 68 candidates over 11 of 39 readable
#: documents, and 19 of those over 5 of the 11 were data rows of that kind.
#: The rule requires a NAME to survive stripping the page number; the threshold
#: is swept rather than chosen (`propose.CONTENTS_NAME_RUN`). It rescues 16 of
#: the 19 and 0 of the 49 real contents entries -- the three it does not rescue
#: carry a real word (`- 8.225m Fjell 41`) and this predicate cannot separate
#: them from a contents entry. It only ever REMOVES members from a run, so it
#: can only add concepts, never take one away. Opt-out `--no-contents-name`.
DEFAULT_CONTENTS_NAME = True
#: Round 3's two spreadsheet rules (D1 and D3), held back through rounds 5 and
#: 6 by a RETRIEVAL regression rather than by the reference: they take the
@ -236,6 +249,7 @@ def _propose_plans(
outline_gate: bool = False,
first_span_from_zero: bool = False,
close_span_gaps: bool = False,
contents_name: bool = False,
pdf_headings: bool = False,
pdf_headings_reserve: bool = False,
ocr: bool = False,
@ -272,6 +286,7 @@ def _propose_plans(
outline_gate=outline_gate,
first_span_from_zero=first_span_from_zero,
close_span_gaps=close_span_gaps,
contents_name=contents_name,
pdf_headings=pdf_headings,
pdf_headings_reserve=pdf_headings_reserve,
ocr=ocr,
@ -307,6 +322,7 @@ def build(
outline_gate: bool = DEFAULT_OUTLINE_GATE,
first_span_from_zero: bool = DEFAULT_FIRST_SPAN_FROM_ZERO,
close_span_gaps: bool = DEFAULT_CLOSE_SPAN_GAPS,
contents_name: bool = DEFAULT_CONTENTS_NAME,
pdf_headings: bool = DEFAULT_PDF_HEADINGS,
pdf_headings_reserve: bool = DEFAULT_PDF_HEADINGS_RESERVE,
ocr: bool = DEFAULT_OCR,
@ -377,6 +393,7 @@ def build(
outline_gate=outline_gate,
first_span_from_zero=first_span_from_zero,
close_span_gaps=close_span_gaps,
contents_name=contents_name,
pdf_headings=pdf_headings,
pdf_headings_reserve=pdf_headings_reserve,
ocr=ocr,
@ -684,6 +701,30 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
"pre-2026-09-11 spans byte for byte"
),
)
build_parser.add_argument(
"--contents-name",
action="store_true",
default=DEFAULT_CONTENTS_NAME,
help=(
"Admit a title into a contents run only when a NAME survives "
"stripping its trailing page number, so a run of data rows is not "
"read as a contents listing and discarded. Measured over the "
"39-document corpus: clause 1 dropped 68 candidates over 11 "
"documents, of which 19 over 5 documents were data rows; the rule "
"rescues 16 of the 19 and 0 of the 49 real contents entries. It "
"only removes members from a run, so it can only add concepts. "
"ON since 2026-09-09"
),
)
build_parser.add_argument(
"--no-contents-name",
action="store_false",
dest="contents_name",
help=(
"The repair's explicit opt-out, for a consumer who needs the "
"pre-2026-09-09 contents-run predicate byte for byte"
),
)
build_parser.add_argument(
"--pdf-headings",
choices=("none", "font", "font-reserve"),
@ -757,6 +798,7 @@ def main(argv: list[str] | None = None) -> int:
outline_gate=args.outline_gate,
first_span_from_zero=args.first_span_from_zero,
close_span_gaps=args.close_span_gaps,
contents_name=args.contents_name,
pdf_headings=args.pdf_headings == "font",
pdf_headings_reserve=args.pdf_headings == "font-reserve",
ocr=args.ocr,

77
src/llm_ingestion_okf/extract.py
View file

@ -66,16 +66,42 @@ _PANDOC_FORMATS: dict[str, str] = {
}
# What each row's behaviour actually rests on, asserted in the suite rather
# than written in a comment that rots. `measured` means real corpus files and a
# hand-counted fasit; `unmeasured` means the corpus contains ZERO files of that
# type, so the row works by construction and has never been checked against a
# document anyone wrote. An unmeasured row must not read as a supported one.
# than written in a comment that rots. THREE classes, and the third exists
# because the first two could not tell the truth about these rows:
#
# measured real CORPUS files and a hand-counted fasit. Someone wrote the
# document for their own purposes and we counted what we got.
# constructed hand-built or generator-built documents with a hand-written
# fasit, and no corpus file at all. The row has now met a
# document end to end -- it is not `unmeasured` -- but a
# document written to exercise it is not a document anyone
# wrote, so it is not `measured` either.
# unmeasured the corpus contains ZERO files of the type AND no document
# has ever been put through the row. It works by construction.
#
# An unmeasured row must not read as a supported one, and neither must a
# constructed one.
#
# THE THREE OFFICE ROWS MOVED unmeasured -> constructed ON 2026-09-09, each on
# its own hand-built document (N = 1, except `pptx` at N = 2):
#
# .odt 1 of 1 declared headings recovered, 1 concept, 0 characters in no
# segment.
# .pptx 2 of 2 declared slide titles recovered on a deck that DECLARES them
# (`<p:ph type="title"/>`), 0 of 2 on one that does not -- the latter
# lands as `Slide 1`/`Slide 2`, which is the converter naming a slide
# it has no title for, not a segmentation failure.
# .rtf 0 declared headings, because the container has no heading style and
# the author's title is bold text. The proposer therefore proposes
# nothing and the document reaches Door B's INBOX as one concept --
# content preserved, structure zero. That is the row's honest result
# and it is the one open finding of the three.
_EVIDENCE: dict[str, str] = {
".docx": "measured",
".xlsx": "measured",
".pptx": "unmeasured",
".odt": "unmeasured",
".rtf": "unmeasured",
".pptx": "constructed",
".odt": "constructed",
".rtf": "constructed",
}
# Load-bearing, all three, and none of them hygiene:
@ -703,6 +729,20 @@ _OPTIONAL_EXTRACTORS: dict[str, Callable[[bytes], str]] = {
# containing a `#` cannot be read as a sheet.
_SHEET_HEADING = re.compile(r"^#{1,6} (?P<name>.*?) \{#sheet-\d+\}$")
# Pandoc's ATTRIBUTE syntax at the end of a heading, which is what the sheet
# and slide anchors above are an instance of. Deliberately NARROW, because the
# known-negative is the whole point: an author writing `Mal for {kundenavn}` or
# `Feltet {"id": 4}` wrote a title, and stripping that would be this same
# defect pointed the other way.
#
# The three narrowings, each doing work: the block must be at the END of the
# title (`$`), it must OPEN with `#` (pandoc's identifier -- `{.class}` and
# `{key=val}` alone are not what any converter here emits, and matching them
# would reach further than measured), and the identifier is the restricted
# character set pandoc actually generates, so a brace holding a space, a quote
# or a colon is not an attribute.
_CONVERTER_ATTRIBUTE = re.compile(r"\s*\{#[A-Za-z0-9_.:-]+\}\s*$")
# A line the converter wrote as part of a pipe table. Whether one of them is
# the table's SEPARATOR is decided by POSITION, never by content: an empty
# spreadsheet row renders as `| | |` and a separator as `|----|----|`, and
@ -713,6 +753,29 @@ _SHEET_HEADING = re.compile(r"^#{1,6} (?P<name>.*?) \{#sheet-\d+\}$")
_TABLE_LINE = "|"
def strip_converter_attribute(title: str) -> str:
"""Remove a trailing pandoc attribute anchor from a heading's title.
ONE definition, read by both title-forming sites: `propose` names a
segment from an ATX heading, `structure` derives a document title from its
leading heading, and a rule living in only one of them would strip the
attribute on one path and leave it on the other -- with the id and the
title then disagreeing about the same concept.
Lives HERE because the attribute is a CONVERTER artefact: `_SHEET_HEADING`
above is the same syntax read for a different purpose, and this module is
the one that knows what pandoc writes. That reading must keep its
attribute, which is why the strip is applied to a title downstream and
never to the extracted text.
RENAMES CONCEPT IDS, by design and with the operator's authorisation
(2026-09-09): a filename is reduced FROM the title, so the two move
together. Measured exposure at the time: 2 of 810 concepts on the default
K2 bundle and 2 of 1108 on Arm B.
"""
return _CONVERTER_ATTRIBUTE.sub("", title)
@dataclass(frozen=True)
class SourceUnits:
"""Where in the ORIGINAL each stretch of the extracted text came from.

93
src/llm_ingestion_okf/propose.py
View file

@ -62,7 +62,7 @@ from pathlib import Path
from typing import Any
from .errors import IngestError
from .extract import extract_text
from .extract import extract_text, strip_converter_attribute
from .materialize import reduce_to_id_grammar
from .segmentation import observed_extractor_version
@ -227,6 +227,16 @@ _OUTLINE = re.compile(r"^\s{0,4}(?P<number>\d{1,2})[.)]?\s+(?P<title>\S.*?)\s*$"
# through `_segment_path` -- an unstripped page number would become part of a
# filename.
_TRAILING_PAGE_NUMBER = re.compile(r"[\s.]+\d{1,4}\s*$")
#: Clause 1's name part. An alphabetic run of at least this many characters is
#: a NAME; anything shorter is a stray letter in a data row. SWEPT over the
#: 43-document corpus at 1, 2 and 3, and it collapses at both ends: at 1 a
#: drawing's door schedule keeps a bare `V` and stays misclassified (13 of 19
#: data rows rescued, not 16), and at 3 the two-letter section name `VA` stops
#: being a name, falls out of run membership, and takes `RIB`, `MMI` and
#: `Tittelfelt` below `CONTENTS_RUN` with it -- one acronym costing four REAL
#: contents entries. At 2 the rescue is 16 of 19 and the regression 0 of 49.
CONTENTS_NAME_RUN = 2
_NAME_PART = re.compile(r"[^\W\d_]{%d,}" % CONTENTS_NAME_RUN)
# D3's grammar, and it reads a CELL rather than a line. A sheet's section label
# is a bare number, optionally joined to another by a separator where two groups
# were merged (`11+12`), and nothing else: no letters, so a row opening with a
@ -292,6 +302,29 @@ def _strip_page_number(title: str) -> str:
return _TRAILING_PAGE_NUMBER.sub("", title)
def bears_a_name(title: str) -> bool:
"""Does a NAME survive stripping the trailing page number?
Clause 1's admission predicate, and the whole of round 9's repair. The
shipped rule asked whether `_TRAILING_PAGE_NUMBER` MATCHED, which is a
question about the number and not about the title -- a drawing's dimension
chain, a P&ID's schematic labels, a door schedule and a borehole log's
coordinate column all end in integers, and none of them names anything.
Measured on the corpus, 19 of the 68 candidates clause 1 discarded over 5
of the 11 documents were rows of that kind.
Module level and importable for the same reason `outline_lines` is: the
reach instrument measures THIS predicate, and an instrument re-implementing
the grammar it measures is measuring a second definition.
Applied to the REST, never to the whole title: `40.23` bears no name and
`Poseprover posisjon 24 7` does. Three corpus data rows carry a real word
(`- 8.225m Fjell 41`) and this predicate cannot separate them from a
contents entry -- that is a stated limit, not a rounding error.
"""
return _NAME_PART.search(_strip_page_number(title).strip()) is not None
def outline_lines(text: str) -> list[tuple[int, int, str]]:
"""Every line the outline grammar admits, as `(line index, integer, title)`.
@ -515,6 +548,7 @@ def find_candidates(
outline_gate: bool = False,
first_span_from_zero: bool = False,
close_span_gaps: bool = False,
contents_name: bool = False,
) -> list[Candidate]:
"""Every boundary the mechanical rules propose, in document order.
@ -550,6 +584,13 @@ def find_candidates(
It is its own flag and not part of an arm because the orphan check is
reached by every file type, and moving it is a decision about all of them.
`contents_name` is round 9's gate and it is OFF at False, where clause 1
behaves exactly as it did before the rule existed. On, a title joins a
contents run only when a NAME survives stripping its page number, so a run
of data rows is no longer read as a contents listing. It can only ever
REMOVE members from a run, which is why it only ever rescues candidates and
never discards one the shipped rule kept.
`close_span_gaps` is OFF at False, where a mark removed after its
neighbour's span was closed takes that text out of the plan entirely. On,
a span runs to the next SURVIVING candidate's start and the last one runs
@ -716,7 +757,11 @@ def find_candidates(
if atx is None and numbered is None:
continue
if atx is not None:
title = atx.group("title")
# The converter's own anchor is not part of the title, and it must
# come off HERE rather than where the filename is reduced: the id
# is reduced FROM the title, so stripping it downstream would leave
# the two naming the same concept differently.
title = strip_converter_attribute(atx.group("title"))
level = len(atx.group("hashes"))
inner = _NUMBERED.match(title)
number = inner.group("number") if inner else None
@ -772,7 +817,9 @@ def find_candidates(
# Arm F clause 1's input, and it must be read HERE: the orphan pass below
# deletes every bodiless heading, which is every entry of a contents list
# but the last, and a run of one is below `CONTENTS_RUN`.
contents_run = _contents_run_positions(marked) if unit_fold else set()
contents_run = (
_contents_run_positions(marked, contents_name=contents_name) if unit_fold else set()
)
for position_in_list, (_, candidate) in enumerate(marked):
if position_in_list in absorbed:
continue
@ -818,7 +865,7 @@ def find_candidates(
contents=position_in_list in contents_run,
)
)
resolved = fold_units(candidates) if unit_fold else candidates
resolved = fold_units(candidates, contents_name=contents_name) if unit_fold else candidates
if first_span_from_zero and resolved and resolved[0].start > 0:
# Applied AFTER the fold, so the concept that SURVIVES is the one that
# opens at 0: the fold can discard a contents run, and moving the start
@ -869,7 +916,9 @@ def _absorbed_tables(
return absorbed
def _contents_run_positions(marked: list[tuple[int, Candidate]]) -> set[int]:
def _contents_run_positions(
marked: list[tuple[int, Candidate]], *, contents_name: bool = False
) -> set[int]:
"""Arm F clause 1, measured on the PRE-orphan list. Same predicate as there.
A run of at least `CONTENTS_RUN` consecutive page-numbered headings, table
@ -885,19 +934,22 @@ def _contents_run_positions(marked: list[tuple[int, Candidate]]) -> set[int]:
concept while its neighbours were not. What still bounds the rule is the run
LENGTH, which is what the `CONTENTS_RUN` sweep bought and is unchanged.
"""
def member(candidate: Candidate) -> bool:
if candidate.rule == RULE_TABLE_BLOCK:
return False
if not _TRAILING_PAGE_NUMBER.search(candidate.title):
return False
return bears_a_name(candidate.title) if contents_name else True
inside: set[int] = set()
index = 0
while index < len(marked):
candidate = marked[index][1]
if candidate.rule == RULE_TABLE_BLOCK or not _TRAILING_PAGE_NUMBER.search(candidate.title):
if not member(candidate):
index += 1
continue
end = index
while (
end < len(marked)
and marked[end][1].rule != RULE_TABLE_BLOCK
and _TRAILING_PAGE_NUMBER.search(marked[end][1].title)
):
while end < len(marked) and member(marked[end][1]):
end += 1
if end - index >= CONTENTS_RUN:
inside.update(range(index, end))
@ -914,7 +966,7 @@ def _contents_run_positions(marked: list[tuple[int, Candidate]]) -> set[int]:
CONTENTS_RUN = 3
def fold_units(candidates: list[Candidate]) -> list[Candidate]:
def fold_units(candidates: list[Candidate], *, contents_name: bool = False) -> list[Candidate]:
"""Arm F: ONE rule, three clauses, and it only MERGES or DISCARDS.
Derived from the three rules the operator wrote across the K3 unit
@ -950,18 +1002,25 @@ def fold_units(candidates: list[Candidate]) -> list[Candidate]:
# work to do: a contents list WITH dot leaders keeps every entry, and that
# run exists only in this list.
drop: set[int] = {position for position, c in enumerate(candidates) if c.contents}
def member(candidate: Candidate) -> bool:
if candidate.rule == RULE_TABLE_BLOCK:
return False
if not _TRAILING_PAGE_NUMBER.search(candidate.title):
return False
return bears_a_name(candidate.title) if contents_name else True
index = 0
while index < len(candidates):
candidate = candidates[index]
if candidate.rule == RULE_TABLE_BLOCK or not _TRAILING_PAGE_NUMBER.search(candidate.title):
if not member(candidate):
index += 1
continue
end = index
while (
end < len(candidates)
and candidates[end].rule != RULE_TABLE_BLOCK
and candidates[end].level == candidate.level
and _TRAILING_PAGE_NUMBER.search(candidates[end].title)
and member(candidates[end])
):
end += 1
if end - index >= CONTENTS_RUN:
@ -1171,6 +1230,7 @@ def build_plan(
outline_gate: bool = False,
first_span_from_zero: bool = False,
close_span_gaps: bool = False,
contents_name: bool = False,
) -> dict[str, Any]:
"""The artifact. Every entry PROPOSED, the plan itself never adjudicated."""
taken: set[str] = set()
@ -1187,6 +1247,7 @@ def build_plan(
outline_gate=outline_gate,
first_span_from_zero=first_span_from_zero,
close_span_gaps=close_span_gaps,
contents_name=contents_name,
)
for candidate in subdivide(text, candidates, max_segment_chars):
entries.append(
@ -1263,6 +1324,7 @@ def run(
outline_gate: bool = False,
first_span_from_zero: bool = False,
close_span_gaps: bool = False,
contents_name: bool = False,
pdf_headings: bool = False,
pdf_headings_reserve: bool = False,
ocr: bool = False,
@ -1338,6 +1400,7 @@ def run(
outline_gate=outline_gate,
first_span_from_zero=first_span_from_zero,
close_span_gaps=close_span_gaps,
contents_name=contents_name,
)
# Nothing to propose is an OUTCOME, and it is not an artifact. An empty
# plan cannot be replayed -- `process_inbox` refuses one, because a plan

8
src/llm_ingestion_okf/structure.py
View file

@ -34,6 +34,8 @@ from collections.abc import Mapping, Sequence
from dataclasses import dataclass, field
from pathlib import Path
from .extract import strip_converter_attribute
# A document number is either an alpha-prefixed identifier (`N500`, `V720`,
# `R610.4`) or a dotted numeric section (`4.2.1`). A BARE integer is
# deliberately not a number: `12-things.md` and `2026-notes.md` are ordinary
@ -168,7 +170,11 @@ def _leading_heading(body: str) -> str | None:
# Only a LEADING heading is the document's title. A heading further
# down is a section of the document, and taking it would retitle every
# document whose body happens to open with prose.
return line[2:].strip() if line.startswith("# ") else None
if not line.startswith("# "):
return None
# Same rule as the segment path reads, from the same function: a
# converter anchor must not become a derived document title either.
return strip_converter_attribute(line[2:].strip()).strip()
return None

123
tests/test_contents_name_part.py Normal file
View file

@ -0,0 +1,123 @@
"""Clause 1 asks whether a title NAMES something, not whether a number ends it.
`_TRAILING_PAGE_NUMBER` reads a run of consecutive titles ending in an integer
as a contents listing and discards it. That predicate cannot tell a contents
entry from a run of DATA ROWS, because a drawing's dimension chain, a P&ID's
schematic labels, a door schedule and a borehole log's coordinate column all
end in integers too. Measured on the 43-document corpus, clause 1 dropped 68
candidates over 11 of 39 readable documents, and 19 of those 68 over 5 of the
11 were data rows whose titles -- `40.23`, `300 1`, `3000 V 1`, `619649.097`
-- name nothing at all.
Round 8 removed the DAMAGE (`--close-span-gaps` carries a removed mark's text
on the mark above), so what is left is classification. The repair is one
clause: a title is a contents entry only if a NAME survives stripping the page
number.
WHERE THE THRESHOLD SITS IS MEASURED, NOT CHOSEN. Swept over the corpus at an
alphabetic run of >= 1, >= 2 and >= 3 characters, and it collapses at both
ends: at >= 1 three data rows keep a stray single letter (`3000 V`) and stay
misclassified (13 of 19 rescued); at >= 3 a real contents list breaks, because
`VA` is a two-letter section name and dropping it out of run membership takes
`RIB`, `MMI` and `Tittelfelt` below `CONTENTS_RUN` with it -- one acronym
costing four real entries. At >= 2 the rescue is 16 of 19 and the regression 0
of 49 real entries. This file holds BOTH sides, and the second is the one that
matters: a rule that rescues a table by also rescuing contents lists has moved
the defect rather than fixed it.
"""
from __future__ import annotations
from llm_ingestion_okf import cli
from llm_ingestion_okf.propose import find_candidates
DEFAULT = dict(
outline_run=cli.DEFAULT_OUTLINE_RUN,
table_grid=cli.DEFAULT_TABLE_GRID,
unit_fold=cli.DEFAULT_UNIT_FOLD,
keep_table_heading=cli.DEFAULT_KEEP_TABLE_HEADING,
sheet_section_rows=cli.DEFAULT_SHEET_SECTION_ROWS,
drop_wrapped_outline=cli.DEFAULT_DROP_WRAPPED_OUTLINE,
outline_gate=cli.DEFAULT_OUTLINE_GATE,
first_span_from_zero=cli.DEFAULT_FIRST_SPAN_FROM_ZERO,
close_span_gaps=cli.DEFAULT_CLOSE_SPAN_GAPS,
contents_name=cli.DEFAULT_CONTENTS_NAME,
)
#: The borehole-log shape, rebuilt from the corpus document's geometry: a
#: coordinate column whose rows are `<northing>.<decimals>` and end in digits.
#: Five consecutive rows, comfortably over `CONTENTS_RUN`, each with a body so
#: the orphan check does not remove them first.
DATA_ROWS = """# Geoteknisk borelogg Testdokument
Dette dokumentet har en kjent struktur: en koordinatkolonne der hver rad er
et tall, og ingen av dem navngir noe som helst.
## 619482.244
Proevepunkt P20, naverboring, fire proever i sjiktet 0-4 meter.
## 619551.473
Proevepunkt P21, naverboring, tre proever i sjiktet 0-3 meter.
## 619575.846
Proevepunkt P22, naverboring, fem proever i sjiktet 0-5 meter.
## 619649.097
Proevepunkt P23, naverboring, to proever i sjiktet 0-2 meter.
## 619712.518
Proevepunkt P24, naverboring, seks proever i sjiktet 0-6 meter.
"""
#: The known-negative, and it carries the exact boundary the sweep found: a
#: real contents listing whose shortest entry is a TWO-letter section name.
#: Every one of these must still be discarded.
REAL_CONTENTS = """# Modellansvar Testdokument
Innhold
## Ansvar fagmodellansvarlig 5
## Kontaktpersoner 7
## RIB 9
## VA 11
## MMI 13
## Tittelfelt 15
Dette avsnittet staar under listen og gir den en kropp.
"""
def _titles(document: str) -> list[str]:
return [candidate.title for candidate in find_candidates(document, **DEFAULT)]
def test_a_run_of_numeric_data_rows_is_not_a_contents_listing() -> None:
"""The known-positive: nothing survives stripping, so nothing is a name."""
titles = _titles(DATA_ROWS)
for row in ("619482.244", "619551.473", "619575.846", "619649.097", "619712.518"):
assert row in titles, f"clause 1 discarded the data row {row!r} as a contents entry"
def test_a_real_contents_listing_is_still_discarded() -> None:
"""The known-negative, and the reason the threshold is 2 and not 3.
`VA` is two letters. At an alphabetic run of >= 3 it stops being a name,
falls out of run membership, and the run around it drops below
`CONTENTS_RUN` -- so the whole listing survives and four real entries are
emitted as concepts. Measured on the corpus, that is exactly what happened.
"""
titles = _titles(REAL_CONTENTS)
for entry in ("Ansvar fagmodellansvarlig", "Kontaktpersoner", "RIB", "VA", "MMI", "Tittelfelt"):
assert not any(title.startswith(entry) for title in titles), (
f"clause 1 emitted the contents entry {entry!r} as a concept"
)

169
tests/test_converter_attribute_titles.py Normal file
View file

@ -0,0 +1,169 @@
"""A converter's own attribute is not part of a document's title.
Converting a spreadsheet, pandoc writes each sheet as
`## <sheet name> {#sheet-<n>}`, and a deck whose slides carry title
placeholders as `## <slide title> {#slide-<n>}`. The brace block is pandoc's
attribute syntax -- an anchor it emits so its own HTML output can link to the
section. It is not something the author of the spreadsheet or the deck wrote.
It reached the artifact twice over: into a concept's `title`, and from there
into the concept's id, because a filename is REDUCED from the title. Measured
2026-09-09 on the two bundles on disk, `2 of 810` concepts on the default K2
bundle and `2 of 1108` on Arm B carried it, and the ids were
`del-ii-bilag-7-prisskjema/prissammenstilling-sheet-1` and
`del-ii-bilag-0-dokumentliste-del-ii/ark1-sheet-1`.
Round 7 measured this and left it, because the repair RENAMES concept ids
another repository has cited in writing, and renaming those is not a
measurement's call. The operator authorised the strip on 2026-09-09; the
exposure is in the round-9 report with both ids side by side, before and
after.
TWO TITLE SITES, ONE RULE. `propose` forms a segment's title from an ATX
heading and `structure` derives a document title from its leading heading, so
a rule living in one of them would strip the attribute on one path and leave
it on the other. Both read `strip_converter_attribute`.
THE KNOWN-NEGATIVE IS THE POINT. The pattern must match pandoc's attribute
form and nothing else: a heading that legitimately contains braces -- a
placeholder in a template, a code identifier -- is a title the author wrote,
and stripping it would be the same defect pointed the other way.
"""
from __future__ import annotations
import importlib.util
from pathlib import Path
import pytest
from llm_ingestion_okf import cli
from llm_ingestion_okf.extract import extract_text, strip_converter_attribute
from llm_ingestion_okf.propose import find_candidates
from llm_ingestion_okf.structure import derive_document_structure
DEFAULT = dict(
outline_run=cli.DEFAULT_OUTLINE_RUN,
table_grid=cli.DEFAULT_TABLE_GRID,
unit_fold=cli.DEFAULT_UNIT_FOLD,
keep_table_heading=cli.DEFAULT_KEEP_TABLE_HEADING,
sheet_section_rows=cli.DEFAULT_SHEET_SECTION_ROWS,
drop_wrapped_outline=cli.DEFAULT_DROP_WRAPPED_OUTLINE,
outline_gate=cli.DEFAULT_OUTLINE_GATE,
first_span_from_zero=cli.DEFAULT_FIRST_SPAN_FROM_ZERO,
close_span_gaps=cli.DEFAULT_CLOSE_SPAN_GAPS,
contents_name=cli.DEFAULT_CONTENTS_NAME,
)
SHEET_DOCUMENT = """## Prissammenstilling {#sheet-1}
Denne posten samler prisene for hele leveransen og er den posten et
spoersmaal om pris maa naa.
## Ark1 {#sheet-2}
Denne posten lister dokumentene i del II og har sin egen kropp.
## Lysbilde med krav {#slide-3}
Dette lysbildet bar en tittelplassholder, saa konvertereren ga det et anker.
"""
#: The known-negative. Braces the AUTHOR wrote, in the two shapes that occur:
#: a template placeholder and a code identifier. Neither is pandoc's attribute
#: form, and neither may be touched.
AUTHORED_BRACES = """## Mal for {kundenavn} og leveransen
Denne overskriften har krollparenteser forfatteren skrev selv, og den staar
paa niva 2 sammen med den neste -- clause 2 folds a level that occurs once
into the level above, which would empty this fixture without saying so.
## Feltet {"id": 4} i nyttelasten
Dette avsnittet beskriver et felt og staar under sin egen overskrift.
"""
def test_the_attribute_is_stripped_from_a_segment_title() -> None:
titles = [candidate.title for candidate in find_candidates(SHEET_DOCUMENT, **DEFAULT)]
for leaked in ("{#sheet-1}", "{#sheet-2}", "{#slide-3}"):
assert not any(leaked in title for title in titles), (
f"the converter attribute {leaked} reached a concept title: {titles}"
)
assert "Prissammenstilling" in titles
assert "Lysbilde med krav" in titles
def test_the_attribute_is_stripped_from_a_derived_document_title() -> None:
derived = derive_document_structure(
"# Prissammenstilling {#sheet-1}\n\nEn kropp.\n", source_file="regneark.md"
)
title = derived.title
assert title == "Prissammenstilling", f"structure derived the title {title!r}"
def test_authored_braces_are_left_alone() -> None:
"""The known-negative: only pandoc's attribute form is an attribute."""
titles = [candidate.title for candidate in find_candidates(AUTHORED_BRACES, **DEFAULT)]
assert any("{kundenavn}" in title for title in titles), titles
assert any('{"id": 4}' in title for title in titles), titles
assert strip_converter_attribute('Feltet {"id": 4} i nyttelasten') == (
'Feltet {"id": 4} i nyttelasten'
)
assert strip_converter_attribute("Mal for {kundenavn}") == "Mal for {kundenavn}"
def test_a_deck_that_declares_slide_titles_is_named_by_them() -> None:
"""The slide form, on a REAL conversion rather than a hand-written string.
The committed `pptx` fixture emits `## Slide 1` / `## Slide 2`, and round 7
read that as the format segmenting badly. Decomposed 2026-09-09 it is not:
that deck's title shapes carry no `<p:ph type="title"/>` placeholder, so the
converter has nothing to name a slide WITH. Given a deck that declares one,
the converter writes `## <authored title> {#slide-N}` -- and the attribute
is exactly what this module strips, so the concept ends up named by the
title the author wrote.
Built in memory from the committed generator rather than added to
`tests/fixtures/k2-office/`, which holds exactly three containers and
asserts that it does.
"""
pytest.importorskip("pypandoc", reason="office conversion needs the [extract] extra")
# Loaded BY PATH: the generator lives under `tests/fixtures/` and is not on
# `sys.path`, so `importorskip` on its name skips silently -- a green run
# over an empty set, which is the one outcome this file exists to prevent.
spec = importlib.util.spec_from_file_location(
"make_k2_office", Path(__file__).parent / "fixtures" / "make_k2_office.py"
)
assert spec is not None and spec.loader is not None
make = importlib.util.module_from_spec(spec)
spec.loader.exec_module(make)
def titled(shape_id: int, name: str, text: str) -> str:
return (
"<p:sp><p:nvSpPr>"
f'<p:cNvPr id="{shape_id}" name="{name}"/><p:cNvSpPr/>'
'<p:nvPr><p:ph type="title"/></p:nvPr>'
"</p:nvSpPr><p:spPr/>" + make._pptx_text_body(text) + "</p:sp>"
)
parts = make.pptx_parts()
parts["ppt/slides/slide1.xml"] = make._pptx_slide(
titled(2, "Tittel", make.TITLE)
+ make._pptx_shape(3, "Ingress", make.INTRO)
+ make._pptx_table(4, "Kravtabell", tuple(make.PAIRS))
)
parts["ppt/slides/slide2.xml"] = make._pptx_slide(
titled(2, "Undertittel", make.GRID_CAPTION)
+ make._pptx_table(3, "Luminansmatrise", make.GRID)
)
data = make.build_container(parts, stored_first="[Content_Types].xml")
text = extract_text("tittelplassholder.pptx", data)
headings = [line for line in text.splitlines() if line.startswith("#")]
# The known-positive for the strip: the converter DOES write the anchor
# here, so a green assertion below is not green over an empty set.
assert any("{#slide-" in line for line in headings), headings
titles = [candidate.title for candidate in find_candidates(text, **DEFAULT)]
assert titles == [make.TITLE, make.GRID_CAPTION], titles

31
tests/test_default_bundle_pin.py
View file

@ -41,12 +41,26 @@ import okf_consume_measure # noqa: E402
#: --bundle-id k2-trinn1-20260903 --okf-version 0.2
#: with no arm flag at all -- the package default, which is the point.
#:
#: Rebuilt 2026-09-11 for `--close-span-gaps`. Digest, from inside the bundle:
#: Rebuilt 2026-09-09 for `--contents-name` (round 9). Digest, from inside the
#: bundle:
#: find . -type f -print0 | sort -z | xargs -0 shasum -a 256 | shasum -a 256
#: -> 8c93e5e3222577a2b3352ca83af980e403d3a571c3a467b83c3d8170b1df2b69
#: -> 21af4a1aa98315cf514c4cbc6b4a9b77ce63960224d6d7b31b34d55cc67fb2ad
#: (The previous default, `K2-bundle-default-20260911`, was
#: 8c93e5e3222577a2b3352ca83af980e403d3a571c3a467b83c3d8170b1df2b69 at 436
#: concepts and stays on disk.)
#: Two independent builds of it differ in NOTHING (`diff -rq`), including
#: `log.md`, which carries the corpus path and never the bundle's own.
DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260911"
#:
#: CONCEPT IDS MOVED IN THIS REBUILD, and not only because the count did.
#: Round 9 strips pandoc's `{#sheet-N}` / `{#slide-N}` anchor where a title is
#: formed, and a concept's filename is reduced FROM its title, so TWO ids on
#: this bundle are renamed:
#: del-ii-bilag-7-prisskjema/prissammenstilling-sheet-1 -> .../prissammenstilling
#: del-ii-bilag-0-dokumentliste-del-ii/ark1-sheet-1 -> .../ark1
#: The first is an id `portfolio-optimiser` has cited in writing. The rename
#: was authorised by the operator on 2026-09-09 after the exposure was counted:
#: 2 of 810 concepts on the previous default and 2 of 1108 on Arm B.
DEFAULT_BUNDLE = Path.home() / "corpora" / "okf-telling-20260829" / "K2-bundle-default-20260912"
GOLD_SET = PROJECT_ROOT / ".claude/projects/2026-09-07-okf-consume-prepass/hit-at-k-questions.json"
requires_default_bundle = pytest.mark.skipif(
@ -58,13 +72,18 @@ requires_default_bundle = pytest.mark.skipif(
),
)
#: Measured 2026-09-11 on the bundle above. The count moved 425 -> 436 with
#: Measured 2026-09-09 on the bundle above. The count moved 425 -> 436 with
#: `--sheet-section-rows --keep-table-heading`; `--first-span-from-zero` and
#: `--close-span-gaps` each moved it by NOTHING, which is the point of both --
#: they add no boundary, they only move a span's start or its end. Round 8's
#: rule closed 43 631 characters (2.51 % of the corpus) that were in no
#: segment, and the count is byte-for-byte the same 436.
EXPECTED_CONCEPTS = 436
#: segment, and the count was byte-for-byte the same 436.
#:
#: 436 -> 453 with round 9's `--contents-name`, which does add concepts: a run
#: of data rows is no longer read as a contents listing and discarded, so the
#: candidates it was taking with it survive. Corpus-wide, 429 -> 447 candidates
#: over 32 -> 33 documents with a plan, and characters in no segment stay 0.
EXPECTED_CONCEPTS = 453
EXPECTED_HITS = 5
#: Rank per question INDEX, `None` for the row that misses on every bundle and
#: every configuration measured so far. The identity is the index; the question

19
tests/test_docs_promises.py
View file

@ -58,19 +58,24 @@ def test_the_readme_no_longer_claims_docx_and_xlsx_fail_fast() -> None:
assert claim.lower() not in text, f"README still claims: {claim}"
def test_the_readme_states_which_rows_are_unmeasured() -> None:
"""An unmeasured row must not read as a supported one.
def test_the_readme_states_which_rows_are_not_measured() -> None:
"""A row that is not `measured` must not read as a supported one.
Three of the five office formats have denominator ZERO in the corpus this
work was measured on. A consumer reading the README should be able to see
that without reading the source.
Reads the CLASS from the table rather than the literal `unmeasured`: round
9 moved those three rows to `constructed`, and a test pinned to one word
would have gone green over an empty set the moment the word changed. Every
class that is not `measured` must be named in the README, whichever it is.
"""
text = README.read_text(encoding="utf-8")
unmeasured = {s.lstrip(".") for s, e in _EVIDENCE.items() if e == "unmeasured"}
assert unmeasured, "the evidence table lists no unmeasured rows"
for suffix in unmeasured:
assert suffix in text, f"README does not mention the unmeasured row {suffix}"
assert "unmeasured" in text.lower()
weaker = {s.lstrip("."): e for s, e in _EVIDENCE.items() if e != "measured"}
assert weaker, "the evidence table lists no rows weaker than measured"
for suffix, evidence in weaker.items():
assert suffix in text, f"README does not mention the {evidence} row {suffix}"
assert evidence in text.lower(), f"README does not use the word {evidence}"
def test_the_readme_still_states_what_stays_out() -> None:

8
tests/test_extract.py
View file

@ -180,11 +180,17 @@ def test_evidence_class_is_asserted_not_commented() -> None:
assert set(_EVIDENCE) == set(_PANDOC_FORMATS), "every row needs an evidence class"
assert {s for s, e in _EVIDENCE.items() if e == "measured"} == {".docx", ".xlsx"}
assert {s for s, e in _EVIDENCE.items() if e == "unmeasured"} == {
# Since 2026-09-09 the three office rows are `constructed`, not
# `unmeasured`: each has now been put through end to end on a hand-built
# document with a hand-written fasit, and none of them has a corpus file.
# The set is asserted EMPTY rather than dropped -- a class with no members
# is a fact about this package, and a future row can re-enter it.
assert {s for s, e in _EVIDENCE.items() if e == "constructed"} == {
".pptx",
".odt",
".rtf",
}
assert {s for s, e in _EVIDENCE.items() if e == "unmeasured"} == set()
def test_the_unparsed_set_is_empty_now_that_every_row_has_a_reader() -> None:

8
tests/test_k2_office_fixtures.py
View file

@ -16,8 +16,10 @@ becoming a memory of a run.
WHAT IT DOES NOT DO. Three synthetic documents in one house style are not a
corpus. A green run here says the row is exercised, never that the format is
covered; `_EVIDENCE` therefore stays `unmeasured` and this suite asserts that
it does.
covered. Round 9 (2026-09-09) measured all three end to end for the first time
and moved them `unmeasured` -> `constructed`, a third evidence class defined in
`extract._EVIDENCE`: the row has met a document, but not one anyone wrote for
their own purposes. This suite asserts that class rather than the old one.
"""
from __future__ import annotations
@ -113,4 +115,4 @@ def test_the_fasit_is_hand_counted_and_committed_beside_the_fixtures() -> None:
def test_three_synthetic_documents_do_not_make_a_row_measured() -> None:
"""One house style, N = 3, written by us -- that is exercise, not coverage."""
assert [_EVIDENCE[key] for key in (".pptx", ".odt", ".rtf")] == ["unmeasured"] * 3
assert [_EVIDENCE[key] for key in (".pptx", ".odt", ".rtf")] == ["constructed"] * 3

9
tests/test_propose_sheet_sections.py
View file

@ -79,14 +79,19 @@ def test_a_run_of_numbered_rows_cuts_the_block_into_sections() -> None:
of the block -- so the LAST section carries the sheet's tail, which is the
honest consequence of a rule that cuts and never discards.
"""
# The title carries NO `{#sheet-1}` since 2026-09-09: the converter anchor
# is stripped where the title is formed, because a concept's id is reduced
# from its title and the two must not name the same concept differently.
# The fixture keeps the anchor -- it is what pandoc writes, and removing it
# from the input would test the strip against text that never has it.
off = okf_propose_segments.find_candidates(SECTIONED_SHEET)
assert [(c.title, c.rule) for c in off] == [
("Kostnadsoversikt {#sheet-1}", okf_propose_segments.RULE_TABLE_BLOCK)
("Kostnadsoversikt", okf_propose_segments.RULE_TABLE_BLOCK)
]
on = okf_propose_segments.find_candidates(SECTIONED_SHEET, sheet_section_rows=True)
assert [(c.number, c.title, c.rule) for c in on] == [
(None, "Kostnadsoversikt {#sheet-1}", okf_propose_segments.RULE_TABLE_BLOCK),
(None, "Kostnadsoversikt", okf_propose_segments.RULE_TABLE_BLOCK),
("01", "01 Felleskostnader", okf_propose_segments.RULE_SHEET_SECTION),
("11+12", "11+12 Rigging og drift", okf_propose_segments.RULE_SHEET_SECTION),
("02", "02 Bygning", okf_propose_segments.RULE_SHEET_SECTION),