open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

docs(k3): round 10, and the fasit that held a working rule off the default [skip-docs]

Browse source 
The measurement record for `6ff84d7`, `191de89` and `36c201c`, in S7 form with
every figure carrying its denominator and an "Honesty limits" section.

Three of this round's own figures disagree with the order's and the
disagreement is stated first: the corpus has 5 `docx` and 43 files (round 9
counted `trinn1` and `trinn2`, which `diff -rq` shows byte-identical), the
lockfile drift is `uv run` without `--frozen` INSIDE the project rather than
outside it, and ruff 0.16.6 reports 148 rather than 144.

The result that decides the `rtf` rule's default is in section 1.7 and it is
not a missing measurement: `diff -rq` bounds the whole exposure at 1 of 33
document directories, that document is reference position 11, and its fasit
picks the arm that yields ONE concept where the rule yields two. The row would
get worse, so the flag ships off with every number that would justify moving
it. Whether that fasit should stand is the operator's, and it is one position.

Leak sweep with the control run first: 0 corpus paths and 0 K2 document names,
against 15 hits for a known-positive word in the same file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 23:27:03 +02:00
commit de9564d06a
1 changed files with 295 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

295
docs/2026-09-09-k3-runde10-rtf-prefiks-shim-ruff.md Normal file
View file

@ -0,0 +1,295 @@
# K3 round 10: the type that declares nothing, the prefix that is not a word,
# a shim a path importer cannot read, and a lint gate nobody had chosen
Date: 2026-09-09. Commits `6ff84d7`, `191de89`, `36c201c` on `main`, unpushed.
Baseline `6ff18fd`; see § 0 for what was reproduced before any code moved.
Every measurement below carries its denominator. Where this round's number
disagrees with the order's, the disagreement is stated first and this round's
number governs.
## 0. Reproduced before anything was touched
| what | order / round 9 | measured here |
|---|---|---|
| `pytest -q` | ≥ 1515 green | **1515 passed, 0 skipped** |
| `ruff check src tests tools` | clean | clean (0.15.22) |
| `ruff format --check .` | (not in the acceptance) | **RED**, 2 files |
| `mypy src` | clean, 21 files | clean, 21 files |
| ` M uv.lock` | 681 insertions / 4 deletions | **681 / 4**, reverted |
| pinned bundle | 453 concepts / 865 md | 453 concepts |
| Arm B | 629 concepts | 629 concepts |
| smoke test | 52 md / 26 concepts, 0 of 5 rejected | 52 md / 26 concepts |
| `okf project` = `okf build` | `diff -r` empty | empty |
**Three divergences from the order's own figures, all measured:**
1. **The corpus has 5 `docx`, not 10, and 43 files, not 86.** `K2/trinn1` holds
43 files (33 `pdf`, 5 `docx`, 2 `xlsx`, 1 `zip`, 1 `smc`, 1 `doc`), and
`diff -rq trinn1 trinn2` is empty — the two steps are byte-identical, and
round 9 counted both. The false-positive denominator is **5 distinct
`docx`**.
2. **The lockfile drift is `uv run` WITHOUT `--frozen` INSIDE the project**, not
`uv run` outside it. `uv.lock` contained **0** lines naming `rapidocr` or
`onnxruntime`, so it predated the `[ocr]` extra and any unlocked resolve
wrote that extra's tree back. Reproduced twice in this session. Round 9's
report does not carry the claim (`grep` for `uv.lock` over `docs/*.md` and
`README.md`: 0 hits, control `grep -c "uv"` on the same file: 3) — only
`STATE.md` did, and it is corrected there.
3. **Ruff 0.16.6 reports 148, not 144** — this round added four files.
## 1. `rtf`: 0 segments → 6 of 6 authored titles over N = 4
### 1.1 What the row was
0 of 0 declared headings, 0 concepts, **1368 of 1368 characters in no
segment**, N = **1**. The container has no heading style, so the author's title
is bold text.
### 1.2 The grammar is markdown, not `rtf`
The converter already emits that title as `**Kravspesifikasjon for
tunnelbelysning**` — a whole bold line in the SAME markdown every office row
produces. Bold inside a paragraph is `text **bold** text`, which is not a whole
line. That is the discriminator, and it means no `rtf`-only heading grammar
exists — the same shape of decision as the PDF font reader emitting ATX.
### 1.3 The false-positive denominator, measured FIRST
Over **47 readable documents** (3 office fixtures, 5 in the operator's folder,
39 of the 43 corpus files), split by whether the document declares a heading of
its own:
| max chars | standalone | no terminal punctuation | FP docs | FP lines | cand docs | cand lines |
|---|---|---|---|---|---|---|
| — | no | no | 2 | 12 | 4 | 10 |
| — | no | **yes** | 1 | 2 | 4 | 6 |
| 40 | no | no | 2 | 11 | 4 | 10 |
| 40 | no | **yes** | 1 | 1 | 4 | 6 |
| 40 | yes | **yes** | 1 | 1 | 4 | 6 |
| 120 | yes | **yes** | 1 | 1 | 4 | 6 |
**Three parameters swept, one carries.** A maximum title length (unlimited /
40 / 60 / 80 / 120) is flat at every value. A must-stand-between-blank-lines
clause is flat — the converter already writes one paragraph per line. Only
refusing a line that ends in terminal punctuation moves anything, taking
false-positive lines from 9–12 to 1–2 and dropping candidates like
`er inngatt mellom:` and `Sted og dato:` — a contract cover page's sentence
fragments, never section names. **A parameter measured flat is not in the
rule.**
### 1.4 The last false positive is closed by a principle already shipped
One survivor: a corpus `docx` declaring two headings of its own and setting
one later line in bold. The repair is not a fourth parameter — it is G1, which
`_gate_outline` and `fold_units` clause 2 already carry: **recovery yields to
declaration.** False positives are then **0 of the 31 declaring documents** by
construction, and **0 of the 27** on the corpus alone.
A second axis was measured and NOT added: `heading_reserve_applies` also yields
to an admitted Arm D run. Over all **4 of 4** documents that reach this rule the
outline gate admits 0 runs, so the axis is flat and coupling to `--outline-run`
would be a knob no measurement asks for. n = 4, which is small.
### 1.5 Both alternatives the order named were measured and felled
A fourth hand-laid fixture **declares** heading styles in an `rtf` stylesheet.
Measured: the vendored converter discards the style and emits the same bold
line, so alternative (i) — "read the declared headings out of the markdown" —
has nothing to read. Alternative (ii), `rtf` → `docx` → markdown, yields **0
ATX headings** on that same document and on the original fixture: the loss is
in the `rtf` READER, before any writer sees the style. No intermediate
container recovers it.
### 1.6 The row, against a fasit written first
Fixtures are hand-laid in `tests/fixtures/make_k2_office.py` — the file's own
policy forbids the converter for committed fixtures, and the policy wins. The
fasit (4 documents, **6 authored titles**, 1 non-title bold run) was committed
before this measurement ran.
| document | axis varied | authored titles | recovered | false | chars in no segment |
|---|---|---|---|---|---|
| `krav-rikt-tekstformat.rtf` | bold + 16 pt | 1 | 1 | 0 | 1368 → 0 |
| `…-fet-alene.rtf` | bold at BODY size | 2 | 2 | 0 | 215 → 0 |
| `…-fet-i-avsnitt.rtf` | bold run mid-sentence | 1 | 1 | 0 | 196 → 0 |
| `…-stil.rtf` | declares `\s1`/`\s2` | 2 | 2 | 0 | 215 → 0 |
| **total, N = 4** | | **6** | **6** | **0** | **1994 → 0** |
The variants live in their own directory: `test_k2_office_fixtures.py` reads N
off `k2-office/` by listing it, because Door B walks a drop directory
recursively.
### 1.7 Reach, and why the flag is OFF
Corpus reach: **2 of 39** readable documents, both `docx`, **0 of 33 `pdf`**
(never through the converter, so inert by construction) and **0 of 2 `xlsx`**.
Operator's folder: 26 → **27** concepts, 52 → **53** md, replacing a mechanical
`tabell-linje-30` with two named concepts.
A K2 bundle built with it: **454 concepts / 867 md**, hit@8
**`[1,1,1,1,1,None]`** — identical to the pinned 453-concept bundle and to Arm
B. The retrieval side costs nothing.
**The default does not move, and the reason is a fasit rather than a missing
measurement.** `diff -rq` between the pinned bundle and the new one names the
whole exposure: **1 of 33 document directories** plus `index.md`. That one
document is **reference position 11**, whose fasit names the whole document as
a single lookup and picks the arm that yields **one** concept. The rule makes
it two. **That row would get
worse**, and "no row gets worse" is the acceptance. The flag ships OFF with
every number above; whether position 11's fasit should stand is the operator's,
and it is one position, not twelve.
## 2. The prefix over-match
### 2.1 The number, re-measured, control FIRST every time
| word | equality | prefix | share | bundle |
|---|---|---|---|---|
| `under` (control) | **79** | 172 | 38.0 % | default-20260912 (453) |
| `undersjøisk` | 0 | 172 | 38.0 % | " |
| `bilateral` | 0 | **400** | **88.3 %** | " |
| `standhaftig` | 0 | 219 | 48.3 % | " |
| `under` (control) | 76 | 174 | 27.7 % | Arm B (629) |
| `undersjøisk` | 0 | 174 | 27.7 % | " |
Round 9's figures reproduce exactly on the bundle they were taken on
(`-20260911`, 436 concepts: 171 and control 79).
**How the two extra known-negatives were FOUND, not chosen:** every 4-character
prefix in the pinned bundle was ranked by document frequency (`bila` 400 of
453, `stan` 219, `skol` 205, `oppd` 185, `doku` 178, `unde` 172 …), and a real
Norwegian word was taken from the widest. They are still this session's choice
and not an independent source.
### 2.2 Three candidates, all three failed on the same row
| candidate | `bilateral` | `standhaftig` | `undersjøisk` | control | ranks, default | ranks, Arm B |
|---|---|---|---|---|---|---|
| shipped, floor 4 | 400 | 219 | 172 | 172 | `1,1,1,1,1,–` | `1,1,1,1,1,–` |
| floor 5 | 0 | 56 | 171 | 171 | **`2`**,1,1,1,1,– | **`–`**,1,1,1,1,– |
| floor 6 | 0 | 0 | 35 | 79 | **`2`**,1,1,1,1,**`2`** | **`–`**,1,1,1,1,1 |
| floor 7 / 8 | 0 | 0 | 0 | 79 | **`2`**,1,1,1,1,1 | **`–`**,1,1,1,1,1 |
| coverage ≥ 0.50 | 0 | 0 | 23 | 97 | **`2`**,1,1,1,1,**`2`** | **`–`**,1,1,1,1,1 |
| coverage ≥ 0.60–0.80 | 0 | 0 | 0 | 82–88 | **`2`**,1,1,1,1,1 | **`–`**,1,1,1,1,1 |
| words ≥ 6 | 0 | 219 | 137 | 79 | 1,1,1,1,1,– | 1,1,1,1,1,– |
| words ≥ 8 / 10 / 12 | 0 | 56–0 | 137–0 | 79 | **`2`**,1,1,1,1,– | **`–`**,1,1,1,1,– |
Only "words ≥ 6" holds every rank, and it fixes `bilateral` alone.
### 2.3 Why they all fail on one row, decomposed
Row 1 has 4 question tokens and its gold document spans 12 concepts. Token by
token, the loose prefix pairs are:
- `hvordan` → `hvor` (4 shared)
- `prisene` → `pris`, `prisskjema`, `prissammenstilling`,
`prissammenstillingstabell` (4 shared)
`pris` is 0.57 of `prisene` and 0.22 of `prissammenstilling`, so **a coverage
rule at 0.60 cuts exactly the match that finds the price sheet.** The
over-match and the wanted match are one mechanism seen from two sides, and no
threshold on length or coverage separates them.
### 2.4 The fourth candidate: the shared prefix must be a WORD
`pris` occurs as a token on its own; `bila` and `stan` never do.
| bundle | `bilateral` | `standhaftig` | `undersjøisk` | control | ranks |
|---|---|---|---|---|---|
| default-20260912 | 400 → **0** | 219 → **56** | 172 → 162 | 172 | `1,1,1,1,1,–` |
| Arm B | 512 → **0** | 235 → **33** | 174 → 160 | 174 | `1,1,1,1,1,–` |
Both acceptance criteria met at once. **ON by default**, opt-out
`--no-stem-prefix`, pinned in `tests/test_default_bundle_pin.py` with its own
known-negative on the shipped bytes.
**What remains is a different answer, not a ceiling.** `undersjøisk` still
reaches 162 because it shares `under`, which IS a word here — a productive
Norwegian prefix. A rule cutting that would cut the behaviour
`MIN_SHARED_PREFIX` exists for. The residual is a statement about Norwegian.
Like `--tie-shared-rank`, this **alters a payload with no bundle changing**: a
consumer pinned to the previous excerpt order needs the opt-out.
## 3. The shim
`sys.modules[__name__] = _impl` replaces the REGISTRY entry. A caller using
`importlib.util.spec_from_file_location` holds the object `module_from_spec`
built, which that line never reaches. Measured under two counting methods on
the caller-held object: **3 of 76** public names by `vars()`, and
`build_payload` absent. (The order carried 4 of 90 and 3 of 75; this round's
counts govern.)
One line, before the alias, copying the public names into this file's globals.
The dunder filter is load-bearing: an unfiltered `vars(_impl)` overwrites
`__name__` before the next statement uses it as the alias key. It restores
attribute **access** and not patch-through — a caller monkeypatching a copy
patches a binding the implementation never reads, which is why the alias stays.
**What the consumer actually asked for is also done**, and it was a line rather
than a code change: the shim docstring and a `CHANGELOG.md` note under 0.7.0
(no new version, tag unmoved) both say to read the module back out of
`sys.modules` after `exec_module`. Sent to `vegnormal-okf` by `coord-send`.
## 4. Ruff
**The defect is not the 148 findings, it is that nobody chose them.**
`[tool.ruff]` set only `line-length` and `target-version`, so the acceptance was
whatever ruff's default happened to be, and the tree read green only as long as
`uv.lock` froze ruff at 0.15.22. 0.16 widened the default set to whole families.
Per rule, before → after: RUF100 50 → 0, I001 20 → 0, ISC004 19, PLW1510 8,
C408 8, EXE001 6, RUF007 5, PLE2515 4, UP031 3, B017 3, UP037/UP007/RUF022/
PLR0402/FURB167/C402 2 each, and ten more at 1. The families outside the
declared set are 0 by selection, and **148 is the number to start from** if they
are adopted — a separate decision, not one to take inside a version-pin commit.
`select = ["E4", "E7", "E9", "F", "I", "RUF100"]`; pin `ruff>=0.16.6,<0.17`
(0.16.6 established by resolving `ruff` unconstrained in a clean venv). `S` is
measured out rather than assumed out: **2657** `S101` on a suite whose every
assertion is an `assert`, and `S603` flags **19** subprocess calls of which one
was ever marked.
**The 2 files 0.16 would reformat are markdown, not Python** — `README.md` and
`docs/2026-09-08-blindsone-below-k-k2.md`. 0.16 formats fenced Python inside
markdown, and the second block is a **quotation** of `COST_VOCABULARY` as it
stood when that measurement was taken. Reformatting a quotation makes it stop
being one, so markdown is excluded from the formatter.
`tools/okf_consume_measure.py` is fenced by the order as run-not-edited; its
three findings are exempted by path with the reason and the debt named, and its
bytes are untouched.
**The lockfile trap is closed, not avoided.** The relock is complete for every
declared extra (703 insertions, 26 deletions), and measured afterwards an
unfrozen `uv run` leaves the file alone.
## 5. Honesty limits
- **n = 12 reference positions**, of which **8 `pdf`, 3 `docx`, 1 `xlsx`** — so
every per-type fraction is a fraction of 8, 3 or 1. Both sheet readings are
reported, never one.
- **Position 11's verdict rests on the first rater alone.** The sheet records
it: the blind positions are 0, 2, 4, 6, 8 and 10, and 11 is not among them.
The decision to hold `--bold-title` off rests on that single rating.
- **n = 6 questions** on the retrieval side, one rater, one gold set, and row 6
misses on every bundle and every configuration measured so far.
- **The three known-negative words in § 2 were chosen by this session** from the
bundle's own vocabulary, not by an independent source. The METHOD (rank every
4-character prefix by document frequency) is stated so it can be re-run.
- **The `rtf` row rests on 4 hand-built documents, not corpus files.** The
corpus contains **zero** `rtf`. The evidence class stays `constructed`, and
`measured` — real corpus files with a hand-counted fasit — is not claimed.
- **"General" rests on ONE corpus of 43 documents**, `K2/trinn2` byte-identical
with `K2/trinn1` and `K1` empty.
- **"Delivered" is not "answered."** Every retrieval figure here is a rank, not
a correct answer.
- **`--stem-prefix` makes a payload depend on the corpus** the way
`rarity_weights` already does. It is deterministic for a given bundle and not
transferable between bundles.
- Blocked on the operator and not on code: **position 0's `|F|`**, the
**`xlsx` fasit's `|F|` = 12**, the **`pptx` fasit**, **routing between bundles
via `--id`**, and now **whether position 11's fasit should stand** against
`--bold-title`.