open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

docs(k3): round 16, the whole title takes the route precision could not

Browse source 
The report, the CHANGELOG entry under [Unreleased] and the README's ranking
section. No version bump and no tag: v0.8.0 was released this same day.

R761, delivered ranks, before -> after: hit@1/8/50 3/6 - 5/6 - 5/6 -> 6/6 -
6/6 - 6/6 at default k and 3/6 - 5/6 - 6/6 -> 6/6 - 6/6 - 6/6 at --k 50, with
the known-positive at rank 1 on all four and the known-negative still not a
hit. --no-title-covered on the new binary reproduces the old row exactly.

Nothing else moves, measured on the bytes: the pinned K2 bundle and Arm B keep
(1,1,1,1,1,5) with payloads byte-identical on 6 of 6 questions each,
n100/n200/n500 payloads byte-identical on 5 each, the 828-file HTML corpus at
828 plans / 0 unreadable / 6 015 md with diff -rq empty, the five-document
folder diff -r-identical at 26/52, and okf project still byte-equal to okf
build.

README's test count corrected to what this round measures: 1583 collected,
1582 passed, 1 skipped. The published 1575 was the passed count.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-10 22:26:57 +02:00
commit 4d1b5222b7
4 changed files with 392 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

59
CHANGELOG.md
View file

@ -7,6 +7,65 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
### Added
- **`--title-covered` (ON by default, opt out with `--no-title-covered`): a
question that accounts for a concept's WHOLE title reads that concept first.**
On the 2 761-concept bundle of one standard, the answering section was
delivered at rank 1 on **3 of 6** scored questions and **none of the reading
side's six flags moved that number** -- the whole sweep sits at 3/6 or worse.
Measured on that bundle, before and after: hit@1/8/50 **3/6 - 5/6 - 5/6 ->
6/6 - 6/6 - 6/6** at default `k` and **3/6 - 5/6 - 6/6 -> 6/6 - 6/6 - 6/6** at
`--k 50`, with the known-positive holding rank 1 at both and the
known-negative still not a hit. S1 4 -> 1, S5 not delivered -> 1, S6 3 -> 1.
- **THE DEFECT IS THAT BOTH LEXICAL SIGNALS ARE UNNORMALISED COVERAGE
COUNTS.** They measure how much of the QUESTION a candidate answers and
nothing measures how much of the CANDIDATE the question accounts for, so a
section titled with the question's subject alone scores what a narrower
section titled with that subject PLUS a qualifier scores, and then loses on
the body count. Decomposed per miss: S1 turns on `hvordan`, an interrogative
pronoun; S5 on `hvilke` and `stilles` in a body 7x the gold's, and on
`betonghvelv ~ betongkonstruksjoner` through the four-character stem
`betong`; S6 on an exact tie broken by `concept_id`.
- **A PARTITION, never a fourth RRF signal, and the arithmetic is why.** RRF
consumes ranks alone, so with shared ranks a rule whose positive group has
`m` members is worth `1/61 - 1/(61 + m)` -- a rule firing on ONE concept of
2 761 is worth 0.00026 against a body gap of 0.0029. **A precise rule is
worth LEAST under this fusion.** Measured as a signal it moves hit@1 not at
all (3/6, both as a third and as a fourth signal); as a partition it reaches
6/6. `lookup_hits` is the same shape for the same measured reason, and it
still wins: the new partition lands below it, with a test and its control.
- **By EQUALITY, never by shared prefix.** Four shared leading characters take
the group from 1 to 6 on one question and 9 to 31 on another, with the
answering section falling to candidate rank 6 and the known-positive to 2.
- **TWO CANDIDATE REPAIRS WERE MEASURED AND FELLED FIRST.** Pivoted length
normalisation of the body signal collapses at every value swept
(b = 0.25/0.5/0.75/1.0 -> hit@8 3/6, 1/6, 1/6, 0/6, and at b = 1.0 the
known-positive falls to rank 49): the median concept holds 22 tokens against
a mean of 60, so length normalisation promotes thousands of tiny concepts.
Title PRECISION as a signal reaches candidate hit@1 5/6 and takes the
known-positive from 1 to 4 every time it does.
- **NOTHING ELSE MOVES AND IT IS MEASURED ON THE BYTES.** The pinned K2 bundle
keeps `(1,1,1,1,1,5)` and its 7 pin tests, Arm B keeps `(1,1,1,1,1,5)`, and
the payloads on both are **byte-identical on 6 of 6 questions**; n100/n200/
n500 payloads are byte-identical on 5 questions each; the 828-file HTML
corpus still gives 828 plans, 0 unreadable and 6 015 md with `diff -rq`
empty; the five-document folder is `diff -r`-identical at 26 concepts / 52
md; `okf project` stays byte-equal to `okf build`. hit@k on n100/n200/n500
is **NOT MEASURED** -- this repository holds no gold set for them, which is
0 gold sets and not 0 hits.
- **THIS IS THE FOURTH READING-SIDE CHANGE THAT MOVES A PAYLOAD WITH NO BUNDLE
CHANGING.** A consumer pinned to the previous excerpt order needs
`--no-title-covered`. The rule fires on **0 of 21** measured cells outside
that one bundle, so "no regression" there means it never fires -- not that
it fires harmlessly. Report:
`docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md`.
- `build_payload`'s signature defaults are now held equal to the consume CLI's
argparse defaults by a test, for every same-named parameter. This is O6's
defect in the other command: `cli.build` defaulted two flags `False` in the
signature and `True` in argparse, and a caller reaching it as a function read
the signature.
## [0.8.0] — 2026-09-10 ## [0.8.0] — 2026-09-10
### Added ### Added

52
README.md
View file

@ -408,6 +408,46 @@ number that had stopped being true in the same commit. A flag's "off by
measurement" is a measurement of a *configuration*, not a property of the flag. measurement" is a measurement of a *configuration*, not a property of the flag.
`docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`. `docs/2026-09-10-k3-runde7-forste-spenn-og-rangeringen.md`.
`--title-covered` is **on** by default since 2026-09-10 (opt out with
`--no-title-covered`), and it is the fourth widening here that alters a payload
with no bundle changing. A concept whose EVERY title token is a token of the
question is read before the concepts the fusion ranked above it. The defect it
repairs is that both lexical signals are unnormalised COVERAGE COUNTS: they
measure how much of the question a candidate answers, and nothing measures how
much of the CANDIDATE the question accounts for, so a section titled with the
question's subject alone scores exactly what a narrower section titled with that
subject plus a qualifier scores — and then loses on the body count, because a
longer title and a longer body can only reach more of the question. Measured on
a 2 761-concept bundle of one standard, where **none of the six flags above
moved the number at all**: hit@1/8/50 **3 of 6 · 5 of 6 · 5 of 6 → 6 of 6 · 6 of
6 · 6 of 6** at default `k`, the same 6 of 6 at `--k 50`, the known-positive
holding rank 1 at both and the known-negative still not a hit. The three
recovered rows go 4 → 1, not-delivered → 1 and 3 → 1.
It is a PARTITION and not a fourth RRF signal, and the arithmetic is the
reason: RRF consumes ranks alone, so with shared ranks a rule whose positive
group has `m` members is worth `1/61 − 1/(61 + m)`, and a rule firing on ONE
concept of 2 761 is worth 0.00026 against a body-signal gap of 0.0029 — **a
precise rule is worth least under this fusion, backwards from what precision is
for**. Measured as a signal the same predicate moves hit@1 not at all; as a
partition it reaches 6 of 6. `lookup_hits` is the same shape for the same
measured reason, and it still wins: this partition lands below it. The title is
read by EQUALITY, never by shared prefix — four shared leading characters take
the group from 1 to 6 on one question and 9 to 31 on another, with the answering
section falling to candidate rank 6 and the known-positive to 2.
Two candidate repairs were measured and felled first. Pivoted length
normalisation of the body signal collapses at every value swept (b = 0.25, 0.5,
0.75, 1.0 give hit@8 3, 1, 1 and 0 of 6, and at 1.0 the known-positive falls to
rank 49), because the median concept holds 22 tokens against a mean of 60 — so
length normalisation promotes thousands of tiny concepts over the section that
treats the subject. Title *precision* as a signal reaches candidate hit@1 5 of 6
and takes the known-positive from rank 1 to 4 every time it does. The reach of
what shipped is narrow and stated as such: it fires on 6 of 8 questions on that
bundle and on **0 of 21** measured cells across the pinned K2 bundle, Arm B and
the three N bundles, whose payloads are byte-identical either way.
`docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md`.
It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a
`sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a `sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a
validated known-positive), `denominators`, `excerpts` and `withheld` — and every validated known-positive), `denominators`, `excerpts` and `withheld` — and every
@ -764,12 +804,14 @@ From a checkout, the test suite runs with:
.venv/bin/python -m pytest .venv/bin/python -m pytest
``` ```
The suite is the verification surface for everything above: **1575 tests**, The suite is the verification surface for everything above: **1583 tests
run on 2026-09-10 against the `v0.8.0` release commit with the `[extract]` collected, 1582 passed and 1 skipped**, run on 2026-09-10 with the `[extract]`
extra installed. extra installed. Both numbers are given because they are two measurements: the
figure published before this round was the PASSED count, and `pytest
--collect-only -q` reported one more.
(The figure stood at 596 until 2026-09-09 — measured 2026-08-21 and never (The figure stood at 596 until 2026-09-09 — measured 2026-08-21 and never
updated as the suite grew — and at 1515 until this release: a count is a updated as the suite grew — at 1515 until the `v0.8.0` release and at 1575
measurement with a date on it.) through it: a count is a measurement with a date on it.)
Without the extra the same suite skips the tests covering the parser path; Without the extra the same suite skips the tests covering the parser path;
that split was last counted on 2026-08-21 as 589 passed and 7 skipped and has that split was last counted on 2026-08-21 as 589 passed and 7 skipped and has
**not** been re-measured since. The tests holding the fail-fast rejection for **not** been re-measured since. The tests holding the fail-fast rejection for

282
docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md Normal file
View file

@ -0,0 +1,282 @@
# K3 round 16 — a question that accounts for the WHOLE title takes that section first
Round 15's bundle of one standard delivered the answering section at rank 1 on
**3 of 6** scored questions, and none of the reading side's six flags moved that
number. This round decomposes the three misses, falsifies two candidate repairs
on the corpus, and ships one rule: `--title-covered`, on by default, opt-out
`--no-title-covered`.
Commits: `14ae36a` (red test) · `6bb2189` (the rule) · `ff06b92` (the two
guarantees this round states) · this report.
## 0 Deviations from the order, stated before anything is built on them
Every premise the order named was reproduced. Two readings differ, and in both
cases the number below is the one measured here.
| premise | order | measured here | command |
|---|---|---|---|
| HEAD | `02f9876` | `02f9876` | `git log -1` |
| tag `v0.8.0` | exists, annotated, on `4d1f9d3` | exists, on `4d1f9d3` | `git tag --points-at 4d1f9d3` |
| version | `0.8.0` | `0.8.0` | `grep -n '^version' pyproject.toml` |
| remote `main` | `078b7df` | `078b7df` | `git ls-remote origin refs/heads/main` |
| worktree | clean | clean, `uv.lock` untouched | `git status --short` |
| suite at HEAD | 1575 passed, 1 skipped | **1575 passed, 1 skipped**, 95.84 s | `pytest -q` |
| collected at HEAD | 1576 | **1576** | `pytest --collect-only -q` |
| R761 build | rc 0, 23.2 s, 2 737 dirs, 2 761 concepts | rc 0, **25 s**, 2 737 dirs, **2 761** concepts | `okf build inn-xml …` |
| R761 at default `k` | 3/6 · 5/6 · 5/6, KP 1 | **identical**, and S1 rank 4 · S5 not delivered · S6 rank 3 | `hitk_sk2.py` |
| R761 at `--k 50` | 3/6 · 5/6 · 6/6, KP 1 | **identical** | `hitk_sk2.py --k 50` |
**Deviation 1.** The order's table gives S1 rank **4**; that is its rank at
default `k`. At `--k 50` it is **3**. Both are measured here; the order states
only the first and the two are not in conflict.
**Deviation 2, and it is about an INSTRUMENT rather than about the code.** The
sweep harness written for this round ranks CANDIDATES and the published figures
are DELIVERED ranks, so the two do not agree and must not be read as one number.
On the same bytes the harness gives S1 5 · S3 2 · S6 3 where the CLI delivers
4 · 1 · 3: the cut runs after the fusion and removes candidates above the gold.
Every harness figure in § 3 is labelled `candidate rank`; every figure in § 4 and
§ 5 came out of `okf consume` itself.
Everything was measured in frozen `git archive` exports under
`/tmp/k3r16/{export,export2}`, each `uv sync --frozen --extra extract`, using
that export's own `.venv/bin/okf`. `~/repos/vegnormal-okf` and
`~/okf-test/dokumenter` were read and never written.
## 1 The diagnosis, per miss
The fusion has three signals. Two are lexical COVERAGE COUNTS — one per question
token the candidate answers to, over the title-and-id text and over the body.
The third is the document prior.
**The third signal carries no information on this bundle, and that is measured
rather than inferred.** `document_score` is keyed on
`concept_id.split("/", 1)[0]`, which is the source document; R761 is one XML
file, so the prior takes **one distinct value over all 2 761 concepts**. With
`tie_shared_rank` on, every concept receives the same constant from it. The
order's hypothesis is CONFIRMED. Its evidence — that `--no-tie-shared-rank`
drops hit@1 to 0/6 — is confirmed too: a signal that separates nothing still
imposes an alphabetical order when the tie-break is by id.
So on this bundle the ranking is decided by two coverage counts, and the
question is which of them turns each miss over. Decomposed per candidate, with
the question tokens each one answers:
**S1** — «Hvordan er Prosesskoden delt inn i hovedprosesser?», tokens
`(hvordan, prosesskoden, delt, inn, hovedprosesser)`.
| rank | concept | title cov | body cov | body tokens answered | bytes |
|---|---|---|---|---|---|
| 1 | `11/oversikt-over-prosesser` | 1 | 3 | prosesskoden, delt, hovedprosesser | 114 728 |
| 2 | `hovedprosess-81-l-smasser` | 1 | 3 | **hvordan**, prosesskoden, hovedprosesser | 2 103 |
| 3 | `hovedprosess-83-…` | 1 | 3 | — | 14 529 |
| 4 | `hovedprosess-84-betong` | 1 | 3 | **hvordan**, prosesskoden, hovedprosesser | 4 410 |
| **5** | **`2-1/hovedprosesser`** | 1 | **2** | prosesskoden, hovedprosesser | 1 047 |
The body signal turns it over, and the token it turns on is `hvordan` — an
interrogative pronoun. The title signal separates nothing: every one of these
scores 1 and sits in a tie group thousands of concepts wide.
**S5** — «Hvilke krav stilles til armering i betongkonstruksjoner?».
| rank | concept | title cov | body cov | body tokens answered | bytes |
|---|---|---|---|---|---|
| 1 | `84-4/betongst-p` | 1 | **5** | **hvilke**, krav, **stilles**, armering, betongkonstruksjoner | 19 658 |
| 2 | `34-43/armering-av-betonghvelv` | **2** | 3 | krav, armering, betongkonstruksjoner | 442 |
| … | | | | | |
| **12** | **`84-3/armering`** | 1 | 3 | krav, armering, betongkonstruksjoner | 2 759 |
Two mechanisms, not one. The rank-1 concept wins on `hvilke` and `stilles`,
both grammatical; its body is 7x the gold's, and a longer body can only reach
MORE distinct question tokens. The rank-2 concept wins on the TITLE, at
`betonghvelv ~ betongkonstruksjoner` through the four-character stem `betong` —
a longer title can only reach more too.
**S6** — «Hva omfatter inspeksjon av bruer og konstruksjoner?».
| rank | concept | title cov | body cov | body tokens answered | bytes |
|---|---|---|---|---|---|
| 1 | `88-14/enkel-inspeksjon` | 1 | 4 | omfatter, inspeksjon, bruer, **konstruksjoner** | 446 |
| 2 | `81-6/masser-…-konstruksjoner-under-vann` | 1 | 3 | omfatter, bruer, konstruksjoner | 715 |
| **3** | **`88-1/inspeksjon`** | 1 | 3 | omfatter, inspeksjon, bruer | 585 |
Rank 2 is a TIE with the gold on both signals, broken by `concept_id` — the
declared tie-break, doing what it is declared to do because nothing else
separates them.
**The order's reading of the pattern is confirmed and can be stated more
precisely.** The three golds carry the bare term as their title
(`Hovedprosesser`, `Armering`, `Inspeksjon`) and lose to a title that is that
term plus a qualifier, or to a much larger neighbour. The mechanism under all
three is one thing: **both signals measure how much of the QUESTION the
candidate answers, and nothing measures how much of the CANDIDATE the question
accounts for.** Coverage is recall with no precision beside it, so surface area
— a longer title, a longer body — can only help.
What does not fit the order's phrasing: on S6 the gold is not beaten by the
larger neighbour at all, it is beaten by `concept_id` order at an exact tie, and
on S1 the deciding token is a pronoun rather than a qualifier. Both are the same
defect seen from different sides.
**It is not a segmentation miss on this bundle, and the control says so.**
`bundle-xml/<doc>/2-1`, `/84-3`, `/88-1` and `/1` all exist as concept
directories. On the PDF arm (`build/sk2-bundle-default`, read only)
`find . -maxdepth 2 -type d -name '22' / '53' / '1'` are all EMPTY — only
`22-1`, `53-1` … exist — so S3/S4/KP are segmentation misses there. That is the
known-positive proving the presence query can find a difference.
## 2 Two candidate repairs, measured and FELLED
Both were measured on the corpus before the shipped rule was written.
**Length normalisation of the body signal — falsified.** The textbook answer to
"unnormalised coverage favours long documents" is pivoted length normalisation,
`cov / (1 - b + b*len/avg)`. Swept at b = 0.25, 0.5, 0.75, 1.0 over the eight
questions (candidate ranks):
| b | hit@1 | hit@8 | S1 | S3 | S4 | S5 | KP |
|---|---|---|---|---|---|---|---|
| off (baseline) | 2/6 | 5/6 | 5 | 2 | 1 | 12 | 1 |
| 0.25 | 1/6 | 3/6 | 3 | 55 | 72 | 42 | 1 |
| 0.5 | 1/6 | 1/6 | 32 | 90 | 61 | 183 | 1 |
| 1.0 | 0/6 | 0/6 | 59 | 96 | 64 | 200 | **49** |
It collapses at every value including the known-positive. The reason is the
corpus's shape: the median concept holds **22** tokens against a mean of 60, so
thousands of tiny concepts mentioning a term once are promoted over the section
that treats it. Length is not what is wrong with the body signal.
**Title precision as a signal — falsified on the control.** `|Q ∩ T| / |T|` over
the title alone reaches candidate hit@1 5/6 in one arrangement, and takes the
known-positive from rank 1 to **4** every time it does. KP's gold is titled
`Bruksområder for Prosesskoden` — a phrase, one of whose three tokens the
question answers — so a ratio that rewards short titles demotes it. The fasit's
own comment says a KP below rank 1 means the instrument is broken, so this is
not a trade to make.
## 3 Why a PARTITION and not a fourth signal
The order asks for this to be answered rather than assumed, and the arithmetic
answers it.
RRF consumes RANKS ALONE, so one signal contributes at most `1/(RRF_K + 1)`.
With shared ranks, a signal whose positive group has `m` members gives the group
position 1 and everyone else position `m + 1` — so its whole separating power is
`1/61 - 1/(61 + m)`. **A rule that fires on ONE concept of 2 761 is worth
0.00026**, while the body-signal gap it must close on S1 is
`1/61 - 1/74 = 0.0029`, an order of magnitude larger. **A precise rule is worth
LEAST under this fusion, exactly backwards from what precision is for.**
Measured, not just derived. The same title-subset predicate, as a signal:
| form | hit@1 (candidate) | hit@8 | S1 | S5 | S6 | KP |
|---|---|---|---|---|---|---|
| baseline | 2/6 | 5/6 | 5 | 12 | 3 | 1 |
| as a 3rd signal (replacing the dead prior) | 3/6 | 6/6 | 5 | 2 | 2 | 1 |
| as a 4th signal | 3/6 | 6/6 | 5 | 2 | 2 | 1 |
| **as a PARTITION** | **6/6** | **6/6** | **1** | **1** | **1** | **1** |
This is the same measurement `concept_scores` already records for `lookup_hits`:
a fourth RRF signal put the named concept at 26/15/19 of 446/1 133/270 and
delivered none of them, so the lookup became a partition. The rule shipped here
is that shape for the same reason, and the figures above are its own — not
borrowed from that one.
## 4 The rule
`title_covered_hits(concepts, question)` returns the concepts whose EVERY title
token is a token of the question. `concept_scores` reads them first, keeping the
fusion's order inside each part, BELOW the lookup partition.
**By equality, never by shared prefix.** `tokens_match` accepts four shared
leading characters. Under a prefix-matching form the group goes 1 → 6 on S1 and
9 → 31 on S5, and the gold falls to candidate rank 6 with the known-positive to
2. Under equality both hold rank 1. The precedent is `tokens_match`'s own rule
that an identifier matches by equality alone.
**Reads `title` and not the concept id.** An id segment is this library's own
reduction of the title, so counting it would let the same words qualify twice,
and on a single-document bundle the id carries a uuid that is in no question.
**Reach, measured, and it is narrow.** Group size per question on R761:
| S1 | S2 | S3 | S4 | S5 | S6 | KP | KN |
|---|---|---|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 9 | 2 | **0** | **0** |
It fires on 6 of 8 questions there and on **0 of 6** K2 questions and **0 of 15**
N-bundle cells (§ 5). It never fires on the known-negative.
## 5 The row, and every other row measured
**R761, delivered ranks from `okf consume`, 2 761 concepts, one source document:**
| | before, default `k` | after, default `k` | before, `--k 50` | after, `--k 50` |
|---|---|---|---|---|
| hit@1 | **3/6** | **6/6** | **3/6** | **6/6** |
| hit@8 | 5/6 | **6/6** | 5/6 | **6/6** |
| hit@50 | 5/6 | **6/6** | 6/6 | **6/6** |
| KP rank | 1 | **1** | 1 | **1** |
| KN | not a hit | **not a hit**, same top-1 | not a hit | not a hit |
| S1 · S5 · S6 | 4 · – · 3 | **1 · 1 · 1** | 3 · 11 · 3 | **1 · 1 · 1** |
**The known-negative on the same bytes.** `--no-title-covered` on the NEW binary
reproduces the old row exactly — S1 4, S5 not delivered, S6 3, hit 3/6 · 5/6 —
identical row for row to the OLD binary at shipped defaults. The pin names its
own cause.
**Nothing else moves, and it is measured on the bytes:**
| row | denominator | before | after |
|---|---|---|---|
| K2 pinned default bundle, hit@8 ranks | 6 questions, 453 concepts | `(1,1,1,1,1,5)` | **identical**, payloads **byte-identical on 6 of 6** |
| Arm B bundle, hit@8 ranks | 6 questions, 629 concepts | `(1,1,1,1,1,5)` | **identical**, payloads **byte-identical on 6 of 6** |
| `tests/test_default_bundle_pin.py` | 7 tests | 7 passed | **7 passed**, file unedited |
| n100-2023 | 5 questions, 446 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| n200-2024 | 5 questions, 1 133 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| n500-2024 | 5 questions, 270 concepts | — | payloads **byte-identical**, rule fires 0 of 5 |
| 828-file HTML corpus | 828 documents | 828 plans, 0 unreadable, 6 015 md | **828 plans, 0 unreadable, 6 015 md**; `diff -rq` empty apart from `log.md`'s converter PATH (same pandoc 3.9) |
| `~/okf-test/dokumenter` | 5 documents | 26 concepts / 52 md | **26 / 52**, `diff -r` **EMPTY** |
| `okf project` vs `okf build` | same folder | byte-equal | **byte-equal**, `diff -r` empty |
| suite | — | 1575 passed, 1 skipped | **1582 passed, 1 skipped**, 1583 collected |
| `ruff check src tests tools` · `ruff format --check .` · `mypy src` | ruff 0.16.6 | clean | **clean** (118 files, 21 source files) |
**hit@k on N100/N200/N500 is NOT MEASURED, and the denominator is why:** this
repository holds no gold set for those three bundles — 0 gold sets, not 0 hits.
What is measured there is that the payload does not change.
## 6 Honesty limits
- **N = 6.** hit@k here rests on ONE fasit with SIX scored questions on ONE
corpus from ONE publisher in ONE schema. A move from 3/6 to 4/6 would be one
question; this round moves three. Three questions is not a retrieval
evaluation.
- **The fasit is another repository's and was not touched.** It was set in
`vegnormal-okf` on 2026-09-10 before any lookup ran, and this round read it
and the instrument and wrote to neither.
- **KP and KN are CONTROLS, not results.** Neither enters hit@k. KP holding rank
1 is what says the instrument still counts; KN staying not-a-hit is what says
the rule invents nothing.
- **The PDF arm's zeros on the same questions are a SEGMENTATION miss**, not a
ranking one, and are not comparable with the XML arm's ranks. The presence
check in § 1 is what separates the two.
- **"Solved" means three measured misses were decomposed and the row moved.** It
does not mean a fourth failure shape does not exist. Coverage is still
unnormalised; this rule adds a precision statement about the title and none
about the body.
- **The rule's reach outside R761 is ZERO in everything measured.** "No
regression on K2 and the three N bundles" is really "the rule never fires
there" — 0 of 21 cells — not "it fires and is harmless". A corpus whose
section titles are phrases rather than terms gets nothing from it.
- **S5's group is 9 concepts, all titled `Armering`**, and the fusion's own
order put the right one first. That is one case, not a property: where a
publisher reuses a title, this rule floats all holders and decides nothing
between them.
- **The sweep harness is not the product.** Its candidate ranks and the CLI's
delivered ranks differ by the cut; the falsifications in § 2 and the
arithmetic in § 3 are harness figures, everything in § 4 and § 5 is not.
- **This is the fourth reading-side change that moves a payload with NO bundle
changing.** A consumer pinned to the previous excerpt order needs
`--no-title-covered`. Measured, that consumer is nobody on K2 or the N
bundles: their payloads are byte-identical either way.

8
src/llm_ingestion_okf/consume.py
View file

@ -1159,14 +1159,14 @@ DEFAULT_STEM_PREFIX = True
#: Round 16. Whether a concept whose WHOLE title the question accounts for is #: Round 16. Whether a concept whose WHOLE title the question accounts for is
#: read before the concepts the fusion ranked above it. ON since 2026-09-11. #: read before the concepts the fusion ranked above it. ON since 2026-09-10.
#: #:
#: THE DEFECT IT REPAIRS. Both lexical signals are unnormalised COVERAGE #: THE DEFECT IT REPAIRS. Both lexical signals are unnormalised COVERAGE
#: COUNTS -- one per question token the candidate answers to -- so nothing in #: COUNTS -- one per question token the candidate answers to -- so nothing in
#: the fusion measures how much of the CANDIDATE the question accounts for. A #: the fusion measures how much of the CANDIDATE the question accounts for. A
#: section titled with the question's subject and nothing else scores exactly #: section titled with the question's subject and nothing else scores exactly
#: what a narrower section titled with that subject PLUS a qualifier scores, #: what a narrower section titled with that subject PLUS a qualifier scores,
#: and then loses to it on the body count. Measured 2026-09-11 on a #: and then loses to it on the body count. Measured 2026-09-10 on a
#: 2 761-concept bundle of one standard, where the answering section carries #: 2 761-concept bundle of one standard, where the answering section carries
#: the bare term as its title on three of six scored questions: `Hovedprosesser` #: the bare term as its title on three of six scored questions: `Hovedprosesser`
#: behind `Hovedprosess 81 ...`, `Armering` behind `Armering av ...`, #: behind `Hovedprosess 81 ...`, `Armering` behind `Armering av ...`,
@ -1976,12 +1976,12 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
default=DEFAULT_TITLE_COVERED, default=DEFAULT_TITLE_COVERED,
help=( help=(
"read a concept whose WHOLE title the question accounts for before " "read a concept whose WHOLE title the question accounts for before "
"the concepts the fusion ranked above it. ON since 2026-09-11. " "the concepts the fusion ranked above it. ON since 2026-09-10. "
"Measured on a 2 761-concept bundle of one standard: hit@1 over six " "Measured on a 2 761-concept bundle of one standard: hit@1 over six "
"questions 3 of 6 -> 6 of 6 with the known-positive holding rank 1, " "questions 3 of 6 -> 6 of 6 with the known-positive holding rank 1, "
"where none of the six existing reading-side flags moved that " "where none of the six existing reading-side flags moved that "
"number at all. The title is read by EQUALITY, never by shared " "number at all. The title is read by EQUALITY, never by shared "
"prefix. See docs/2026-09-11-k3-runde16-hele-tittelen-tar-ruten.md" "prefix. See docs/2026-09-10-k3-runde16-hele-tittelen-tar-ruten.md"
), ),
) )
parser.add_argument( parser.add_argument(